Thursday, January 08, 2009

Performance testing/tuning metrics update version

long time ago.... i wrote a initial draft for the key performance metrics i always considered during performance testing and tuning:
http://joychester.xhblog.com/archives/2008/318932.shtml

I want to emphasize and thinning them this time:
Performance testing metrics:
1. Test duration and Time period
2. User load, User distribution on all test scenarios(diagram better)
3. Server response time (top5 + Detail 90% line)
4. Response time highlight for Top 5 transactions and trend(diagram)
5. Error Rate=errors/total number of requests(or particular action's failed rate)
6. Throughput(KB/sec, hits/sec)
7. All servers OS resource monitoring
- CPU% in average and its trend: total CPU%, User CPU%, system CPU%
- Memory% committed in use
- Disk I/O (DB server, App server and file server)
8. Raw data for all response data(attachment)
9. Server error logs---Apache/Jboss error log, if any, Dev should investigate and solve them first(either application, configuration or test scripts issue)!


Performance tuning measurement supplementary
1. SQL profiler log and trend(average,Max, where duration> 500ms order by duration desc)
2. Thread Dump
3. GC trend, or even heap dump
4. Profiling logs--methods level time consuming
5. Apache Access logs
6. JMS table(including DLQs), ActiveMQ monitoring when there are some asynchronous processes on the back-end
7. User accounts or some data sensitive parameters should be provided into consideration and correlated with the response time
- different accounts has different volume of data
- different key words search may grab different volume of data
8. other kind of perf testing result attached, for example change configuration testing or single step isolation testing result

No comments:

Post a Comment