Tuesday, April 14, 2009

Performance test strategy in my mind (after we choose scrum)

1.User story based performance testing

Testing Scope examples:
Code refractory, algorithm changed, internal design or infrastructure change, such as JDK 1.5.06 upgrade to JDK 1.5.16, Front-end tuning,single thread using Watir automation scripts to see if any slowness or client side memory leak…

Test pattern:
Write new component/unit scripts to add load just related to this user story on local performance environment

2.Performance related configuration change testing

Testing Scope examples:

Apache tuning, Tomcat tuning, JVM GC tuning, DB index tuning…

Test pattern:
Go through the most common/critical scenarios(just a simple load scripts),and monitoring the footprint and all performance measurements

3.Weekly performance testing

Testing Scope examples:
For tracking the application overall performance trend, compare with before picture and after picture, in order to guarantee no degradation on performance

Test pattern:
Combine all existing component performance scripts;
Based on the latest build on local perf environment, and getting performance measurements with the latest test scenarios(User patterns + User distribution), do a comparison or judgment, in order to prevent any performance degradation due to new code check in or any resource contention

4. Load/Stress/Capacity testing

Test pattern:
Draw a diagram of throughput curve;
Figure out the best support load, and threshold of our application;
Make sure how our application behaves under heavy load or different load

5. Long duration performance testing

Test scope examples:
No memory leak? No deadlock? No unexpected Crash?

Test Pattern:
Run the combined performance test scenario all night long, check your result next morning

6.Release performance testing

Test scope examples:
No surprise and guarantee that 90% page server response time is under 3 seconds on stage environment, Help to judge the overall performance of this release

Test Pattern:
The same as weekly testing strategy, should be a formal weekly performance test

You may add more kind of tests, I may ask myself before testing:"What do you want to get from this kind testing? Is it worth or just waste of my time? Do we have a better Strategy?"

No comments:

Post a Comment