Wednesday, April 15, 2009

Performance testing process in one sprint(after we choose Scrum)

1> Performance Planning:
--- Requirement analysis and make performance test strategy (How to test in this sprint, what kind of user story need to do the user story based test)
--- Define performance criteria( such as 90% percentile under 3 seconds page server response time under a certain load)
--- Design User patterns(what) and User distribution(How) to form a diagram scenario to describe or demo this easily

2> Performance Environment set up
---At least do weekly build on our local performance environment based on our work load and progress

3> Test Data Preparation (we intent to not using production data, but making the same size of data in our database)
--- Using store procedure to generate data into our DB
--- Using External scripts to generate data: For example, JMeter to create some "fake" data during one night so that you can test tomorrow; BTW, using Jmeter to generate tons of data is another way of load testing itself :)

4> Test Scripts design and debugging
--- Parametrization, correlation
--- Error Handling and Content Check points
--- Logic control flow, generate random number or string processing

5> Scenario design
---user load(combine all the scripts based on user patterns and user distributions, draw a UCML diagram if you like)
---think time, download resource or not?
---simulate browser cache or not?
---test schedule
......

6> Trial Run(Errors Clean up)
--- Single thread run for whole scenario: Focusing on SQL profiling result and Do initial DB layer tuning
--- Trail run with concurrent users(relative low load) in order to clean up any errors caused by scripts or application

7> Iterative Testing and Tuning(Time boxed testing & tuning phase)
--- Run the full scenario under the same user load as last sprint, guarantee that no any degradation
--- Trying different load to see the capacity limit in current sprint
--- Get a new benchmark with best support load on your performance environment
--- If any performance issue found, then Test and tune module by module (testing duration may only 10-20 mins with less think time)

8> Report analysis
--- Based on test and tuning result, monitoring logs, error logs...

9> Log defects in to bug tracking system
--- Log the defects in order to priorities and track for next sprint

It is just a guideline for tester, you can freely change the sequence and priority of each task. Of Course, You need to collaborate developer friends closely, you are not alone, check out:
http://joychester.blogspot.com/2009/01/performance-testers-you-do-need-friends.html

Thank my teammates for always support and trust! We can make our performance even better!

No comments:

Post a Comment