Tuesday, April 28, 2009

Main purpose on doing performance tests in my mind

1> Help to find out the (Top) bottlenecks, so that we are able to focus on and make a plan to tune them one by one;
2> Help team/PM to make a decision on release by measuring the trend of performance, guarantee no degradation on each release. (I prefer to taking the throughput trend and overall performance data rather than individually response time as a degradation indicator)
3> Help dev to do the hot spot tuning, we will "agilely" and "aggressively" change our test strategy during tuning phase :)
check out My previous blog on performance test strategies:
http://joychester.blogspot.com/2009/04/performance-test-strategy-in-my-mind.html
4> Validate some "guessing" or "pilot" projects
5> Capacity planning, scalability testing(horizontally and vertically)
6> Bonus! Finding out functional or security bugs in application occasionally
。。。
Feel free to add yours:)

Thursday, April 23, 2009

一双破袜子引发的争论



In Million Dollar baby, I like these lines between Eastwood and Freeman so much! :)

Background: Freeman put his feet on Eastwood's desk, then Eastwood came in...

Eastwood:
Where are your shoes?(你鞋呢?)

Freeman:
I am airing out my feet.(我脚透透气)

Eastwood:
You got big holes in your socks(袜子那么大的洞)

Freeman:
Oh, they're not that big(哦,这还不算大的)

Eastwood:
Didn't I give you money for some new ones?(我之前没给你钱让你买双新的?)

Freeman:
These are my sleeping socks, My feet like a little air at night.(这是我睡觉时的袜子,脚丫喜欢在晚上透透气)--居然喜欢穿袜子睡觉??

Eastwood:
How come you're wearing them in the daytime, then?(那你白天怎么穿晚上的袜子?)

Freeman:
Because my daytime socks got too many holes in them.(噢,白天袜子的洞更大)

Eastwood:
Well, if i give you some more money, you buy some new socks, please??(靠,如果我多给你点钱,你能不能买几双新的,拜托)

Freeman:
Well, i'd be tempted, but i couldn't say for sure. Might find its way to the track.(呦,这倒是个好主意,不过我不能保证,这钱真的会用到袜子上)

Eastwood:
。。。(无语)

“疯子”和“魔方”



《当幸福来敲门》一直有一个问题没有搞懂,为什么影片会有一个“疯子”一直跟Chris“过不去”?是巧合还是别有用心?(Sorry,叫疯子并非歧视,是因为不知道他的称呼。。。)


“疯子”是一个标尺,心理的标尺,标志着Chris崩溃的底线,也许疯子十几年之前的状态正和现在的Chris一样??很显然,结果是“疯子”没有把握住,而Chris 挺住了!其实有时候Chris的境遇并不如“疯子”,但他在挣扎,坚持这自己的希望,在等待幸福来临时的那一刻,很显然,正和海报上的图画一样,他当时所有的一切就是儿子和他赖以生存的“time machine”!

我想,这就是“疯子”在电影里存在的必要吧 :)

小小的“魔方”,威力巨大,它代表着机遇,如果当时在出租车上高级经理人没有玩魔方,或者说Chris自己不会玩,也许经理人不会对这个素不相识的“穷光蛋”有什么好的印象。

正是魔方让他对Chris刮目相看,一切就是那么巧合,这就是对机遇的最佳诠释--在正确的时间,相遇在正确的地点,遇到正确的人,做了正确的事,一切就有了转机!
享受当幸福来临的时刻吧!It really belongs to you!

Thursday, April 16, 2009

今天,你XXX了吗?

突然想到一个句式:
--今天,你云计算了吗?
--今天,你敏捷了吗?
--今天,你Google了吗?
--今天,你百度了吗?
--今天,你twitter了吗?
。。。
--今天,你吃饭了吗?
--今天,你OUT了吗?

结论,看到娃哈哈有个啤酒饮料的广告,发现自己真的OUT了

Wednesday, April 15, 2009

Performance testing process in one sprint(after we choose Scrum)

1> Performance Planning:
--- Requirement analysis and make performance test strategy (How to test in this sprint, what kind of user story need to do the user story based test)
--- Define performance criteria( such as 90% percentile under 3 seconds page server response time under a certain load)
--- Design User patterns(what) and User distribution(How) to form a diagram scenario to describe or demo this easily

2> Performance Environment set up
---At least do weekly build on our local performance environment based on our work load and progress

3> Test Data Preparation (we intent to not using production data, but making the same size of data in our database)
--- Using store procedure to generate data into our DB
--- Using External scripts to generate data: For example, JMeter to create some "fake" data during one night so that you can test tomorrow; BTW, using Jmeter to generate tons of data is another way of load testing itself :)

4> Test Scripts design and debugging
--- Parametrization, correlation
--- Error Handling and Content Check points
--- Logic control flow, generate random number or string processing

5> Scenario design
---user load(combine all the scripts based on user patterns and user distributions, draw a UCML diagram if you like)
---think time, download resource or not?
---simulate browser cache or not?
---test schedule
......

6> Trial Run(Errors Clean up)
--- Single thread run for whole scenario: Focusing on SQL profiling result and Do initial DB layer tuning
--- Trail run with concurrent users(relative low load) in order to clean up any errors caused by scripts or application

7> Iterative Testing and Tuning(Time boxed testing & tuning phase)
--- Run the full scenario under the same user load as last sprint, guarantee that no any degradation
--- Trying different load to see the capacity limit in current sprint
--- Get a new benchmark with best support load on your performance environment
--- If any performance issue found, then Test and tune module by module (testing duration may only 10-20 mins with less think time)

8> Report analysis
--- Based on test and tuning result, monitoring logs, error logs...

9> Log defects in to bug tracking system
--- Log the defects in order to priorities and track for next sprint

It is just a guideline for tester, you can freely change the sequence and priority of each task. Of Course, You need to collaborate developer friends closely, you are not alone, check out:
http://joychester.blogspot.com/2009/01/performance-testers-you-do-need-friends.html

Thank my teammates for always support and trust! We can make our performance even better!

Tuesday, April 14, 2009

Performance test strategy in my mind (after we choose scrum)

1.User story based performance testing

Testing Scope examples:
Code refractory, algorithm changed, internal design or infrastructure change, such as JDK 1.5.06 upgrade to JDK 1.5.16, Front-end tuning,single thread using Watir automation scripts to see if any slowness or client side memory leak…

Test pattern:
Write new component/unit scripts to add load just related to this user story on local performance environment

2.Performance related configuration change testing

Testing Scope examples:

Apache tuning, Tomcat tuning, JVM GC tuning, DB index tuning…

Test pattern:
Go through the most common/critical scenarios(just a simple load scripts),and monitoring the footprint and all performance measurements

3.Weekly performance testing

Testing Scope examples:
For tracking the application overall performance trend, compare with before picture and after picture, in order to guarantee no degradation on performance

Test pattern:
Combine all existing component performance scripts;
Based on the latest build on local perf environment, and getting performance measurements with the latest test scenarios(User patterns + User distribution), do a comparison or judgment, in order to prevent any performance degradation due to new code check in or any resource contention

4. Load/Stress/Capacity testing

Test pattern:
Draw a diagram of throughput curve;
Figure out the best support load, and threshold of our application;
Make sure how our application behaves under heavy load or different load

5. Long duration performance testing

Test scope examples:
No memory leak? No deadlock? No unexpected Crash?

Test Pattern:
Run the combined performance test scenario all night long, check your result next morning

6.Release performance testing

Test scope examples:
No surprise and guarantee that 90% page server response time is under 3 seconds on stage environment, Help to judge the overall performance of this release

Test Pattern:
The same as weekly testing strategy, should be a formal weekly performance test

You may add more kind of tests, I may ask myself before testing:"What do you want to get from this kind testing? Is it worth or just waste of my time? Do we have a better Strategy?"

It reminds me something...

It reminds me 3.5 years ago:



He is that kind of person~~~

I am an organizer:


I can pass:


I can shoot:


I can "Dunk":


I Have My Friend behind:


Sometimes, I will feel upset:


But most of time, I feel Great!


Being Trusted, Supported,Recognized is Sweetest thing in my life!


Thanks and Best Wishes to You,
Truly,Deeply,Silently!!

Monday, April 13, 2009

How to debug Performance Scripts Issue in Loadrunner

Here are my thoughts:

1. Do you have content check points in your current load testing scripts?
If not, adding check points on the most critical actions for tracking; This is very important for the performance scripts design:
You may want to Check these:
http://joychester.blogspot.com/2009/01/lr-scripts-debuging-on-controller-try.html
http://joychester.blogspot.com/2009/03/content-check-for-web-service-calls-2.html
http://joychester.blogspot.com/2009/02/content-check-for-webservicecalls.html

2. Try to isolate the problem,not mix everything up at the same time.
You can run just one performance scripts at a time which has the problem, and you can run it under different user load, so that you may reproduce the issue easily or just guarantee that this script has no problem;

3. Checking the very first error you met in your scripts.
Usually it is the first thing you need to be caution when you running the load testing. Because it may trigger a tons of following errors, if you did not do error handling well

4. Try to look at the parameters/correlation value you use within the scripts.
This is a check point when you find there is an error on your scripts, make sure you use the write parameters/correlation value before you start to do other deeper investigation. It is not easy to find if you do not know the application very well

5. Of course you should run your scripts in Vugen successfully before running it in Controller.
But I want to remind is that you have to validate all the parameters or situations you may meet, such as using different accounts, different data, also do the Branch coverage if you have "if-clause" in your scripts. So you may get my idea--"Do your unit test on your test scripts" :)

6. Sometimes or more often,The errors may just related to the load you add to the system, not caused by scripts itself.
This errors are only reproduce under a certain load, so it might not be a scripts issue... You can do a load test on your scripts to verify your idea--10 users load, 20 users load, 30... till you get such an error.
But one kind of error is not related to the load, for example, you just input one column which need to be unique value, but in your scripts does not implement this well, so that in Vugen(single user load) or even low load you will not get kind of error, however, if two thread generate this value at the same time, then you may get duplicated value for these two users...thus, you will get an system error,but actually this is absolutely a scripts issue! Be careful!

7. Last and the best practice--After running your scripts in Vugen, please look at your application whether the scripts make the action happen or not, you should believe your eyes, right? :)

Tuesday, April 07, 2009

How to generate random number in a dynamic world

Besides the random number by Loadrunner parameter itself, usually you need to create a Random number of which range is based on the correlated result, for example, you want to check a check-box which number is dynamically changed:



int checkbox_num1=0;
char Randomitem[10];
char RandomStatus[10];
char SRanStatus[10];
...

web_reg_save_param("Checkbox_name1",
"LB=TableSelectCheckbox-_dash_",
"RB=)\" >",
"ORD=ALL",
LAST);

web_submit_data(...
.... );

checkbox_num1 = atoi(lr_eval_string("{Checkbox_name1_count}"));

srand(time(NULL));

if (checkbox_num1>0) {

itoa(rand() % checkbox_num1 +1,Randomitem,10);

sprintf(RandomStatus,"{Checkbox_name1_%s}",Randomitem);

sprintf(SRanStatus, "%s", lr_eval_string(RandomStatus));

lr_save_string(SRanStatus, "CheckboxORD");

}

else if (checkbox_num1==0) {//there is no any suitable value found

lr_exit(LR_EXIT_ITERATION_AND_CONTINUE, LR_AUTO);

}

Loadrunner: How to insert a charactor(s) into a string

I want to replace the '\' into '\\' in one string, here is my implementation in Loadrunner:


char * position;
int i=0;
int j=0;
Char * SRanStatus2_backup;
Char SRanStatus2[50]="abcc\dada-com";

SRanStatus2_backup= (char *)malloc(50 * sizeof(char));

memset(SRanStatus2, 0, sizeof(SRanStatus2));

memset(SRanStatus2_backup, 0, sizeof(SRanStatus2_backup));

position = SRanStatus2;

while ((*position)!= NULL) {

if ((*position)!='\\') { //if ((*position)!='\x07') {

SRanStatus2_backup[j]=SRanStatus2[i];
i++;
j++;
}

else if ((*position)=='\\') { //else if ((*position)=='\x07') {
SRanStatus2_backup[j]='\\';
SRanStatus2_backup[j+1]='\\';
j=j+2;
i++;
}

position++;
}

SRanStatus2_backup[j]='\0';

lr_save_string(SRanStatus2_backup, "CheckboxORD_2_backup");

free(SRanStatus2_backup);
free(SRanStatus2);

return 0;

Friday, April 03, 2009

A Joke on a Slow response page

My friend and I take a look at one website, when we got to one page, it takes almost 2 mins loading...

Me: So damn slow page, and no useful information. Let's go to another page to have a try

I was taking my mouse to try to click next link, while my friend stop me at once!

My friend speak seriously: NO! NO! NO! Let's treasure this Hard-earned page, take more seconds on it to retrieve our time losses, even it give us nothing information...

we just stared at that slow page for a few seconds quietly, and never come back in the future