we need some sanity test from performance perspective,basically need to get end-end performance result on daily build automatically:
1. end-end response time
2. each page size info
3. clean cache between each test round
so the first item is relative easy to achieve and more straight forward by Watir
for second item, after checking out http://www.newsqa.com/delete-cache-and-cookies-in-ruby/, it works well on Windows XP_SP3+IE6 browser (did not try any other platform)
for the third item, current Iuse "ie.html.length" to get page size, any other better idea? I did not dig this deeper right now.
Or i may consider install a HTTPWatch +Watir to do all the things, haha :)
Friday, August 28, 2009
Wednesday, August 26, 2009
A bonus get from QA who helps us test website Performance
I always borrow our QA team to help us validate performance on staging env when they do release testing as a bonus, here is an example email I sent to the group:
Hi, All QA friends,
Once you find slowness on your side(for example 60 seconds to login, …), please do so :
1. Can you reproduce it? Or just randomly replicate?
2. Username/password/account
3. Which server are you touching? (you need some sniffer tools to determine this if there is a cluster on back-end)
4. Time stamp you encounter the problem(PST time preferred/Beijing time zone is also OK)
5. What steps you are following? (replicate steps if have)
Of course we need to set up some monitoring/profiling tools behind or run ping -t IP to guarantee Network is OK during testing phase.
Hi, All QA friends,
Once you find slowness on your side(for example 60 seconds to login, …), please do so :
1. Can you reproduce it? Or just randomly replicate?
2. Username/password/account
3. Which server are you touching? (you need some sniffer tools to determine this if there is a cluster on back-end)
4. Time stamp you encounter the problem(PST time preferred/Beijing time zone is also OK)
5. What steps you are following? (replicate steps if have)
Of course we need to set up some monitoring/profiling tools behind or run ping -t IP to guarantee Network is OK during testing phase.
Monday, August 24, 2009
Performance test scope requirement collection table
I would like to provide a table which include a relative simple and high level categories of performance factors to PM/Arch/Dev leader/(Func)QA leader/ module owner for collecting comments or requirements in numbers/priority from Func team point of view.
Here is an example(Click to view big):

So it can help you to filter some work easily this way :) and then prioritize other remaining tasks/user stories.
Here is an example(Click to view big):
So it can help you to filter some work easily this way :) and then prioritize other remaining tasks/user stories.
Tuesday, August 11, 2009
How to Create Date offset in LR
just for my work, i have to create Date offset in LR myself, hopefully it is helpful for you as well, it is just a simple example, you can use yours:)
lr_save_datetime("%d", DATE_NOW, "Day1");
lr_save_datetime("%m", DATE_NOW, "Mon1");
lr_save_datetime("%Y", DATE_NOW, "Year1");
if ((atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}")))<= 29)
{
lr_save_int(atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}")), "param1");
lr_save_int(atoi(lr_eval_string("{Mon1}")), "param2");
lr_save_int(atoi(lr_eval_string("{Year1}")), "param3");
lr_output_message("parameter is %s-%s-%s", lr_eval_string("{param3}"),lr_eval_string("{param2}"),lr_eval_string("{param1}"));
}
else if ((atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}")))> 29 && atoi(lr_eval_string("{Mon1}"))<12)
{
lr_save_int(atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}"))-29, "param1");
lr_save_int(atoi(lr_eval_string("{Mon1}"))+1, "param2");
lr_save_int(atoi(lr_eval_string("{Year1}")), "param3");
lr_output_message("parameter is %s-%s-%s", lr_eval_string("{param3}"),lr_eval_string("{param2}"),lr_eval_string("{param1}"));
}
else if ((atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}")))> 29 && atoi(lr_eval_string("{Mon1}"))==12)
{
lr_save_int(atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}"))-29, "param1");
lr_save_int(01, "param2");
lr_save_int(atoi(lr_eval_string("{Year1}"))+1, "param3");
lr_output_message("parameter is %s-%s-%s", lr_eval_string("{param3}"),lr_eval_string("{param2}"),lr_eval_string("{param1}"));
}
lr_save_datetime("%d", DATE_NOW, "Day1");
lr_save_datetime("%m", DATE_NOW, "Mon1");
lr_save_datetime("%Y", DATE_NOW, "Year1");
if ((atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}")))<= 29)
{
lr_save_int(atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}")), "param1");
lr_save_int(atoi(lr_eval_string("{Mon1}")), "param2");
lr_save_int(atoi(lr_eval_string("{Year1}")), "param3");
lr_output_message("parameter is %s-%s-%s", lr_eval_string("{param3}"),lr_eval_string("{param2}"),lr_eval_string("{param1}"));
}
else if ((atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}")))> 29 && atoi(lr_eval_string("{Mon1}"))<12)
{
lr_save_int(atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}"))-29, "param1");
lr_save_int(atoi(lr_eval_string("{Mon1}"))+1, "param2");
lr_save_int(atoi(lr_eval_string("{Year1}")), "param3");
lr_output_message("parameter is %s-%s-%s", lr_eval_string("{param3}"),lr_eval_string("{param2}"),lr_eval_string("{param1}"));
}
else if ((atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}")))> 29 && atoi(lr_eval_string("{Mon1}"))==12)
{
lr_save_int(atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}"))-29, "param1");
lr_save_int(01, "param2");
lr_save_int(atoi(lr_eval_string("{Year1}"))+1, "param3");
lr_output_message("parameter is %s-%s-%s", lr_eval_string("{param3}"),lr_eval_string("{param2}"),lr_eval_string("{param1}"));
}
Thursday, July 09, 2009
How to get page size data in LR scripts part2--extract common code into header file
Step1:
Create your common method in Loadrunner installation folder, which is usually in ..\Program Files\HP\LoadRunner\include
In calpagesize.h file, here is the source code which simply return the “current page size” you want to measure:
int calpagesize(int Pre_size){
int Temptotalsize=0;
int currentResponseSize = 0;
Temptotalsize = web_get_int_property(HTTP_INFO_TOTAL_RESPONSE_STAT);
return currentResponseSize = Temptotalsize - Pre_size;
}
Step2:apply the calpagesize() method:
Adding #include “calpagesize.h” in your globals.h file
Step3: Adding calculate page size code(in red line) into action:
Take Yahoo home page for example:
Action()
{
int Pre_page_size=0;
int cur_page_size=0;
Pre_page_size=web_get_int_property(HTTP_INFO_TOTAL_RESPONSE_STAT); //get accumulate page size data before sending next request
web_url("www.yahoo.com",
"URL=http://www.yahoo.com/",
"Resource=0",
"RecContentType=text/html",
"Referer=",
"Snapshot=t1.inf",
"Mode=HTML",
LAST);
if (flag==1){ //flag==1 means trigger the page size func code
cur_page_size = calpagesize(Pre_page_size); //return yahoo home page page size
lr_save_int(atoi(lr_eval_string("{Avg_01_PageSize}"))+cur_page_size, "Avg_01_PageSize"); //save into parameter ” Avg_01_PageSize”
}
}
So normally, we may need to add 4 lines before and after the page you want to calculate the size.
finally, you can calculate the page size in Vuser_end():
lr_message("Avg_01_PageSize = %d", atoi(lr_eval_string("{Avg_01_PageSize}"))/atoi(lr_eval_string("{Iterations}")));
enjoy!
Cheng
Create your common method in Loadrunner installation folder, which is usually in ..\Program Files\HP\LoadRunner\include
In calpagesize.h file, here is the source code which simply return the “current page size” you want to measure:
int calpagesize(int Pre_size){
int Temptotalsize=0;
int currentResponseSize = 0;
Temptotalsize = web_get_int_property(HTTP_INFO_TOTAL_RESPONSE_STAT);
return currentResponseSize = Temptotalsize - Pre_size;
}
Step2:apply the calpagesize() method:
Adding #include “calpagesize.h” in your globals.h file
Step3: Adding calculate page size code(in red line) into action:
Take Yahoo home page for example:
Action()
{
int Pre_page_size=0;
int cur_page_size=0;
Pre_page_size=web_get_int_property(HTTP_INFO_TOTAL_RESPONSE_STAT); //get accumulate page size data before sending next request
web_url("www.yahoo.com",
"URL=http://www.yahoo.com/",
"Resource=0",
"RecContentType=text/html",
"Referer=",
"Snapshot=t1.inf",
"Mode=HTML",
LAST);
if (flag==1){ //flag==1 means trigger the page size func code
cur_page_size = calpagesize(Pre_page_size); //return yahoo home page page size
lr_save_int(atoi(lr_eval_string("{Avg_01_PageSize}"))+cur_page_size, "Avg_01_PageSize"); //save into parameter ” Avg_01_PageSize”
}
}
So normally, we may need to add 4 lines before and after the page you want to calculate the size.
finally, you can calculate the page size in Vuser_end():
lr_message("Avg_01_PageSize = %d", atoi(lr_eval_string("{Avg_01_PageSize}"))/atoi(lr_eval_string("{Iterations}")));
enjoy!
Cheng
Thursday, May 14, 2009
You got GrooveUtil.dll load error on LR9.5?
When I record scripts using LR9.5 on my laptop which install the office2007 there: error happens:

here is my temp solution:
Find GrooveUtil.dll under C:\Program Files\Microsoft Office\Office12, and rename this GrooveUtil.dll into other name :)
click to view big picture

My friend Alfred found out another solution for this issue, just disable the Groove GFS help on IE Add-ons:
here is my temp solution:
Find GrooveUtil.dll under C:\Program Files\Microsoft Office\Office12, and rename this GrooveUtil.dll into other name :)
click to view big picture
My friend Alfred found out another solution for this issue, just disable the Groove GFS help on IE Add-ons:
Add page size funcion code into your LR scripts
As my ramp up project, I spent some time to complete a sample on how to get page size after applying small piece of code
Choosing Ajax(click and scripts) protocol;
pre-condition: set your log level as "send message only when an error occurs"
Within globe()
set one flag to trigger the page size func code on demand:
int flag=0;
Within vuser_init()Action, you can initial such parameter as:
lr_save_int(0, "accu_google_frontpage_PageSize");
Within Action() i take google landing page as a example:
within end() action, calculate the average number of each page size:
So you will get each page size freely now!
Thanks my friend Jeff,Alfred and Calvin previous sample code and ideas!
Choosing Ajax(click and scripts) protocol;
pre-condition: set your log level as "send message only when an error occurs"
Within globe()
set one flag to trigger the page size func code on demand:
int flag=0;
Within vuser_init()Action, you can initial such parameter as:
lr_save_int(0, "accu_google_frontpage_PageSize");
Within Action() i take google landing page as a example:
int ThisTimeHttpResponseSize=0;
char *message;
int i=0;
int loops=2;
lr_save_int(loops,"loops_goo");
while(i小于loops){//if there is some loops here
message=(char *)malloc(140*sizeof(char));
memset(message, 0, sizeof(char));
web_browser("www.google.com",
DESCRIPTION,
ACTION,
"Navigate=http://www.google.com/",
LAST);
i++;
if (flag==1) {//enable pagesize func, if flag==1;
lr_set_debug_message(LR_MSG_CLASS_JIT_LOG_ON_ERROR, LR_SWITCH_OFF);
//web_get_int_property Returns the accumulated size, including header and body,
ThisTimeHttpResponseSize = web_get_int_property(HTTP_INFO_DOWNLOAD_SIZE);
strcat(message, "PageSize_of_google_frontpage_");
strcat(message, lr_eval_string("{Iteration}"));
strcat(message, " = %d");
lr_message(message, ThisTimeHttpResponseSize);
lr_set_debug_message(LR_MSG_CLASS_JIT_LOG_ON_ERROR, LR_SWITCH_ON);
//multiple iteration accumulated
lr_save_int(atoi(lr_eval_string("{accu_google_frontpage_PageSize}"))+ThisTimeHttpResponseSize, "accu_google_frontpage_PageSize");
}
free(message);
}
within end() action, calculate the average number of each page size:
if (flag==1) {
lr_save_int(atoi(lr_eval_string("{Iteration}")), "Iterations");
lr_set_debug_message(LR_MSG_CLASS_JIT_LOG_ON_ERROR, LR_SWITCH_OFF);
lr_message("Iterations=%s",lr_eval_string("{Iterations}"));
lr_message("Avg_google_frontpage_PageSize = %d", atoi(lr_eval_string("{accu_google_frontpage_PageSize}"))/(atoi(lr_eval_string("{loops_goo}"))*atoi(lr_eval_string("{Iterations}"))));
lr_set_debug_message(LR_MSG_CLASS_JIT_LOG_ON_ERROR, LR_SWITCH_ON);
}
So you will get each page size freely now!
Thanks my friend Jeff,Alfred and Calvin previous sample code and ideas!
Wednesday, May 13, 2009
Tester,not only for counting bugs
What is one Tester's Mission?
--Counting bugs?
Are the company pay for bugs we have found?
Are the company recongize one's performance by how many bugs you already find?
If yes, two options:
1> you can go to dev, say "hi, dude, please bury some bugs and let me find them". Of course if that person who may not want to continue his career in this company or he really a "brother" to you.
2> or you may find a "right" partner who will often make mistakes, then you will be a million easily.
--Help dev to make less mistakes?
I agree!Even if i can not easily find a bug now, i am glad to see our application so stable and trustable...
I am happy to work with even he may write a bugless code, you can help him continuously improvement, meanwhile you can just dig deeper treasure which may need more effort or deeper understanding on application functionalities and infrastructure. More important, this, of course, makes customer happy(they may hardly find out bugs as well).You make the company win!
--Counting bugs?
Are the company pay for bugs we have found?
Are the company recongize one's performance by how many bugs you already find?
If yes, two options:
1> you can go to dev, say "hi, dude, please bury some bugs and let me find them". Of course if that person who may not want to continue his career in this company or he really a "brother" to you.
2> or you may find a "right" partner who will often make mistakes, then you will be a million easily.
--Help dev to make less mistakes?
I agree!Even if i can not easily find a bug now, i am glad to see our application so stable and trustable...
I am happy to work with even he may write a bugless code, you can help him continuously improvement, meanwhile you can just dig deeper treasure which may need more effort or deeper understanding on application functionalities and infrastructure. More important, this, of course, makes customer happy(they may hardly find out bugs as well).You make the company win!
Tuesday, May 05, 2009
Not Only Keeping walking, But Keeping Thinking
Monday, May 04, 2009
Run, Rockets, Run
Subscribe to:
Posts (Atom)