Performance engineering internal group(team) responsibility

Here is a draft of internal performance group(team) responsibility chart in my mind, so much things to do, we need to prioritize each section better to maximum its value :

Responsibility	Tasks	Role	Delivery
Capacity planning	1. Current user pattern/distribution 2. Gap analysis 3. Predict trend of usage 4. Top slow transactions 5. Top usage of transactions	QA lead, Performance QA(owner) PM, OPS(supportive) SDM, TL(consultant)	Access log analysis result RFP distribution pattern
Performance Environment Setup	1. Setup Performance environment 2. Data preparation 3. Server configuration	SCM/Dev (owner) Performance QA (supportive)	Testable performance environment
Performance test planning and design	1. Performance test strategy design 2. Performance testing scripts/scenario design 3. Keep performance test scripts/ load distribution update to date	Performance QA(owner) Performance Dev(supportive) SDM, TL, QA lead(consultant)	Performance test scripts/ scenarios
Code review and design inspection	1. code review and Performance inspection checklist	Performance Dev , TL(owner) Performance QA(supportive) Arch(consultant)	Code review retro
Performance testing Execution	1. User-story based performance test 2. Weekly performance test 3. Release performance testing 4. End-end performance test 5. Performance initiatives 6. Performance test report	Performance QA(owner) Performance Dev(supportive)	Performance test result report
Performance result analysis	1. Server log issues analysis 2. Performance test result analysis 3. Conclusion of performance Grade and suggestion of further investigation	Performance Dev(owner) Performance QA(supportive)	Identify any potential Performance bottlenecks Escalate Performance issues to the team
Performance improvement planning	1. making follow up plan for escalated performance issues	SDM, TL, QA lead(owner) Performance QA/Dev(supportive)	Create tech user stories for each performance issue Triage the priority of each performance user story
Performance tuning	1. Java back-end performance tuning 2. DB tuning 3. Front-end performance tuning 4. Performance Validation	SDM, TL, Dev (owner) Performance QA(supportive)	Tuning analysis summary Performance improvement comparison result Code review Check-in code into new release
Production Performance issues Follow up	1. Log performance issue in Rally 2. Server log analysis 3. Reproduce performance issue on local 4. Find the root cause of bottleneck	Performance Dev(owner) Performance QA(supportive)	Performance issue analysis result Reproduce suggestions

Analysis Jmeter result log with Google Charts by Ruby

A thought from long time ago, but never pick it up...

Thanks to http://googlecharts.rubyforge.org/, it give me the inspiration on how to combine ruby and Google Charts, then i just make it by ruby

Thanks to my intern Tom as well, he has helped me to re-factoring of my previous code, he made the charts a little bit fancy than mine:)

enjoy it!

Note: the data of bellowing Diagram is not real data , just for testing purpose :)



Ruby Source Code V1.0:

require 'gchart'

def getresdata(block1)
# scan for the first numeric token
str = block1.scan(/\d+/)[0];
return str;
end

# format RGB
def fill(str)
i = 6 - str.length();
while(i > 0)
str = '0' + str;
i -= 1;
end
return str;
end

def format_color(num, boundary)
return "000000" if num < boundary =" 0"> 16777215;

color = boundary;
increment = (16777216 - boundary)/num;

retStr = "%x" % [boundary];
retStr = fill(retStr);

while(num > 1)
color += increment;
str = "%x" % [color];
str = fill(str);

num -= 1;
retStr += ',' + str;
end

return retStr;
end

def Draw_Perf_data(file)
url_sum = Hash.new();
IO.foreach(file) { |line|
next if !line.include?("lb="); # skip when this is xml head line

line.strip!(); # trim
url = line.scan(/lb="\w+/)[0]; # fetch the 'lb=' token
puts url.sub!("lb=\"", ""); # fetch the value of lb

if url_sum.has_key?(url)
url_sum[url].push(getresdata(line).to_i); # exist, then add cell
else
url_sum[url] = Array.new(1, getresdata(line).to_i); #!exist, then create
end
}

datas = Array.new();
actions = Array.new();

url_sum.each { |key, val|
datas.push(val);
actions.push(key);
}

colors = format_color(3, 0);
arrayabc = Gchart.line(:size => "800x375",:title => "Request Log", :data => datas, :line_colors => colors, :legend => actions, :axis_with_labels => ['y'], :custom => "chg=100,25");

puts(arrayabc);
return arrayabc;
end


######################
# Call the method, just input your Jmeter result log as a parameter, note it should be the default format of your logs

Draw_Perf_data("123.txt");

here is my configuration for my JMeter result file:

Performance env health check file

Previously, before i start perf testing, i usually went through the application to see if application works well. Checking if each server's java process running successfully or not at first will save us time(based on my working experience)

By just one click, I can tell the specific server contains Java process or not.

here is source file(BTW, you should input Gawk into the PsTools as well):

set PS_TOOLS_HOME=D:\\PsTools
set JVM_IP1=192.168.1.1

%PS_TOOLS_HOME%\pslist \\%JVM_IP1% -u "domain\username" -p "pwd" -e java | %PS_TOOLS_HOME%\gawk "{if($1 !~ /java/) {} else {print $0, \"\nGot you Java! :-)\"}}"
echo %JVM_IP1% Health check finished

Web access log analysis with Awk for Performance test modeling

I try to extract some end users' actions from Web access log, so that i can design my performance testing scenarios more accurately, not by assumption.

The scripts' purpose is to filter all the .do actions from access.log and i can use the output file to do more complexed analysis.

I named my script as "W3A.txt", here is the source:

 Begin {i=1}  
 $7~/\.do/ {  
 {if (i==ARGIND) {  
 sub(/;jsessionid=[0-9a-zA-Z]+.worker[0-9]/,"",$7 );  
 sub(/\?[a-zA-Z0-9_=&-\/:\?%]+/,"?",$7);  
 print $4,$7,$11 > i  
 }  
 else i++  
 }  
 { if (i> ARGIND)  
 exit 1  
 }  
 }  
 END{}  

(P.S. This log will give me each action's date/time, URL, response time)

how to use it?

$ awk -f W3A.txt access.2009-11-*_perf.log

Generated result files:

1.log
2.log
...

P.S.it can process multiple log files, and also generating result logs based on each log you put into the augments separately. if you just want to generate single result file, i think it is much easier than i did here :)
Hope helps!

JMeter "leaks" by storing the raw data into the heap

almost 1 year ago i wrote about load generated by JMeter decreasing after long time running, looking at this :http://joychester.blogspot.com/2009/01/jmeter-load-get-decreasing-after-long.html

After looking at the Heapdump of JMeter after on day running, we got the suspect:



I prefered to using the Aggregate listener as always, so it indeed to store every single response data into Heap, it may be one of the reason cause Jmeter slower and slower to generate the load itself.

Jmeter __javascript() got blocked

One of my Friends are using JMeter 2.3.2 __javascript() func to generate Random number, but after using VisualVM to monitoring Jmeter, we found such a threaddump:

"Thread Group 1-18" prio=6 tid=0x379c3800 nid=0xf3c runnable [0x39f2f000..0x39f2fd18]
java.lang.Thread.State: RUNNABLE
at org.mozilla.javascript.gen.c123605._c0()
at org.mozilla.javascript.gen.c123605.call()
at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:340)
at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:2758)
at org.mozilla.javascript.gen.c123605.call()
at org.mozilla.javascript.gen.c123605.exec()
at org.mozilla.javascript.Context.evaluateString(Context.java:1132)
at org.apache.jmeter.functions.JavaScript.execute(JavaScript.java:94)
- locked <0x0b9991e0> (a org.apache.jmeter.functions.JavaScript)
at org.apache.jmeter.engine.util.CompoundVariable.execute(CompoundVariable.java:138)
at org.apache.jmeter.engine.util.CompoundVariable.execute(CompoundVariable.java:107)
....

"Thread Group 1-17" prio=6 tid=0x390f4c00 nid=0x288 waiting for monitor entry [0x39edf000..0x39edfd98]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.jmeter.functions.JavaScript.execute(JavaScript.java:73)
- waiting to lock <0x0b9994a8> (a org.apache.jmeter.functions.JavaScript)
at org.apache.jmeter.engine.util.CompoundVariable.execute(CompoundVariable.java:138)
at org.apache.jmeter.engine.util.CompoundVariable.execute(CompoundVariable.java:107)
at org.apache.jmeter.testelement.property.FunctionProperty.getStringValue(FunctionProperty.java:87)
at org.apache.jmeter.testelement.property.AbstractProperty.hashCode(AbstractProperty.java:221)
at java.util.HashMap.getEntry(Unknown Source)
at java.util.HashMap.containsKey(Unknown Source)
at java.util.HashSet.contains(Unknown Source)
at org.apache.jmeter.testelement.AbstractTestElement.isTemporary(AbstractTestElement.java:376)
at org.apache.jmeter.testelement.AbstractTestElement.recoverRunningVersion(AbstractTestElement.java:351)
at org.apache.jmeter.threads.JMeterThread.notifyTestListeners(JMeterThread.java:561)
at org.apache.jmeter.threads.JMeterThread.access$200(JMeterThread.java:60)
....

"Thread Group 1-15" prio=6 tid=0x37149000 nid=0x878 waiting for monitor entry [0x39e3f000..0x39e3fa98]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.jmeter.functions.JavaScript.execute(JavaScript.java:73)
- waiting to lock <0x0b998000> (a org.apache.jmeter.functions.JavaScript)
at org.apache.jmeter.engine.util.CompoundVariable.execute(CompoundVariable.java:138)
at org.apache.jmeter.engine.util.CompoundVariable.execute(CompoundVariable.java:107)
....

Here is the source code for your reference :)

public synchronized String execute(SampleResult previousResult, Sampler currentSampler)
throws InvalidVariableException {

JMeterContext jmctx = JMeterContextService.getContext();
JMeterVariables vars = jmctx.getVariables();

String script = ((CompoundVariable) values[0]).execute();
// Allow variable to be omitted
String varName = values.length < 2 ? null : ((CompoundVariable) values[1]).execute().trim();
String resultStr = "";

then you know why...
__javascript() itself got Performance problem, So be Careful to use __javascript when you perform load testing

Generating Heapdump On JDK5(v16+)

Teammates and I are doing one thing to get our life a little bit easier--Generating Heapdump On JDK5(V16+); P.S. unfortunately you could not using this way if you are using lower version of JDK5, like JDK1.5.0_11... A bug reported here for your information, you have to use JDK5(V16+): http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6454676

Thanks Andy and Adams' great idea and Support!!
0. download PsTools and SendSignal and gawk into your target machine as PS_TOOLS_HOME
1. Add -XX:+HeapDumpOnCtrlBreak in your JVM arguments
2. Using SendSignal to simulate the Ctl+Break signal to the service

here is the HeapGen.bat file, hope this helps you too:

 @echo off  
 rem -------------------------------------------------------------------------  
 rem Invoking Remote JVM to do Heap Dump Script for Win32  
 rem 1. Please replace your directory of PsTools  
 rem 2. Please set IP before execute it  
 rem -------------------------------------------------------------------------  
 set PS_TOOLS_HOME=D:\\PsTools  
 set JVM_IP=10.279.33.33  
 set HPROF_HOME=D:\abc\efg\bin  
 echo ===================================================================================  
 echo .  
 echo Your Directory of PsTools is %PS_TOOLS_HOME%  
 echo .  
 echo The Remotely Controlled Machine is %JVM_IP%  
 echo .  
 echo Heap dump is created in a file in the working  
 echo directory of the VM namely %HPROF_HOME%  
 echo .  
 echo =================================================================================== 
 echo .  
 %PS_TOOLS_HOME%\pslist \\%JVM_IP% -u "${Domainname}\${yourusername}" -p "${yourpassword}" -e java | %PS_TOOLS_HOME%\gawk "{if($1 ~ /java/) system(\" %PS_TOOLS_HOME%\\psexec \\\\%JVM_IP% -d -c %PS_TOOLS_HOME%\\SendSignal \" $2)}"  
 echo .  
 echo ===================================================  
 echo .  
 echo Heap Dump file has been created  
 echo .  
 echo ===================================================  
 echo .   

How to Clean IE Cache with Watir?

we need some sanity test from performance perspective,basically need to get end-end performance result on daily build automatically:

1. end-end response time
2. each page size info
3. clean cache between each test round

so the first item is relative easy to achieve and more straight forward by Watir

for second item, after checking out http://www.newsqa.com/delete-cache-and-cookies-in-ruby/, it works well on Windows XP_SP3+IE6 browser (did not try any other platform)

for the third item, current Iｕｓｅ　"ie.html.length" to get page size, any other better idea? I did not dig this deeper right now.

Or i may consider install a HTTPWatch +Watir to do all the things, haha :)

A bonus get from QA who helps us test website Performance

I always borrow our QA team to help us validate performance on staging env when they do release testing as a bonus, here is an example email I sent to the group:

Hi, All QA friends,
Once you find slowness on your side(for example 60 seconds to login, …), please do so :

1. Can you reproduce it? Or just randomly replicate?
2. Username/password/account
3. Which server are you touching? (you need some sniffer tools to determine this if there is a cluster on back-end)
4. Time stamp you encounter the problem(PST time preferred/Beijing time zone is also OK)
5. What steps you are following? (replicate steps if have)

Of course we need to set up some monitoring/profiling tools behind or run ping -t IP to guarantee Network is OK during testing phase.

Performance test scope requirement collection table

I would like to provide a table which include a relative simple and high level categories of performance factors to PM/Arch/Dev leader/(Func)QA leader/ module owner for collecting comments or requirements in numbers/priority from Func team point of view.

Here is an example(Click to view big):



So it can help you to filter some work easily this way :) and then prioritize other remaining tasks/user stories.

How to Create Date offset in LR

just for my work, i have to create Date offset in LR myself, hopefully it is helpful for you as well, it is just a simple example, you can use yours:)

lr_save_datetime("%d", DATE_NOW, "Day1");
lr_save_datetime("%m", DATE_NOW, "Mon1");
lr_save_datetime("%Y", DATE_NOW, "Year1");

if ((atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}")))<= 29)

{


lr_save_int(atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}")), "param1");
lr_save_int(atoi(lr_eval_string("{Mon1}")), "param2");
lr_save_int(atoi(lr_eval_string("{Year1}")), "param3");

lr_output_message("parameter is %s-%s-%s", lr_eval_string("{param3}"),lr_eval_string("{param2}"),lr_eval_string("{param1}"));


}

else if ((atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}")))> 29 && atoi(lr_eval_string("{Mon1}"))<12)

{

lr_save_int(atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}"))-29, "param1");
lr_save_int(atoi(lr_eval_string("{Mon1}"))+1, "param2");
lr_save_int(atoi(lr_eval_string("{Year1}")), "param3");

lr_output_message("parameter is %s-%s-%s", lr_eval_string("{param3}"),lr_eval_string("{param2}"),lr_eval_string("{param1}"));

}


else if ((atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}")))> 29 && atoi(lr_eval_string("{Mon1}"))==12)

{

lr_save_int(atoi(lr_eval_string("{Day1}"))+atoi(lr_eval_string("{round1}"))-29, "param1");
lr_save_int(01, "param2");
lr_save_int(atoi(lr_eval_string("{Year1}"))+1, "param3");

lr_output_message("parameter is %s-%s-%s", lr_eval_string("{param3}"),lr_eval_string("{param2}"),lr_eval_string("{param1}"));

}

How to get page size data in LR scripts part2--extract common code into header file

Step1:
Create your common method in Loadrunner installation folder， which is usually in ..\Program Files\HP\LoadRunner\include

In calpagesize.h file, here is the source code which simply return the “current page size” you want to measure:

int calpagesize(int Pre_size){

int Temptotalsize=0;
int currentResponseSize = 0;

Temptotalsize = web_get_int_property(HTTP_INFO_TOTAL_RESPONSE_STAT);

return currentResponseSize = Temptotalsize - Pre_size;

}

Step2:apply the calpagesize() method:

Adding #include “calpagesize.h” in your globals.h file

Step3: Adding calculate page size code(in red line) into action:

Take Yahoo home page for example:
Action()
{
int Pre_page_size=0;
int cur_page_size=0;


Pre_page_size=web_get_int_property(HTTP_INFO_TOTAL_RESPONSE_STAT); //get accumulate page size data before sending next request

web_url("www.yahoo.com",
"URL=http://www.yahoo.com/",
"Resource=0",
"RecContentType=text/html",
"Referer=",
"Snapshot=t1.inf",
"Mode=HTML",
LAST);

if (flag==1){ //flag==1 means trigger the page size func code

cur_page_size = calpagesize(Pre_page_size); //return yahoo home page page size

lr_save_int(atoi(lr_eval_string("{Avg_01_PageSize}"))+cur_page_size, "Avg_01_PageSize"); //save into parameter ” Avg_01_PageSize”

}

}
So normally, we may need to add 4 lines before and after the page you want to calculate the size.

finally, you can calculate the page size in Vuser_end():

lr_message("Avg_01_PageSize = %d", atoi(lr_eval_string("{Avg_01_PageSize}"))/atoi(lr_eval_string("{Iterations}")));

enjoy!
Cheng

You got GrooveUtil.dll load error on LR9.5?

When I record scripts using LR9.5 on my laptop which install the office2007 there: error happens：


here is my temp solution:

Find GrooveUtil.dll under C:\Program Files\Microsoft Office\Office12, and rename this GrooveUtil.dll into other name ：）

click to view big picture


My friend Alfred found out another solution for this issue, just disable the Groove GFS help on IE Add-ons:

Add page size funcion code into your LR scripts

As my ramp up project, I spent some time to complete a sample on how to get page size after applying small piece of code

Choosing Ajax(click and scripts) protocol；
pre-condition： set your log level as "send message only when an error occurs"
Within globe()
set one flag to trigger the page size func code on demand:
int flag=0;

Within vuser_init()Action, you can initial such parameter as:
lr_save_int(0, "accu_google_frontpage_PageSize");

Within Action() i take google landing page as a example:

int ThisTimeHttpResponseSize=0;
char *message;
int i=0;
int loops=2;

lr_save_int(loops,"loops_goo");

 while(i小于loops){//if there is some loops here

  message=(char *)malloc(140*sizeof(char));

  memset(message, 0, sizeof(char)); 

  web_browser("www.google.com", 
  DESCRIPTION, 
  ACTION, 
  "Navigate=http://www.google.com/", 
  LAST);

  i++;
 
  if (flag==1) {//enable pagesize func, if flag==1;
  
   lr_set_debug_message(LR_MSG_CLASS_JIT_LOG_ON_ERROR, LR_SWITCH_OFF);
  
   //web_get_int_property Returns the accumulated size, including header and body, 
  
   ThisTimeHttpResponseSize = web_get_int_property(HTTP_INFO_DOWNLOAD_SIZE);
   
   strcat(message, "PageSize_of_google_frontpage_");
   strcat(message, lr_eval_string("{Iteration}"));
   strcat(message, " = %d");
   lr_message(message, ThisTimeHttpResponseSize);
  
   lr_set_debug_message(LR_MSG_CLASS_JIT_LOG_ON_ERROR, LR_SWITCH_ON);
  
   //multiple iteration accumulated
   lr_save_int(atoi(lr_eval_string("{accu_google_frontpage_PageSize}"))+ThisTimeHttpResponseSize, "accu_google_frontpage_PageSize");

  }
 
 free(message);

 }

within end() action, calculate the average number of each page size:

if (flag==1) {

 lr_save_int(atoi(lr_eval_string("{Iteration}")), "Iterations");
 lr_set_debug_message(LR_MSG_CLASS_JIT_LOG_ON_ERROR, LR_SWITCH_OFF);
 lr_message("Iterations=%s",lr_eval_string("{Iterations}"));
 lr_message("Avg_google_frontpage_PageSize = %d", atoi(lr_eval_string("{accu_google_frontpage_PageSize}"))/(atoi(lr_eval_string("{loops_goo}"))*atoi(lr_eval_string("{Iterations}"))));
 lr_set_debug_message(LR_MSG_CLASS_JIT_LOG_ON_ERROR, LR_SWITCH_ON);

 }

So you will get each page size freely now!

Thanks my friend Jeff,Alfred and Calvin previous sample code and ideas!

Tester,not only for counting bugs

What is one Tester's Mission?

--Counting bugs?
Are the company pay for bugs we have found?
Are the company recongize one's performance by how many bugs you already find?

If yes, two options:
1> you can go to dev, say "hi, dude, please bury some bugs and let me find them". Of course if that person who may not want to continue his career in this company or he really a "brother" to you.
2> or you may find a "right" partner who will often make mistakes, then you will be a million easily.

--Help dev to make less mistakes?
I agree!Even if i can not easily find a bug now, i am glad to see our application so stable and trustable...
I am happy to work with even he may write a bugless code, you can help him continuously improvement, meanwhile you can just dig deeper treasure which may need more effort or deeper understanding on application functionalities and infrastructure. More important, this, of course, makes customer happy(they may hardly find out bugs as well).You make the company win!

Not Only Keeping walking， But Keeping Thinking

两张有趣的图片，内容并无新意，关键是省去N多文字：

“过犹不及”：


“只要功夫深，铁杵（不一定）磨成针”：
Not Only Keeping walking， But Keeping Thinking to be an expert(If you really want it)！



Note: pictures from http://headrush.typepad.com

Run, Rockets, Run

Rockets Win the first Game, I made a right Guess :)

Twitter proves!!

Click picture bellowing to look at the details:

Main purpose on doing performance tests in my mind

1> Help to find out the (Top) bottlenecks, so that we are able to focus on and make a plan to tune them one by one;
2> Help team/PM to make a decision on release by measuring the trend of performance, guarantee no degradation on each release. (I prefer to taking the throughput trend and overall performance data rather than individually response time as a degradation indicator)
3> Help dev to do the hot spot tuning, we will "agilely" and "aggressively" change our test strategy during tuning phase :)
check out My previous blog on performance test strategies:
http://joychester.blogspot.com/2009/04/performance-test-strategy-in-my-mind.html
4> Validate some "guessing" or "pilot" projects
5> Capacity planning, scalability testing（horizontally and vertically）
6> Bonus! Finding out functional or security bugs in application occasionally
。。。
Feel free to add yours：）

一双破袜子引发的争论

In Million Dollar baby, I like these lines between Eastwood and Freeman so much! :)

Background: Freeman put his feet on Eastwood's desk, then Eastwood came in...

Eastwood:
Where are your shoes?(你鞋呢？)

Freeman:
I am airing out my feet.（我脚透透气）

Eastwood:
You got big holes in your socks（袜子那么大的洞）

Freeman:
Oh, they're not that big（哦，这还不算大的）

Eastwood:
Didn't I give you money for some new ones?（我之前没给你钱让你买双新的？）

Freeman:
These are my sleeping socks, My feet like a little air at night.(这是我睡觉时的袜子，脚丫喜欢在晚上透透气)--居然喜欢穿袜子睡觉？？

Eastwood:
How come you're wearing them in the daytime, then?（那你白天怎么穿晚上的袜子？）

Freeman:
Because my daytime socks got too many holes in them.（噢，白天袜子的洞更大）

Eastwood:
Well, if i give you some more money, you buy some new socks, please??（靠，如果我多给你点钱，你能不能买几双新的，拜托）

Freeman:
Well, i'd be tempted, but i couldn't say for sure. Might find its way to the track.（呦，这倒是个好主意，不过我不能保证，这钱真的会用到袜子上）

Eastwood:
。。。（无语）

“疯子”和“魔方”

《当幸福来敲门》一直有一个问题没有搞懂，为什么影片会有一个“疯子”一直跟Chris“过不去”？是巧合还是别有用心？（Sorry，叫疯子并非歧视，是因为不知道他的称呼。。。）


“疯子”是一个标尺，心理的标尺，标志着Chris崩溃的底线，也许疯子十几年之前的状态正和现在的Chris一样？？很显然，结果是“疯子”没有把握住，而Chris 挺住了！其实有时候Chris的境遇并不如“疯子”，但他在挣扎，坚持这自己的希望，在等待幸福来临时的那一刻，很显然，正和海报上的图画一样，他当时所有的一切就是儿子和他赖以生存的“time machine”！

我想，这就是“疯子”在电影里存在的必要吧 ：）

小小的“魔方”，威力巨大，它代表着机遇，如果当时在出租车上高级经理人没有玩魔方，或者说Chris自己不会玩，也许经理人不会对这个素不相识的“穷光蛋”有什么好的印象。

正是魔方让他对Chris刮目相看，一切就是那么巧合，这就是对机遇的最佳诠释--在正确的时间，相遇在正确的地点，遇到正确的人，做了正确的事，一切就有了转机！
享受当幸福来临的时刻吧！It really belongs to you！

今天，你XXX了吗？

突然想到一个句式：
--今天，你云计算了吗？
--今天，你敏捷了吗？
--今天，你Google了吗？
--今天，你百度了吗？
--今天，你twitter了吗？
。。。
--今天，你吃饭了吗？
--今天，你OUT了吗？

结论，看到娃哈哈有个啤酒饮料的广告，发现自己真的OUT了

Performance testing process in one sprint(after we choose Scrum)

1> Performance Planning:
--- Requirement analysis and make performance test strategy (How to test in this sprint, what kind of user story need to do the user story based test)
--- Define performance criteria( such as 90% percentile under 3 seconds page server response time under a certain load)
--- Design User patterns(what) and User distribution(How) to form a diagram scenario to describe or demo this easily

2> Performance Environment set up：
---At least do weekly build on our local performance environment based on our work load and progress

3> Test Data Preparation (we intent to not using production data, but making the same size of data in our database)
--- Using store procedure to generate data into our DB
--- Using External scripts to generate data: For example, JMeter to create some "fake" data during one night so that you can test tomorrow; BTW, using Jmeter to generate tons of data is another way of load testing itself ：）

4> Test Scripts design and debugging：
--- Parametrization, correlation
--- Error Handling and Content Check points
--- Logic control flow, generate random number or string processing

5> Scenario design：
---user load(combine all the scripts based on user patterns and user distributions, draw a UCML diagram if you like)
---think time, download resource or not?
---simulate browser cache or not?
---test schedule
......

6> Trial Run(Errors Clean up)：
--- Single thread run for whole scenario: Focusing on SQL profiling result and Do initial DB layer tuning
--- Trail run with concurrent users（relative low load） in order to clean up any errors caused by scripts or application

7> Iterative Testing and Tuning(Time boxed testing & tuning phase)：
--- Run the full scenario under the same user load as last sprint, guarantee that no any degradation
--- Trying different load to see the capacity limit in current sprint
--- Get a new benchmark with best support load on your performance environment
--- If any performance issue found, then Test and tune module by module (testing duration may only 10-20 mins with less think time)

8> Report analysis：
--- Based on test and tuning result, monitoring logs, error logs...

9> Log defects in to bug tracking system：
--- Log the defects in order to priorities and track for next sprint

It is just a guideline for tester, you can freely change the sequence and priority of each task. Of Course, You need to collaborate developer friends closely, you are not alone, check out:
http://joychester.blogspot.com/2009/01/performance-testers-you-do-need-friends.html

Thank my teammates for always support and trust! We can make our performance even better!

Performance test strategy in my mind (after we choose scrum)

1.User story based performance testing

Testing Scope examples:
Code refractory, algorithm changed, internal design or infrastructure change, such as JDK 1.5.06 upgrade to JDK 1.5.16, Front-end tuning,single thread using Watir automation scripts to see if any slowness or client side memory leak…

Test pattern:
Write new component/unit scripts to add load just related to this user story on local performance environment

2.Performance related configuration change testing

Testing Scope examples:
Apache tuning, Tomcat tuning, JVM GC tuning, DB index tuning…

Test pattern:
Go through the most common/critical scenarios(just a simple load scripts),and monitoring the footprint and all performance measurements

3.Weekly performance testing

Testing Scope examples:
For tracking the application overall performance trend, compare with before picture and after picture, in order to guarantee no degradation on performance

Test pattern：
Combine all existing component performance scripts;
Based on the latest build on local perf environment, and getting performance measurements with the latest test scenarios(User patterns + User distribution), do a comparison or judgment, in order to prevent any performance degradation due to new code check in or any resource contention

4. Load/Stress/Capacity testing

Test pattern:
Draw a diagram of throughput curve;
Figure out the best support load, and threshold of our application;
Make sure how our application behaves under heavy load or different load

5. Long duration performance testing

Test scope examples:
No memory leak? No deadlock? No unexpected Crash?

Test Pattern:
Run the combined performance test scenario all night long, check your result next morning

6.Release performance testing

Test scope examples:
No surprise and guarantee that 90% page server response time is under 3 seconds on stage environment, Help to judge the overall performance of this release

Test Pattern:
The same as weekly testing strategy, should be a formal weekly performance test

You may add more kind of tests, I may ask myself before testing:"What do you want to get from this kind testing? Is it worth or just waste of my time? Do we have a better Strategy?"

It reminds me something...

It reminds me 3.5 years ago:



He is that kind of person~~~

I am an organizer:


I can pass:


I can shoot:


I can "Dunk":


I Have My Friend behind:


Sometimes, I will feel upset:


But most of time, I feel Great!


Being Trusted, Supported,Recognized is Sweetest thing in my life!


Thanks and Best Wishes to You,
Truly,Deeply,Silently!!

How to debug Performance Scripts Issue in Loadrunner

Here are my thoughts:

1. Do you have content check points in your current load testing scripts?
If not, adding check points on the most critical actions for tracking; This is very important for the performance scripts design:
You may want to Check these:
http://joychester.blogspot.com/2009/01/lr-scripts-debuging-on-controller-try.html
http://joychester.blogspot.com/2009/03/content-check-for-web-service-calls-2.html
http://joychester.blogspot.com/2009/02/content-check-for-webservicecalls.html

2. Try to isolate the problem，not mix everything up at the same time.
You can run just one performance scripts at a time which has the problem, and you can run it under different user load, so that you may reproduce the issue easily or just guarantee that this script has no problem;

3. Checking the very first error you met in your scripts.
Usually it is the first thing you need to be caution when you running the load testing. Because it may trigger a tons of following errors, if you did not do error handling well

4. Try to look at the parameters/correlation value you use within the scripts.
This is a check point when you find there is an error on your scripts, make sure you use the write parameters/correlation value before you start to do other deeper investigation. It is not easy to find if you do not know the application very well

5. Of course you should run your scripts in Vugen successfully before running it in Controller.
But I want to remind is that you have to validate all the parameters or situations you may meet, such as using different accounts, different data, also do the Branch coverage if you have "if-clause" in your scripts. So you may get my idea--"Do your unit test on your test scripts" :)

6. Sometimes or more often,The errors may just related to the load you add to the system, not caused by scripts itself.
This errors are only reproduce under a certain load, so it might not be a scripts issue... You can do a load test on your scripts to verify your idea--10 users load, 20 users load, 30... till you get such an error.
But one kind of error is not related to the load, for example, you just input one column which need to be unique value, but in your scripts does not implement this well, so that in Vugen(single user load) or even low load you will not get kind of error, however, if two thread generate this value at the same time, then you may get duplicated value for these two users...thus, you will get an system error,but actually this is absolutely a scripts issue! Be careful!

7. Last and the best practice--After running your scripts in Vugen, please look at your application whether the scripts make the action happen or not, you should believe your eyes, right? :)

How to generate random number in a dynamic world

Besides the random number by Loadrunner parameter itself, usually you need to create a Random number of which range is based on the correlated result, for example, you want to check a check-box which number is dynamically changed:

int checkbox_num1=0;
char Randomitem[10];
char RandomStatus[10];
char SRanStatus[10];
...

web_reg_save_param("Checkbox_name1",
  "LB=TableSelectCheckbox-_dash_",
  "RB=)\" >",
  "ORD=ALL",
  LAST);

web_submit_data(...
              .... );

checkbox_num1 = atoi(lr_eval_string("{Checkbox_name1_count}"));

    srand(time(NULL));

 if (checkbox_num1>0) {
 
 itoa(rand() % checkbox_num1 +1,Randomitem,10);

 sprintf(RandomStatus,"{Checkbox_name1_%s}",Randomitem);

 sprintf(SRanStatus, "%s", lr_eval_string(RandomStatus));

 lr_save_string(SRanStatus, "CheckboxORD");

 }

 else if (checkbox_num1==0) {//there is no any suitable value found

  lr_exit(LR_EXIT_ITERATION_AND_CONTINUE, LR_AUTO);

 }

Loadrunner: How to insert a charactor(s) into a string

I want to replace the '\' into '\\' in one string, here is my implementation in Loadrunner:

 char * position;
 int i=0;
 int j=0;
 Char * SRanStatus2_backup;
 Char SRanStatus2[50]="abcc\dada-com";

 SRanStatus2_backup= (char *)malloc(50 * sizeof(char));

 memset(SRanStatus2, 0, sizeof(SRanStatus2));

 memset(SRanStatus2_backup, 0, sizeof(SRanStatus2_backup));

 position = SRanStatus2;

 while ((*position)!= NULL) {

  if ((*position)!='\\') { //if ((*position)!='\x07') {

   SRanStatus2_backup[j]=SRanStatus2[i];
   i++;
   j++;
  }
   
  else if ((*position)=='\\') { //else if ((*position)=='\x07') {
   SRanStatus2_backup[j]='\\';
   SRanStatus2_backup[j+1]='\\';
   j=j+2;
   i++;
  }

  position++;
 }

 SRanStatus2_backup[j]='\0';

 lr_save_string(SRanStatus2_backup, "CheckboxORD_2_backup");

 free(SRanStatus2_backup);
 free(SRanStatus2);

 return 0;

A Joke on a Slow response page

My friend and I take a look at one website, when we got to one page, it takes almost 2 mins loading...

Me: So damn slow page, and no useful information. Let's go to another page to have a try

I was taking my mouse to try to click next link, while my friend stop me at once!

My friend speak seriously: NO！ NO！ NO！ Let's treasure this Hard-earned page, take more seconds on it to retrieve our time losses, even it give us nothing information...

we just stared at that slow page for a few seconds quietly, and never come back in the future

JVM (GC) tuning useful links and Netbeans Heapwalker demo project

I want to add some useful document links which bring me up on this big topic--JVM (GC) tuning:
http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html
http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html
http://java.sun.com/performance/reference/whitepapers/tuning.html
http://java.sun.com/j2se/1.5.0/docs/guide/vm/gc-ergonomics.html
http://java.sun.com/j2se/1.5/pdf/jdk50_ts_guide.pdf
http://java.sun.com/j2se/1.5.0/docs/guide/vm/server-class.html

For Heap dump analyzer, apart from Eclipse Memory Analyzer, there is another Cool free analyzer tool,Netbeans HeapWalker:
http://profiler.netbeans.org/docs/help/6.0/heapwalker.html

here is one very interesting HeapWalker Demo, for QA people who is interesting in trying to tune the performance memory issues, you may go through this message and get some sense of performance tuning by yourself :)

http://wiki.netbeans.org/HeapWalkerDemo

Dining Philosophers Game--how to make Deadlock issue

Try this game if you do not want to write a code to implement the deadlock issue:
snapshot:


play at below :
http://wwwhomes.doc.ic.ac.uk/~jnm/book/book_applets/Diners.html

利用Firebug实现Google Maps中国的地理译码

2007年8月在北京参加首届研究生地理信息系统论坛，在论坛上有幸介绍了一下陈汝烨和我在创建roommap网站时期的一个地理译码的过程，现在Google Maps中文的地理译码已经公开了一些地理译码信息，之前需要自己去“hack”这部分信息：
http://www.codechina.org/doc/google/gmapapi/#Geocoding_Etc

所谓地理译码：是把地址(如"1600 Amphitheatre Parkway, Mountain View, CA")转换为地理坐标(如经度-122.083739，纬度37.423021)的流程，您可以用它把数据库里面的街道地址或用户提供的地址信息标记在地图上。

现在将我们俩当时（2007年）的思路共享给大家：

摘 要：目前Google Maps 中国的API已经提供了地图所需要的基本服务，比如任何网站可以引用浏览Google maps中国地图，以及一些简单应用比如显示气象图等，但使用者不能进行地理译码。地理译码是把地址（如“中国科学院遥感应用研究所”）转换为地理坐标（如经度-122.083739，纬度37.423021）的过程。通过它，您就可以把数据库中的街道地址或用户提供的地址信息标记在地图上。但是，到目前为止，在Google maps US 已经实现了地理译码的API（GClientGeocoder 类中有相应的方法实现）。Google Maps中国地图中，用户可以对地名搜索并进行地理坐标定位，然而，中文 Google Maps API尚不提供地理译码功能。不过，我们可以根据Google提供的规范开发自己的地理译码器。本文介绍了利用Google maps网络已有资源，如何实现对中文地理位置名称到地理经纬度坐标的转换，从而在自己的网站中利用Google maps中国对任何国内的地理位置进行搜索定位。

作者采用Firebug监测工具，分析Google中国地图程序产生的Http响应，从中解析经纬度坐标信息，最终实现Google Maps中国的地理译码功能。

首先登录http://ditu.google.cn/网站，将需要查询的地名提交并提交。查看Firebug中获得命名为maps的Http响应（response），从相应的html文本中发现名为markers: [] 数组， [] 内每个id: 的内容就表示一个对象或称一个地理信息结果。由此按照markers: [] 数组内对象（或id）的数量，我们可以把地理译码过程分成一下三种情况：

1. markers: [] 数组中有多个对象（或id）。比如搜索“扬州大学“，那么地图上可以得到多个结果，包含扬州大学的多个校区的地址。在这种情况下markers: [] 数组中每个对象（或id）存在一个geocode的属性，在这个属性中我们发现了一个没有经过加密的该地址的地理坐标。因此，我们可以直接用正则表达式获取到改对象（或id）的经纬度信息，从而遍历得到每个地址的地理位置。

2. markers: [] 数组中仅有一个对象（或id）。比如搜索“中国科学院遥感应用研究所”，只有一个精确地址。这种情况下的http响应结果中包含一个markers数组，数组长度为1，这个数组包含的元素对象有laddr和mapabcPoiid两个属性。laddr就是中文地址描述，而作者发现Firebug监测到浏览器发送了标识为mapabcPoi的请求，从地址中中可以看到 Google地图的地理数据用的mapabc.com的数据，那么推断mapabcPoiid是该地理点在mapabc数据库中的标识符。作者查看mapabcPoi请求的响应内容（response），得到类似这样的代码notpcoStrbase64=…发现这个结果应该是base64编码后的结果，所以作者利用base64解码后就可以获得该地址的地理坐标了。

3. 找到多个不确定的结果。这种情况下，返回结果中包含一系列的建议的地址，比如搜索“上海马戏城” ，结果提示您找的是不是“某某路上的马戏城”。从响应的html文本中利用正则表达式取得这些建议的地址，然后再重新搜索就可以得到建议地址的坐标了

《注：代码部分已经由陈汝烨同学利用javascript实现》

结语：有时候，结果并不重要，过程更重要 ：）

LR Vugen become slow when scripts is running with parameter notepad file opened

Today, I notice that LR(9.0) Vugen become slow when scripts is running with parameter notepad opened.

Vugen seems like a "snail" to execute your scripts line by line, once you close the parameter notepad file, then it become normal.

I am sure it is a bug in LR 9.0 , at least it is not fit my expectation :)

content check for web service calls (2)

I have written one way to do the XML content check for web service call:
http://joychester.blogspot.com/2009/02/content-check-for-webservicecalls.html

Here is another simple way to do this in Loadrunner if you find hard to write a Query(Xpath) in lr_xml_find:

Char Position;
......

lr_start_transaction("web service transaction");

web_service_call( "StepName=echo_101",

"SOAPMethod=MyService.MyServiceSOAP12port0.echo",

"ResponseParam=response",

"Service=MyService",

"Snapshot=t1156241380.inf",

BEGIN_ARGUMENTS,

"xml:part1=*******",

END_ARGUMENTS,

BEGIN_RESULT,

"part1/EchoOutput=Param_EchoOutput2",

END_RESULT,

LAST);

//want to match if the response data include Success Key word
Position= (char *)strstr(lr_eval_string("{response}"), "Success");

if (Position!=NULL) {

lr_end_transaction("web service transaction", LR_PASS);

}

else {
lr_output_message("web service call error occured!Help!:)"
}

Nike Hyperdunk Supreme Kobe

科比不是我的偶像，他的球技出众，领袖气质，虽不讨厌，但打球的风格不是我欣赏的那种（俺的偶像是皮蓬，纳什）：）

今天无意中在Nike factory看到了这双黑的出众的Nike Hyperdunk Supreme Kobe's PE， 是在去年10月份上市的，比起奥运版的白灰色，我觉得这款更具科比的个性--720CNY拿下(原价1250CNY)：
“黑夜给了我黑色的眼睛，我却用它来寻找光明”:



还有一点，如果有你非常喜欢的鞋子，而且觉得贵，请忍耐半年的时间，然后去折扣店淘，祝你好运！