preview of "EXPerT"-- End to End performance test "framework"

I am not sure it can be called "framework"(it is too "tiny" to call it "framework", and currently Framework word becomes abuse...), my "intention" is to setup a platform to run my sanity performance test with Watir. And I met Httpwatch last year which can integrate with Watir to automate it and get detail Performance data, it reminds me an amazing way to Perform our End to End Performance testing for Daily tracking.

I made a dummy version few months ago, and it proves my thoughts very well. Tom(My intern) and I work together to make it more fancy and robust during last month.


I name it as "EXPerT", it is stand for "Extreme Performance Testing", will publish it soon, and hope it can helps more people to make a better performance website

Features overview:
1> Launching Web browser (IE by default)
2> Httpwatch detail Performance measurement(right now we are focusing on Basic Edition data)
3> Generating Detail HTML report
4> Error Handling
5> Test Case management
6> Local and Global Parametrization support
7> Integrate with Google Chart to demo your perf result
8> threshold setting for Response time/page size/ect
9> Easy to write your own methods to extent your toolkit
10> accessing DB operation provided
11> Taking screen shot when page got Error or anywhere you want
12> Performance data statistics (pending)

Screen shots:
Summary page information(i set the performance threshold for response time to 3 seconds and page size to 200KB)


I will keep on posting on this stuff. Happy Performance testing!

Next step i may integrate it with Dynatrace-ajax as well, it will help to get more accurate Front-end performance insight! That will be exciting to me! :)

My own Pic

Watir -Click_no_wait() does not trigger the Popup window

Due to Click_no_wait() does not work on my IE, based on the discussion and suggestion from this thread:http://jira.openqa.org/browse/WTR-320 , I have to roll back my ruby version from ruby186-27 to ruby186-26, here is my process:

1. Uninstall ruby186-27
2. Install ruby186-26
3. Run "gem update --system"
But Shit happens：

...
rubygems-update requires builder >= 0

4. manually download rubygems-1.3.5.zip
5. upzip rubygems-1.3.5.zip to C:\
6. Run "C:\rubygems-1.3.5>ruby setup.rb" --RubyGems 1.3.5 installed
7. Run "gem update --system" --Nothing to update
8. Run "gem install watir"

Done. i will see if it works now...God bless!
-- update , it did works! :)

Updated from 2010/09/25:
It has fixed this issue on ruby186-27 now, i have not tried that yet:
http://jira.openqa.org/browse/WTR-320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Regular expression in Watir method

We can make usage of regular expression in Watir to easily access the dynamic object, for example:
you want to pick a option in select_list which id is : hello_12345, 12345 is dynamic number, so you can write like this:

cframe.select_list(:id, /hello_\d+/).select("Select one option");

Memory Pool--Code Cache can be "GCed"

The code cache GC has been detected :)

Open expire headers to reduce 304 requests

Open expire headers on Apache http server:

[IfModule mod_expires.c]
ExpiresActive On

ExpiresByType image/* "access plus 30 days"
ExpiresByType application/x-javascript "access plus 30 days"
ExpiresByType text/javascript "access plus 30 days"
ExpiresByType text/css "access plus 30 days"
ExpiresByType image/x-icon "access plus 360 days"

[/IfModule]

before enabling Expire header:


After enabling Expire header:


you can see all of the static files, like images, .js and .css are read from browser cache directly instead of sending additional requests to the server and get 304 not modified response status code. it improves our performance and saving bandwidth as well.

Enable Gzip on Apache Http server

Enable Gzip, modify the httpd.conf:

Apache Httpd.conf change:

LoadModule deflate_module modules/mod_deflate.so

LoadModule headers_module modules/mod_headers.so

.....

#Enable Gzip



# we do not need to 9 level, due to CPU% is high but no further compression benefit
DeflateCompressionLevel 7



# Insert filter
AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/javascript

AddOutputFilterByType DEFLATE application/xml application/xhtml+xml application/javascript application/x-javascript

AddOutputFilter DEFLATE css js


# Netscape 4.x has some problems...

BrowserMatch ^Mozilla/4 gzip-only-text/html


# Netscape 4.06-4.08 have some more problems

BrowserMatch ^Mozilla/4\.0[678] no-gzip



# MSIE masquerades as Netscape, but it is fine

BrowserMatch \bMSIE !no-gzip !gzip-only-text/html



# Don't compress images and already compressed files

SetEnvIfNoCase Request_URI \\.(?:gif|jpg|cab|jpe?g|exe|bmp|rar|zip|swf|png)$ no-gzip dont-vary



# Make sure proxies don't deliver the wrong content, need load mod_headers.so

Header append Vary User-Agent env=!dont-vary





or if you want things simple(may bring some bugs on certain browsers):


DeflateCompressionLevel 7
AddOutputFilterByType DEFLATE text/html text/plain text/xml text/javascript text/css

move your mouse automaticly :)

Access directory of JDK6/bin and run jrunscript.exe (note: only for JDK6)
Input scripts:

r = java.awt.Robot();
r.setAutoDelay(500);
while(true){r.mouseMove(500+100*Math.random(), 500+100*Math.random());}

share and enjoy it!

simple thread dump analysis Awk scripts

Here are the Thread.state we usually see and pay attention to:
* RUNNABLE
* BLOCKED
* WAITING
* TIMED_WAITING

If want to know what kind of status of thread dump i picked up from app server when CPU% is pretty high or want to find blocking threads, this is useful give you some insights, you can develop yours in 10 mins:)

BEGIN {RS="\n\"";IGNORECASE = 1;count1=0;count2=0;count3=0;count4=0;count5=0}

($0~ /workmanager\(/){

if ($0~ /State: runnable/) {

count1=count1+1;

}


if ($0~ /State: TIMED_WAITING on/){

count2=count2+1;

}

if ($0~ /State: WAITING\n/) {

count3=count3+1;

}

if ($0~ /State: WAITING on /) {

count4=count4+1;

}

if ($0~ /State: Blocked on /) {

count5=count5+1;

}


{

sum=sum+1;
}

}

END{
print "# of runnable threads is:" count1;
print "# of timed_Waiting on condition threads is: " count2;
print "# of Waiting threads is: " count3;
print "# of Waiting on condition threads is: " count4;
print "# of Blocked threads is: " count5;
print "Total num of threads: " sum;
}
-------------------
Sample output:

# of runnable threads is:10
# of timed_Waiting on condition threads is: 85
# of Waiting threads is: 0
# of Waiting on condition threads is: 3
# of Blocked threads is: 2
Total num of threads: 100

BTW, i found "timed_Waiting on" and "Blocked" threads number is huge, then you got problem on synchronization probably, dig into it.

Performance engineering internal group(team) responsibility

Here is a draft of internal performance group(team) responsibility chart in my mind, so much things to do, we need to prioritize each section better to maximum its value :

Responsibility	Tasks	Role	Delivery
Capacity planning	1. Current user pattern/distribution 2. Gap analysis 3. Predict trend of usage 4. Top slow transactions 5. Top usage of transactions	QA lead, Performance QA(owner) PM, OPS(supportive) SDM, TL(consultant)	Access log analysis result RFP distribution pattern
Performance Environment Setup	1. Setup Performance environment 2. Data preparation 3. Server configuration	SCM/Dev (owner) Performance QA (supportive)	Testable performance environment
Performance test planning and design	1. Performance test strategy design 2. Performance testing scripts/scenario design 3. Keep performance test scripts/ load distribution update to date	Performance QA(owner) Performance Dev(supportive) SDM, TL, QA lead(consultant)	Performance test scripts/ scenarios
Code review and design inspection	1. code review and Performance inspection checklist	Performance Dev , TL(owner) Performance QA(supportive) Arch(consultant)	Code review retro
Performance testing Execution	1. User-story based performance test 2. Weekly performance test 3. Release performance testing 4. End-end performance test 5. Performance initiatives 6. Performance test report	Performance QA(owner) Performance Dev(supportive)	Performance test result report
Performance result analysis	1. Server log issues analysis 2. Performance test result analysis 3. Conclusion of performance Grade and suggestion of further investigation	Performance Dev(owner) Performance QA(supportive)	Identify any potential Performance bottlenecks Escalate Performance issues to the team
Performance improvement planning	1. making follow up plan for escalated performance issues	SDM, TL, QA lead(owner) Performance QA/Dev(supportive)	Create tech user stories for each performance issue Triage the priority of each performance user story
Performance tuning	1. Java back-end performance tuning 2. DB tuning 3. Front-end performance tuning 4. Performance Validation	SDM, TL, Dev (owner) Performance QA(supportive)	Tuning analysis summary Performance improvement comparison result Code review Check-in code into new release
Production Performance issues Follow up	1. Log performance issue in Rally 2. Server log analysis 3. Reproduce performance issue on local 4. Find the root cause of bottleneck	Performance Dev(owner) Performance QA(supportive)	Performance issue analysis result Reproduce suggestions