Little's Law on Performance Test

Little's Law, as a part of Queuing theory, which was introduced to me by Wilson Mar about 2 years ago. We can make use of it to help performance test planning and modeling.

I also would like to usually calculate the arrival rate of system :

L = λW
λ = L/W

Where,
W = Average response time + Think time
L = Virtual Users we simulate in load generation tool

After calculating the arrival rate λ , I will compare arrival rate λ with Throughput X to see if the system can catch up the incoming load, if not, then start tuning it!

One example:

W =(0.352 + 0); -- which means no think time for test
L = 8;

then,
λ = L/W = 8/(0.352 + 0) = 22.73

Meanwhile, get Throughput measured by load generation tool
Throughput = 22
Not that bad currently :)

Some Update after several years I met with such an elegant equation, following results are what i have worked with my perf team to prove the Little's law:

Little's Law for Finding Response Time: 

MeanResponseTime(AVG) = MeanNumberInSystem(VUs) / MeanThroughput(TPS)

Assumptions:

• Stable system, in the long terms
• No Obvious System Bottlenecks
• Similar Actions in the system

Test And Measurement -- Get XXX Pages: 

VUs	AVG(ms)	90%(ms)	TPS	Calculated VUs using Little's Law
1	58	72	16.98	0.99
12	74	108	161.63	11.96
25	83	121	297.99	24.73
50	138	209	360.67	49.77
75	197	349	379.59	74.78
100	261	492	381.21	99.50
125	326	636	381.80	124.71

Predicting and Modeling: 

• Find relations between VU and Avg Response Time By Curve fitting:

y = 0.0067x2 + 1.3698x + 53.613

• Response Time Validation: 

Vus	Predict Result	Test Result
35	109.7	101
65	170.9	172
90	231.1	238
110	285.3	290

• Find relations between VU and Throughput By Curve fitting:

y = 82.39ln(x) + 7.8124

• Throughput Validation: 

Vus	Predict Result	Test Result
35	300.7	342.61
65	351.7	376.35
90	378.5	374.49
110	395	377.74

Simulate Moving your mouse by AutoIT and WebDriver

You can use AutoIT within WebDriver to move your mouse freely, Create your own APIs to simulate your mouse move and click very easily, enjoy it! :)

 autoitx ai = autoitx.INSTANCE;  
 WebDriver driver = new FirefoxDriver();  
 driver.get("http://www.autoitscript.com/autoit3/downloads.shtml");  
 FirefoxWebElement downloadpic = (FirefoxWebElement) driver.findElement(By.cssSelector("img[alt=\"Download AutoIt\"]"));  
 int X = (int) downloadpic.getLocationOnScreenOnceScrolledIntoView().getX();  
 int Y = (int) downloadpic.getLocationOnScreenOnceScrolledIntoView().getY();  
 int offsetX = (int) downloadpic.getSize().getWidth();  
 int offsetY = (int) downloadpic.getSize().getHeight();
 int posx = X + offsetX/2;  
 int posy = Y + offsetY/2;  
 ai.AU3_MouseMove(posx, posy, 20);  
 ai.AU3_MouseClick("left", posx, posy, 1, 10);  

Reference for my previous post on WebDriver and AutoIT: http://joychester.blogspot.com/2011/01/deal-with-file-download-in-webdriver-by.html

Set up Browser caching with Caution to speed up your website

"If you use tech improperly , things will be messed up!"

There are good start tips for Browser caching strategy posted by HttpWatch long time ago: http://blog.httpwatch.com/2007/12/10/two-simple-rules-for-http-caching/
It works great in most of situation.

However, if you have your website more complex, such as using ajax, the post above may not be sufficient.

I found interesting post from caching on IE for caching Ajax requests:

This means that any Ajax derived content in IE is never updated before its expiration date – even if you use a forced refresh (Ctrl+F5)—which means even you add “Pragma: no-cache” header with your request.
The only way to ensure you get an update is to manually remove the content from the cache.

Detail info goes to Fact#2 of this post: http://blog.httpwatch.com/2009/08/07/ajax-caching-two-important-facts/

Another conclusion for browser caching:

“In order to ensure consistent caching behaviour with IE and Firefox you should:
Always specify an Expires header. It will normally be set to -1 for immediate expiration of HTML pages or a date well into the future for other resources such as images, CSS and Javascript;
If you want to force a page to be reloaded, even with the Back button, then use Cache-Control: no-cache, no-store”

Detail info goes to this post: http://blog.httpwatch.com/2008/10/15/two-important-differences-between-firefox-and-ie-caching/

How to set up Cache-control and Expire header on Apache web Server based on your derision and deep understanding of your application:
http://www.askapache.com/htaccess/speed-up-your-site-with-caching-and-cache-control.html

Deal with File download in WebDriver by AutoIT

"This is annoying , but you have to face it sometimes"

Solution:
1> Install AutoIT V3.2, not the latest version, like V3.3.6, BTW, you had better install AutoIT to its default folder:"C:\Program Files\AutoIt3"
2> Download jWinAuto
3> Add all .jars to your libs, including jna.jar and jAutoIt.jar within jWinAuto
4> Write your testing code
I put the Watir scripts in comments for each steps, FYI:

 import org.openqa.selenium.By;  
 import org.openqa.selenium.WebDriver;  
 import org.openqa.selenium.firefox.FirefoxDriver;  
 import com.compdev.jautoit.autoitx.autoitx;  
 public class AutoIT {  

      public static void main(String[] args) throws InterruptedException {  

           //ai = WIN32OLE.new("AutoItX3.Control")  
           autoitx ai = autoitx.INSTANCE;  
            //ie=FireWatir::Firefox.new  
           WebDriver driver = new FirefoxDriver();  
            //ie.goto("http://www.autoitscript.com/autoit3/downloads.shtml")  
           driver.get("http://www.autoitscript.com/autoit3/downloads.shtml");  
            //ie.image(:alt,'Download AutoIt').click  
            driver.findElement(By.cssSelector("img[alt=\"Download AutoIt\"]")).click();  
            //res=ai.WinWait("Opening autoit-v3-setup.exe","",10)  
            int res = ai.AU3_WinWait("Opening autoit-v3-setup.exe","",10);  
            // if res = 1, then it shows you find the window with proper title  
            System.out.print("res" + res);  
            Thread.sleep(3000);  
            //res=ai.WinActivate("Opening autoit-v3-setup.exe")  
            ai.AU3_WinActivate("Opening autoit-v3-setup.exe","");  
            ai.AU3_Send("{TAB}",0);  
            Thread.sleep(3000);  
            ai.AU3_Send("{TAB}",0);  
            Thread.sleep(3000);  
            ai.AU3_Send("{ENTER}",0);  
            //Start file downloading...
      }  
 }  

So you can use AutoIT in WebDriver to deal with your Win32(disgusting) pop-ups or other situations, easily! Want to know how powerful the AutoIT is, please refer to its function reference :)

BTW, just realize WebDriver is implementing a batch of GUI event driven APIs, Cool!

Check JVM runtime compiler by VisualVM

One of my friends asked me how to determine what kind of runtime compiler of a JVM is using, the 'Server' or 'Client'?

there is some default value listed on Ergonomics, but if you want to make sure of it on your server?

It is not that hard to check, the easier way is to use VisualVM on Overview tab:

Usually, you had better add "-server" parameter to ensure a server mode on your machine.

The very first OOM in 2011

Here is how my first OOM of 2011 being detected:

2011/01/04, 2PM:
My defualt JVM setting:

 
...
 wrapper.java.additional.5=-server  
 wrapper.java.additional.6=-Xss128k  
 wrapper.java.additional.7=-Xms1536m  
 wrapper.java.additional.8=-Xmx1536m  
 wrapper.java.additional.9=-XX:MaxPermSize=64m  
 wrapper.java.additional.10=-Xmn512m  
 wrapper.java.additional.11=-XX:ParallelGCThreads=5  
 wrapper.java.additional.12=-XX:+UseConcMarkSweepGC  
...

2011/01/04, 2:01PM:
Changes from -Xmn=512m to -XX:NewRatio=2

 
...
 wrapper.java.additional.5=-server  
 wrapper.java.additional.6=-Xss128k  
 wrapper.java.additional.7=-Xms1536m  
 wrapper.java.additional.8=-Xmx1536m  
 wrapper.java.additional.9=-XX:MaxPermSize=64m  
 wrapper.java.additional.10=-XX:NewRatio=2  
 wrapper.java.additional.11=-XX:ParallelGCThreads=5  
 wrapper.java.additional.12=-XX:+UseConcMarkSweepGC  
...

2011/01/04, 2:10PM:
Load testing running, OOM happened:

 INFO  | jvm 1  | 2011/01/03 23:24:13 | 2011-01-03 23:24:13, 19560420@qtp-9708927-58, ERROR, com.XXX.XXXXX.concept.ConceptMetadataServlet - ConceptMetadataServlet.doGet() exception  
 INFO  | jvm 1  | 2011/01/03 23:24:13 | java.lang.OutOfMemoryError  
 INFO  | jvm 1  | 2011/01/03 23:24:13 |      at java.util.zip.Deflater.init(Native Method)  
 INFO  | jvm 1  | 2011/01/03 23:24:13 |      at java.util.zip.Deflater.<init>(Deflater.java:123)  
 INFO  | jvm 1  | 2011/01/03 23:24:13 |      at java.util.zip.GZIPOutputStream.<init>(GZIPOutputStream.java:46)  
 INFO  | jvm 1  | 2011/01/03 23:24:13 |      at org.mortbay.servlet.GzipFilter$GzipStream.doGzip(GzipFilter.java:525)  
 INFO  | jvm 1  | 2011/01/03 23:24:13 |      at org.mortbay.servlet.GzipFilter$GzipStream.<init>(GzipFilter.java:417)  
 INFO  | jvm 1  | 2011/01/03 23:24:13 |      at org.mortbay.servlet.GzipFilter$GZIPResponseWrapper.newGzipStream(GzipFilter.java:392)  
 INFO  | jvm 1  | 2011/01/03 23:24:13 |      at org.mortbay.servlet.GzipFilter$GZIPResponseWrapper.getOutputStream(GzipFilter.java:342)  
....
NFO   | jvm 1    | 2011/01/03 23:24:14 | #
INFO   | jvm 1    | 2011/01/03 23:24:14 | # A fatal error has been detected by the Java Runtime Environment:
INFO   | jvm 1    | 2011/01/03 23:24:14 | #
INFO   | jvm 1    | 2011/01/03 23:24:14 | # java.lang.OutOfMemoryError: requested 87744 bytes for Chunk::new. Out of swap space?
INFO   | jvm 1    | 2011/01/03 23:24:14 | #

2011/01/04, 2:20PM:

reproduced OOM again, exactly the same one...

2011/01/04, 2:30PM:

buy some drink and continue to do the search on the web

2011/01/04, 2:36PM:

Search keywords:java.lang.OutOfMemoryError, org.mortbay.servlet.GzipFilter$GzipStream.doGzip
It is said to be a Known issue; and it is tagged as fixed already in Jetty version 6.1.23

2011/01/04, 2:45PM:

Changes back from -XX:NewRatio=2 to -Xmn=512m
No OOM happens again after a few rounds of tests....
An idea comes into my mind: "NewRatio seems to be another Factor to this OOM?!"

2011/01/04, 3:25PM:

Search KeyWords: NewRatio, Out of swap space
Another Known issue being stamped: "-XX:NewRatio completely ignored when combined with -XX:+UseConcMarkSweepGC"
Also a Regression bug after above issue is said to be "fixed"

Happy New Year!!