Tuesday, December 14, 2010

Track my Performance Data with WebDriver test by MongoDB

In my WebDriver test, I will always track performance data during automation test on daily bases, it helps to reuse automation scripts which QA team donated to track slowness cases, and guarantee the test coverage very well.
Here is a sample tracking diagram (X-axis: Date, Y-axis: Response time):


Meanwhile, the data is End-End response time,not only back-end html generation time, which load testing tool usually measures.

Testing Code:
MongoDBReader conn = new MongoDBReader();  
WebDriver driver = new FirefoxDriver();  

long starttime = new Date().getTime();  
driver.get("http://www.google.com/");  
conn.InstertPermLog(starttime, "Landing_Page");  

driver.close();  

MongoDBReader source code:
 import java.net.UnknownHostException;  
 import java.util.Date;  
 import java.text.DateFormat;  
 import java.text.SimpleDateFormat;  
 import com.mongodb.BasicDBObject;  
 import com.mongodb.DB;  
 import com.mongodb.DBCollection;  
 import com.mongodb.Mongo;  
 import com.mongodb.MongoException;  
 /* sample usage  
  * MongoDBReader conn = new MongoDBReader();  
  * ...  
  * long starttime = new Date().getTime();  
  * driver.get("http://www.google.com/");  
  * conn.InstertPermLog(starttime, "Landing_Page");  
  */  
 public class MongoDBReader {  
      private DB db;  
      private DBCollection coll;  
      public MongoDBReader(String hostname) throws UnknownHostException, MongoException{  
           // Connect to Mongo DB server  
           Mongo m = new Mongo( hostname );  
           // Get Mongo DB Connection  
           db = m.getDB("perfdb");  
           // Prepare Collection  
           coll = db.getCollection("perfTestColl");  
      }  
      public MongoDBReader() throws UnknownHostException, MongoException {  
           //Connect to the localhost MongoDB by Default  
           String hostname = "localhost";  
           // Connect to Mongo DB server  
           Mongo m = new Mongo( hostname );  
           // Get Mongo DB Connection  
           db = m.getDB("perfdb");  
           // Prepare Collection  
           coll = db.getCollection("perfTestColl");  
      }  
      public DB getDB(){  
           return db;  
      }  
      public DBCollection getCollection(){  
           return coll;  
      }  
      public void InstertPermLog(long startTime, String transactionName){  
           long endtime = new java.util.Date().getTime();  
           long responsetime = endtime - startTime;  
           DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss.S");  
        Date date = new Date();  
           BasicDBObject doc = new BasicDBObject();  
           doc.put("TimeStamp", dateFormat.format(date));  
           doc.put("Rtime", responsetime);  
           doc.put("transaction", transactionName);  
           coll.insert(doc);  
      }  
 }  

MongoDB Shell console:
> db.perfTestColl.find();
{ "_id" : ObjectId("4d078fbb3599030f8a412be1"), "TimeStamp" : "2010/12/14 23:39:
39.546", "Rtime" : NumberLong(4609), "transaction" : "Landing_Page" }
{ "_id" : ObjectId("4d078fc33599030f8b412be1"), "TimeStamp" : "2010/12/14 23:39:
47.953", "Rtime" : NumberLong(1641), "transaction" : "Landing_Page" }
{ "_id" : ObjectId("4d078fcb3599030f8c412be1"), "TimeStamp" : "2010/12/14 23:39:
55.578", "Rtime" : NumberLong(1360), "transaction" : "Landing_Page" }

Wednesday, December 08, 2010

O'Reilly Velocity China 2010

I am starting to do Performance Testing since 2006. And I have followed Steve about Front-end Performance optimization from 2007, although i am mainly focus on the testing and back-end tuning at most of time.
A big Fan of him, because always can get many fresh ideas and best practices after reading his blog, if you haven't read his blog, suggest to catch up with it :) (Seems twitter has become his major updated place right now...)


Thursday, November 25, 2010

Sorting algorithm in Perl

My Girl Friend likes the language that easy to write but hard to read, just like Perl which makes me crazy :)

Here are 3 simple examples which is illustrating her algorithm basic skills(BTW, she is a math talented person i ever met during college)

Insertion Sorting
 @arry1 = (12,32,3,25,67,24,56);  
 @arry2 = shift @arry1;  
 $j=$#arry2;  
 foreach $arry (@arry1){  
  if ($arry2[$j]<= $arry) {  
   push @arry2, $arry;  
  }  
  elsif ($j==0 ) {  
   unshift @arry2, $arry;  
  }  
  else {  
   foreach (reverse(@arry2)){  
    if ($_ > $arry){  
    $j-=1;  
    }  
   }  
   if ($j >=0){  
   $i = $j+1;  
   @arry2= (@arry2[0..$j], $arry, @arry2[$i..$#arry2]);  
   }  
   else {  
    unshift @arry2, $arry;   
   }  
  }  
  $j=$#arry2;  
 }  

 foreach (@arry2){  
   print "$_ ";   
  }  
Quick Sorting:

 @arry1=(12,32,3,25,67,24,56);  
 $i=0;  
 $j=$#arry1;  
 &quick_sorting($i,$j);  
  sub quick_sorting {  
   my($m,$n)= @_;  
   while ($m <$n){  
    if ($arry1[$m]<= $arry1[$n]) {  
     $n-=1;  
     }  
    else {  
      ($arry1[$m], $arry1[$n]) = ($arry1[$n], $arry1[$m]);  
      $m+=1;  
     }  
    }  
   if ($m - $_[0]> 1) {  
    &quick_sorting($_[0], $m-1);  
    }  
   if ($_[1] -$n > 1){  
    &quick_sorting($n+1, $_[1]);  
    }     
  }  
 foreach $arry (@arry1){  
  print "$arry\n";   
 }  

Bubble Sorting
 @arry1 = (12,32,3,25,67,24,56);  
 $i=0;  
 $j=$#arry1;  
  for ( ; $j >=0; $j-=1){  
   for ($i=0; $i<$j; $i+=1 ){  
   if ($arry1[$i]-$arry1[$i+1] gt 0) {  
     ($arry1[$i], $arry1[$i+1])=($arry1[$i+1], $arry1[$i]);  
     }  
   }  
  }  
  foreach $arry (@arry1){  
   print "$arry\n";   
  }  

Wednesday, November 24, 2010

Aggregate Jmeter Raw data into CSV file

I am doing daily automated Jmeter load test to track performance trend using command line, due to only raw data available , so i need to write some code to parser and Aggregate it.
PS: I am using Ruby1.9 to handle this, Ruby1.8.7 and below do not implement the append row to CSV file yet.

You need to Modify the settings for Results file configuration in jmeter.properties in advance:

 jmeter.save.saveservice.output_format=csv  
 jmeter.save.saveservice.assertion_results=none  
 jmeter.save.saveservice.data_type=false  
 jmeter.save.saveservice.label=true  
 jmeter.save.saveservice.response_code=true  
 jmeter.save.saveservice.response_data=false   
 jmeter.save.saveservice.response_data.on_error=false  
 jmeter.save.saveservice.response_message=false  
 jmeter.save.saveservice.successful=false  
 jmeter.save.saveservice.thread_name=true  
 jmeter.save.saveservice.time=true  
 jmeter.save.saveservice.subresults=false  
 jmeter.save.saveservice.assertions=false  
 jmeter.save.saveservice.latency=true  
 jmeter.save.saveservice.samplerData=false  
 jmeter.save.saveservice.responseHeaders=false  
 jmeter.save.saveservice.requestHeaders=false  
 jmeter.save.saveservice.encoding=false  
 jmeter.save.saveservice.bytes=true  
 jmeter.save.saveservice.url=false  
 jmeter.save.saveservice.filename=false  
 jmeter.save.saveservice.hostname=false  
 jmeter.save.saveservice.thread_counts=false  
 jmeter.save.saveservice.sample_count=false  
 jmeter.save.saveservice.idle_time=false
 jmeter.save.saveservice.print_field_names=false

Ruby Source code:

 require 'csv'
 require 'time'
 #Generate agg result from Jmeter raw data
 def parserCSVperfdata(filename)  
  h = {};  
  h_count = {}  
  result = [];  
  i =0;  
  CSV.foreach(filename) do |rec|  
   if (h.has_key?(rec[2]))  
    h[rec[2]] = h[rec[2]].to_i + rec[1].to_i  
    h_count[rec[2]] = h_count[rec[2]].to_i + 1  
    next;  
   else  
    h[rec[2]] = rec[1]  
    h_count[rec[2]] = h_count[rec[2]].to_i + 1  
   end  
  end  
  h.each_key { |key|  
   h[key] = h[key].to_i/h_count[key].to_i  
   result[i] = key;  
   result[i+1] = ",";  
   result[i+2] = h[key];  
   result[i+3] = "\n";  
   i=i+4  
  }  
  return h.sort;  
 end 

 #insert agg result into target CSV file
 def insertlog(src,dest)  
  # get current date and collect the data today  
  t = Time.now  
  cdate = t.strftime("%Y-%m-%d")  
  #append row to CSV file, using Ruby1.9 here for mode = 'a'  
  csvwriter = CSV.open(dest,'a')  
  #append array of string, need to convert date string to array  
  csvwriter << cdate.split  
  src.each do |row|  
   csvwriter << row  
  end  
 end  

Testing code:

 t = Time.now  
 mdate = t.strftime("%Y%m%d")  
 aggresultfile = "D://Perf//loadtest.csv";  
 singlethreadresultfile = "D://Perf//singleuser.csv"  
 # Generate JMeter agg result on daily bases  
 arraydir = Dir.entries("D://tools//jakarta-jmeter-2.4//bin");  
 arraydir.each { |item|  
       item.scan(/#{mdate}_[0-9]+_rawdata\.csv/) { |match|  
           aggdata = parserCSVperfdata("D://tools//jakarta-jmeter-2.4//bin//#{match}");  
           insertlog(aggdata,aggresultfile);  
      }  
 }  

Enjoy!

Monday, November 22, 2010

Set current date and time variable in Batch file

Check your system date format at first:
The current date is: 11/23/2010 Tue
so the format is mm/dd/yyyy weekday

It might be different from other computer, some of are put weekday in the front of the date:
The current date is: Tue 11/23/2010

Here we take 11/23/2010 Tue as an example:
@echo off  
SET inputdate=%date:~6,4%%date:~0,2%%date:~3,2%  
SET inputtime=%time:~0,2%%time:~3,2%%time:~6,2%  
echo %inputdate%_%inputtime%_Rawdata.jtl

 Output:20101122_234425_Rawdata.jtl  

PS: no space between ‘=’ and ‘%’, it costs me 10 mins around this...

Thursday, November 11, 2010

How to attach add-ons to your FirefoxDirver

A Typical sample like this, in case i forget how to do :)
 //Pre-condition: Right Clicking the button to save the add-on to some location  
final String firebugPath = "C:\\addon-1843-latest.xpi";  
FirefoxProfile profile = new FirefoxProfile();      
profile.addExtension(new File(firebugPath));  

WebDriver driver = new FirefoxDriver(profile);  

Another Sample is to set proxy for the FirefoxDriver:
FirefoxProfile profile = new FirefoxProfile();  
profile.setPreference("network.proxy.http", "YourDomain");  
profile.setPreference("network.proxy.http_port", port);  
profile.setPreference("network.proxy.type", 1);   
profile.setPreference("network.proxy.share_proxy_settings", true);  
WebDriver driver = new FirefoxDriver(profile);  

Friday, November 05, 2010

Find the Difference

Ruby is cool, it can write some code like this:
 arrayA = ['a345','v123','765','b666', '333', 'aaaa'];  
 arrayB = ['234','b666','777','v123','c123'];  

 puts arrayA - arrayB;  
 puts arrayB - arrayA;  

Or if you want to practice your head, you can type the code like me :
 def getdiff( array1, array2)  

  array1.sort!  
  array2.sort!  

  a = array1.pop  
  b = array2.pop  

  while (a != nil && b!= nil)  

    if a == b  
     a = array1.pop  
     b = array2.pop  
     next; 
 
    elsif a < b  
     puts b;  
     b = array2.pop();  
     next;  

    elsif a > b  
     puts a;  
     a = array1.pop();  
     next;
  
    end  
  end  

  while(b == nil && a != nil)  
    puts a  
    a = array1.pop();  
  end  

  while(a == nil && b != nil)  
    puts b  
    b = array2.pop();  
  end  

  puts "Mission complete!"  

 end  

 arrayA = ['a345','v123','765','b666', '333', 'aaaa'];  
 arrayB = ['234','b666','777','v123','c123'];  

 getdiff(arrayA, arrayB);  

Thursday, November 04, 2010

Using WebDriver and Dynatrace-Ajax to help your Performance test

I have WebDriver Automation test suite in place, So i do not want to create new scripts like Watir or Selenium to do performance test with dynatrace-ajax, just like this post by Watir : http://blog.dynatrace.com/2009/11/04/5-steps-to-automate-browser-performance-analysis-with-watir-and-dynatrace-ajax-edition/

or This one by Selenium 1.0: http://blog.dynatrace.com/2010/05/21/how-to-use-your-selenium-tests-for-automated-javascriptajax-performance-analysis/

I have created Raft which is a Automation fixture by integrating WebDriver with TestNG to help to Automation Test : https://github.com/joychester/Raft/wiki

So I can use WebDriver scripts to help with Performance sanity test now, which save a lot of effort to find out performance problem in early stage and also reuse the scripts already created by QA team :)
First you need to launch Dynatrace-ajax client from : Start-> All Programs-> dynatrace-> dynatrace Ajax Edition

Then you need to add "DT_IE_XXX" Environment Variables to start your test.

This is my Runnner.bat file to start my WebDriver tests:
 @echo off  
 set DT_IE_AGENT_ACTIVE=true  
 set DT_IE_SESSION_NAME=dynaTrace_Automation
 java -cp ..\Classes;..\lib\*; raft.engine.TestEngine > log.txt  

Here is a Sample test script:
 @Test(priority = 4)  
   public void aTest3() throws InterruptedException{  
     WebDriverPlus driver = WebDriverPlus.newWebDriverPlusInstance("IE");  
     //InternetExplorerDriver driver = new InternetExplorerDriver();  
     Thread.sleep(2000);  
     driver.get("http://www.google.com/");  
     Thread.sleep(2000);  
     driver.findElement(By.name("q")).sendKeys("dynatrace");  
     Thread.sleep(2000);  
     JavascriptExecutor js = (JavascriptExecutor) driver;  
     js.executeScript("_dt_addMark(\"start_search\")");  
     driver.findElement(By.name("btnG")).click();  
     js.executeScript("_dt_addMark(\"end_search\")");  
     System.out.println("Its Test3");  
     driver.quit();  
   }  

PS: Here i am using Raft's own method newWebDriverPlusInstance() which can load different browser Type to do your test.
  WebDriverPlus driver = WebDriverPlus.newWebDriverPlusInstance("IE");   
Of course, you can use InternetExplorerDriver directly in sample code which is commented.

Here is a screen shot of Dynatrace-ajax summary generated by my WebDriver test:

Enjoy!

Wednesday, November 03, 2010

Please help to vote Issue 835 of WebDriver

My teammate Jersey found this bug:"click() does not work when the link is wrapped in two lines"
http://code.google.com/p/selenium/issues/detail?id=941

However, there is a duplicate issue there: http://code.google.com/p/selenium/issues/detail?id=835

So please help to vote for this issue and make it get fixed ASAP :)

Monday, October 18, 2010

Asking PageFactory.initElements() for Help

Some people asked me what kind of things are done by PageFactory.initElements()?
Let's see a simple example to see the difference,PS: code just for Demo, not for formal testing:-):

CommonPage.class(PageObject)
 public class CommonPage {  
      private WebDriver driver;  
      @FindBy(how = How.NAME, using = "q")  
   private WebElement searchtext;  
      @FindBy(how = How.NAME, using = "btnG")  
   private WebElement searchbutton;  
      public CommonPage(WebDriver driver) {  
             this.driver = driver;  
             System.out.println("cool stuff!");  
      }  
      public CommonPage getpages(){  
           driver.get("http://www.google.com/");  
           searchtext.sendKeys("hello, common");  
           return this;  
      }  
      public void searchaction(){  
           searchbutton.click();  
      }  
 }  

Version1, using PageFactory.initElements(WebDriver, Class):
Testcode:
 public class PageFactoryTest {  
      /**  
       * @param args  
       */  
      public static void main(String[] args) {  
           // TODO Auto-generated method stub  
                WebDriver driver = new FirefoxDriver();  
                CommonPage searchpage = PageFactory.initElements(driver, CommonPage.class); 
                searchpage.getpages().searchaction();   
                driver.quit();  
           }  
      }  

Version2, just using New:
Testcode:
 public class PageFactoryTest {  
      /**  
       * @param args  
       */  
      public static void main(String[] args) {  
           // TODO Auto-generated method stub  
                WebDriver driver = new FirefoxDriver();  
                CommonPage searchpage = new CommonPage(driver);  
                searchpage.getpages().searchaction();  
                driver.quit();  
           }  
      }  

So both PageFactory.initElements(driver, CommonPage.class); and new CommonPage(driver); are calling the construct method in order to pass the Driver to Commonpage class.

The difference between these 2 versions is to initialize looking up the WebElement (Fields) you declared in Commonpage.
so if you define your WebElements using @FindBy annotation instead of defining it within your each method directly(eg. driver.findElement(By)), then you need to using PageFactory.initElements() for help.