Thursday, November 25, 2010

Sorting algorithm in Perl

My Girl Friend likes the language that easy to write but hard to read, just like Perl which makes me crazy :)

Here are 3 simple examples which is illustrating her algorithm basic skills(BTW, she is a math talented person i ever met during college)

Insertion Sorting
 @arry1 = (12,32,3,25,67,24,56);  
 @arry2 = shift @arry1;  
 $j=$#arry2;  
 foreach $arry (@arry1){  
  if ($arry2[$j]<= $arry) {  
   push @arry2, $arry;  
  }  
  elsif ($j==0 ) {  
   unshift @arry2, $arry;  
  }  
  else {  
   foreach (reverse(@arry2)){  
    if ($_ > $arry){  
    $j-=1;  
    }  
   }  
   if ($j >=0){  
   $i = $j+1;  
   @arry2= (@arry2[0..$j], $arry, @arry2[$i..$#arry2]);  
   }  
   else {  
    unshift @arry2, $arry;   
   }  
  }  
  $j=$#arry2;  
 }  

 foreach (@arry2){  
   print "$_ ";   
  }  
Quick Sorting:

 @arry1=(12,32,3,25,67,24,56);  
 $i=0;  
 $j=$#arry1;  
 &quick_sorting($i,$j);  
  sub quick_sorting {  
   my($m,$n)= @_;  
   while ($m <$n){  
    if ($arry1[$m]<= $arry1[$n]) {  
     $n-=1;  
     }  
    else {  
      ($arry1[$m], $arry1[$n]) = ($arry1[$n], $arry1[$m]);  
      $m+=1;  
     }  
    }  
   if ($m - $_[0]> 1) {  
    &quick_sorting($_[0], $m-1);  
    }  
   if ($_[1] -$n > 1){  
    &quick_sorting($n+1, $_[1]);  
    }     
  }  
 foreach $arry (@arry1){  
  print "$arry\n";   
 }  

Bubble Sorting
 @arry1 = (12,32,3,25,67,24,56);  
 $i=0;  
 $j=$#arry1;  
  for ( ; $j >=0; $j-=1){  
   for ($i=0; $i<$j; $i+=1 ){  
   if ($arry1[$i]-$arry1[$i+1] gt 0) {  
     ($arry1[$i], $arry1[$i+1])=($arry1[$i+1], $arry1[$i]);  
     }  
   }  
  }  
  foreach $arry (@arry1){  
   print "$arry\n";   
  }  

Wednesday, November 24, 2010

Aggregate Jmeter Raw data into CSV file

I am doing daily automated Jmeter load test to track performance trend using command line, due to only raw data available , so i need to write some code to parser and Aggregate it.
PS: I am using Ruby1.9 to handle this, Ruby1.8.7 and below do not implement the append row to CSV file yet.

You need to Modify the settings for Results file configuration in jmeter.properties in advance:

 jmeter.save.saveservice.output_format=csv  
 jmeter.save.saveservice.assertion_results=none  
 jmeter.save.saveservice.data_type=false  
 jmeter.save.saveservice.label=true  
 jmeter.save.saveservice.response_code=true  
 jmeter.save.saveservice.response_data=false   
 jmeter.save.saveservice.response_data.on_error=false  
 jmeter.save.saveservice.response_message=false  
 jmeter.save.saveservice.successful=false  
 jmeter.save.saveservice.thread_name=true  
 jmeter.save.saveservice.time=true  
 jmeter.save.saveservice.subresults=false  
 jmeter.save.saveservice.assertions=false  
 jmeter.save.saveservice.latency=true  
 jmeter.save.saveservice.samplerData=false  
 jmeter.save.saveservice.responseHeaders=false  
 jmeter.save.saveservice.requestHeaders=false  
 jmeter.save.saveservice.encoding=false  
 jmeter.save.saveservice.bytes=true  
 jmeter.save.saveservice.url=false  
 jmeter.save.saveservice.filename=false  
 jmeter.save.saveservice.hostname=false  
 jmeter.save.saveservice.thread_counts=false  
 jmeter.save.saveservice.sample_count=false  
 jmeter.save.saveservice.idle_time=false
 jmeter.save.saveservice.print_field_names=false

Ruby Source code:

 require 'csv'
 require 'time'
 #Generate agg result from Jmeter raw data
 def parserCSVperfdata(filename)  
  h = {};  
  h_count = {}  
  result = [];  
  i =0;  
  CSV.foreach(filename) do |rec|  
   if (h.has_key?(rec[2]))  
    h[rec[2]] = h[rec[2]].to_i + rec[1].to_i  
    h_count[rec[2]] = h_count[rec[2]].to_i + 1  
    next;  
   else  
    h[rec[2]] = rec[1]  
    h_count[rec[2]] = h_count[rec[2]].to_i + 1  
   end  
  end  
  h.each_key { |key|  
   h[key] = h[key].to_i/h_count[key].to_i  
   result[i] = key;  
   result[i+1] = ",";  
   result[i+2] = h[key];  
   result[i+3] = "\n";  
   i=i+4  
  }  
  return h.sort;  
 end 

 #insert agg result into target CSV file
 def insertlog(src,dest)  
  # get current date and collect the data today  
  t = Time.now  
  cdate = t.strftime("%Y-%m-%d")  
  #append row to CSV file, using Ruby1.9 here for mode = 'a'  
  csvwriter = CSV.open(dest,'a')  
  #append array of string, need to convert date string to array  
  csvwriter << cdate.split  
  src.each do |row|  
   csvwriter << row  
  end  
 end  

Testing code:

 t = Time.now  
 mdate = t.strftime("%Y%m%d")  
 aggresultfile = "D://Perf//loadtest.csv";  
 singlethreadresultfile = "D://Perf//singleuser.csv"  
 # Generate JMeter agg result on daily bases  
 arraydir = Dir.entries("D://tools//jakarta-jmeter-2.4//bin");  
 arraydir.each { |item|  
       item.scan(/#{mdate}_[0-9]+_rawdata\.csv/) { |match|  
           aggdata = parserCSVperfdata("D://tools//jakarta-jmeter-2.4//bin//#{match}");  
           insertlog(aggdata,aggresultfile);  
      }  
 }  

Enjoy!

Monday, November 22, 2010

Set current date and time variable in Batch file

Check your system date format at first:
The current date is: 11/23/2010 Tue
so the format is mm/dd/yyyy weekday

It might be different from other computer, some of are put weekday in the front of the date:
The current date is: Tue 11/23/2010

Here we take 11/23/2010 Tue as an example:
@echo off  
SET inputdate=%date:~6,4%%date:~0,2%%date:~3,2%  
SET inputtime=%time:~0,2%%time:~3,2%%time:~6,2%  
echo %inputdate%_%inputtime%_Rawdata.jtl

 Output:20101122_234425_Rawdata.jtl  

PS: no space between ‘=’ and ‘%’, it costs me 10 mins around this...

Thursday, November 11, 2010

How to attach add-ons to your FirefoxDirver

A Typical sample like this, in case i forget how to do :)
 //Pre-condition: Right Clicking the button to save the add-on to some location  
final String firebugPath = "C:\\addon-1843-latest.xpi";  
FirefoxProfile profile = new FirefoxProfile();      
profile.addExtension(new File(firebugPath));  

WebDriver driver = new FirefoxDriver(profile);  

Another Sample is to set proxy for the FirefoxDriver:
FirefoxProfile profile = new FirefoxProfile();  
profile.setPreference("network.proxy.http", "YourDomain");  
profile.setPreference("network.proxy.http_port", port);  
profile.setPreference("network.proxy.type", 1);   
profile.setPreference("network.proxy.share_proxy_settings", true);  
WebDriver driver = new FirefoxDriver(profile);  

Friday, November 05, 2010

Find the Difference

Ruby is cool, it can write some code like this:
 arrayA = ['a345','v123','765','b666', '333', 'aaaa'];  
 arrayB = ['234','b666','777','v123','c123'];  

 puts arrayA - arrayB;  
 puts arrayB - arrayA;  

Or if you want to practice your head, you can type the code like me :
 def getdiff( array1, array2)  

  array1.sort!  
  array2.sort!  

  a = array1.pop  
  b = array2.pop  

  while (a != nil && b!= nil)  

    if a == b  
     a = array1.pop  
     b = array2.pop  
     next; 
 
    elsif a < b  
     puts b;  
     b = array2.pop();  
     next;  

    elsif a > b  
     puts a;  
     a = array1.pop();  
     next;
  
    end  
  end  

  while(b == nil && a != nil)  
    puts a  
    a = array1.pop();  
  end  

  while(a == nil && b != nil)  
    puts b  
    b = array2.pop();  
  end  

  puts "Mission complete!"  

 end  

 arrayA = ['a345','v123','765','b666', '333', 'aaaa'];  
 arrayB = ['234','b666','777','v123','c123'];  

 getdiff(arrayA, arrayB);  

Thursday, November 04, 2010

Using WebDriver and Dynatrace-Ajax to help your Performance test

I have WebDriver Automation test suite in place, So i do not want to create new scripts like Watir or Selenium to do performance test with dynatrace-ajax, just like this post by Watir : http://blog.dynatrace.com/2009/11/04/5-steps-to-automate-browser-performance-analysis-with-watir-and-dynatrace-ajax-edition/

or This one by Selenium 1.0: http://blog.dynatrace.com/2010/05/21/how-to-use-your-selenium-tests-for-automated-javascriptajax-performance-analysis/

I have created Raft which is a Automation fixture by integrating WebDriver with TestNG to help to Automation Test : https://github.com/joychester/Raft/wiki

So I can use WebDriver scripts to help with Performance sanity test now, which save a lot of effort to find out performance problem in early stage and also reuse the scripts already created by QA team :)
First you need to launch Dynatrace-ajax client from : Start-> All Programs-> dynatrace-> dynatrace Ajax Edition

Then you need to add "DT_IE_XXX" Environment Variables to start your test.

This is my Runnner.bat file to start my WebDriver tests:
 @echo off  
 set DT_IE_AGENT_ACTIVE=true  
 set DT_IE_SESSION_NAME=dynaTrace_Automation
 java -cp ..\Classes;..\lib\*; raft.engine.TestEngine > log.txt  

Here is a Sample test script:
 @Test(priority = 4)  
   public void aTest3() throws InterruptedException{  
     WebDriverPlus driver = WebDriverPlus.newWebDriverPlusInstance("IE");  
     //InternetExplorerDriver driver = new InternetExplorerDriver();  
     Thread.sleep(2000);  
     driver.get("http://www.google.com/");  
     Thread.sleep(2000);  
     driver.findElement(By.name("q")).sendKeys("dynatrace");  
     Thread.sleep(2000);  
     JavascriptExecutor js = (JavascriptExecutor) driver;  
     js.executeScript("_dt_addMark(\"start_search\")");  
     driver.findElement(By.name("btnG")).click();  
     js.executeScript("_dt_addMark(\"end_search\")");  
     System.out.println("Its Test3");  
     driver.quit();  
   }  

PS: Here i am using Raft's own method newWebDriverPlusInstance() which can load different browser Type to do your test.
  WebDriverPlus driver = WebDriverPlus.newWebDriverPlusInstance("IE");   
Of course, you can use InternetExplorerDriver directly in sample code which is commented.

Here is a screen shot of Dynatrace-ajax summary generated by my WebDriver test:

Enjoy!

Wednesday, November 03, 2010

Please help to vote Issue 835 of WebDriver

My teammate Jersey found this bug:"click() does not work when the link is wrapped in two lines"
http://code.google.com/p/selenium/issues/detail?id=941

However, there is a duplicate issue there: http://code.google.com/p/selenium/issues/detail?id=835

So please help to vote for this issue and make it get fixed ASAP :)

Monday, October 18, 2010

Asking PageFactory.initElements() for Help

Some people asked me what kind of things are done by PageFactory.initElements()?
Let's see a simple example to see the difference,PS: code just for Demo, not for formal testing:-):

CommonPage.class(PageObject)
 public class CommonPage {  
      private WebDriver driver;  
      @FindBy(how = How.NAME, using = "q")  
   private WebElement searchtext;  
      @FindBy(how = How.NAME, using = "btnG")  
   private WebElement searchbutton;  
      public CommonPage(WebDriver driver) {  
             this.driver = driver;  
             System.out.println("cool stuff!");  
      }  
      public CommonPage getpages(){  
           driver.get("http://www.google.com/");  
           searchtext.sendKeys("hello, common");  
           return this;  
      }  
      public void searchaction(){  
           searchbutton.click();  
      }  
 }  

Version1, using PageFactory.initElements(WebDriver, Class):
Testcode:
 public class PageFactoryTest {  
      /**  
       * @param args  
       */  
      public static void main(String[] args) {  
           // TODO Auto-generated method stub  
                WebDriver driver = new FirefoxDriver();  
                CommonPage searchpage = PageFactory.initElements(driver, CommonPage.class); 
                searchpage.getpages().searchaction();   
                driver.quit();  
           }  
      }  

Version2, just using New:
Testcode:
 public class PageFactoryTest {  
      /**  
       * @param args  
       */  
      public static void main(String[] args) {  
           // TODO Auto-generated method stub  
                WebDriver driver = new FirefoxDriver();  
                CommonPage searchpage = new CommonPage(driver);  
                searchpage.getpages().searchaction();  
                driver.quit();  
           }  
      }  

So both PageFactory.initElements(driver, CommonPage.class); and new CommonPage(driver); are calling the construct method in order to pass the Driver to Commonpage class.

The difference between these 2 versions is to initialize looking up the WebElement (Fields) you declared in Commonpage.
so if you define your WebElements using @FindBy annotation instead of defining it within your each method directly(eg. driver.findElement(By)), then you need to using PageFactory.initElements() for help.

Tuesday, October 12, 2010

Windows OS performance measurements by 'typeperf' in Ruby

I am writing a Windows resource monitoring script with ‘typeperf’ command in Ruby, so that i can centralize to get all my testing server's OS Health status during performance testing.
I simply get measurements like CPU%, Mem utlization, Disk I/O throughput and Network I/O throughput from Perfmon Counters.

perf_mon_util.rb Class Source code:
 class PerfMonUtil  
  @hostIPs = [];  
  def initialize(hostarray)  
   hostarray = hostarray.sub(/\s/, '');  
   @hostIPs = hostarray.split(',');  
   puts "monitor IP(s): " + @hostIPs * ",";  
  end  
  # the unit of duration is set to minutes by default  
  def perfmon(duration = 5,intv = 1, filepath = "C:\\temp_mon\\")  
   puts @hostIPs;  
   puts intv;  
   sampler_count = (duration.to_i * 60) / intv.to_i;  
   @hostIPs.each { |inputIP|  
    machineIP = inputIP.strip;  
    filename = filepath + machineIP.gsub(/\./,'_') + "-" + Time.now.strftime("%Y%m%d-%H%M%S");  
    Thread.new {  
        system("typeperf",  
        "\\processor(_total)\\% processor time", #CPU%  
        "\\processor(_total)\\% user time",  
        "\\processor(_total)\\% privileged time",  
        "\\Memory\\Available Kbytes", #MEM_Utilization  
        "\\physicaldisk(_total)\\% disk read time", #DISK I/O  
        "\\physicaldisk(_total)\\% disk write time",  
        "\\network interface(*#1)\\bytes total/sec", #NetWork I/O  
        "-s", "#{machineIP}","-si","#{intv}", "-sc", "#{sampler_count}", "-o", filename)  
        Process.exit!(1);  
     }  
   }  
  end  
 end  

Perf_mon_runner.rb source code:
 require 'perf_mon_util'  
 case ARGV.length  
 when 0  
  puts "Please assign monitored machines' name, you can access help by type Perf_mon_runner.rb -help"  
  puts "Process exiting..."  
  Process.exit!(1);  
 when 1  
  if ARGV[0] == "-help"  
   puts "usage samples listed: Perf_mon_runner.rb MonitredIPs Duration Interval FilePath"  
   puts "Perf_mon_runner.rb \"ip1,ip2,ip3\" 1 20 \"D:\\\\temp\\\\\""  
   puts "Monitoring object list(Measurements including):"  
   puts " \\processor(_total)\\% processor time"  
   puts " \\processor(_total)\\% user time"  
   puts " \\processor(_total)\\% privileged time"  
   puts " \\Memory\\Available Kbytes"  
   puts " \\physicaldisk(_total)\\% disk read time"  
   puts " \\physicaldisk(_total)\\% disk write time"  
   puts " \\network interface(*#1)\\bytes total/sec"  
   Process.exit!(1);  
  end  
  puts "put your monitoring files in C:\\temp_mon\\"  
  CPU_Utilization= PerfMonUtil.new(ARGV[0]);  
  CPU_Utilization.perfmon();  
 when 2  
  puts "put your monitoring files in C:\\temp_mon\\"  
  CPU_Utilization= PerfMonUtil.new(ARGV[0]);  
  CPU_Utilization.perfmon(ARGV[1]);  
 when 3  
  puts "put your monitoring files in C:\\temp_mon\\"  
  CPU_Utilization= PerfMonUtil.new(ARGV[0]);  
  CPU_Utilization.perfmon(ARGV[1],ARGV[2]);  
 when 4  
  puts "put your monitoring files in " + ARGV[3]  
  CPU_Utilization= PerfMonUtil.new(ARGV[0]);  
  CPU_Utilization.perfmon(ARGV[1],ARGV[2],ARGV[3]);  
 end  

How to use it in command-line:
Perf_mon_runner.rb ["YourIP1,YourIP2"] [Monitor_Duration] [Sample_Interval] [FilePath]

Real Usage sample:
Perf_mon_runner.rb "YourIP1,YourIP2" 1 20 "D:\\temp\\"

PS: The Monitoring result will write in the file path as .CSV format by default.

i am using Gruff to draw the graph of monitoring result:

the code is pretty straightforward:
 require 'csv'  
 require 'gruff'  
 def parserperfmon(csvfile,targetfile)  
  arraycpu = Array.new;  
  i = 0;  
  CSV.foreach(csvfile) do |rec|  
    # CPU% data is put into arraycpu
    arraycpu[i] = rec[1]  
    i = i+1;  
  end  
  #column title is removed
  arraycpu.shift  
  #String need to be converted to float type in arraycpu
  arraycpu.collect! { |item|  
   item.to_f  
  }  
  g = Gruff::Line.new  
  g.title = "CPU% Monitoring Graph";  
  g.data("CPU%", arraycpu)  
  g.write(targetfile)
 end  

 filename = "D:\\TEMP\\10_201_10_173-20101125-153102\.csv"  
 targetname ="D:\\TEMP\\perf_mon.png"
 parserperfmon(filename,targetname);  

will consider to add : \\System\\Processor Queue Length to monitor the Queue length:
An instantaneous count of threads that are in the processor queue.

Saturday, October 09, 2010

A Hangzhou Trip















A bird's eye-view of West LakeOn the top of the hill













I like the smile face in the middle :-) The question is who did that...













 "Taking look at my tail,  i am changing the color"