Saturday, June 12, 2010

String match using Regular expression in both Java and Ruby

sometimes, I will looking for some dynamic data/elements from HTMLsource to make my automation scripts more robust and reliable, so using regular expression to do string match is very useful for me to get them:

Example in Java( for selenium usage):

import java.util.regex.Pattern;
import java.util.regex.Matcher;

public class RegexTestHarness {

public static void main(String[] args){


Pattern pattern =
Pattern.compile("information-abc-([0-9]+)");

Matcher matcher =
pattern.matcher("information-abc-9887234-information-abc-223333889-information-abc-12233");

while (matcher.find()) {
for (int i=1; i<=matcher.groupCount(); i++) {
String groupStr = matcher.group(i);
System.out.println(groupStr);
}
}

}
}


Example in Ruby( for Watir usage):

def regex_match(search_pattern)
source_string = "information-abc-9887234-information-abc-223333889-information-abc-12233";

source_string.scan(/#{search_pattern}/) { |match| puts match;}

end

regex_match("information-abc-([0-9]+)");


Love Ruby, it is simple :)

Friday, June 11, 2010

World CUP 2010 , Right here, Right now!

Two talented team i will support:

- Argentina
- Mexico

Let's enjoy the Game from Today!:)

Wednesday, June 09, 2010

SoapUI MockOperation Dispatching in Groovy script

SoapUI provides several ways to dispatch mock service response, you can find the detail information from: http://www.soapui.org/Service-Mocking/simulating-complex-behaviour.html.

If you want to manage your mock response dispatch more flexible, then the Script way is your choice.

Here is the sample script you can select your mock response according to the different requests:

def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context);
def holder = groovyUtils.getXmlHolder(mockRequest.getRequestContent());

//Get node value from Xpath
def OID1 = holder.getNodeValue("//v2:getRequest[1]/v2:OIDs[1]");
def OID2 = holder.getNodeValue("//v2:getRequest[1]/v2:OIDs[2]");

//if condition 1 is matched, return mockresopnse1
if (OID1 == "C1308C7C-097F"&& OID2 == "") {

mockOperation.setDefaultResponse("mockresopnse1");

}
//else if condition 2 is matched, return mockresopnse2
else if (OID1 == "A097B4E5-0159" && OID2 == "17001212-AABF") {

mockOperation.setDefaultResponse("mockresopnse2");

}


PS: Due to lack of documentation and sample scripts on this, I spent me several hours to do some investigation on it, the only thing to do is "reading the APIs and have a try".
Hope it helps,And Thanks for Lynn and Vance's help on this as well! :)

Tuesday, May 25, 2010

Bad response on Firefox with enabling Gzip

While functional testing with enabling gzip(We are using Apache/2.0.49) bring me a bad news, there is randomly returns bad response instead of 200 or 304 response code with Firefox/3.6.3:



Apache access.log has indicated the .css file return 304 Not modified, however, FF seems neither receive 304 response code successfully nor retrieve it from local cache. So the page looks wired.

Another issue is that the page will display the irrecognizable characters in its response, something like this:
���������HTTP/1.1 200 OK Date: Tue, 25 May 2010 07:30:02 GMT Server: Apache/2.0.49 (Win32) DAV/2 mod_ssl/2.0.55 OpenSSL/0.9.8e mod_jk/1.2.15 X-Powered-By: Servlet 2.4; JBoss-4.0.4.GA (build: CVSTag=JBoss_4_0_4_GA date=200605151000)/Tomcat-5.5 Vary: Accept-Encoding Content-Encoding: gzip Keep-Alive: timeout=15, max=88 Connection: Keep-Alive Transfer-Encoding: chunked Content-Type: text/html;charset=UTF-8 a ¬� ������2d3 �VQo...


Meanwhile, IE seems works fine. Does it mean its a Firefox issue? Or it is a Apache 2.0.* gzip issue? (As i did not see such issues reported by Apache 2.2 version)

After taking simple search, there are other people met the same problem as well:
http://forums.mozillazine.org/viewtopic.php?f=38&t=1121915&p=9413795#p9413795

I tried Apache2.2.10, but it seems preventing 304 Not modified response, which is a bug of apache:
https://issues.apache.org/bugzilla/show_bug.cgi?id=45023

Anyone met the similar issues? Any better solutions?

Friday, May 21, 2010

"Firewall(?)" blocks "Accept-Encoding" header

recently, I did Gzip testing with loadrunner with "Accept-Encoding: gzip, deflate" request headers, so want to make sure that Gzip rocks as i told to my teammates.

My test scenario is running from remote server which located in US DC, it sent requests to China servers.

However, i noticed the test result is not changed, then i doubt about Gzip impact, keep look at apache access logs by adding \"%{Accept-Encoding}i\" to LogFormat in httpd.conf. There is no "gzip, deflate" header in access log found:
"GET / HTTP/1.1" 200 2788 0 "-" "Mozilla/5.0 (compatible; MSIE 6.0; Windows NT 5.1)" "-"


Then I did another round of test from local desktop, then everything is fine:
"GET / HTTP/1.1" 200 2788 0 "-" "Mozilla/5.0 (compatible; MSIE 6.0; Windows NT 5.1)" "gzip, deflate"


So "Firewall" blocks "Accept-Encoding" header is just my assumption, but not quite sure... At least notice your test result is making sense, otherwise, you are wasting of time, do not let such test result misleading!

Update:
Find a person with similar issue as mine: http://forums.mozillazine.org/viewtopic.php?f=9&t=1483185
There is Outpost Firewall which disable gzip...

Another latest post on "pushing beyond Gzipping" by Yahoo! talking about forcing Gzip when Accept-Encoding request header is mangled or stripped.

Wednesday, May 12, 2010

Simple Performance profiling samples by Btrace

If you want to dig into why certain method runs slow than you expected, then you can do performance profiling by Btrace, Here is some simple example i used before, hope this helps, BTW, I used JVisualVM Btrace plugin, it is really easy of use, but Be caution with bugs :)

Calculate method execution time:

 /* BTrace Script Template */
 package com.sun.btrace.samples;  

 import com.sun.btrace.annotations.*;  
 import static com.sun.btrace.BTraceUtils.*;  
 import com.sun.btrace.aggregation.*;  
 @BTrace  
 public class MethodResponseTime {  
      /* put your code here */  
   @TLS  
   private static long starttime;  
   private static Aggregation average = newAggregation(AggregationFunction.AVERAGE);  
   private static Aggregation globalcount = newAggregation(AggregationFunction.COUNT);  
      @OnMethod(  
      clazz = "com.myapp.service.pipeline.PipelineServlet",  
      method = "execute"  
      )  
      public static void onCall(){  
        println("enter this method!");  
     starttime = timeNanos();  
      }  
   @OnMethod(  
      clazz = "com.myapp.operations.PrePopulateResponse",  
      method = "execute",  
   location = @Location(Kind.RETURN)  
      )  
      public static void onReturn(){  
        println("Method End!");  
     int duration = (int)(timeNanos()- starttime)/1000000;  
     println(duration);  
     addToAggregation(average,duration);  
     addToAggregation(globalcount,duration);  
      }  
   @OnTimer(20000)  
   public static void onEvent(){  
     println("----------------");  
     printAggregation("Average", average);  
     printAggregation("Global Count", globalcount);  
     println("----------------");  
   }  
 }  


------------------------------------------------------------

Execution time between code lines:

 /* BTrace Script Template */  
 package com.sun.btrace.samples;  

 import com.sun.btrace.annotations.*;  
 import static com.sun.btrace.BTraceUtils.*;  
 import com.sun.btrace.aggregation.*;  
 @BTrace  
 public class CodeLines {  
 @TLS private static long startTime;  
 @OnMethod(  
 clazz="com.myapp.impl.CommonResourceChangePollingImpl",  
 location=@Location(value=Kind.LINE, line=97)  
 )  
 public static void onEnter(){  
 //println("enter this method");  
 startTime= timeNanos();  
 }  
 @OnMethod(  
 clazz="com.myapp.impl.CommonResourceChangePollingImpl",  
 location=@Location(value=Kind.LINE, line=100)  
 )  
 public static void onReturn(){  
 //println("method end!");  
 int duration = (int)(timeNanos()-startTime)/1000000;  
 println(duration);  
 addToAggregation(average,duration);  
 addToAggregation(globalCount, duration);  
 }  
 @OnTimer(30000)  
 public static void onEvent() {  
 println("---------------------------------------------");  
 printAggregation("Average", average);  
 printAggregation("Global Count", globalCount);  
 println("---------------------------------------------");  
 }  
 }  

PS: it calculates from code line 97 to 100 , but not including 100 execution time itself

--------------------------------------------------------------
Call dump trace:

 /* BTrace Script Template */  
 package com.sun.btrace.samples;  

 import com.sun.btrace.annotations.*;  
 import static com.sun.btrace.BTraceUtils.*;  
 import com.sun.btrace.aggregation.*;  
 @BTrace  
 public class CallTree {  
 @OnMethod(  
 clazz="com.myapp.impl.CommonResourceChangePollingImpl",  
 method="getResourceChanges",  
 location=@Location(value=Kind.LINE, line=-1)  
 )  
 public static void online(@ProbeClassName String pcn, @ProbeMethodName String pmn, int line) {  
 print(strcat(pcn, "."));  
 print(strcat(pmn, ":"));  
 println(line);  
 }  
 }  


-----------------------------------------------------------
Regular expression match on Clazz:

 /* BTrace Script Template */  
 package com.sun.btrace.samples;  

 import com.sun.btrace.annotations.*;  
 import static com.sun.btrace.BTraceUtils.*;  
 import com.sun.btrace.aggregation.*;  
 @BTrace  
 public class RegexMatch {  
 @OnMethod(  
 clazz="/com\\.mycompany\\.service\\..*/",  
 location=@Location(value=Kind.LINE, line=-1)  
 )  
 public static void online(@ProbeClassName String pcn, @ProbeMethodName String pmn, int line) {  
 print(strcat(pcn, "."));  
 print(strcat(pmn, ":"));  
 println(line);  
 }  
 }  

Monday, May 10, 2010

Create short URL by tinyurl service

Google charts usually create a long url which is really meaningless for us to read. and I want to convert by some external services instead of creating a new one, I tried Shorturl but failed due to network service problem or something.

here is one simple way to generate using http://tinyurl.com/ service, easy of use :)


require 'open-uri'
require 'uri'

retrun_URL = 'http://chart.apis.google.com/chart?chxl=0:|load+google+landing+page|click_suggestion|1:|0|1|2|3|4|5&chxt=x,y&chco=76A4FB&chd=s:94&chtt=Detail+Page+Response+Time+for+GoogleSuggest.html&cht=lc&chs=900x300&chxr=1,5.977,5.401'

encoding_url = URI.escape(retrun_URL, Regexp.new("[^#{URI::PATTERN::UNRESERVED}]"))

tiny_url = open("http://tinyurl.com/api-create.php?url=#{encoding_url}").read

puts tiny_url


Sample OUTPUT:
>>http://tinyurl.com/1c2

Note: If you did not encoding/escape the URL, then the service will call it "bad URI(is not URI?)...URI::InvalidURIError"

Friday, May 07, 2010

Class loading issue by using Jaxb

After Monitoring our JVM by Jconsole/JvisalVM, I found the Perm Gen is not that stable and will perform Full GC after a short interval, meanwhile, the trend of total class loaded number is going up almost linear...

However Jconsole/JvisalVM only give me a rough idea that we got a problem on class loading or we just guess we may "dynamically creating new classes on the fly".




The simplest way to retrieve class loading information is to check the Verbose output check-box on Jconsole, so that you can trace the class loading event on command window where you started your service.


Here is some of loaded class information which caused the trouble we identify, Each new JAXB context would dynamically generate new classes instead of reusing the existing one:

[Loaded com.whatever.v2_0.RespondedDateRangeSummary$JaxbAccessor
F_responseOID from __JVM_DefineClass__]
[Loaded com.whatever.v2_0.RespondedDateRangeSummary$JaxbAccessor
F_respondDate from __JVM_DefineClass__]
[Loaded com.whatever.v2_0.RespondedDateRangeSummary$JaxbAccessor
F_respondDate from __JVM_DefineClass__]
[Loaded com.whatever.v2_0.RespondedDateRangeSummary$JaxbAccessor
F_statusRespondDate from __JVM_DefineClass__]
[Loaded com.whatever.v2_0.RespondedDateRangeSummary$JaxbAccessor
F_statusRespondDate from __JVM_DefineClass__]
[Loaded com.whatever.v2_0.RespondedDateRangeSummary$JaxbAccessor
F_statusOID from __JVM_DefineClass__]
[Loaded com.whatever.v2_0.RespondedDateRangeSummary$JaxbAccessor
F_statusOID from __JVM_DefineClass__]
[Loaded com.whatever.v2_0.RespondedDateRangeSummary$JaxbAccessor
F_statusCode from __JVM_DefineClass__]
[Loaded com.whatever.v2_0.RespondedDateRangeSummary$JaxbAccessor
F_statusCode from __JVM_DefineClass__]


Here is one potential solution on this similar problem, you can try that:
http://fleets.wordpress.com/2010/01/08/jaxb-memory-leak-with-metro-too-many-loaded-classes/

And also in HornetQ performance tuning " Avoiding Anti-Patterns", the first item is that "Re-use connections / sessions / consumers / producers", so we should Always re-use them instead of create new one. And also you could add -XX:+TraceClassLoading and -XX:+TraceClassUnloading for your running production to trace those information instead.

Sunday, April 25, 2010

Class Relation diagram in ExperT automation suite

I have posted "ExperT" automation for end-end performance test with Watir and Httpwatch for a while: http://joychester.blogspot.com/2010/02/preview-of-expert-end-to-end.html,
Here I made one simple class diagram for their relationship between these main classes:


And created Github account already, hope to open source the code soon...

Friday, April 23, 2010

Infinite Getting "Cannot open connection" error, high CPU%

I met one exception on our env recently:
2010-04-22 00:20:44,836 ERROR (WorkManager(4)-1767:)[org.hibernate.event.def.AbstractFlushingEventListener] Could not synchronize database state with session
org.hibernate.exception.JDBCConnectionException: Could not execute JDBC batch update
......
Caused by: java.sql.BatchUpdateException: I/O Error: Read timed out
at net.sourceforge.jtds.jdbc.JtdsStatement.executeBatch(JtdsStatement.java:966)


And this will cause infinite loop to get a connection, taking over 90% CPU time:
....
org.hibernate.exception.GenericJDBCException: Cannot open connection
....


after looking around the jtds site,it quoted "jTDS 1.2.3 has fixed this issue" from its release notes:

[1843801] infinite loop if DB connection dies during a batch

Seems it is exactly the bug I met!

Did not have chance to evaluate this fix, however, jtds-1.2.3 is a 2-year release after all since jtds-1.2.2.

Upgrade to latest version maybe a great choice if you met same problem.