Class Relation diagram in ExperT automation suite

I have posted "ExperT" automation for end-end performance test with Watir and Httpwatch for a while: http://joychester.blogspot.com/2010/02/preview-of-expert-end-to-end.html,
Here I made one simple class diagram for their relationship between these main classes:


And created Github account already, hope to open source the code soon...

Infinite Getting "Cannot open connection" error, high CPU%

I met one exception on our env recently:

2010-04-22 00:20:44,836 ERROR (WorkManager(4)-1767:)[org.hibernate.event.def.AbstractFlushingEventListener] Could not synchronize database state with session
org.hibernate.exception.JDBCConnectionException: Could not execute JDBC batch update
......
Caused by: java.sql.BatchUpdateException: I/O Error: Read timed out
at net.sourceforge.jtds.jdbc.JtdsStatement.executeBatch(JtdsStatement.java:966)

And this will cause infinite loop to get a connection, taking over 90% CPU time:

....
org.hibernate.exception.GenericJDBCException: Cannot open connection
....

after looking around the jtds site,it quoted "jTDS 1.2.3 has fixed this issue" from its release notes:

[1843801] infinite loop if DB connection dies during a batch

Seems it is exactly the bug I met!

Did not have chance to evaluate this fix, however, jtds-1.2.3 is a 2-year release after all since jtds-1.2.2.

Upgrade to latest version maybe a great choice if you met same problem.

Typical performance data degradation over time

It becomes slower and slower over time...



After restarting the SQL server service, everything becomes normal:


note:
series 1-- before restarting SQL server service
series 2-- after restarting SQL server service

It seems "SQL cache Memory" keeping growth leads to this degradation from my monitoring result, but even not quite sure why...

Gather SQL execution time stat from DMV within performance testing period

after each round of performance testing, i will gather SQL execution time stat from DMV during testing period. it gives me insight on top slowness SQLs which i can focus on first:

SELECT creation_time
,last_execution_time
,total_physical_reads
,total_logical_reads
,total_logical_writes
, execution_count
, total_worker_time
, total_elapsed_time
, total_elapsed_time / execution_count avg_elapsed_time
,SUBSTRING(st.text, (qs.statement_start_offset/2) + 1,
((CASE statement_end_offset
WHEN -1 THEN DATALENGTH(st.text)
ELSE qs.statement_end_offset END
- qs.statement_start_offset)/2) + 1) AS statement_text

FROM sys.dm_exec_query_stats AS qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) st

where creation_time > '2010-04-20 14:30:57.553' and creation_time <'2010-04-20 15:35:57.553'

ORDER BY total_elapsed_time / execution_count DESC;

thanks for this thread, i take the SQL from here and just add creation_time filter: http://www.sql-server-performance.com/articles/per/tsql_statement_performance_p1.aspx

HornetQ JMS Queue testing using Sender and Consumer

Here is a dummy demo of how to test JMS performance by JMeter sender(Point-to-Point） and Consumer.
I am taking ExampleQueue as an example in HornetQ itself, by using JMeter sender to add load to the ExampleQueue, and write some code to create a dummy consumer listening to ExampleQueue to print out received text：
first you need to add .jar into jmeter\lib which you can get from HornetQ lib:

hornetq-core-client.jar
hornetq-jms-client.jar
hornetq-transports.jar
netty.jar
jnp-client.jar
jboss-jms-api.jar

Sender by Jmeter(click to see big pic):


Consumer by Jmeter(You need to come up with JMSCorrelationID in JMS header otherwise you may get an Error):

ERROR - jmeter.protocol.jms.sampler.FixedQueueExecutor: Correlation id is null. Set the JMSCorrelationID header

Here is my configuration(click to see big pic):


multi-thread Sender by Java code sample:

package msq;
import java.util.Hashtable;
import javax.jms.JMSException;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.MessageProducer;
import javax.jms.Queue;
import javax.jms.Session;
import javax.jms.TextMessage;
import javax.naming.NamingException;
import java.util.concurrent.Executors;
import java.util.concurrent.ExecutorService;


public class JMS_Q_Sender_threads {

static int TASK_NUM=10;


static class MyTask implements Runnable{

public void run(){

try{
// Step 1. Create an initial context to perform the JNDI lookup.
Hashtable env = new Hashtable();
env.put(Context.PROVIDER_URL, "jnp://localhost:1099");
env.put(Context.INITIAL_CONTEXT_FACTORY, "org.jnp.interfaces.NamingContextFactory");
env.put(Context.URL_PKG_PREFIXES, "org.jboss.naming:org.jnp.interfaces");
Context ctx = new InitialContext(env);

// Step 2. Lookup the connection factory
ConnectionFactory cf = (ConnectionFactory)ctx.lookup("/ConnectionFactory");

// Step 3. Lookup the JMS queue
Queue queue = (Queue)ctx.lookup("/queue/ExampleQueue");

// Step 4. Create the JMS objects to connect to the server and manage a session
Connection connection = cf.createConnection();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);

// Step 5. Create a JMS Message Producer to send a message on the queue
MessageProducer producer = session.createProducer(queue);

// Step 6. Create a Text Message and send it using the producer
final int numMessages = 5;
for (int i = 0; i < numMessages; i++){
TextMessage message = session.createTextMessage("Hello, HornetQ!");
producer.send(message);
System.out.println(Thread.currentThread().getId()+": Sent message: " + message.getText());
}
// Finally, we clean up all the JMS resources
connection.close();
} catch (Exception e){}

}


}
public static void main(final String[] args) throws NamingException, JMSException
{
MyTask task=new MyTask();
ExecutorService pool=Executors.newFixedThreadPool(5);
for (int i=0;i < TASK_NUM ; i++){
pool.submit(task);
}
pool.shutdown();
}

}

Consumer by Java code sample:
package msq;
import java.util.Hashtable;
import javax.jms.JMSException;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.MessageConsumer;
import javax.jms.Queue;
import javax.jms.Session;
import javax.jms.TextMessage;
import javax.naming.NamingException;

public class JMS_Q_Consumer {
public static void main(final String[] args) throws NamingException, JMSException
{
// Step 1. Create an initial context to perform the JNDI lookup.
Hashtable env = new Hashtable();
env.put(Context.PROVIDER_URL, "jnp://localhost:1099");
env.put(Context.INITIAL_CONTEXT_FACTORY, "org.jnp.interfaces.NamingContextFactory");
env.put(Context.URL_PKG_PREFIXES, "org.jboss.naming:org.jnp.interfaces");
Context ctx = new InitialContext(env);

// Step 2. Lookup the connection factory
ConnectionFactory cf = (ConnectionFactory)ctx.lookup("/ConnectionFactory");

// Step 3. Lookup the JMS queue
Queue queue = (Queue)ctx.lookup("/queue/ExampleQueue");

// Step 4. Create the JMS objects to connect to the server and manage a session
Connection connection = cf.createConnection();
Session session2 = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
MessageConsumer messageConsumer = session2.createConsumer(queue);

// Step 5. Start the Connection so that the server starts to deliver messages
connection.start();

int j = 0;
// final int num = 5;
// while(j <= num){
final long duration = 60000;

long start = System.currentTimeMillis();

while (System.currentTimeMillis() - start <= duration){
// Step 6. Receive the message
TextMessage messageReceived = (TextMessage)messageConsumer.receive(5000);
if (messageReceived != null) {
j++;
System.out.println(j + "Received message: " + messageReceived.getText());
}
else {
System.out.println("no message detected");
break ;
}

}
long end = System.currentTimeMillis();

double rate = 1000 * (double)j / (end - start);

System.out.println("We consumed " + j + " messages in " + (end - start) + " milliseconds");

System.out.println("Actual consume rate was " + rate + " messages per second");
// Finally, we clean up all the JMS resources
connection.close();
}

}
You can create any type of consumer based on your requirement and get the performance data accordingly.
And also I used Jconsole Mbeans to help monitoring the Queue JMS message at the mean time. So that we can notice if there is anything wrong with the Queue ASAP.

Translate From Hash to Array in ruby

it is code for putting transaction name and its response time in pair , so that it can be used to show on the Google chart afterwords.

#Hash implementation：

@Detail_CHART = Hash.new
if @Detail_CHART.has_key?(transname)
new_name = transname + i.to_s
while @Detail_CHART.has_key?(new_name)
i += 1;
new_name = transname + i.to_s
end
@Detail_CHART.store(new_name, restime);
else @Detail_CHART.store(transname, restime);
end

However, i found ruby Hash keep its own order instead of original order of input, so i have to change hash to Array implementation.

#Array implementation：

@Detail_action = Array.new
@Detail_time = Array.new
if @Detail_action.include?(transname)
new_name = transname + i.to_s;
while @Detail_CHART.has_key?(new_name)
i += 1;
new_name = transname + i.to_s;
end
@Detail_action.push(new_name);
@Detail_time.push(restime);
else
@Detail_action.push(transname);
@Detail_time.push(restime);
end