Wednesday, December 23, 2009

Performance engineering internal group(team) responsibility

Here is a draft of internal performance group(team) responsibility chart in my mind, so much things to do, we need to prioritize each section better to maximum its value :

Responsibility

Tasks

Role

Delivery

Capacity planning

1. Current user pattern/distribution

2. Gap analysis

3. Predict trend of usage

4. Top slow transactions

5. Top usage of transactions

QA lead, Performance QA(owner)

PM, OPS(supportive)

SDM, TL(consultant)

Access log analysis result

RFP distribution pattern

Performance Environment Setup

1. Setup Performance environment

2. Data preparation

3. Server configuration

SCM/Dev (owner)

Performance QA (supportive)

Testable performance environment

Performance test planning and design

1. Performance test strategy design

2. Performance testing scripts/scenario design

3. Keep performance test scripts/ load distribution update to date

Performance QA(owner)

Performance Dev(supportive)

SDM, TL, QA lead(consultant)

Performance test scripts/ scenarios

Code review and design inspection

1. code review and Performance inspection checklist

Performance Dev , TL(owner)

Performance QA(supportive)

Arch(consultant)

Code review retro

Performance testing Execution

1. User-story based performance test

2. Weekly performance test

3. Release performance testing

4. End-end performance test

5. Performance initiatives

6. Performance test report

Performance QA(owner)

Performance Dev(supportive)

Performance test result report

Performance result analysis

1. Server log issues analysis

2. Performance test result analysis

3. Conclusion of performance Grade and suggestion of further investigation

Performance Dev(owner)

Performance QA(supportive)

Identify any potential Performance bottlenecks

Escalate Performance issues to the team

Performance improvement planning

1. making follow up plan for escalated performance issues

SDM, TL, QA lead(owner)

Performance QA/Dev(supportive)

Create tech user stories for each performance issue

Triage the priority of each performance user story

Performance tuning

1. Java back-end performance tuning

2. DB tuning

3. Front-end performance tuning

4. Performance Validation

SDM, TL, Dev (owner)

Performance QA(supportive)

Tuning analysis summary

Performance improvement comparison result

Code review

Check-in code into new release

Production Performance issues Follow up

1. Log performance issue in Rally

2. Server log analysis

3. Reproduce performance issue on local

4. Find the root cause of bottleneck

Performance Dev(owner)

Performance QA(supportive)

Performance issue analysis result

Reproduce suggestions

Tuesday, December 15, 2009

Analysis Jmeter result log with Google Charts by Ruby

A thought from long time ago, but never pick it up...

Thanks to http://googlecharts.rubyforge.org/, it give me the inspiration on how to combine ruby and Google Charts, then i just make it by ruby

Thanks to my intern Tom as well, he has helped me to re-factoring of my previous code, he made the charts a little bit fancy than mine:)

enjoy it!

Note: the data of bellowing Diagram is not real data , just for testing purpose :)



Ruby Source Code V1.0:

require 'gchart'

def getresdata(block1)
# scan for the first numeric token
str = block1.scan(/\d+/)[0];
return str;
end

# format RGB
def fill(str)
i = 6 - str.length();
while(i > 0)
str = '0' + str;
i -= 1;
end
return str;
end

def format_color(num, boundary)
return "000000" if num < boundary =" 0"> 16777215;

color = boundary;
increment = (16777216 - boundary)/num;

retStr = "%x" % [boundary];
retStr = fill(retStr);

while(num > 1)
color += increment;
str = "%x" % [color];
str = fill(str);

num -= 1;
retStr += ',' + str;
end

return retStr;
end

def Draw_Perf_data(file)
url_sum = Hash.new();
IO.foreach(file) { |line|
next if !line.include?("lb="); # skip when this is xml head line

line.strip!(); # trim
url = line.scan(/lb="\w+/)[0]; # fetch the 'lb=' token
puts url.sub!("lb=\"", ""); # fetch the value of lb

if url_sum.has_key?(url)
url_sum[url].push(getresdata(line).to_i); # exist, then add cell
else
url_sum[url] = Array.new(1, getresdata(line).to_i); #!exist, then create
end
}

datas = Array.new();
actions = Array.new();

url_sum.each { |key, val|
datas.push(val);
actions.push(key);
}

colors = format_color(3, 0);
arrayabc = Gchart.line(:size => "800x375",:title => "Request Log", :data => datas, :line_colors => colors, :legend => actions, :axis_with_labels => ['y'], :custom => "chg=100,25");

puts(arrayabc);
return arrayabc;
end


######################
# Call the method, just input your Jmeter result log as a parameter, note it should be the default format of your logs

Draw_Perf_data("123.txt");




here is my configuration for my JMeter result file: