Wednesday, March 18, 2009

Garbage Collector Ergonomics digest

the “ParallelGC” will report one issue: "Out-of-Memory Exceptions", I found one material related to this issue and they give some suggestions to avoid "OOM" issue:

Quote:

The parallel garbage collector (UseParallelGC) throws an
out-of-memory exception if an excessive amount of time is being
spent collecting a small amount of the heap.

To avoid this exception, you can increase the size of the heap.
You can also set the parameters -XX:GCTimeLimit=time-limit and
-XX:GCHeapFreeLimit=space-limit where:

time-limit:
The upper limit on the amount of time spent in garbage
collection in percent of total time (default is 98).

space-limit:
The lower limit on the amount of space freed during a garbage
collection in percent of the maximum heap (default is 2).


Here is the article called Garbage Collector Ergonomics
http://java.sun.com/j2se/1.5.0/docs/guide/vm/gc-ergonomics.html

I also take the Java tuning White paper as a reference, which emphasis on the most important points of Java tuning:
http://java.sun.com/performance/reference/whitepapers/tuning.html

For heapsize part, it describe as following:
The maximum heap size of a Java application is limited by three factors: the process data model (32-bit or 64-bit) and the associated operating system limitations, the amount of virtual memory available on the system, and the amount of physical memory available on the system. The size of the Java heap for a particular application can never exceed or even reach the maximum virtual address space of the process data model. For a 32-bit process model, the maximum virtual address size of the process is typically 4 GB, though some operating systems limit this to 2 GB or 3 GB. The maximum heap size is typically -Xmx3800m (1600m) for 2 GB limits), though the actual limitation is application dependent. For 64-bit process models, the maximum is essentially unlimited. For a single Java application on a dedicated system, the size of the Java heap should never be set to the amount of physical RAM on the system, as additional RAM is needed for the operating system, other system processes, and even for other JVM operations. Committing too much of a system's physical memory is likely to result in paging of virtual memory to disk, quite likely during garbage collection operations, leading to significant performance issues. On systems with multiple Java processes, or multiple processes in general, the sum of the Java heaps for those processes should also not exceed the the size of the physical RAM in the system.

The next most important Java memory tunable is the size of if the young generation (also known as the NewSize). Generally speaking the largest recommended value for the young generation is 3/8 of the maximum heap size. Note that with the throughput and low pause time collectors it may be possible to exceed this ratio. For more information please see the discussions of the Young Generation Guarantee in the Tuning Garbage Collection with the 5.0 Java Virtual Machine document.


For GC algorithms you can choose from:
* The -XX:+UseParallelGC parallel (throughput) garbage collector, or
* The -XX:+UseConcMarkSweepGC concurrent (low pause time) garbage collector (also known as CMS)
* The -XX:+UseSerialGC serial garbage collector (for smaller applications and systems)


first two choices are most common for large server applications. The white paper give us a lot of tuning examples, I have to say "tuning is an art more than science, there is no silver bullet". Please keep your tuning work for always...

If you have any other questions:

1> remember FAQ, although it is based on JDK1.4.2, very helpful!
http://java.sun.com/docs/hotspot/gc1.4.2/faq.html

2> read "Tuning Garbage Collection with the 5.0 Java[tm] Virtual Machine" again:

http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html

Friday, March 13, 2009

<微软如何测试软件>国内可以买到了:)

今天发现这本书已经在卓越网上有的卖了,有意思,而且也不算太贵,比起亚马逊网站的报价:$29.69, 卓越的报价是¥136=$20 :)



Interview with Alan Page on This Book:
http://www.youtube.com/watch?v=ushDnjJTLi0&hl=en&fs=1
http://www.youtube.com/watch?v=PussKcpJrik&hl=en&fs=1

More detail book info at Joyo:
http://www.amazon.cn/mn/detailApp?ref=BR&uid=168-2551957-2466640&prodid=enbk626384#

JavaScript Memory Leak Detector band on IE7

以前测试browser端(IE/FireFox/Safari)的memory leak问题,经常采用的方法就是手工走遍整个页面的flow,在经过每个页面的时候,反复刷新该页(5次以上)。。。观察task manager上的memory trend是否增长,从而初步判断出哪张页面可能存在内存泄漏的问题。不过真正找到leaks的原因还是需要费一番周折,而且需要经验的积累才能较快的定位问题。

也曾经用过Drip这个小工具,可以自动刷新页面,找到有内存泄漏的页面,并且可以看出DOM 中有问题的对象:
How to fix the leaks:
http://www.outofhanwell.com/ieleak/index.php?title=Fixing_Leaks

最近发现GPDE小组(Global Product Development - Europe team)去年已经发布了Beta版本的JavaScript Memory Leak Detector,作为IE7的一个插件帮助开发来找到Leaks:

http://blogs.msdn.com/gpde/pages/javascript-memory-leak-detector.aspx

Here is the brief intro of this tool:
JavaScript Memory Leak Detector (download) is a debugging tool to detect memory leaks in JavaScript code. It can be installed as an Internet Explorer band and it is expressly designed to find memory leaks in JavaScript code that runs in Internet Explorer. Cool, hmm...

一般情况下,如果用户习惯不是反复操作同一张页面,那么少许的内存泄漏(根据典型用户的机器配置来综合考虑)也可以暂时忽略。

Wednesday, March 04, 2009

生活中的“隔离测试”

Ad-Hoc testing:
今天回到家,发现房间的台灯A坏了,昨天用的还好好的呢。。。。

Log one defect here for tracking

Try to reproduce this issue and find more clues/isolation test in order to do further discussion and decision making

Pre-conditions
1.假设在本实验阶段,没有出现停电,短路,断路等异常情况,一切环境在万分和谐的情况下顺利进行:)
2.台灯A和台灯B上的台灯需要可以相互兼容,灯泡的最高功率应均小于两个台灯支持的功率上限

Isolation steps:
1》拿了另外房间里一个完好的台灯B来尝试,发现插座是好的,也通着电;

2》把坏的台灯A上的灯泡拿下来,放在好的灯座B上,打开台灯B,发现台灯B不亮----灯泡A本身有问题,但也不能排除灯座A本身也存在问题

3》进一步排除灯座A的问题,将灯泡B安装在灯座A上,打开台灯A,发现灯泡B依然闪亮---灯座A本身没有问题,可以断定因灯泡A(老化或质量)问题引起。

Action plan
等周末去超市买个新灯泡换上,问题解决!

Close this bug, make it as Closed,add your comments

Sunday, March 01, 2009

急性胃肠炎初判

症状一:肚子难受,隔气,但还不时的想吃东西
症状二:突然的上吐下泻(基本吃啥吐啥)
症状三:头晕发热
症状四:胃部(肚子的左下部位)有持续的微痛。

那么这些症状都符合的话,这说明你可能得了急性胃肠炎。。。。

如果没有身体脱水的话,可以吃药来治疗:

1. 消炎药:头孢克洛分散片
2. 止泻药:香连片
3. 止痛养胃药:蒙脱石散

饮食尽量不要吃肉类以及高蛋白的食物,要以清淡,新鲜,易于消化的蔬菜,米粥为主。平时多喝开水,最好加点盐,补充拉肚所丢失的水分以及盐分:)

大家切记,身体健康始终是第一位的!无论什么时候,都不要跟自己的身体开玩笑!

Tuesday, February 24, 2009

Apache Gzip level adjustment

Gzip level does impact our performance, because itself is a conflicting stuff—-consuming CPU% to reduce the file size in order to reduce the long duration network time.

so we need take these three things into consideration:
1.Network speed
2.Apache http Server capacity,especially CPU
3.File Compression ratio(.js, .Css, .html,.xml)

I did a quick testing charts based on different Gzip level and high speed Network:

Chart1—response time under different Gzip level
Gzip level | response time | Apache CPU%
Level 9 | 0.85s | 98%
Level 7 | 0.60s | 97%
Level 6 | 0.53s | 96%
Level 5 | 0.45s | 95%
Level 3 | 0.42s | 93%
Level 2 | 0.40s | 93%
Level 1 | 0.38s | 93%

Chart2—X.js Compression ratio under different Gzip level
orginal size of X.js=140K

Gzip level | After compression size
Level 9 | 30.1K
Level 7 | 30.1K
Level 6 | 30.3K
level 5 | 31.2K
Level 4 | 33K

I choose:
DeflateCompressionLevel 6

so from testing result, we can see level 9 is not always right :) just for your reference, make your own Gzip level.

Friday, February 20, 2009

Apache Http server Cache

I did some investigation on our web server after looking at the Content Caching mechanism.
there are several thing to confirm:
1. using Gzip or not?
2. What kind of files should we cache?
3. Which Cache method should i choose?
4. What kind of configurations to each Cache method?

Testing preparation: prepare one simple web page and Jmeter as load generating tool


Disk cache config:

IfModule mod_cache.c
IfModule mod_disk_cache.c
CacheRoot c:/cacheroot
CacheEnable disk /
CacheDirLevels 3
CacheDirLength 2
/IfModule
/IfModule

Mem cache config:
IfModule mod_cache.c
IfModule mod_disk_cache.c
CacheDefaultExpire 86400
CacheEnable mem /
MCacheSize 100000
MCacheMaxObjectCount 10000
MCacheMinObjectSize 100
MCacheMaxObjectSize 6400000
MCacheRemovalAlgorithm LRU
/IfModule
/IfModule

we just want to cache some static files on web server, like .js,.css and some images, which are not modified often.

First, we confirm to use Gzip to improve our web site performance(even if there is some bugs on it). If only apply Cache on Apache http server, the response time is not good if you have a few large static file to load...

Disable Gzip:
1> with Mod_mem_cache: 4160ms in average response time
2> with Mod_disk_cache: 4185ms in average response time

Enable Gzip without any Apache cache:
3> 2885ms in average response time, however, the CPU% is almost 90% with highest Gzip level(Level 9)

Note: To enable Gzip, add Accept-Encoding: gzip,deflate into JMeter header manager;


So I intent to use both Apache cache and Gzip to see any great improvement on response time and CPU% on web server.

But,finding that we using Mod_mem_cache, while Gzip can not be effected... do not know why
For Mod_disk_cache, it can be living with Gzip, and more important, it will reduce CPU% of apache server! that is great!
So, it might do a pre-Gzip job first, then cache it into the cache root directive, here is the result data for your reference:

Enable Gzip:
1> with Mod_disk_cache: 2793ms in average response time
CPU%=15%
if you want to see if Mod_disk_cache works or not, one way is to look at the cache root directive folder, if there is some strange files created after or during testing, then it works!

for Mod_disk_cache configration: you have to adjust these two interesting items:
CacheDirLevels
CacheDirLength
because different combination can give different numbers, more info please see http://httpd.apache.org/docs/2.2/mod/mod_disk_cache.html

one reminder, htcacheclean is used to keep the size of mod_disk_cache's storage within a certain limit.
you can use cmd command line to run it as a daemon mode:
C:/AppServ/Apache2.2/bin/htcacheclean -t -i -d30 -pD:/cacheroot -l1M
or
create a .bat then add a schedule task:
C:/AppServ/Apache2.2/bin/htcacheclean -t -n -v -pD:/cacheroot -l1M

In one word: we want to enable mod_disk_cache+Gzip on apache http server, currently :)

Wednesday, February 18, 2009

Content check for Web_service_call's response XML in Loadrunner

You never know what you gonna get if you do not have a content check for your response data.
If only it is a non-manual test---automation test, UI based performance test or web service call performance test and something else, you need to do validation for each response data!

A simple example for your reference, hope you can have a better solution to check the response data:

Note: in following web_service_call request, i have changed "<" into "{" and ">" into "}" due to some display issue by this blog:

web_service_call( "StepName=get****ByID_101",
"SOAPMethod=******",
"ResponseParam=response_1",
"Service=VendorServiceService",
"Snapshot=t1234933987.inf",
BEGIN_ARGUMENTS,
"xml:arg0="
"{arg0}"
"{externalID}2{/externalID}"
"{uniqueID}{/uniqueID}"
"{/arg0}",
END_ARGUMENTS,
BEGIN_RESULT,
END_RESULT,
LAST);

find_cnt =lr_xml_find("XML={response_1}",
"Query=//meetingRoomSetups[2]/name",
"Value=banquet",
"IgnoreCase=Yes",
"NotFound=error",
LAST);

if (find_cnt ==0) {
lr_out_message(...);
lr_exit(...);
}

So you can simply use lr_xml_find()to check the critical or unique value in the response XML,just like web_reg_find() to do content check for Web requests.

of Course you can use Checkpoint for content check, it is easy to fill in the expect result in Checkpoint tab from tree view in Vugen

Monday, February 16, 2009

假如分开了,还会回来吗?

37岁的奥尼尔,在精彩的poping和霸气十足的扣篮中赢得了所有人的尊重。

这种“表演”才是全明星赛的真谛,尽情的享受篮球带给自己和球迷的快乐,大姚需要尽快的适应,腼腆不会给你带来加分和任何乐趣。。。
过去的事就随风去吧,不必总是活在自己的记忆里,科比和奥尼尔不约而同对记者提出“还会不会在一起共同比赛”的问题谁了声“No!”
分开就是分开了,大家都还有自己的一片天,把记忆永久的封存在最美好的一刻吧,合适的时候回忆起来,总会那么美好,大家还是好朋友好兄弟,分开的日子更融洽!:)

“如果有一天你决定离开了,你还会回来吗?”

Thursday, February 12, 2009

Get JMS_messaging table trend by easy SQL

I have not find a very powerful tool (and free) to monitor the JMS_messaging table status, so i intend to write some easy SQL to monitoring the trend of it. Anyway it is just a temp solution to solve my job...

Basically the SQL is to Get the JMS_messaging table destination contents by a fixing time(like every 30 seconds or something) and meanwhile i want to get the trend of JMS_messaging table to see how many DLQs in the table before,during and after my load test.

declare @i int
declare @nowtime datetime
set @i=0

while @i<20
begin

set @nowtime=getdate()

SELECT destination, count(destination)
FROM JMS_MESSAGES
group by destination

set @i=@i+1
print @i
print convert(varchar(max), @nowtime, 121)

waitfor delay '00:00:30'

end;

so i will start this SQL just before i start my tests, then start the load test, after load test then check the results and messages in SQL server 2005.
you can adjust the loop counts(@i) and time delay value according to your load tests duration.