Virtual Machines

Sun - http://java.sun.com/javase/downloads/index.jsp

IBM - http://www-128.ibm.com/developerworks/java/jdk/

Bea JRockit - http://commerce.bea.com/products/weblogicjrockit/jrockit_prod_fam.jsp

Tools

GCViewer - http://www.tagtraum.com/gcviewer.html

StackTrace - http://www.adaptj.com/root/main/stacktrace

Sun JDK: jstack (jstack.exe on Windows)

http://www.tagtraum.com/ contains also a lot of the parameters one may add to a JVM.

Settings for Openfire

Openfire Installation Guide

Disable IPv6

As still some users do not use IPv6, it is save to disable it by adding as the third line in bin/openfire:

INSTALL4J_ADD_VM_PARAMS="-Djava.net.preferIPv4Stack=true"

Heap Settings

Add "-Xms32m -Xmx128m -Xss128k -Xoss128k -XX:ThreadStackSize=128" to INSTALL4J_ADD_VM_PARAMS to specify an initial heap of 32 MB, a maximum heap of 128 MB and a stack size of 128 kB for native and java threads. Do not append a k to the ThreadStackSize value.

Monitor the JVM Memory Usage

Add "-XX:+PrintGCDetails -Xloggc:$/WF/logs/gc.log" to INSTALL4J_ADD_VM_PARAMS to track every garbage collecion and allocation failure.

Heap dump on OutOfMemory error

Add "-XX:+HeapDumpOnOutOfMemoryError" to INSTALL4J_ADD_VM_PARAMS to create a heap dump every time an OutOfMemory error occurs. There should never occur one, but if it does the dump contains useful information to debug the issue.

Note: If you did not adjust the heap settings and try to connect 10000 users to Openfire then you will also get a heap dump which is not really interesting.

DNS Settings

If s2s connections to servers with dynamic IP addresses are possible one may want to change the DNS cache to a low value instead of forever. One can set the "networkaddress.cache.ttl=1800" value in

"path-to-JRE\lib\security\java.security" (Windows) / "path-to-JRE/lib/security/java.security" (Linux) or add "-Dsun.net.inetaddr.ttl=1800" as an additional INSTALL4J_ADD_VM_PARAMS parameter.

"1800" seconds are 30 minutes, this should be a fine value.

JM-898 should fix this issue.

UTF-8 Settings

Add "-Dfile.encoding=UTF-8" to the INSTALL4J_ADD_VM_PARAMS parameter. This fixes a few issues: JM-1351, JM-1190, JM-1101.

Debugging

To identify deadlocks or looping threads you need to create a stack trace.

Creating a stack trace

Use `jstack openfire-pid >/tmp/javacore.txt´ to write one into /tmp/javacore.txt.

On Linux `kill -3 openfire-pid´ will cause the JVM to write one to STDOUT, one will likely find in in nohup.out, stdout.log or somewhere else, depending on your start script.

One can also use StackTrace (graphical UI) to create one, the link is at the top of the page.

On Windows all methods may fail if the process runs as SYSTEM and your user does not have SYSTEM rights.

Thread identification

To identify looping threads you should also get one or more stack traces and try to identify the looping thread.

Linux native to java thread mapping

This short manual is for Linux and it may not work on your Linux installation, depending on your kernel and ps versions:

You may want to try to run `ps -T -p openfire-pid -o pid,tid,pri,time | grep -v '00:00:00'´ - this could (depending on your linux ps implementation) display the thread which is using a lot of CPU just like:

PID TID PRI TIME

8313 8313 23 00:00:01

8313 8314 21 00:13:23 <== high "time" value, look at the TID, I use 8314 as a sample value.

8313 8315 23 00:00:04

... very long list, run the command above or `ps -T -p openfire-pid -o pid,tid,pri,time | grep 8314´ again to make sure that this value increases.

Then you would have to look for the "tid" with the highest "time" value and convert it from dec to hex (8314=0x207A ; Linux: `printf "%X\n" 8314´; Windows: `calc´ with scientific view).

You should find somewhere in the javacore a line with ".. nid=0x207A .." - this should be the looping thread.

Windows native to java thread mapping

Get "pslist" from MS Technet, Process Utilities, PsList.

Run "pslist.exe -dmx Openfire >c:\temp\openfire.threads.txt" to create a list of threads within Openfire. Also the Openfire PID will be written to the file (you can alternatively look in the Task Manager for it).

Run "jstack openfire-pid >c:\temp\openfire.javacore.txt" to create a core file. This may fail depending on service permissions - for Spark it usually works fine.

Then you need to map the interesting "Tid" in `openfire.threads.txt´ to a "nid" in `openfire.javacore.txt´ by converting it to hex(Tid 252 ==> nid=0xFC).