The Holy Java

Building the right thing, building it right, fast

Posts Tagged ‘profiling’

Profiling Tomcat Webapp with VisualVM and NetBeans – Pitfalls

Posted by Jakub Holý on February 25, 2012

Profiling a webapp running on Tomcat with VisualVM or NetBeans wasn’t as easy as expected, so this is a brief record of what to avoid to succeed.

Environment: Mac OS X, Java JDK 1.6.0_29, Netbeans 7.1, VisualVM 1.3.3 (installed separately), Tomcat 6.

The Pitfalls:

VisualVM

  • VisualVM Sampler and Profiler: To be able to drill down to the slow methods you need to take a snapshot before you stop the sampling/profiling (there is a [Snapshot] button above the Hot Spots list). This is not very intuitive and the interface doesn’t communicate it.
  • VisualVM Profiler: Excludes Thread.sleep() & Object.wait() time (as opposed to the NetBeans profiler where you can choose to include/exclude them) => if you method is spending lot of time waiting for a lock, you won’t discover it
  • You might need to allow unsafe, passwordless JMX connections in your Tomcat config, see Resources

NetBeans Profiler

  • While VisualVM was able to dynamically connect to my Tomcat, NetBeans wasn’t able to do it and hasn’t provided any notification about the failure. The only visible manifestation was that it wasn’t showing and collecting any data. Solution: Use the “Direct” attach invocation, i.e. starting the target java application with the NetBeans profiling agentlib.

Tools Overview

VisualVM

Extremely useful tool, included in JDK since 6.0 (command line: jvisualvm) or on the VisualVM page.

  • Automatically discovers local Java processes and can connect to them (if they run Java 6+)
  • Monitoring – threads, heap, permgen, CPU, classes
  • Sampler – low-overhead profiling tool (takes thread snapshot at regular intervals and compares their stack traces to find out where most time is spent)
  • Plugins, such as MBeans, Tracers
  • Profiler – a simpler version of NetBeans profiler (comparison here); it can dynamically attach to running (Java 6+) processes and instrument classes on-the-fly. The not very visible checkbox Settings in the right-top corner can be used to set the classes to start profiling from (syntax: my.package.** to include subpackages or my.package.* or my.pkg.MyClass) and the packages not to / only to profile (syntax differs here: my.package.* to include subpackages or my.package. or my.pkg.).

Resources

Posted in Languages, Tools | Tagged: , , | Comments Off

Joshua Bloch: Performance Anxiety – on Performance Unpredictability, Its Measurement and Benchmarking

Posted by Jakub Holý on December 10, 2010

Joshua Bloch had a great talk called Performance Anxiety (30min, via Parleys; slides also available ) at Devoxx 2010, the main message as I read it was

  1. Nowadays, performance is completely non-predictable. You have to measure it and employ proper statistics to get some meaningful results.
  2. Microbenchmarking is very, very hard to do correctly. No, you misunderstand me, I mean even harder than that! :-)
  3. From the resources: Profiles and result evaluation methods may be very misleading unless used correctly.

Read the rest of this entry »

Posted in Languages | Tagged: , , , | 6 Comments »

Eclipse Profile configuration: The launch requires at least one data collector

Posted by Jakub Holý on May 13, 2010

I just installed TPTP into my Eclipse 3.5 under Ubuntu 9.04 and tried to profile a class. The Profile Configuration opened with a red warning reading “the launch requires at least one data collector to be selected“. Clicking the configuration’s Monitor tab reveals a more detailed error (and nothing to select):

IWATO435E An error occured when connecting to the host.

A quick check of the error log (Window – Show View – Other… – General – Error Log) reveals the cause:

RAServer generated the following output:  [Error Stream]:ACServer: error while loading shared libraries: /home/jholy/development/tools/eclipse-ide/pulse2-2.4.2/Common/plugins/org.eclipse.tptp.platform.ac.linux_ia32_4.4.202.v201002100300/agent_controller/bin/../lib/libtptpUtils.so.4: file too short

Checking the content of the lib/ folder revealed an interesting thing:

-rw-r–r– 1 jholy jholy   17 2010-02-16 23:16 libtptpUtils.so
-rw-r–r– 1 jholy jholy   21 2010-02-16 23:16 libtptpUtils.so.4
-rwxr-xr-x 1 jholy jholy 100K 2010-02-16 23:16 libtptpUtils.so.4.5.0

As also the content of the two small files suggests (they contain a name of the corresponding file with a longer name), the *.so and *.so.4 files should have been links but the installer failed to create them.

Solution

List all files in the lib/ folder, you will see that there are many real files like libtptpUtils.so.4.5.0 and libxerces-c.so.26.0 and many should-be-links files. The solution is, of course, to replace all those files that shoud be links with actual links.

For me the solution was:

$ cd .../plugins/org.eclipse.tptp.platform.ac.linux_ia32_4.4.202.v201002100300/agent_controller/lib
# Move out the files that are OK
lib$ mkdir tmp
lib$ mv libswt-* libcbe.so tmp/
# Fix the links
lib$ for FILE in `ls *.so`; do ln -sf "${FILE}.4.5.0" $FILE; ln -sf "${FILE}.4.5.0" "${FILE}.4"; done
# Move the correct files back
lib$ mv tmp/* .
lib$ rmdir tmp
# Fix links for files with *.26 instead of *.4.5.0
lib$ ln -sf libxerces-c.so.26.0 libxerces-c.so.26
lib$ ln -sf libxerces-c.so.26.0 libxerces-c.so
lib$ ln -sf libxerces-depdom.so.26.0 libxerces-depdom.so.26
lib$ ln -sf libxerces-depdom.so.26.0 libxerces-depdom.so
lib$ rm libxerces-depdom.so.4 libxerces-c.so.4
# Done!

Try to open the profile configuration now, the IWATO435E should have disappeared and you should be able to select a data collector.

If not, restart Eclipse, try again, check the error log.

My environment

  • Ubuntu 9.04
  • Eclipse 3.5
  • TPTP – see above

Related

There is a similar post of the same problem but with different cause: Get Eclipse TPTP to run on Ubuntu Karmic Koala – the cause was: “the Agent Controller was built against old C++ libraries which were no longer available on my system (Ubuntu 9.10 Karmic Koala, amd64)”.

Posted in eclipse | Tagged: , , , , | Comments Off

RAD: Profiling a portlet

Posted by Jakub Holý on November 14, 2006

Recently I needed to profile a portlet to find its time performance bottlenecks. Since I developed it in RAD using its WebSphere 5.1 Test Environment, I thought I’d run the server in the Profiling mode and would get the results. The profiler worked very hard, but at the end no results were displayed.  I tried to profile a simple (non server) Java app. – again it seemed to do something but no results vere displayed. But finally I’ve found how to get the data using hprof:

Analyzing portlet performance with hprof
  1. Create a normal Java app. that will call the portlet’s code or a JUnit (or e.g. MockObjectTestCase, if you need  JNDI and the like set up) test
  2. Write a main method of the app. and call there the portlet code. If it’s a test case, don’t forget to call setUp before and tearDown after that.
  3. At the end of main call System.exit(0) – otherwise it may happen that the thread will not actually finish and thus the output won’t be completely generated (usually there will only be a header and names of threads that have started). Killing it via the red square button stops the threads but doesn’t generate the output.
  4. Run the app./test case as a “Java Application” using a Sun JRE 1.4 (IBM JRE would fail) and passing the VM the option “-Xrunhprof:cpu=samples,thread=y,file=cpu_profiling.hprof,depth=32

The important points: Call System.exit at the end and use Sun’s JRE 1.4.

Analyzing the output of hprof

The best is to find a tool that analyzis the output of hprof. If you want to do it yourself, look first at the end of the file – there is a list of the most time consuming methods, each with a trace number. Find a stack trace of the method in the previous part of the file using that number. Note that the stack traces are not ordered w.r.t. the trace number.

Additional notes

I’ve also tried Eclipse 3.2 and its profiling tool (TPTP). For a simple java application it displayed no output untill I changed the default options, setting polling frequency to 1sec (default: auto). I had also some problems trying to run it with JRE 5.0 but 1.4 was ok. So perhaps the RAD’s profiling would display soem results if set correctly.

Posted in eclipse, Languages | Tagged: , , , | Comments Off