![]() |
Java Interactive Profiler |
hprof
tool
that ships with the JDK. There are, however, a few differences:
hprof
is not an interactive
profiler. It starts when your program starts and ends when the
JVM exits. In many cases this doesn't give you a true
measure of performance since the Just In Time compiler doesn't
compile code on the first pass. In addition, this type of profiler
is not useable at all in web applications since you end up
profiling the web container as well as the web application.
JIP, on the other hand, allows you to turn the profiler
on and off while the JVM is running.JVMPI
(Java Virtual Machine Profiling Interface)
which requires the use of native components. JIP, however,
is pure Java. It takes advantage of the Java5 feature
which allows you to hook the classloader. JIP adds
aspects to every method of every class that you want
to profile. These aspects allow it to capture performance
data.hprof
will cause a program to run 20 times slower. JIP, on the other
hand, is lightweight. A VM with profiling turned on is about
twice as slow as one without a profiler. When the profiler
is turned off, there is almost no overhead associated with
using JIP.hprof
, for
example, will show you the relative amount of time that is
spent in different parts of your code, but hprof
has so much overhead, that you cannot use it to get real
world timing measurements. JIP, on the other hand, actually
tracks the amount of time used to gather performance data
and factors that time out of its analysis. This allows you
to get close to real world timings for every class in your
code. So there is no need to litter your code with
System.currentTimeMillis()
!To run the profiler, you need the following:
profile.jar
These files are loaded by the application classloader and should not be in the extentions loader path. The jar files need to be in the same directory. The properties file can be anywhere.
To use the profiler, you need to use the following JVM arguments:
-javaagent:[DIR]\profile.jar -Dprofile.properties=[DIR2]\profile.properties
where [DIR] is the directory that contains the profile.jar
Note: Due to a bug in the JDK on OS X, [DIR] must be a fully qualifed path.and [DIR2] is the directory that contains the profile.properties
By default (if you don't give a -Dprofile.properties
),
profiling starts out turned on and the remote
interface (use to profile interactively) is turned off (see
the profile.properies
for more information). In
this case JIP works just like hprof
, although it is
much faster.
When using with stock Tomcat, set the java agent by using the
env. variable JAVA_OPTS
. For example, on Windows
use the following:
SET JAVA_OPTS=-javaagent:[DIR]\profile.jar -Dprofile.properties=[DIR2]\profile.properties
Where [DIR1] and [DIR2] are described as above.
In some cases, like webapps, you probably want to start out with the profiler turned off. When you get to some case you want to test, you'd like to turn it on, run the test case, turn the profiler off and dump the results. To do these things, you first of all need to change the profile properties file:
profiler=off
remote=on
port=15599
To interact with the profiler you need the client.jar
which is
in the /client
directory of the JIP distribution.
File.bat localhost 15599 c:\tmp\test-profile.txt
Start.bat localhost 15599
Finish.bat localhost 15599
example.bat
,
example-ant-1.5.sh
and example.sh
. All use the
example of compiling JIP with ant. example.bat
and
example-ant-1.5.sh
work only with ant 1.5 and example.sh
works only with ant 1.6 (this is because the way that ant gets invoked
changed between the two releases).
profile.jar
in my classpath, my extensions classpath,
my bootstrap classpath, or someother classpath?
Most profilers have some native
component. This is because most profilers use the
JVMPI
(Java Virtual Machine Profiling Interface)
which requires the use of native components. JIP, however,
is pure Java. It takes advantage of the Java5 feature
which allows you to hook the classloader. JIP adds
aspects to every method of every class that you want
to profile. These aspects allow it to capture performance
data.
JIP's speed is mainly due to the fact that it doesn't profile every class. JIP doesn't profile classes that are in the bootstrap classpath (all of the core Java classes) or classes that are in the extension classpath (i.e., files in-Djava.ext.dirs=
). These are classes that you can't change, so it's not every useful to know what their performace is like. It's usually enough to know that a method that is calling one of these classes is slow. You can usually figure out how to optimize things from there.
This IBM DeveloperWorks article gives an overview of the mechanisms that JIP uses.
profile.jar
in my classpath, my extensions classpath,
my bootstrap classpath, or someother classpath?
It is best not to put profile.jar
in any of these places. The classes in this JAR are loaded
by the application classloader (the classloader that loads classes that are in the classpath), but
they don't needed to be in the classpath itself.
JIP doesn't profile classes that are loaded by the bootstrap or extensions classloaders. This is one of the reasons that it is so fast. On a technical level, the classes in JIP are loaded by the javaagent which uses the application classloader. This means that while JIP is capable of instrumenting classes loaded by all classloaders, the instrumentation itself is composed of calls to classes in the JIP package, which are loaded by the application classloader, which aren't visible to classes loaded by the bootstrap and extensions classloaders. For more on this look here.
There are two mechanisms to control which classes and packages JIP profiles. The main one is by usingClassLoaderFilters
. AClassLoaderFilter
tells JIP to profile classes based on which classloader was used to load them. Why is this necessary? To start with, JIP cannot profile classes loaded by the Bootstrap or Extensions classloaders. Also, which classloader your application uses can vary by environment. For example, standalone applications use the Application classloader. You can see which class this is by callingClassLoader.getSystemClassLoader()
(usuallysun.misc.Launcher$AppClasLoader
). But if you're running in a container like Tomcat or JBoss, the application classloder is used to load the container's classes. The container will use another classloader to load application classes. Since you usually don't want to see all of what is going on inside Tomcat or JBoss, you can use aClassLoaderFilter
to tell JIP to instrument just your classes. There are several prebuildClassLoaderFilters
that can be used:You specify which class loader filter you want to use via the profile properties file:
com.mentorgen.tools.profile.instrument.clfilter.StandardClassLoaderFilter
com.mentorgen.tools.profile.instrument.clfilter.WebAppClassLoaderFilter
com.mentorgen.tools.profile.instrument.clfilter.AntTaskClassLoaderFilter
net.sourceforge.jiprof.instrument.clfilter.JBossEJBClassLoaderFilter
net.sourceforge.jiprof.instrument.clfilter.JBossServletClassLoaderFilter
net.sourceforge.jiprof.instrument.clfilter.JBossUniversalClassLoaderFilter
Writing your own is as simple as implementing theClassLoaderFilter.1=com.mentorgen.tools.profile.instrument.clfilter.StandardClassLoaderFilter
com.mentorgen.tools.profile.instrument.clfilter.ClassLoaderFilter
interface. To do that, you'll need to find out which classloader is loading your code, which can be easily accomplished by adding this somewhere in your code:If you create aSystem.out.println(Thread.currentThread().getContextClassLoader().getClass().getName());
ClassLoaderFilter
if an enviroment that it's supported, please create a feature request for it and attach your BSD licensed code. It will be greatfully included in the next release! There is a simpler method for doing this that doesn't involve writting any code. TheGenericClassLoaderFilter
allows you to specify when Class Loaders to filter on in the profile properties file. To do this, first change your class loader filter:(If you don't specify a class loader filter, JIP defaults toClassLoaderFilter.1=net.sourceforge.jiprof.instrument.clfilter.GenericClassLoaderFilter
GenericClassLoaderFilter
, so you could just as easily remove that property from the file). Once you know which classloader(s) you want to use, put them in the profile properties file:If the classloader(s) your interested in implements an interface, you can specify that instead (in other words, the classes you specify don't need to be subclasses ofaccept-class-loaders=org.apache.catalina.loader.StandardClassLoader,org.apache.catalina.loader.WebappClassLoader
java.lang.ClassLoader
). In the example above, both Tomcat classloaders implement a common interface, so you could do this:There is an additional mechansim that provides a finer level of control over which classes are profiled and which aren't. This is done via include and exclude lists. A list is a comma sperated list of packages or fully qualified class names. For example:accept-class-loaders=org.apache.catalina.loader.Reloader
This will exclude from the profile all classes and packages in theexlude=com.sun.tools.javac,org.apache.xerces.impl.XMLScanner
com.sun.tools.javac
package. In addition, the classorg.apache.xerces.impl.XMLScanner
will be excluded as well.
Include lists work the same way, only they tell the profiler what classes can be included. Note that the lists are applied to the classes that theClassLoaderFilter
accepts. So it further narrows what is profiled. Note also that if you use both an include and an exclude list, the include list is applied first. Be very careful when using both include and exclude. If a method that you want to see is called from a class that has been excluded, you will not see the call at all! Include lists are very tricky in this regard because they implicitly exclude classes and packages.
Specifyfile=[name]
in the profile properties file. The default isprofile.txt
which will be output in the JVM's working directory. You can even specify what type of output you'd like using theoutput
property. This defaults totext
but you can set if toxml
orboth
. If you specify both, JIP will substitute "xml" as the extension for the file name of the xml file. In other words, if you sayyou'll get two output files,
file=profile.txt output=bothprofile.txt
andprofile.xml
.
In a multiuser environment, even the performance of staticly compiled languages can vary. Java uses a runtime "Just In Time" compiler that observes how the code is being executed in order to use an optimal compilation strategy. This can lead to a variance in the time it takes a short running program to execute. Other factors such and disk and network IO can also contribute to the amount of time it takes a program to execute. These kinds of factors should be easy to spot in JIP's output. You should be aware that there are two things that JIP doesn't measure:
- The execution of static initializers. JIP uses an internal call stack to measure the net execution time of methods. Static initializers aren't executed as part of a program's flow of control, but rather are executed when the JVM loads the class, so JIP doesn't measure them.
- JMV startup time and time the JVM spends loading classes.
When you tell JIP to exclude a class or package, no instrumentation is done to those classes. Less profiling means lower overhead which makes everything run faster. While JIP does make an attempt to factor out its own overhead when taking measurements, nothing is perfect.
Information on interactive profiling can be found here.
First, make sure that JIP is actually running. If it is, you should see this output when the JVM starts:If you don't see this, make sure that the JVM is being invoked correctly. If you can see that in the output, make sure that you're using the rightJava Interactive Profiler: starting
ClassLoaderFilter
and/oraccept-class-loaders
. Then look at what you're including and excluding and make sure it makes sense. If all of this looks good, add the following to your profile properties file:Then try to profile again. When the program starts up, you should see a bunch of messages indebug=on
stdout
that look like this:INST org/apache/tools/ant/RuntimeConfigurable [sun.misc.Launcher$AppClassLoader] INST org/apache/tools/ant/Location [sun.misc.Launcher$AppClassLoader] INST org/apache/tools/ant/helper/ProjectHelper2$RootHandler [sun.misc.Launcher$AppClassLoader] INST org/apache/tools/ant/util/JAXPUtils [sun.misc.Launcher$AppClassLoader] skip org/apache/xerces/jaxp/SAXParserFactoryImpl [sun.misc.Launcher$AppClassLoader] skip org/apache/xerces/jaxp/SAXParserImpl [sun.misc.Launcher$AppClassLoader]
INST
indicates that the class in question has been instrumented.skip
means that it has not. If you don't see any of this, then you're not pointing to the profile properties file that you think you are. If you're using a relative path to this file, you might want to try making it absolute. If all you see areskips
, then something is wrong with the class loader filter you're using or the include/exclude lists. If you're still having problems and you're using Unix or OS X, make sure you have permissions to the directory where the output file is configured to go. If you're still having problems, post a message on the help forum. We're always happy to help out!
The way that JIP profiles is by tracking call stacks. Each call stack is associated with a thread. There are circumstances under which you end up with more than one call stack associated with a given thread. This often happens in web apps where worker threads are pooled and reused across mutliple client requests. JIP calls these interactions. You also might start a profile in the middle of a thread executing. Once the flow of control makes it back to the place where it started, profiling must stop. Every time this happens, it's called an interaction.
You cannot profile classes that are loaded by the Extension classloader. When JIP instruments
a class, it inserts a call to the runtime profiler. The runtime profiler class itself is loaded
by the javaagent interface using the Application classloader. Extensions loaded classes cannot
"see" classes loaded by the Application classloader. If you're trying to profile a
standalone app and want to profile a class that's being loaded by the extensions classloader
(i.e., -Djava.ext.dirs=[some-dir-with-my-jar]
), move the jar file
to the classpath.
You can do this with profile properties. Settingmax-method-count
to any positive integer will expressly limit the number of entries in this section of the output. -1 is the default, which means no limit. Setting this tocompact
will only display methods with a gross execution time of 10 (milliseconds) or more. You can control this with themethod.compact.threshold.ms
property.
There are a number of ways to do this, all of which use profile properties. The first isthread-depth
which defaults to -1, which means no limit. Setting this to a positive integer will limit how deep the call tree is. If you set this property tocompact
, JIP will use a node's gross time to determine if it should be displayed. This value default is 10 (milliseconds) but can be changed usingthread.compact.threshold.ms.
Yup:output-summary-only=yes
If you tell JIP to output to a directory (i.e., file=my-output-dir
),
JIP will output uniquely named files to that directory every time you run a profile.
Nope. You can see how many objects of each class were created using track.object.alloc=on
but that's no substitute for a good heap analysis tool.
Sure:output-method-signatures=yes
JIP has an output option for outputting an XML document (output=xml
).
This contains all of the information that you get in a text output, but in a terse, but
easy to parse XML format. This makes JIP a great tool to use if you have need to do
custom analysis. In fact, someone created a GUI front-end for JIP that uses the XML
output. It's pretty cool.
Yes! It's calledjipViewer
and can be found in theprofile
directory. It requires that when you profile your app, you generate an XML file output (output=xml
oroutput=both
).![]()
Yes! JIP allows you to plug in your own runtime profiler. Here's how to do it:JIP comes with 2 profilers.
- Create a class with the following methods (no need to implement an interface, nothing to inherit from -- total fredom!):
(public static void initProfiler() public static void start(String className, String methodName) public static void end(String className, String methodName) public static void alloc(String className) public static void beginWait(String className, String methodName) public static void endWait(String className, String methodName) public static void unwind(String className, String methodName, String exception)
beginWait
andendWait
are called when an object is waiting on a semaphore.) You might want to also add a shutdownHook so that your class will get notified when the JVM shutsdown.- In your profile properties file, change
profiler-class
to point to your class.com.mentorgen.tools.profile.runtime.Profile
is the one that's normally used.net.sourceforge.jiprof.timeline.TimeLineProfiler
isn't really a profiler at all -- it's a tool to aid in the understanding of concurrency issues.
Yes! Change you profile properties to have the following:The output looks like this:profiler-class=net.sourceforge.jiprof.timeline.TimeLineProfiler
Ledgend:... START [9] ProjectComponent:getProject()Lorg/apache/tools/ant/Project; (org.apache.tools.ant) END [9] ProjectComponent:getProject()Lorg/apache/tools/ant/Project; (org.apache.tools.ant) START [9] Project:log(Lorg/apache/tools/ant/Task;Ljava/lang/String;I)V (org.apache.tools.ant) Time: 3260 ms. START [9] Project:fireMessageLogged(Lorg/apache/tools/ant/Task;Ljava/lang/String;I)V (org.apache.tools.ant) ALLOC [9] BuildEvent (org.apache.tools.ant) START [9] BuildEvent:
(Lorg/apache/tools/ant/Task;)V (org.apache.tools.ant) START [9] ProjectComponent:getProject()Lorg/apache/tools/ant/Project; (org.apache.tools.ant) END [9] ProjectComponent:getProject()Lorg/apache/tools/ant/Project; (org.apache.tools.ant) START [9] Task:getOwningTarget()Lorg/apache/tools/ant/Target; (org.apache.tools.ant) END [9] Task:getOwningTarget()Lorg/apache/tools/ant/Target; (org.apache.tools.ant) END [9] BuildEvent: (Lorg/apache/tools/ant/Task;)V (org.apache.tools.ant) ...
Time - This is the time, relative to the start of profiling, when the events listed were captured.
START - The given method was entered.
END - The given method was exited.
W:START - The given method started to wait on a mutex.
W:END - The given method ended its wait on a mutex.
ALLOC - An instance of the given class was created.
THROWS - An exception was thrown.
[x] - This is the id of the thread associated with the event. Notes on usage: -output-method-signatures
can be used to display the full method signature.
-clock-resolution
can be used to toggle the unit of time used to generate the timeline. You can use eitherms
for milliseconds orns
for nanoseconds. (Note that this property is not used by the standard profiler).
- As with the standard JIP profiler, tracking object allocations can be turned on or off by usingtrack.object.alloc=on|off
.
- Currently, the Timeline profiler only supports one output format, so theoutput
property is ignored.
- You cannot use both the standard profiler and the timeline profiler at the same time.
- Output from the Timeline profiler can be very, very large.
- Currently, the interactive profiling is not supported.
- The ordering of method calls is correct within the context of a thread. Ordering is not guaranteed between threads.
Nope. JIP has no way to monitor what's happing in the heap.
If you're using the standard profiler, you can manipulate it programatically at runtime with the following static calls:Profile.clear(); // initialises the profiler Profile.start(); // starts the profiler Profile.stop(); // stops the profiler Profile.setFileName(String fileName); // changes the output file name Profile.shutdown(); // shuts the profiler down.
Profile.shutdown()
causes JIP to generate output. It's recommended that you call this right after callingProfile.stop();
. You could also add a shutdownHook that callsProfile.shutdown()
. Note also that when doing explicit profiling, you should probably start the VM withprofiler=off
in your profile properties.