Using Eclipse Memory Analyzer Tool(MAT) to investigate memory heap dumps

There are occasions where memory usage repeatedly hits the limit, leading to some poor soul having to investigate and possibly fix the issue. Fortunately there are a wealth of profiling tools available to investigate, and locate troubling code. The first step is to analyze a heap dump taken, preferably, when the application is close to falling over. If you are an eclipse user, like me, then MAT is an excellent tool for such a job.

One of our production Java applications was sufferring from such an issue. jmap, and if that is not available, then jcmd, can be used to create a heap dump, as below

jmap -dump:format=b,file=somefile.hprof <pid>

#for jcmd usage, run as user owning the JVM process

jcmd <pid> GC.heap_dump somefile.hprof

The .hprof file can then be opened with Eclipse MAT, and it does some initial data analysis for you. Although, the instinct is to look for memory leaks, and if does not seem to be the problem, it may well be a good idea to look elsewhere else e.g. session data not being removed, or data caching.

Clicking "Dominator Tree" provdies a view, such as below

From the first line in the tree, you can see the object causing the memory issue. Reading the fasterXML, it seems ObjectMapper heavily caches serializers, deserializers and other related objects. So, it seems each created deserializer is repeatedly added to the cache. In this case, it has taken a few minutes to identify the problem. However, fixing the issue may not always be as easy.

The Rest template was written to do

#if the restTemplate.getMessageConverters() already contains a 
#MappingJackson2HttpMessageConverter then get the ObjectMapper 
#or create a new ObjectMapper(), and always do bewlow

mapper.registerModule(new JavaTimeModule());
mapper.registerModule(authenticationModule());

At first glance it seems the deserializer, found in, authenticationModule() is repeatedly added to the objectMapper. For some reason the Deserializers cache treats it as a different object, and adds it to the cache. So, what to do... Well, firstly annotate the serializable object, rather than explicitly setting an object, on the ObjectMapper i.e.

@JsonDeserialize(using = MarkLogicAuthenticationTokenDeserializer.class)
public class MarkLogicAuthenticationToken extends

Secondly, just use a new ObjectMapper() instance for the MappingJackson2HttpMessageConverter i.e. do not register any modules on it.

Next step, build/deploy, and find a safe place to hide...

If the application server exposes a JMX port, you can create realtime monitoring as well. For quick and easy analysis, I tend to just use Java VisualVM or MissionControl. With VisualVM, I get to monitor heap usage, in terms of number of objects, CPU, in realtime. Although it is an expensive operation, it provides an indication of memory usage/object count. This is quite useful if you want to just get a quick snapshot of the environment. Using the information below

Filtering the results on "Deserializer, "I can see that the DeserializersCache only contains 12 objects, and this has been consistent over the past two weeks of monitoring.

Using Prometheus/Grafana usage stats suggest some level of stability

MAT provides a wide range of tools, and I particularly like the idea of using OQL to query heap objects; nice.