Skip to content

Instantly share code, notes, and snippets.

@jonbartels
Last active October 4, 2024 19:10
Show Gist options
  • Save jonbartels/73f623b0063d447312d47e893924a967 to your computer and use it in GitHub Desktop.
Save jonbartels/73f623b0063d447312d47e893924a967 to your computer and use it in GitHub Desktop.
Thread Dumps for Mirth Connect

When to take thread dumps

  • You suspect long running, slow, or stuck connections
  • You suspect infinite loops
  • You have problems that are only fixed by redeploying channels or restarting Mirth
  • You have high memory usage
  • When Mirth just seems generally backed up and not right

How to take a thread dump

jstack is best but there are many methods - https://www.baeldung.com/java-thread-dump

What is in a thread dump

The threads that your JVM has loaded. They might be waiting, running, blocked, or in other states. In Mirth, the thread dump will have channel names as the thread name. The thread dump will NOT contain any PII nor PHI. If you/your employer consider channel names sensitive then do not share thread dumps publicly

Analysis

The author prefers manual analysis. There are a few hotspots to look for:

  • Threads with the thread name of a channel that is misbehaving -> Read the dump to see what operation is being performed when the channel is not running well
  • Threads with com.mirth in the stack trace -> There are background threads for Jetty, Quartz, and the JVM. If its not a trace with com.mirth packages somewhere in it, you probably can't do anything about it.
  • Threads that are RUNNABLE are actively doing something -> A channel that is not performing well with a runnable thread may be in a complex computation or long loop
  • Threads that are WAITING for a lock could be deadlocked -> If a thread is waiting for a lock, you need to grab the hex code for that lock and find what other thread from your stack trace holds that lock. Then review that thread to see what it is doing that is making it hold on to the lock
  • Threads in some sort of IO (network or disk) operation may be dealing with a slow connection -> The most common issues are network IO for TCP, HTTP, or DB calls
  • "qtp" threads are from poller operations -> problems in "qtp" threads will be from polling readers

https://fastthread.io/ is a good analyser that can help find cyclic locks.

Examples

Coming as soon as I remember where I put my last thread dump

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment