Arthas Java Diagnostics for Live Production Troubleshooting
Use Arthas to diagnose a running JVM without restarting it: attach, find hot threads, trace slow methods, watch arguments and decompile loaded classes safely.

Production is slow, one endpoint, and only on one pod. You cannot reproduce it locally, the logs do not say why, and attaching a debugger to a live service is out of the question. The usual tools each cover part of it: a thread dump shows what threads are doing now, a profiler needs a restart or a special flag, and adding a log line means a deploy that may clear the very condition you are chasing. Arthas, an open source diagnostic tool from Alibaba, attaches to a running JVM without a restart and lets you ask questions of it interactively: which method is slow, what arguments it received, what it returned, which jar a class came from. This post walks through attaching, orienting yourself, finding a slow call, looking at live data, inspecting loaded classes, the other commands worth knowing, and the safety rules that apply when the target is production. Examples use Arthas 4 and JDK 25.
Attach
Arthas is a single jar. Download it on the host (or in the container, if its filesystem allows), run it as the same user as the target process and pick the JVM from the list:
curl -O https://arthas.aliyun.com/arthas-boot.jar
java -jar arthas-boot.jar[INFO] arthas-boot version: 4.0.5
[INFO] Found existing java process, please choose one and input the serial number of the process, eg : 1. Then hit ENTER.
* [1]: 23871 dev.javahacks.shop.ShopApplication
[2]: 23990 org.apache.catalina.startup.Bootstrap
1
[INFO] arthas home: /home/app/.arthas/lib/4.0.5/arthas
[INFO] Try to attach process 23871
[INFO] Attach process 23871 success.
[arthas@23871]$It attaches through the JVM's attach mechanism, loads an agent into the target and gives you a prompt. The attach works only if you are the same OS user that owns the process, and in a container if the JVM's temp directory is shared, which is why a non-root user with a read-only filesystem needs a writable /tmp. If you run in Kubernetes, kubectl exec -it <pod> -- sh and run the jar from there, or use an ephemeral debug container that shares the process namespace. Detach cleanly with stop when you finish; that unloads the instrumentation.
Get oriented
The dashboard command is a live overview, refreshed every few seconds:
[arthas@23871]$ dashboard -n 1
ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTED DAEMON
31 http-nio-8080-exec-3 main 5 RUNNABLE 63.2 0:14 false true
28 http-nio-8080-exec-1 main 5 RUNNABLE 21.7 0:05 false true
9 G1 Young RemSet Sampling - 9 RUNNABLE 1.1 0:00 false true
Memory used total max usage GC
heap 612M 1024M 1024M 59.8% gc.g1_young_generation.count 41Threads sorted by CPU, memory areas and GC counts in one screen. Here two request threads are doing the work, so the problem is in request handling, not in the collector. Two commands drill into threads. thread -n 3 prints the stack of the three busiest, and thread -b finds the thread that blocks others:
[arthas@23871]$ thread -n 1
"http-nio-8080-exec-3" Id=31 cpuUsage=63.2% RUNNABLE
at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:4234)
at java.util.regex.Matcher.match(Matcher.java:1270)
at dev.javahacks.shop.search.QueryParser.normalize(QueryParser.java:52)
at dev.javahacks.shop.search.SearchController.search(SearchController.java:31)A hot thread inside a regex match called from QueryParser.normalize is already a lead, obtained without stopping the service. The thread-dump approach on its own, with jstack, is compared in using jcmd, jstack and jmap safely in production, and deadlocks in particular in finding deadlocks in a thread dump.
Find the slow call
trace instruments a method and prints the time spent in each call it makes, as a tree, for matching invocations. It answers "where inside this method does the time go" without code changes:
[arthas@23871]$ trace dev.javahacks.shop.search.SearchController search '#cost > 200'
Press Q or Ctrl+C to abort.
Affect(class count: 1 , method count: 1) cost in 143 ms, listenerId: 1
`---ts=2026-11-06 14:03:11;thread_name=http-nio-8080-exec-3;id=31;is_daemon=true
`---[1204.5ms] dev.javahacks.shop.search.SearchController:search()
+---[0.8ms] dev.javahacks.shop.search.QueryParser:parse() #29
+---[1190.7ms] dev.javahacks.shop.search.QueryParser:normalize() #31
`---[12.1ms] dev.javahacks.shop.search.SearchRepository:find() #34The condition #cost > 200 prints only calls slower than 200 ms, which keeps the output to the interesting ones. The tree shows normalize took 1,190 ms of a 1,204 ms request, so the database is innocent. Here the cause is a regular expression with catastrophic backtracking on some input. Because the method came from your own code, you now know exactly where to look and what input to look for, which the next command supplies.
stack does the inverse: it prints the call path that leads to a method, useful for finding who calls an unexpected method. Both commands add overhead to the instrumented method for as long as they run, so stop them with Q as soon as you have what you need.
See the data
watch observes the arguments, return value or exception of a method call, with an OGNL expression choosing what to print:
[arthas@23871]$ watch dev.javahacks.shop.search.QueryParser normalize '{params, returnObj, throwExp}' -x 2 -n 3 '#cost > 200'
method=dev.javahacks.shop.search.QueryParser.normalize location=AtExit
ts=2026-11-06 14:03:11; [cost=1190.7ms] result=@ArrayList[
@Object[][
@String[aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!],
],
@String[aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!],
null,
]The argument is a long run of a characters followed by !, the classic input that sends a backtracking regex into exponential time. The flags: -x 2 expands objects two levels deep, -n 3 stops after three matches, and the condition limits it to the slow calls. You found the bug's trigger from production traffic in about two minutes. Reproduce it in a unit test with that input, fix the pattern, deploy.
A caution that comes with this power: watch prints real data. Arguments may contain tokens, personal data and passwords. Treat the terminal and any recorded session as sensitive, and limit what you print with narrow expressions. The same goes for tt, which records invocations for later replay.
Check what is loaded
Arthas answers classloading questions that are otherwise guesswork. Which jar did a class come from, and is it the version you think?
[arthas@23871]$ sc -d dev.javahacks.shop.search.QueryParser
class-info dev.javahacks.shop.search.QueryParser
code-source /app/shop.jar!/BOOT-INF/classes!/
class-loader +-org.springframework.boot.loader.launch.LaunchedClassLoader@7b1d7fffsc (search class) with -d shows the code source and the class loader chain. sm lists a class's methods. jad decompiles a loaded class back to source, which answers "is the running code what was deployed" in one command:
[arthas@23871]$ jad --source-only dev.javahacks.shop.search.QueryParser normalizeclassloader -t prints the loader hierarchy as a tree. These are the same questions as in debugging classpath and class loading problems, asked of a live process rather than a restarted one. Comparing the jad output with the repository is also how you spot a stale jar deployed to one pod of a fleet, a surprisingly common explanation for "only one pod is wrong".
Other useful commands
| Command | Use |
|---|---|
vmoption | Read and change some JVM flags live, such as HeapDumpOnOutOfMemoryError |
logger --name ROOT --level debug | Change a log level without a restart |
heapdump /tmp/heap.hprof | Write a heap dump for later analysis |
profiler start / profiler stop | Sample CPU with async-profiler and write a flame graph |
ognl '@System@getenv("APP_ENV")' | Evaluate an expression against the live process |
monitor -c 5 Class method | Aggregate call count, failure count and average cost every 5 seconds |
redefine | Replace a class with a recompiled version (see the safety section) |
profiler is often the quickest route from "it is slow" to a flame graph, and monitor turns a hunch into numbers: calls per interval, failures and mean latency for one method.
Safety
An agent that can read any argument and change classes is powerful, so use it with care in production.
- Overhead.
trace,watchandmonitorinstrument bytecode and add cost to the target methods, andwatchwith deep expansion on a hot method can slow a service noticeably. Use conditions,-nlimits and short runs, and always end withstoporreset. - Access. Anyone who can attach can read memory and call methods. Restrict who may run it, keep it off the image by default (download it only during an incident) and never expose the telnet or HTTP ports it can open beyond localhost; the built-in port is unauthenticated unless you configure a password. The security basics are in Java security basics for backend developers.
- Data. Output and session logs contain live data. Do not paste them into a ticket unredacted.
- Mutation.
redefine,ognlcalls with side effects andloggerchanges alter the running process. A hot-patched class is not in your repository and disappears at restart. Use it to confirm a hypothesis, then fix the code properly. - Change control. Treat the session like a production change: someone else knows you are attached, and what you ran is written down.
Where this does not apply
If you can reproduce the problem locally, a debugger and a profiler on your machine are safer and richer. If your platform forbids attaching to processes (a read-only image with no /tmp, a distroless container with no shell) you will need an ephemeral debug container or the JVM's built-in tools instead. For continuous visibility rather than incident work, metrics and tracing are the right layer. Arthas shines as the incident tool: one running JVM, an unknown cause, and no time for a redeploy. The commands are documented at arthas.aliyun.com and the source is on GitHub.


