rajeshkumar created the topic: How to automatically recover Tomcat from crashes

How to automatically recover Tomcat from crashes

Tomcat occasionally crashes if you do frequent hot-deploys or if you are running it on a machine with low memory. Every time tomcat crashes someone has to manually restart it, so I wrote a script which automatically detects that tomcat has crashed and restarts it.

Here’s the pseudo logic:

[code language=”css”]
1. every few minutes {
2. check tomcat status;
3.
4. if (status is “not running”) {
5. start tomcat;
6. }
7. }
[/code]

Here’s a shell script to implement the above logic. It assumes that you are running on a unix/linux system and have /etc/init.d/tomcat* script setup to manage tomcat.

Adjust the path to “/etc/init.d/tomcat” in the script below to reflect the correct path on your computer. Sometimes it is called /etc/init.d/tomcat5 or /etc/init.d/tomcat6 depending on your tomcat version. Also make sure that the message “Tomcat Servlet Container is not running.” matches with the message that you get when you run the script when tomcat is stopped.

[code language=”css”]
# #! /bin/sh
# SERVICE=/etc/init.d/tomcat
# STOPPED_MESSAGE=”Tomcat Servlet Container is not running.”
#
# if [ “`$SERVICE status`” == “$STOPPED_MESSAGE”];
# then
# {
# $SERVICE start
# }
# fi
[/code]

To run the script every 10 minutes:

1. Save the above script to “/root/bin/recover-tomcat.sh”.

2. Add execute permission:

1. chmod +x /root/bin/recover-tomcat.sh

chmod +x /root/bin/recover-tomcat.sh

3. Add this to root’s crontab, type the following as root:

1. crontab -e

crontab -e

4. Add the following lines to the crontab:

1. # monitor tomcat every 10 minutes
2. */10 * * * * /root/bin/recover-tomcat.sh

What if I don’t have /etc/init.d/tomcat* script on my computer?

Tomcat creates a pid file, typically in the TOMCAT_HOME/bin directory. This file contains the process id of the tomcat process running on the machine. The pseudo logic in that case would be:

[code language=”css”]
1. if (the PID file does not exist) {
2. // conclude that tomcat is not running
3. start tomcat
4. }
5. else {
6. read the process id from the PID file
7. if (no process that id is running) {
8. // conclude that tomcat has crashed
9. start tomcat
10. }
11. }
[/code]

You can implement the above logic as follows. The following is experimental and is merely a suggested way, test it on your computer before using it.

[code language=”css”]
1. # adjust this to reflect tomcat home on your computer
2. TOMCAT_HOME=/opt/tomcat5
3.
4. if [ -f $TOMCAT_HOME/bin/tomcat.pid ]
5. then
6. echo “PID file exists”
7. pid=”`cat $TOMCAT_HOME/bin/tomcat.pid`”
8. if [ “X`ps -p $pid | awk ‘{print $1}’ | tail -1`” = “X”]
9. then
10. echo “Tomcat is running”
11. else
12. echo “Tomcat had crashed”
13. $TOMCAT_HOME/bin/startup.sh
14. fi
15. else
16. echo “PID file does not exist. Restarting…”
17. $TOMCAT_HOME/bin/startup.sh
18. fi
[/code]

Why would tomcat crash?

The most common reason is low memory. For example, if you have allocated 1024MB of max memory to tomcat and enough memory is not available on that machine. Other reasons may involve repeated hot-deploys causing memory leaks, rare JVM bugs causing the JVM to crash.

Regards,
Rajesh Kumar
Twitt me @ twitter.com/RajeshKumarIn

Rajesh Kumar

I’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I am working at Cotocus. I blog tech insights at DevOps School, travel stories at Holiday Landmark, stock market tips at Stocks Mantra, health and fitness guidance at My Medic Plus, product reviews at I reviewed , and SEO strategies at Wizbrand. 

Please find my social handles as below;

Rajesh Kumar Personal Website
Rajesh Kumar at YOUTUBE
Rajesh Kumar at INSTAGRAM
Rajesh Kumar at X
Rajesh Kumar at FACEBOOK
Rajesh Kumar at LINKEDIN
Rajesh Kumar at PINTEREST
Rajesh Kumar at QUORA
Rajesh Kumar at WIZBRAND