If you run your own virtual or dedicated server, chances are you want to run a specific program or some programs that make up the service you want to use or offer. If the program you want to run is part of the Linux distribution of your choice there usually is not much more to it than installing the package and configuring the program. However, if the program comes from an external source or you are writing it yourself, you need to ensure it starts automatically when the server is booting. Additionally, during the development or early testing phases of your own program, there might be errors in the code leading to a crash of your application, and you might want to make sure that it gets restarted automatically in such a case. A few years ago, the solution to the first issue was quite simple. All you needed to do was to create an init script that would then handle the starting and stopping of the server. However, many Linux distributions recently changed how they handle the boot process. Some are still using init, others may be using upstart or even systemd now. Providing the files necessary for all of these systems can be quite a hassle and while upstart and systemd support restarting programs on unexpected termination, implementing this with init is possible but requires to change the init configuration itself.
For my own needs, I have become attached to using a supervisor for this task - the program and the documentation can be found on http://supervisord.org, mostly Linux distributions provide pre-built packages in their repositories. The supervisor itself is a daemon that is run by the system's process management so it gets run by init or its counterparts. To run your own program, you have to add it to Supervisord's configuration and it will ensure that it gets started on boot and restarted in case it crashes.
For example, I will be using a very small custom web application written in Python using the bottle framework. Since this article is not about web programming, I am keeping it simple:
1 from bottle import route, run
2
3 @route('/')
4 def index():
5 return 'Hello World'
6
7 run(host='0.0.0.0', port=8080)
All this does is run a webserver on port 8080 and displaying Hello World in your web browser when you navigate to it. If the above code is saved to a file app.py, you can run it using python app.py and it will just run forever (or until it crashes). Now would be a good time to configure the supervisor to run this application for us. The supervisor provides a command line tool called supervisorctl to check the status of configured applications and to start or stop them if needed. Running supervisorctl status will show you... nothing, as we have not set up anything yet. We create a new file called hello.conf which will contain everything the supervisor needs to know to run our application and place it in /etc/supervisor/conf.d/. The most basic configuration defines a new program to run with a given name and a command to be executed as well as a user name that the program should be run with - if you leave this, your program will run as root which is almost always a bad idea:
[program:hello]
command=/usr/bin/python /home/markus/app.py
user=markus
Observe, that usually it is a good idea to provide absolute paths in such configurations. After the file has been saved, you can use supervisorctl reread to cause the supervisor to reread its configuration file. If everything is right, the output of the command should tell you that a program named hello is now available:
# supervisorctl reread
hello: available
We can now start the program by running
# supervisorctl start hello
hello: started
Check the status again to see if it's actually running now:
# supervisorctl status
hello RUNNING pid 32675, uptime 0:00:46
As you can see, it tells us that the program is running, its PID, and the time elapsed since it started. To simulate a crash, we will forcefully terminate the program and check if the supervisor restarts it as expected:
# kill -9 32675
# supervisorctl status
hello RUNNING pid 32686, uptime 0:00:04
The supervisor homepage gives you much more information about possible values to configure the supervisor itself and the programs it runs. You can even configure the location of log files and automatic rotation for them in case they grow over a given size. More details about this can be found here: http://supervisord.org/configuration.html. Finally, there is no more reason to run your applications in detached screen sessions...
