Published: Mon 13 February 2017
in Linux
The Linux init system.
I decided that I'll dig down into init systems in Linux and learn more about them. I'm running Ubuntu 16.04, so this might look different on other distributions.
The init system in Linux is mainly responsible for starting essential service processes, mounting file systems and possibly other tasks. The main init systems are systemd, System V init and Upstart. Ubuntu uses systemd.
The init system starts after the Kernel starts its first user space process - init. Indeed, let's see what's running with PID 1:
$ ps 1
PID TTY STAT TIME COMMAND
1 ? Ss 0:04 /sbin/init splash
It's /sbin/init. Let's see what this file is:
$ ls -l /sbin/init
lrwxrwxrwx 1 root root 20 Sep 28 18:40 /sbin/init -> /lib/systemd/systemd
From this output, we can figure out that Ubuntu is using systemd. systemd is a fairly new project (initial release was 6 years ago), but it looks like its widely adopted now. systemd would take care of running various services like your ssh server, your web server and various other ones which are more "under the hood" oriented.
systemd organizes itself with unit files which contain the description of various units and their dependencies. The units are organized in configuration files, which live in various directories. The main directories are:
- System unit directory:
/usr/lib/systemd/. Your distribution maintains this, so don't edit it. - System configuration directory:
/etc/systemd. Make your local changes here.
Well, these are not all directories that contain unit files. Here's the full set of paths that systemd uses:
$ systemctl -p UnitPath show
UnitPath=/etc/systemd/system /run/systemd/system /run/systemd/generator /usr/local/lib/systemd/system /lib/systemd/system /usr/lib/systemd/system /run/systemd/generator.late
I won't go into details about what the unit files contain, but instead look at two services that I was curious about - ssh and apache. Who runs them? When are they run? How can I verify that they are running?
Let's start with ssh. The main command to interface with systemd is systemctl. We can use it to list all services that are running, by calling systemctl list-units. Let's look for ssh in here:
$ systemctl list-units | grep ssh
ssh.service loaded active running OpenBSD Secure Shell server
Indeed, we have ssh running. Now, let's look at its status and its config file.
$ systemctl status ssh
● ssh.service - OpenBSD Secure Shell server
Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2017-02-09 19:27:57 PST; 4 days ago
Main PID: 786 (sshd)
Tasks: 1
Memory: 6.4M
CPU: 199ms
CGroup: /system.slice/ssh.service
└─786 /usr/sbin/sshd -D
Feb 12 18:23:50 virtbox sshd[5791]: Accepted password for petko from 192.168.1.86 port 57805 ssh2
Feb 12 18:23:50 virtbox sshd[5791]: pam_unix(sshd:session): session opened for user petko by (uid=0)
...
So here it is. The ssh service is running as process 786. We can see that this process is listening on port 22:
$ sudo netstat -tulpn | grep 786
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 786/sshd
tcp6 0 0 :::22 :::* LISTEN 786/sshd
Indeed, it is. systemctl has another useful command that allows us to print the configuration file for a unit. It works like this:
$ systemctl cat ssh
# /lib/systemd/system/ssh.service
[Unit]
Description=OpenBSD Secure Shell server
After=network.target auditd.service
ConditionPathExists=!/etc/ssh/sshd_not_to_be_run
[Service]
EnvironmentFile=-/etc/default/ssh
ExecStart=/usr/sbin/sshd -D $SSHD_OPTS
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
RestartPreventExitStatus=255
Type=notify
[Install]
WantedBy=multi-user.target
Alias=sshd.service
So here you can see where is the configuration file located.
Alright, enough ssh. Let's move on to apache. First, a little history though. Before systemd, apparently the main init system in Linux was System V. System V is different than systemd, because it executes services in sequential order, while systemd can be parallel. System V also can't start services on "as-needed" basis. So I guess that's why systemd was implemented. systemd has its config files in /etc/init.d. That's where Apache installs its command files as well - it doesn't create systemd unit files. However, systemd knows how to execute the System V init files. I won't go into details of how System V init works, but basically it executes commands on different runlevels and at each runlevel they are executed in sequential order.
Let's see how apache looks like in systemd:
$ systemctl list-units | grep apache
apache2.service loaded active running LSB: Apache2 web server
It's running. Now let's get its status:
$ systemctl status apache2
● apache2.service - LSB: Apache2 web server
Loaded: loaded (/etc/init.d/apache2; bad; vendor preset: enabled)
Drop-In: /lib/systemd/system/apache2.service.d
└─apache2-systemd.conf
Active: active (running) since Mon 2017-02-13 12:49:55 PST; 8h ago
Docs: man:systemd-sysv-generator(8)
Tasks: 55
Memory: 6.5M
CPU: 17.042s
CGroup: /system.slice/apache2.service
├─9097 /usr/sbin/apache2 -k start
├─9100 /usr/sbin/apache2 -k start
└─9101 /usr/sbin/apache2 -k start
Feb 13 12:49:54 virtbox systemd[1]: Starting LSB: Apache2 web server...
Feb 13 12:49:54 virtbox apache2[9071]: * Starting Apache httpd web server apache2
Feb 13 12:49:54 virtbox apache2[9071]: AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globall
Feb 13 12:49:55 virtbox apache2[9071]: *
Feb 13 12:49:55 virtbox systemd[1]: Started LSB: Apache2 web server.
Look at something interesting here. The file responsible for starting apache is listed as /etc/init.d/apache2. That's the file indeed. It's adapted into systemd by using the systemd-sysv-generator. So that is systemd running. We can run a cat to see that:
$ systemctl cat apache2
# /run/systemd/generator.late/apache2.service
# Automatically generated by systemd-sysv-generator
[Unit]
Documentation=man:systemd-sysv-generator(8)
SourcePath=/etc/init.d/apache2
Description=LSB: Apache2 web server
Before=multi-user.target
Before=multi-user.target
Before=multi-user.target
Before=graphical.target
Before=shutdown.target
After=local-fs.target
After=remote-fs.target
After=network-online.target
After=systemd-journald-dev-log.socket
After=nss-lookup.target
Wants=network-online.target
Conflicts=shutdown.target
[Service]
Type=forking
Restart=no
TimeoutSec=5min
IgnoreSIGPIPE=no
KillMode=process
GuessMainPID=no
RemainAfterExit=yes
ExecStart=/etc/init.d/apache2 start
ExecStop=/etc/init.d/apache2 stop
ExecReload=/etc/init.d/apache2 reload
# /lib/systemd/system/apache2.service.d/apache2-systemd.conf
[Service]
Type=forking
RemainAfterExit=no
This is the file that systemd created in order to integrate the System V init command into its system.
What are some other interesting systemctl commands? Let's list them:
# Start / stop /restart a service.
$ sudo systemctl restart apache2
# List all services:
$ systemctl list-units --type=service
# List dependencies:
$ systemctl list-dependencies sshd.service
# See low level properties of a unit:
$ systemctl show sshd.service
And one last cool command:
$ systemd-analyze
Startup finished in 3.989s (kernel) + 7.673s (userspace) = 11.663s
This command prints the time it took to startup our system.
So that's all for today. There's definitely more to explore in systemd land - the syntax of unit files, how systemd executes them and so on. I'll leave that for some other time.