The Boot Process: From Power-On to Login

In this module, you'll learn to:

• Name each stage of the boot process and what can go wrong in it
• Read a server's boot history and timing with journalctl and systemd-analyze
• Check a risky change before a reboot, and know your options when a cloud server won't come back

Before you start: finish Part 1, especially Files, Inodes & Permissions. Open a shell on your lab VM with multipass shell lab.

Five handovers from power-on to login

Booting is a relay race. Each stage does a small job, then hands control to the next one. When a server "won't boot", the first question is always: which stage did it reach?

On a cloud server you don't see the console unless you ask for it. The server simply never answers SSH. AWS, Azure, and Google Cloud all offer a way to read the console output, and a serial console you can type into, for exactly this situation.

A server's boot story

uname -r                    # the running kernel version
ls /boot                    # installed kernels (vmlinuz-*) and initramfs images (initrd.img-*)
cat /proc/cmdline           # the options GRUB passed to the kernel
systemctl get-default       # the target systemd boots towards

sudo dmesg -T | head -30    # the kernel's own messages, with readable times
journalctl -b               # everything logged since this boot
journalctl --list-boots     # earlier boots this machine remembers
journalctl -b -1 -p err     # errors only, from the previous boot

journalctl -b -1 is the most useful line here. After an unexpected reboot, the end of the previous boot's log usually tells you why: an out-of-memory kill, a kernel error, or a clean shutdown someone requested.

Try it. Reboot the lab VM, reconnect, and confirm the journal now remembers two boots:

sudo reboot
# wait a few seconds, then on your laptop:
multipass shell lab
journalctl --list-boots

What made the boot slow

systemd-analyze breaks a boot down by stage and by service:

systemd-analyze                  # time spent in firmware, loader, kernel, and userspace
systemd-analyze blame | head     # the slowest services
systemd-analyze critical-chain   # the chain of services the boot actually waited on

blame lists everything that took time, but services start in parallel, so a slow service doesn't always delay the boot. critical-chain shows only the path the boot waited on, which is where a fix actually helps.

Kernels, GRUB, and targets

Kernel updates install alongside the old kernel rather than replacing it. If a new kernel fails, GRUB's menu can still boot the previous one. The new kernel only runs after a reboot, which is why /var/run/reboot-required appears after a kernel update.

GRUB's settings live in /etc/default/grub. After editing it, regenerate the real configuration. Never edit /boot/grub/grub.cfg directly, because the next update overwrites it:

sudo update-grub                                # Ubuntu, Debian
sudo grub2-mkconfig -o /boot/grub2/grub.cfg     # RHEL, Rocky

systemd boots towards a target, a named group of services. Two special targets matter when things go wrong:

• rescue.target: a single-user system with filesystems mounted but no network services. For repairs.
• emergency.target: the bare minimum, with the root filesystem often read-only. Where systemd drops you when something essential, such as an /etc/fstab mount, fails.

To boot into one from the GRUB menu, edit the kernel line and add systemd.unit=rescue.target. Resetting a lost root password works the same way: on Ubuntu you boot with init=/bin/bash, on RHEL family systems with rd.break, then remount the root filesystem read-write and run passwd.

Why /etc/fstab can stop a boot

During boot, systemd mounts every filesystem listed in /etc/fstab. If a listed disk is missing, it waits (90 seconds by default), and then, unless the entry is marked nofail, it gives up and drops into emergency mode. On a cloud server, that means no network and no SSH: the server just looks dead.

cat /etc/fstab
# device            mount point  type  options   dump  pass
# UUID=1b2c...      /            ext4  defaults  0     1

Each line names a device (preferably by UUID=, which stays the same even if disk names change), a mount point, a filesystem type, and options. Storage & Filesystems covers each field. For now, the rule that saves outages: test every /etc/fstab change before you reboot.

sudo findmnt --verify     # checks every fstab line for problems
sudo mount -a             # actually tries to mount everything listed

If mount -a fails now, the next boot fails too.

The reboot scheduled for tonight

A teammate added a mount for a new data disk and scheduled a reboot for 10 p.m. to apply a kernel update. Before they go home, they ask you to check that the server will come back. Reproduce their change, keeping a backup first:

sudo cp /etc/fstab /etc/fstab.bak
echo "UUID=3f1c2a9e-7b4d-4e2a-9c1f-5d8e6a7b2c10 /data ext4 defaults 0 2" \
  | sudo tee -a /etc/fstab

Your job: decide whether the server would survive tonight's reboot, without rebooting it, and make it safe. Try it before opening a hint.

Key terms

• Firmware (UEFI or BIOS): the code built into the machine that finds a disk to boot from.
• GRUB: the boot loader that chooses a kernel and the options it starts with.
• initramfs: a small temporary root filesystem that finds and mounts the real one.
• PID 1 (systemd): the first process; it starts everything else.
• Target: a named state systemd boots towards, such as multi-user.target or rescue.target.
• Emergency mode: the minimal shell systemd falls back to when something essential fails.
• nofail: an /etc/fstab option that lets the boot continue if that disk is missing.

Pop quiz

Five questions: three on the ideas in this module, and two scenarios where you apply them. The order changes every time you take it, and 4 out of 5 passes.