Security Hardening, AppArmor & TLS

In this module, you'll learn to:

• Harden a fresh server against a short, practical checklist
• Record who changes sensitive files with auditd, and read AppArmor and SELinux denials
• Inspect TLS certificates with openssl, and recognise problems caused by a wrong clock

Before you start: finish Users, sudo & SSH and DNS, Firewalls & Packet Capture. Open a shell on your lab VM with multipass shell lab.

Smaller, patched, and watched

Hardening isn't one setting. It's a habit built on four ideas:

• Least privilege: every user and service gets only the access it needs (Users, sudo & SSH, systemd).
• Smaller attack surface: if a service isn't needed, it isn't running, and if a port doesn't need to be open, it isn't.
• Patched: security updates go on quickly and automatically (Packages).
• Watched: changes to sensitive files are recorded, and logs leave the server (Logs & Monitoring).

Published baselines, such as the CIS Benchmarks, turn these ideas into hundreds of specific checks. You rarely apply all of them by hand; automation and audit tools do that. But you should understand the important ones.

Harden a fresh server

Work through this on the lab VM. Every step is safe for your SSH session.

# 1. What's exposed? Every listening port should have a reason.
sudo ss -tlnp

# 2. Default-deny firewall. Allow SSH FIRST, or you lock yourself out.
sudo ufw allow OpenSSH
sudo ufw default deny incoming
sudo ufw enable
sudo ufw status verbose

# 3. Security updates install automatically?
systemctl is-enabled unattended-upgrades
cat /etc/apt/apt.conf.d/20auto-upgrades

# 4. SSH: keys only, no root sign-in (see Module 04)
sudo sshd -T | grep -Ei '^(passwordauthentication|permitrootlogin)'

# 5. Who can become root?
getent group sudo
sudo ls /etc/sudoers.d/

# 6. Programs that run with their owner's privileges (setuid): know every one
sudo find / -xdev -perm -4000 -type f 2>/dev/null

# 7. An automated audit for everything else, with suggestions
sudo apt install -y lynis
sudo lynis audit system --quick | tail -30

Lynis prints a hardening index and a list of suggestions. Treat it as a to-do list to think about, not a score to maximise: some suggestions won't apply to your server.

Who changed that file?

auditd records security-relevant events from the kernel: who changed a file, who ran a command as root, who changed a user account. Unlike application logs, a user can't simply skip it.

sudo apt install -y auditd
sudo auditctl -w /etc/passwd -p wa -k identity       # watch writes and attribute changes
sudo auditctl -w /etc/sudoers.d/ -p wa -k sudoers

sudo useradd audit-test                              # a change to /etc/passwd
sudo ausearch -k identity -i | tail -20              # who, when, which command
sudo userdel audit-test

auditctl rules last until reboot. Permanent rules go in a file under /etc/audit/rules.d/.

Read denials, don't disable the system

On top of normal permissions, mandatory access control confines programs to what their policy allows, even when they run as root. Ubuntu uses AppArmor, and RHEL family systems use SELinux. When one blocks something, the program sees "Permission denied" even though the file permissions look fine, which is the case Files, Inodes & Permissions warned about.

# AppArmor (Ubuntu)
sudo aa-status                                   # which profiles are loaded, enforcing or complaining
sudo journalctl -k | grep 'apparmor="DENIED"'    # what was blocked, for which program

# SELinux (RHEL, Rocky, Amazon Linux)
getenforce                                       # Enforcing, Permissive, or Disabled
sudo ausearch -m avc -ts recent                  # recent denials
sudo restorecon -Rv /srv/www                     # fix wrong file labels, the most common cause

A denial message names the program, the action, and the file. The usual fix is small: a file in an unusual location needs the right label (restorecon, semanage fcontext), or the profile needs one extra rule. Setting SELinux to Permissive while you investigate is fine. Turning it off for good removes a layer of protection, and it's rarely needed.

Certificates and the clock

Most "HTTPS is broken" incidents are an expired certificate, a missing intermediate certificate, the wrong name on the certificate, or a server whose clock is wrong. openssl checks each of these:

# The certificate a server presents: who it's for, who issued it, valid from and to
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null \
  | openssl x509 -noout -subject -issuer -dates

# Will it still be valid in 30 days? (exit code 0 = yes)
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null \
  | openssl x509 -noout -checkend $((30*24*3600))

# Is this server's clock right, and is it synchronised?
timedatectl

Certificate validity is checked against the local clock. A server whose clock has drifted months ahead sees every certificate on the internet as expired, and one that's behind sees new certificates as "not yet valid". Time sync (here systemd-timesyncd, or chrony) isn't optional.

Every HTTPS call fails

After maintenance work on this server, every HTTPS request it makes fails, and apt update complains too. From your laptop, the same sites work perfectly. Reproduce it:

sudo timedatectl set-ntp false
sudo date -s "2031-01-01 09:00"
curl -sS https://example.com -o /dev/null
sudo apt update 2>&1 | tail -3

Your job: find out why trusted certificates are being rejected, and fix it so it can't drift again. Try it before opening a hint.

Key terms

• Attack surface: everything an attacker could interact with: open ports, services, accounts.
• Default-deny firewall: block all incoming traffic except what you explicitly allow.
• auditd: the kernel audit system, which records who changed sensitive files.
• AppArmor and SELinux: mandatory access control that confines programs, even ones running as root.
• Denial: a log entry showing what AppArmor or SELinux blocked, for which program.
• Certificate chain and expiry: who vouches for a certificate, and the dates it's valid between.
• Time sync: keeping the clock right with NTP, which certificate checks depend on.

Pop quiz

Five questions: three on the ideas in this module, and two scenarios where you apply them. The order changes every time you take it, and 4 out of 5 passes.