Compiling a Custom Linux Kernel & Adding systemd (Part 2)

In Part 1, we built a minimal Linux system using BusyBox and a prebuilt kernel. You got a shell running inside a container, mounted pseudo-filesystems, and saw the boot sequence from init to prompt.

That was the “hello world” of custom Linux. Now we’re doing the real thing.

Today we’re going to compile our own kernel from source, configure only the hardware support we actually need, and compress it to under 10MB. Then we’ll rip out that hand-written init script and replace it with systemd — the same init system that runs on virtually every modern Linux distribution. Finally, we’ll get a real service running (OpenSSH) so you can actually log into your custom system remotely.

This is the post that turns “I sort of understand Linux” into “I know exactly what every layer does.” Let’s go.

Prerequisites

You need everything from Part 1, plus a few extras:

• A Linux machine (kernel compilation doesn’t work well on macOS/Windows — use a VM if needed)
• 4GB+ RAM (compilation is memory-hungry)
• 10GB free disk space (source tree + build artifacts + tools)
• 30-45 minutes (most of that is compile time — you can grab coffee)

Install the build dependencies:

sudo apt-get update
sudo apt-get install -y build-essential libncurses-dev bison flex libssl-dev \
    libelf-dev bc rsync cpio wget xz-utils systemd-container

That systemd-container package is key — it gives us systemd-nspawn, which is like Docker but uses your custom kernel directly. More on that later.

Step 1: Download the Kernel Source

mkdir -p ~/kernel-build && cd ~/kernel-build

# Download latest stable kernel (6.12.x as of writing)
wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.12.tar.xz

# Extract
tar -xf linux-6.12.tar.xz
cd linux-6.12

The full source tree is about 1.4GB uncompressed. That’s 30+ million lines of code spanning every CPU architecture, every driver, every filesystem Linux supports. Your job is to trim that down to what you actually need.

Step 2: Configure the Kernel — What to Keep, What to Kill

This is where most people either panic or get obsessed. Don’t do either. The kernel config is just a giant list of yes/no questions: “Do you need Bluetooth support?” “Do you need Apple ADB keyboard support?” “Do you need the Siemens R3964 line discipline?”

You answer maybe 200 of these. The kernel has thousands.

Start from a Minimal Base

# Start with the tinyconfig — bare minimum, almost nothing enabled
make tinyconfig

tinyconfig gives you a kernel that can boot and… that’s about it. No networking, no filesystem beyond the bare minimum, no device drivers worth mentioning. It compiles in under 2 minutes and produces a ~2MB compressed image. But it’s useless.

We need to add just enough to be practical.

What We Need

Here’s my checklist for a kernel that boots in a container, has networking, and can run systemd:

# Enable 64-bit support (if on x86_64)
scripts/config --enable CONFIG_64BIT

# Essential for booting
scripts/config --enable CONFIG_PRINTK
scripts/config --enable CONFIG_BUG
scripts/config --enable CONFIG_MULTIUSER
scripts/config --enable CONFIG_FUTEX

# Filesystems — we need at least ext4 and proc
scripts/config --enable CONFIG_EXT4_FS
scripts/config --enable CONFIG_PROC_FS
scripts/config --enable CONFIG_SYSFS
scripts/config --enable CONFIG_TMPFS
scripts/config --enable CONFIG_DEVTMPFS
scripts/config --enable CONFIG_DEVTMPFS_MOUNT

# Networking — systemd won't work without this
scripts/config --enable CONFIG_NET
scripts/config --enable CONFIG_INET
scripts/config --enable CONFIG_UNIX
scripts/config --enable CONFIG_NETDEVICES
scripts/config --enable CONFIG_NET_CORE
scripts/config --enable CONFIG_PACKET
scripts/config --enable CONFIG_UNIX_DIAG

# cgroups — systemd is built on these
scripts/config --enable CONFIG_CGROUPS
scripts/config --enable CONFIG_CGROUP_FREEZER
scripts/config --enable CONFIG_CGROUP_DEVICE
scripts/config --enable CONFIG_CGROUP_CPUACCT
scripts/config --enable CONFIG_CGROUP_PERF
scripts/config --enable CONFIG_CGROUP_BPF
scripts/config --enable CONFIG_MEMCG
scripts/config --enable CONFIG_BLK_CGROUP

# Namespaces — container/security isolation
scripts/config --enable CONFIG_NAMESPACES
scripts/config --enable CONFIG_USER_NS
scripts/config --enable CONFIG_PID_NS
scripts/config --enable CONFIG_NET_NS
scripts/config --enable CONFIG_UTS_NS
scripts/config --enable CONFIG_IPC_NS

# systemd needs these specific kernel features
scripts/config --enable CONFIG_FHANDLE
scripts/config --enable CONFIG_EPOLL
scripts/config --enable CONFIG_SIGNALFD
scripts/config --enable CONFIG_TIMERFD
scripts/config --enable CONFIG_EVENTFD
scripts/config --enable CONFIG_INOTIFY_USER
scripts/config --enable CONFIG_FANOTIFY
scripts/config --enable CONFIG_AUTOFS_FS
scripts/config --enable CONFIG_PROC_SYSCTL

# devtmpfs — automatic device node creation
scripts/config --enable CONFIG_DEVTMPFS
scripts/config --enable CONFIG_DEVTMPFS_MOUNT

# Kernel module support (so we can load .ko files later if needed)
scripts/config --enable CONFIG_MODULES
scripts/config --enable CONFIG_MODULE_UNLOAD

# Security features
scripts/config --enable CONFIG_SECCOMP
scripts/config --enable CONFIG_SECCOMP_FILTER

# tmpfs at /run and /tmp (systemd expects these)
scripts/config --enable CONFIG_TMPFS_POSIX_ACL

# Make sure we can boot in a container (no framebuffer, no keyboard, etc.)
scripts/config --disable CONFIG_VT
scripts/config --disable CONFIG_INPUT
scripts/config --disable CONFIG_SERIO
scripts/config --disable CONFIG_DRM
scripts/config --disable CONFIG_SOUND
scripts/config --disable CONFIG_USB
scripts/config --disable CONFIG_WIRELESS
scripts/config --disable CONFIG_WLAN
scripts/config --disable CONFIG_BT
scripts/config --disable CONFIG_WIRELESS_EXT
scripts/config --disable CONFIG_MAC80211

The “Why” Behind These Choices

I’m not listing these randomly. Every single enable here has a reason:

• cgroups and namespaces — systemd is fundamentally a cgroup manager. Without these kernel features, systemd cannot start. Period. If you ever wonder why systemd “requires” so much from the kernel, this is why.
• FHANDLE, EPOLL, SIGNALFD, TIMERFD, EVENTFD — these are Linux-specific system calls that systemd uses heavily for event loop management. They’re more efficient than traditional select()/poll() and systemd assumes they exist.
• DEVTMPFS + DEVTMPFS_MOUNT — without this, you’d need a udevd running to populate /dev. With it, the kernel auto-populates device nodes at boot. For a minimal system, this is essential.
• SECCOMP — lets systemd sandbox services. Even in a toy system, it’s good practice and costs almost nothing.

Everything I’m disabling (VT, INPUT, SOUND, USB, BLUETOOTH, WIRELESS) is stuff a container doesn’t need. Your kernel is running inside a container — the host kernel handles the real hardware. We just need the kernel to manage processes, memory, and networking.

The Interactive Way (Optional)

If you want to see what you’re enabling and tweak things visually:

make menuconfig

This opens an ncurses interface where you can browse categories, read help text for each option, and see what’s enabled. I recommend doing this at least once just to see how massive the kernel config really is. It’s humbling.

Step 3: Compile

# How many CPU cores do you have?
nproc

# Compile using all cores (replace 8 with your nproc output)
make -j8 2>&1 | tail -20

On a modern 8-core machine, this takes about 15-20 minutes. On a 2-core VM, plan for 45 minutes. Go make coffee. Actually, go make lunch.

When it finishes, check what you got:

# The raw kernel image
ls -lh vmlinux

# The compressed boot image (this is what we care about)
ls -lh arch/x86/boot/bzImage

My build produced:

vmlinux:    ~32MB (uncompressed, with debug symbols)
bzImage:   ~7.8MB (compressed, bootable)

Under 10MB. And that’s with networking, cgroups, namespaces, and all the systemd prerequisites. The average distribution kernel is 80-120MB. We just threw away 90%+ of the code and kept only what we need.

Stripping It Further

If you want to go even leaner:

# Strip debug symbols from the modules
find . -name '*.ko' -exec strip --strip-debug {} \;

# Or compile without debug info from the start
make clean
scripts/config --disable CONFIG_DEBUG_INFO
make -j8

Without debug symbols, my bzImage dropped to 6.2MB. That’s a full Linux kernel with networking and cgroup support in less space than a single high-res photo.

Step 4: Build the Root Filesystem with systemd

This is where Part 2 gets interesting. In Part 1, our init was a 20-line shell script. Now we’re replacing it with the same init system that runs on Fedora, Ubuntu, Arch, and Debian.

Why systemd Gets Hate (And Why It Doesn’t Deserve All Of It)

I know, I know. The “systemd controversy” is older than most junior sysadmins. But here’s the thing: systemd is just a process manager that understands cgroups, sockets, and dependencies. Yes, it does more than traditional init systems. But “more” isn’t automatically bad — it’s only bad when you don’t understand what it’s doing. Which is exactly why we’re building it from scratch.

When you’ve compiled a kernel, set up cgroups by hand, and watched systemd start on top of it, you’ll understand exactly what it does. No magic, no mystery.

Create the Root Filesystem

cd ~/kernel-build
mkdir -p rootfs

# Use debootstrap to get a minimal Debian root filesystem
sudo debootstrap --variant=minbase --arch=amd64 trixie rootfs http://deb.debian.org/debian

# Or if you don't want to use debootstrap, copy from Part 1 and add systemd manually:
# The debootstrap approach is faster and gives us a real package manager

Install systemd and OpenSSH

# Chroot into our new root filesystem
sudo chroot rootfs /bin/bash

# Inside the chroot:
apt-get update
apt-get install -y --no-install-recommends systemd systemd-sysv openssh-server

# Clean up apt cache to keep the image small
apt-get clean

# Set a root password (you'll need this for SSH)
echo "root:customlinux" | chpasswd

# Exit the chroot
exit

Create a systemd Service

Let’s create a simple service that proves systemd is actually managing things:

# Create a custom service that logs boot time
sudo tee rootfs/etc/systemd/system/boot-timer.service > /dev/null << 'EOF'
[Unit]
Description=Boot Timer Service
After=network.target

[Service]
Type=oneshot
ExecStart=/bin/bash -c 'echo "System booted at $(date)" > /var/log/boot-timer.log'
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
EOF

# Enable it
sudo chroot rootfs systemctl enable boot-timer.service

This service runs once at boot, writes the timestamp to a log file, and that’s it. Simple, verifiable, and it proves systemd is working.

Set Up the Root Filesystem Structure

# Make sure the necessary directories exist
sudo mkdir -p rootfs/{run,tmp,proc,sys,dev,sys/fs/cgroup}

# Create the machine-id (systemd requires this)
sudo systemd-id128 new > rootfs/etc/machine-id
sudo chmod 0444 rootfs/etc/machine-id

Step 5: Boot It with Your Custom Kernel

Here’s the moment of truth. We’re going to boot our custom-compiled kernel with a root filesystem that runs systemd:

cd ~/kernel-build

# Boot using systemd-nspawn (container runtime that uses your actual kernel)
sudo systemd-nspawn \
    --boot \
    --kernel=linux-6.12/arch/x86/boot/bzImage \
    --root=rootfs \
    --machine=custom-linux

If everything worked, you’ll see something like:

Spawning container custom-linux on /home/you/kernel-build/rootfs.
Press Ctrl-] three times within 1s to kill container.
[  OK  ] Started Journal Service.
[  OK  ] Started D-Bus System Message Bus.
[  OK  ] Reached target Network.
[  OK  ] Started Boot Timer Service.
[  OK  ] Reached target Multi-User System.

Debian GNU/Linux trixie/sid custom-linux tty1

custom-linux login:

That’s systemd running on your custom-compiled kernel. Log in with root / customlinux.

Verify Everything Works

Once logged in, run through these checks:

# Confirm it's our custom kernel
uname -a
# Should show 6.12.0-custom (or similar)

# Confirm systemd is PID 1
ps -p 1 -o comm=
# Should print: systemd

# Check that our custom service ran
cat /var/log/boot-timer.log
# Should show: System booted at <timestamp>

# Check cgroups are working
systemd-cgls
# Should show a tree of cgroups with your services

# Check systemd unit status
systemctl status boot-timer.service
# Should show: active (exited)

# Verify networking
ip addr show
# Should show lo (loopback) at minimum
ping -c 1 127.0.0.1

Test SSH

# Start the SSH daemon (inside the container)
systemctl start sshd

# Check it's running
systemctl status sshd

# From another terminal on your host, SSH in:
ssh root@$(sudo systemd-nspawn --machine=custom-linux --pipe /bin/hostname -I | awk '{print $1}')

You just SSH’d into a Linux system running a kernel you compiled yourself, managed by systemd, with a custom service you wrote. That’s not a toy — that’s the same architecture that runs production servers worldwide, just distilled down to its essentials.

Step 6: The Before/After Comparison

Remember the comparison table from Part 1? Here’s the updated version:

The kernel itself is dramatically smaller. The total filesystem is larger because Debian’s userspace is bigger than BusyBox — but you get a real package manager, real services, and real systemd. That’s the tradeoff.

Things That Will Go Wrong (And How to Fix Them)

Based on my own experience (and the mistakes I made building this the first time):

“Kernel panic: not syncing: VFS: unable to mount root fs” You forgot to enable CONFIG_EXT4_FS or your rootfs is corrupted. Check your filesystem config and make sure the rootfs directory is valid.

“Failed to connect to bus: No such file or directory” systemd can’t find its socket. Make sure /run/systemd exists and D-Bus is installed. Inside the chroot: apt-get install dbus.

“systemd[1]: Failed to mount cgroup” Missing cgroup kernel features. Go back to Step 2 and make sure all the CONFIG_CGROUP_* options are enabled. Also check that cgroup2 filesystem is supported.

SSH connection refused The SSH daemon isn’t running or isn’t installed. systemctl status sshd will tell you. Make sure you installed openssh-server in the chroot and that root login is permitted in /etc/ssh/sshd_config.

Kernel compiles but is 80MB+ You didn’t start from tinyconfig or you enabled too many drivers. Run make tinyconfig and re-enable only the options in Step 2. Use make menuconfig to check what’s enabled under “Device Drivers” — that’s usually where the bloat hides.

Why This Matters at 3 AM

You’re on call. A production server kernel panics. The error mentions cgroups and PID 1. In the old version of you, that’s a mystery — you reboot and hope. In the new version of you, you know:

• PID 1 is the init system — if it crashed, the whole system goes down. That’s by design.
• cgroups are how the kernel groups processes — if a cgroup is misconfigured, services can’t start.
• The kernel panics because something in userspace told it to do something impossible — the stack trace tells you exactly what.
• You can compile a debug kernel, boot it, and reproduce the issue because you’ve done it before.

That’s the difference between the sysadmin who reboots and the sysadmin who debugs. And it starts with understanding what’s actually under the hood.

Clean Up

When you’re done experimenting:

# Stop the container
sudo machinectl terminate custom-linux

# Remove the build artifacts (optional — they take ~10GB)
rm -rf ~/kernel-build/linux-6.12

Keep the rootfs directory if you want to boot it again later without recompiling.

Related reads:

• Building Your Own Linux from Scratch (Part 1)
• Your OS Has Been Hiding Things From You (Windows & Linux Edition)
• The Art of Reading Logs Like a Detective: Finding Needles in Haystacks
• Stop Doing Things Manually: 5 Scripts