FreeBSD's Loader

What the Userland Built

Linux: LILO (1992 to 2015)

Linux: GRUB (2005 to present)

The Point The Unix Way — Episode 14 Between the firmware that knows almost nothing and the kernel that must know everything, there is a small program with a rather strange job. The bootloader is the one piece of software whose design quietly encodes what the operating system above believes itself to be. Three answers to the same question. Each tells a different story. Three stages. boot0 is a 446-byte master boot record. boot1 reads the partition table and locates loader. loader(8) is a Forth interpreter, around 600 KB on amd64. Forth, because in 1992 someone decided the bootloader should be programmable without becoming an operating system. Forth gives you a small, deterministic, stack-based language with no runtime assumptions about memory layout or operating system services. It is enough to express "look at this dataset, decide which kernel to load, present a menu, hand off." The loader understands UFS and ZFS natively. It can mount a ZFS dataset as root. It knows what a Boot Environment is. It presents the list of available Boot Environments as a menu before the kernel starts. None of this is bolted on. It is the design. The pre-install bootloader for ZFS-on-root systems writes the loader into a small partition outside the ZFS pool, but the loader itself reads the pool metadata, walks the dataset hierarchy, and offers the user the choice of which root dataset to boot. This is the enabling primitive on which the entire FreeBSD Boot Environment workflow stands. The loader exposed Boot Environments as a first-class concept. The userland question was: how does an admin create, list, activate, destroy, and switch between them? The first answer was a shell script called manageBE, written in the early days of ZFS-on-FreeBSD. Functional, but not pleasant to use. In 2012, a Polish FreeBSD admin called vermaden wrote beadm(8) in POSIX sh(1) and awk(1), deliberately mimicking the Solaris and Illumos beadm interface so that anyone coming from those systems would feel at home. The original announcement and discussion lives on the FreeBSD Forums in the thread "HOWTO ZFS Madness" (number 31662, still readable today). The script was, and remains, around fifteen hundred lines of POSIX shell and awk: small, auditable, dependency-free, runnable on any FreeBSD system since 9.x. For six years, vermaden's beadm was the working standard for managing Boot Environments on FreeBSD. It was packaged in ports, recommended by the documentation, and built into countless admin workflows. In 2018, FreeBSD 12.0 shipped bectl(8), a reimplementation in C that landed in base. The motivation was straightforward: a tool used as universally as beadm had become deserved to be in base, with all the testing and consistency guarantees that base implies. The C rewrite gave it tighter integration with libbe, the FreeBSD library that already encoded the Boot Environment data model, and direct ZFS access without the cost of subprocess fan-out. The transition was not entirely smooth. Early bectl versions had bugs that beadm did not, in part because beadm had been shaken out by six years of production use across hundreds of admins. There are documented cases from the 12.x era in which running beadm against a bectl-confused dataset state cleanly resolved the inconsistency. The bugs have largely been fixed, and bectl is now the recommended tool. beadm continues to be developed and continues to ship features that bectl has not adopted. The most interesting is the REROOT option: after creating a Boot Environment, upgrading the running system, and discovering that something is not right, REROOT swaps userspace into the pre-upgrade Boot Environment without a full reboot. The kernel stays up; the userspace is reloaded from the chosen dataset. The whole operation takes seconds rather than the half-minute of a reboot. It is the kind of feature that exists because someone was solving an actual operational problem and writing the code that fixed it, rather than waiting for the perfect abstraction. That trajectory, from manageBE to beadm to bectl, is a model of how Unix tools mature. A clever script meets an unfilled need. A wider community adopts it and shapes it. A reimplementation in base preserves what worked and fixes what did not, while the original keeps innovating in places where base cannot move as quickly. Both tools coexist, and the choice between them is technical rather than political. Werner Almesberger wrote LILO at ETH Zürich in 1992 and maintained it until 1998. John Coffman took over until 2007, when development passed to Joachim Wiedorn. Active development ended in December 2015 with version 24.2, and was never resumed. The Linux Loader's design choice was a blocklist. The position of the kernel image on disk was recorded as a list of physical sectors at install time. Move the kernel, the bootloader points at gibberish. Update the kernel, run /sbin/lilo before rebooting or the machine refuses to come back. This was reasonable in 1992. Disks were small, kernels were rebuilt rarely, and the BIOS understood little beyond INT 13h. The assumption was a slow, static world. By 2010 the assumption had become a trap. GPT and UEFI made blocklists structurally awkward. RAID and LVM moved blocks behind the back of the bootloader. The Linux kernel started shipping new versions every couple of months. The ritual that LILO required after each change had become the leading cause of unbootable systems for users who forgot it. The other extreme. GRUB began in 1995 as a research bootloader by Erich Boleyn, was adopted by the GNU project in 1999, and was rewritten from scratch as GRUB 2 in 2005. The result is a small operating system that runs before the operating system. GRUB 2 ships filesystem drivers for ext2, ext3, ext4, btrfs, XFS, F2FS, JFS, ReiserFS, FAT, NTFS, ISO9660, AFFS, HFS, UDF, ZFS (read-only, partial), and several others. It can decompress kernels in zlib, lzma, lz4, and zstd. It runs scripts in its own POSIX-shell-flavoured language. It supports themes, fonts, gfxmode, network boot, UEFI Secure Boot, multiboot, and chainloading. Its core is around 200 KB; its loadable modules add several megabytes. The assumption is a fragmented world. Linux distributions disagree about filesystem choice, kernel layout, root-on-everything, and boot configuration. GRUB has to understand every variant because the world above it does not converge. The cost of that assumption is everything complexity costs. The benefit is that GRUB will boot almost anything you put in front of it. The omission is interesting. GRUB has read-only ZFS support, sufficient to load a kernel from a ZFS dataset. It does not have Boot Environment awareness, because the conversation about ZFS-as-an-OS-management-layer never happened on Linux. ZFSBootMenu exists precisely to fill that gap, by replacing GRUB entirely with a small kexec-loaded Linux kernel whose only job is to present a Boot Environment menu and hand off to the chosen one. The question is not which bootloader is best. It is what the bootloader assumes about the system above it. FreeBSD's loader assumes the system above it is coherent enough to be worth talking to: that the bootloader, the filesystem, the kernel, and the userland are designed by people who can sit in the same room. From that assumption, things like Boot Environment selection at the loader become possible, and tools like beadm and bectl become natural. LILO assumed a slow, static world in which the admin would re-run the installer after each change. The world was no longer that world by 2005, and the bootloader could not move with it. GRUB assumes a fragmented world in which the bootloader must understand every variant of every filesystem because the distributions above it do not converge. The cost is permanent complexity, the benefit is that GRUB boots anything. Three loaders. Three theories of the operating system. Each design is correct for the world it assumes. The interesting work is choosing which world to live in. Read the full article on vivianvoss.net → By Vivian Voss — System Architect & Software Developer. Follow me on LinkedIn for daily technical writing. Templates let you quickly answer FAQs or store snippets for re-use. as well , this person and/or