| 26 min

Bringing in Rust: A no_std Allocator Under the Kernel

Carve the C heap allocator out of PurgatoryOS and replace it with a no_std Rust implementation wired in via extern C FFI. Set up aarch64-unknown-none-softfloat, build-std core + alloc, the GlobalAlloc trait, a spin-locked free list, and a Makefile that links libpurgatory_rs.a into the existing kernel.elf — without rewriting a single line of C above the allocator.

Diagram of a C kernel calling into a no_std Rust static library through the extern C FFI boundary to allocate memory

The last post ended with a kernel that has a real privilege boundary. Two tasks live in EL0, four syscalls give them supervised access to the UART, and eret is the only way back across. For the first time in this series, the word kernel has a stronger meaning because now we have code that other code is forbidden from touching.

That makes this the right moment for the detour I’ve been deferring since Post 8. The C allocator we wrote some posts ago is perfectly functional, but also genuinely unsafe. It trusts every caller to return pointers it was given, to free them exactly once, and never to keep a stale copy after a kfree. We enabled the hardware to prevent user tasks from corrupting the kernel, but nothing prevents the kernel from corrupting itself. A double-free in kernel/heap.c is undefined behaviour. The free-list chain quietly splits, two future kmalloc calls return the same address, and the bug surfaces three modules later with a stack trace that points nowhere useful. Anyone who has shipped a C codebase has lived this.

In this post, we rewrite the allocator in no_std Rust and call it from the existing C kernel via the extern "C" boundary. We do not rewrite the scheduler, the MMU, the UART driver, or the syscall dispatcher. Those stay in C. We replace only the kernel/heap.c with a Rust crate that exposes the same two symbols (kmalloc, kfree) and the same ABI, and we let the linker silently splice it in. By the end of the post, every kmalloc in the kernel (the scheduler’s PCB allocations, the per-task stacks we set up in Post 10, everything) is serviced by Rust code that the compiler has statically proven is free of double-frees, aliasing mistakes, and the two subtle bugs from Post 8’s What Broke section. The C above the boundary is completely unchanged.

This is the smallest possible beachhead, and that’s the point. It is much easier to bring Rust in one module at a time than to rewrite a kernel. It is also easier to reason about: we will keep unsafe confined to a handful of lines, with everything else in safe Rust, and the C side will have no idea anything changed.


What We’re Building Today

The visible behaviour of PurgatoryOS will stay the same after this post. It still boots, it still runs task_a and task_b in EL0, and the UART output is byte-for-byte identical to that of the previous post. What changes is what’s happening underneath. Five new files (seven, counting the Cargo.lock that cargo will produce for you) take over the heap:

  • rust/Cargo.toml: the crate manifest that declares the no_std static library target.
  • rust/.cargo/config.toml: the cargo config that pins the aarch64-unknown-none-softfloat target and enables -Z build-std=core,alloc.
  • rust/aarch64-unknown-none-softfloat.json: the cargo config that pins the aarch64-unknown-none-softfloat target and enables -Z build-std=core,alloc.
  • rust/src/lib.rs: the #![no_std] crate root with the panic handler and the #[global_allocator] wiring.
  • rust/src/allocator.rs: the port of the current C free-list algorithm, plus a spin-locked wrapper and the GlobalAlloc trait implementation.
  • rust/src/ffi.rs: the four-line extern "C" shim: kmalloc, kfree that C will actually link against.

And one change you would have made anyway:

  • Makefile: a new target that runs cargo build to produce libpurgatory_rs.a, a tiny rule that excludes kernel/heap.c from the build (the Rust library now owns that symbol), and one extra argument in the final $(LD) invocation.

None of these touches breaks anything in C. #include "kernel/heap.h" still works. Every existing kmalloc(size) / kfree(ptr) call site behaves the same. The scheduler, the syscall dispatcher, and task_create are all untouched. The physical address space is also unchanged except for a few kilobytes of .text moving into the main kernel block:

bios@confessions ~/memory-map · Physical address space after Post 11 — Rust .text shares Block 0 with the kernel
PL011 UART (MMIO) 0x0009000000 → 0x0009001000 (4 KB)
Block 0 · Kernel .text (C) 0x0040000000 → 0x004000C000 (48 KB)
Block 0 · Rust .text 0x004000C000 → 0x004000E000 (8 KB)
Block 0 · Kernel .rodata / .data / .bss 0x004000E000 → 0x0040028000 (104 KB)
Block 0 · Boot stack (16 KB) 0x0040028000 → 0x004002C000 (16 KB)
Block 1 · .text.user / rodata.user 0x0040200000 → 0x0040202000 (8 KB)
Block 1 · Heap (1 MB) 0x0040202000 → 0x0040302000 (1 MB)
PL011 UART (MMIO)
Block 0 · Kernel .text (C)
Block 0 · Rust .text
Block 0 · Kernel .rodata / .data / .bss
Block 0 · Boot stack (16 KB)
Block 1 · .text.user / rodata.user
Block 1 · Heap (1 MB)
unmapped

The heap bytes of the 1 MB pool bounded by the linker symbols __heap_start and __heap_end are in the exact same place. What changed is that the code servicing those bytes is now about 2 KB of Rust .text instead of 1.5 KB of C .text.

One call, from C into Rust and back

Here is a single kmalloc(64) call from the scheduler’s task_create, traced across the new FFI boundary. Step through it to see which side of the line owns the bytes at each instant. The ownership tag on the right is the whole point of the exercise.

bios@confessions ~/ffi-boundary
C · callerkernel.elf
running
call
Rust · calleelibpurgatory.a
unloaded
CC calls kmalloc(128)1 / 7

task_create in kernel/scheduler.c runs a bog-standard kmalloc(128) to allocate a PCB. No change from Post 10: the C compiler emits a bl kmalloc, with size in x0 per AAPCS64. Whatever lives at the symbol `kmalloc` will be called.

// kernel/scheduler.c — unchanged
pcb_t *pcb = kmalloc(sizeof(pcb_t));
values at the boundary
none·C caller
x0 (size)128
return slot<pending>
allocatornot yet entered
irq stateenabled

AAPCS64 puts the first u64 argument in x0. Nothing about the call site knows — or needs to know — that the callee is Rust.

Seven steps, one kmalloc. The only one of those steps that can be unsound is the last unsafe return, which spans three lines. Everything else is safe because the Rust compiler is checked.


Why Rust, Why Here

This is a fair question, because our C allocator works. It’s already tested and proven, so why go through the trouble of introducing a whole second language and a whole second build system for the sake of 180 lines of replacement code?

The answer is the thing no amount of C testing can give you. In a 2019 talk, Microsoft’s Security Response Centre reported that ~70% of all vulnerabilities Microsoft fixes over a given year are memory-safety bugs. In 2020, Chromium’s security team reported exactly the same ratio: 70% of serious Chrome bugs are memory-safety issues, with half being use-after-free. The US government’s CISA has since made the case formally, urging vendors to adopt memory-safe languages for new code. These numbers are not because C programmers are careless; they’re because C’s guarantees end at the type checker, and an allocator has no way to stop a caller from misusing its pointers.

Rust doesn’t eliminate this class of bug by being careful. It eliminates it by refusing to compile programs that can exhibit it. Box<T> calls dealloc exactly once, at Drop time, and moves when you pass it around, so the bytes have a single owner at every point in the program. &mut T cannot coexist with any other reference to the same T, so there’s no way to observe a freed block through a stale pointer. You cannot write a double-free in safe Rust; the compiler rejects it as a use-after-move.

None of that, crucially, costs anything at runtime. The machine code emitted by rustc for this allocator is roughly the same size and speed as gcc’s for the C version. The guarantees are compile-time; the binary is not.

This shift has been underway in systems programming generally. Rust has been accepted into the Linux kernel with a policy document “Rust kernel policy”, which treats it as a first-class source language, and Android 16 already ships a Rustimplementation of ashmem in production. Google, Microsoft, Meta, Amazon, and Cloudflare all have Rust in kernel-adjacent positions. This series is a toy OS, but the engineering bet is the same: pay the upfront cost of #![no_std], and get the long-tail debugging cost back, many times over.

The argument against, and it’s a fair one, is that it builds complexity. We are adding rustup, a nightly toolchain, cargo, and the -Z build-std flag to a project that previously needed only aarch64-elf-gcc and qemu-system-aarch64. The rest of this post is about keeping that cost small: one Cargo crate, one Makefile target, no changes to link.ld.


The Toolchain

We need to add two tools on top of what you already have:

# The nightly toolchain (we need it for -Z build-std).
rustup toolchain install nightly
rustup component add rust-src --toolchain nightly

# The bare-metal AArch64 target.  This is tier 3 in rustc's support
# matrix — it builds and runs, but rustup won't auto-install std for it,
# which is exactly what we want.
rustup target add aarch64-unknown-none-softfloat --toolchain nightly

First, aarch64-unknown-none-softfloat is as opposed to plain aarch64-unknown-none, which uses the soft-float calling convention. The rustc platform-support docs explain that the split exists precisely for kernels: we haven’t set up the FP registers in the EL1 boot path, and a hard-float ABI would try to pass some arguments in v0–v7, which haven’t been enabled. The soft-float variant keeps all arguments in the general-purpose registers x0–x7, which is exactly what AAPCS64 and our C code already assume. Using the wrong variant is one of those bugs that appears as a silent register clobber two hours into debugging.

Second, rust-src enables -Z build-std. Since aarch64-unknown-none-softfloat is tier 3, there is no pre-built libcore or liballoc on rustup’s servers. -Z build-std takes the standard-library source from rust-src and builds core and alloc alongside your crate, with the same codegen options. The result is a .a that’s been compiled with -mcpu=cortex-a53, the same flag we’ve been using for C the whole series.

No PATH surgery, no cross-compiler builds. If rustup --version and cargo +nightly --version work, you are done.


The Crate

The Rust side of the kernel lives in a single directory at the repo root:

purgatory/
├── Makefile
├── link.ld
├── arch/
├── drivers/
├── kernel/          # all the C modules from Posts 1–10
├── include/
├── user/
└── rust/            # ← new
    ├── Cargo.toml
    ├── .cargo/
    │   └── config.toml
    └── src/
        ├── lib.rs
        ├── allocator.rs
        └── ffi.rs

rust/Cargo.toml declares a staticlib crate, because that’s the form the C linker can consume:

# rust/Cargo.toml

[package]
name    = "purgatory_rs"
version = "0.1.0"
edition = "2021"

[lib]
name       = "purgatory_rs"
crate-type = ["staticlib"]   # produce libpurgatory_rs.a (not .rlib)

[dependencies]
spin = { version = "0.9", default-features = false, features = ["mutex", "spin_mutex"] }  # no_std spin-lock

[profile.release]
panic        = "abort"   # no unwinding — we're in a kernel
opt-level    = 2         # match the C CFLAGS
lto          = true      # allow inlining across our own modules
codegen-units = 1        # also for LTO
overflow-checks = true   # keep them on — we'd rather panic than wrap

The only external dependency is spin, whose Mutex is a no-alloc, no_std-safe spinlock. We turn off the defaults to skip the RwLock, barriers, and futures we don’t need, and turn on just the two features that actually carry Mutex into our .a, mutex (the generic façade) and spin_mutex (the busy-loop implementation). Forgetting the second one is the typical first-build failure: cargo build succeeds, but spin::Mutex isn’t in scope, and the compiler complains that Mutex::<FreeList>::new() is not a function.

The .cargo/config.toml is the knob that pins the target and turns on build-std, so we don’t have to type long flags every build:

# rust/.cargo/config.toml

[build]
target = "aarch64-unknown-none-softfloat"

[unstable]
build-std          = ["core", "alloc"]
build-std-features = ["compiler-builtins-mem"]  # memcpy/memset symbols

[target.aarch64-unknown-none-softfloat]
rustflags = [
    "-C", "target-cpu=cortex-a53",
    "-C", "panic=abort",
    "-C", "force-unwind-tables=no",
]

The compiler-builtins-mem feature is important and easy to miss. On -unknown-none targets, rustc doesn’t assume memcpy, memset, memmove, or memcmp exist; the C library normally provides them. Turning the feature on tells compiler-builtins to ship its own, target-compiled versions inside libpurgatory_rs.a. Without it, the linker fails with an undefined reference to memcpy as soon as any Rust struct larger than 16 bytes moves.


The Crate Root: lib.rs

The root is short because it does nothing but: declare no_std, pull in core and alloc, define the panic handler that no_std requires, and wire up the global allocator.

// rust/src/lib.rs

#![no_std]
#![feature(alloc_error_handler)]

extern crate alloc;   // needed for Box/Vec/etc. inside Rust

mod allocator;
mod ffi;

use core::panic::PanicInfo;

/// # Safety
/// Called by rustc when anything in this crate panics. A kernel has nowhere
/// to unwind to, so we halt the CPU. The `wfi` keeps the core idle rather
/// than busy-looping — easier on power, friendlier on emulators.
#[panic_handler]
fn panic(info: &PanicInfo) -> ! {
    // UART access through a raw volatile write — we deliberately do NOT call
    // back into C here. If kprint panicked inside us, recursive panics would
    // double-fault. Writing straight to the PL011 data register is the only
    // thing we trust from inside a panic path.
    let uart = 0x0900_0000 as *mut u32;
    for b in b"\n[rust panic] ".iter() {
        unsafe { core::ptr::write_volatile(uart, *b as u32); }
    }
    if let Some(loc) = info.location() {
        // File name and line number only — formatting a PanicInfo allocates,
        // which we obviously can't do while holding the allocator's lock.
        for b in loc.file().as_bytes().iter().take(64) {
            unsafe { core::ptr::write_volatile(uart, *b as u32); }
        }
    }
    loop {
        unsafe { core::arch::asm!("wfi"); }
    }
}

/// Called if the global allocator returns null.  Our `kmalloc` never
/// returns null from inside Rust (it returns to C, which checks), but the
/// `alloc` crate's Box/Vec code paths need this symbol to exist.
#[alloc_error_handler]
fn oom(_: core::alloc::Layout) -> ! {
    panic!("kernel heap exhausted");
}

Two things to notice:

  • The panic handler does not call kprint because that would cross back into C, and if C panics were somehow possible, they’d recurse. We write bytes directly to the PL011 MMIO address we already know by heart from Post 4. And we use wfi to halt, not a busy loop; the CPU goes into the low-power state, the timer-interrupt post already set up.
  • The extern crate alloc line lets us use Box, Vec, BTreeMap, and String on the Rust side of the kernel. We won’t do that in this post, but the moment we register the #[global_allocator], we get all of the standard collections for free inside Rust, without a single additional line of code. The Embedded Rust Book’s no_std chapter walks through exactly this split between core (always available) and alloc (available once you supply a global allocator).

The Same Algorithm, Safer: allocator.rs

The allocator itself is a direct port of the current free-list allocator. Same first-fit search, same split threshold and same forward coalescing. The difference is the code’s structure, which is worth pausing on before the listing.

In C, every block header was a raw block_header_t * that we dereferenced unconditionally. A bug in split or coalesce could write a nonsense next pointer into a header, and the next walk would crash somewhere far away. In Rust, the same walk is a safe iterator over &mut Block references; the unsafety is confined to two lines at construction time.

// rust/src/allocator.rs

use core::alloc::{GlobalAlloc, Layout};
use core::ptr::{self, NonNull};
use spin::Mutex;

/// 8-byte alignment — AArch64's minimum for 64-bit loads/stores.
/// Identical to Post 8's HEAP_ALIGN.
const HEAP_ALIGN: usize = 8;

/// The on-heap block header.  Layout matches kernel/heap.c's
/// block_header_t byte-for-byte so a pointer we hand to C and later
/// receive back from C lands on the right type.
///
/// Rust field ordering is deterministic with `#[repr(C)]`, so this is
/// a safe one-for-one mapping.
#[repr(C)]
struct Block {
    size:    usize,             // payload bytes, not counting this header
    is_free: u32,               // 1 = free, 0 = allocated (u32 for C compat)
    _pad:    u32,               // keep AArch64 alignment
    next:    Option<NonNull<Block>>,
}

impl Block {
    /// The 24-byte rounded header size — same as HEADER_SIZE in heap.h.
    const HDR: usize = {
        let raw = core::mem::size_of::<Block>();
        (raw + HEAP_ALIGN - 1) & !(HEAP_ALIGN - 1)
    };

    /// Address of the payload the caller will see.
    fn payload_ptr(&mut self) -> *mut u8 {
        unsafe {
            (self as *mut Self as *mut u8).add(Self::HDR)
        }
    }
}

/// The allocator's internal state — a singly-linked list of blocks.
/// The entire chain lives *inside* the heap itself; we only keep one
/// pointer in .bss.
struct FreeList {
    head: Option<NonNull<Block>>,
}

// SAFETY: FreeList is only ever accessed inside a spin::Mutex, which
// serialises access. The raw pointers it holds never escape a
// MutexGuard's scope to another thread.
unsafe impl Send for FreeList {}

impl FreeList {
    const fn empty() -> Self {
        FreeList { head: None }
    }

    /// Called once at boot to lay the initial "one giant free block"
    /// across [start, end).  Unsafe because we're promising the range
    /// is actually usable memory.
    unsafe fn init(&mut self, start: *mut u8, end: *mut u8) {
        let total = end as usize - start as usize;
        assert!(total > Block::HDR + HEAP_ALIGN, "heap region too small");

        let block = start as *mut Block;
        (*block).size    = total - Block::HDR;
        (*block).is_free = 1;
        (*block)._pad    = 0;
        (*block).next    = None;

        self.head = NonNull::new(block);
    }

    /// First-fit allocation.  Returns a payload pointer on success,
    /// null if no suitable block is found.
    fn alloc_raw(&mut self, size: usize) -> *mut u8 {
        if size == 0 { return ptr::null_mut(); }
        let size = (size + HEAP_ALIGN - 1) & !(HEAP_ALIGN - 1);

        let mut cur = self.head;
        while let Some(mut node) = cur {
            // SAFETY: nodes are constructed at init() and split() and
            // never dangle while held in this list.  We're the sole
            // writer because of the enclosing Mutex.
            let block = unsafe { node.as_mut() };

            if block.is_free == 1 && block.size >= size {
                let remainder = block.size - size;

                // Split only if the remainder can hold a full header
                // plus HEAP_ALIGN bytes of payload.  Identical to the
                // `>` guard from Post 8's C version.
                if remainder > Block::HDR + HEAP_ALIGN {
                    // SAFETY: split_addr is inside the same 1 MB heap
                    // region we were handed at init().
                    let split_addr = unsafe {
                        block.payload_ptr().add(size)
                    } as *mut Block;

                    unsafe {
                        (*split_addr).size    = remainder - Block::HDR;
                        (*split_addr).is_free = 1;
                        (*split_addr)._pad    = 0;
                        (*split_addr).next    = block.next;
                    }

                    block.size = size;
                    block.next = NonNull::new(split_addr);
                }

                block.is_free = 0;
                return block.payload_ptr();
            }

            cur = block.next;
        }

        ptr::null_mut()
    }

    /// Mark a previously-allocated pointer as free, and coalesce
    /// forward if the next block is also free.  UB if ptr was not
    /// previously returned by alloc_raw.
    unsafe fn free_raw(&mut self, ptr: *mut u8) {
        if ptr.is_null() { return; }

        let block = (ptr as *mut u8).sub(Block::HDR) as *mut Block;
        (*block).is_free = 1;

        if let Some(mut next) = (*block).next {
            let nref = next.as_mut();
            if nref.is_free == 1 {
                (*block).size += Block::HDR + nref.size;
                (*block).next  = nref.next;
            }
        }
    }
}

// ── The global allocator singleton ─────────────────────────────────

pub struct Heap {
    inner: Mutex<FreeList>,
}

impl Heap {
    pub const fn new() -> Self {
        Heap { inner: Mutex::new(FreeList::empty()) }
    }

    /// Initialise from the linker-exported heap range.  Called once
    /// from the kernel, on the boot path, before any allocation.
    pub unsafe fn init_from_linker(&self) {
        extern "C" {
            static __heap_start: u8;
            static __heap_end:   u8;
        }
        let start = &__heap_start as *const u8 as *mut u8;
        let end   = &__heap_end   as *const u8 as *mut u8;
        self.inner.lock().init(start, end);
    }

    pub fn alloc_raw(&self, size: usize) -> *mut u8 {
        self.inner.lock().alloc_raw(size)
    }

    pub unsafe fn free_raw(&self, ptr: *mut u8) {
        self.inner.lock().free_raw(ptr)
    }
}

// ── GlobalAlloc impl (so Box/Vec work inside Rust) ─────────────────

unsafe impl GlobalAlloc for Heap {
    unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
        // Our free-list only handles HEAP_ALIGN alignment; anything
        // stricter is a caller error inside Rust.  Callers from C go
        // through kmalloc, which never asks for more than 8-byte.
        debug_assert!(layout.align() <= HEAP_ALIGN);
        self.alloc_raw(layout.size())
    }

    unsafe fn dealloc(&self, ptr: *mut u8, _layout: Layout) {
        self.free_raw(ptr);
    }
}

#[global_allocator]
pub static ALLOCATOR: Heap = Heap::new();

Five things are worth highlighting against the Post 8 C version:

  1. #[repr(C)] on Block is what makes the layout identical byte-for-byte to block_header_t in include/kernel/heap.h. If you ever want to have both the C and the Rust allocator coexist at runtime (or debug a half-freed pool from GDB), the layouts match.
  2. Option<NonNull<Block>> replaces the raw struct block_header *next of the C version. NonNull says “never null”; Option says “may be absent”. Together they occupy exactly 8 bytes (Rust’s null-pointer optimisation), so the header stays the same size as before.
  3. The unsafe keyword is a flag, not a scope. Every unsafe fn and unsafe { } block in the code above is a pin on the map: this is where memory safety depends on the programmer, not the compiler. We have six of them. They are audit targets. In the C version, every function in the file was an audit target.
  4. The split guard (remainder > Block::HDR + HEAP_ALIGN) is the same expression as the C version’s. If you missed Post 8’s What Broke section, this is the line that prevents unsigned underflow when the remainder is too small to hold a header. Rust would have caught the underflow at runtime via overflow-checks = true, but the guard is still the right fix.
  5. spin::Mutex is both a lock and a type gate. The .lock() call returns a MutexGuard<'_, FreeList> that dereferences to &mut FreeList. The borrow checker will refuse any code path that re-enters .lock() while the guard is live, which would lead to a runtime deadlock. That’s a bug category that the C version cannot make structurally impossible.

One footnote on the lock choice. Because we’re single-core and because IrqGuard masks interrupts around every FFI entry, the spin loop in spin::Mutex can never actually spin. We could replace it with a RefCell and save a handful of instructions, but Send/ Sync reasoning gets messier. The trade-off favours the mutex for clarity.


The Boundary: ffi.rs

This is the whole of the C-visible surface: three functions, forty lines. It is deliberately the smallest thing that works.

bios@confessions ~/rust/src/ffi.rs
use crate::allocator::ALLOCATOR;
use core::arch::asm;
 
/// RAII: mask IRQs on construction, restore on Drop.
struct IrqGuard { prev: u64 }
 
impl IrqGuard {
  fn new() -> Self {
      let prev: u64;
      unsafe {
          asm!("mrs {0}, daif", out(reg) prev);
          asm!("msr daifset, #2");  // mask IRQ
      }
      Self { prev }
  }
}
 
impl Drop for IrqGuard {
  fn drop(&mut self) {
      unsafe {
          asm!("msr daif, {0}", in(reg) self.prev);
      }
  }
}
 
/// C entry point: initialise the heap from linker symbols.
#[unsafe(no_mangle)]
pub extern "C" fn rust_heap_init() {
  unsafe { ALLOCATOR.init_from_linker(); }
}
 
/// C entry point: allocate.  Takes over from kernel/heap.c's kmalloc.
#[unsafe(no_mangle)]
pub extern "C" fn kmalloc(size: usize) -> *mut u8 {
  let _g = IrqGuard::new();
  ALLOCATOR.alloc_raw(size)
}
 
/// C entry point: free.
#[unsafe(no_mangle)]
pub extern "C" fn kfree(ptr: *mut u8) {
  let _g = IrqGuard::new();
  unsafe { ALLOCATOR.free_raw(ptr); }
}

The entire C-facing surface consists of three extern "C" functions. Everything behind them is pure, safe Rust. If a later version of the allocator rewrites the internals (slab, buddy, bitmap), the FFI shim does not change.


Wiring It into the Build

The Makefile change is three stanzas. One to build the Rust lib, one to skip kernel/heap.c (the Rust library now owns that symbol), and one to tell the final link to include the Rust .a.

# ── Makefile (additions only) ───────────────────────────────────────────────

RUST_DIR    := rust
RUST_TARGET := aarch64-unknown-none-softfloat
RUST_LIB    := $(RUST_DIR)/target/$(RUST_TARGET)/release/libpurgatory_rs.a

# 1.  Tell cargo to rebuild the staticlib whenever any .rs or Cargo.toml
#     changes.  `cargo build` is fast enough to run unconditionally, but we
#     still list prereqs so `make` understands the dep chain.
$(RUST_LIB): $(shell find $(RUST_DIR)/src -name '*.rs' 2>/dev/null) \
             $(RUST_DIR)/Cargo.toml \
             $(RUST_DIR)/.cargo/config.toml
	cd $(RUST_DIR) && cargo +nightly build --release

# 2.  Drop kernel/heap.c out of the C object list — Rust now owns kmalloc/kfree.
C_OBJS := $(filter-out build/kernel/heap.o, $(C_OBJS))

# 3.  Final link: keep everything you had, append the Rust archive.
$(TARGET): $(OBJS) $(RUST_LIB) link.ld
	$(LD) $(LDFLAGS) -o $@ $(OBJS) $(RUST_LIB)

link.ld does not need to change. The Rust .text, .rodata, .data, and .bss go into the generic wildcards in the kernel sections (*(.text .text.*) etc.) alongside everything else that’s not explicitly .text.user or .rodata.user. The allocator’s static state is zero-initialised at construction (Heap::new() is a const fn), so it ends up in .bss and gets the same bss_init clear the rest of the kernel enjoys.

The only extra call we need from kernel_main is one line, to give the Rust side a chance to read __heap_start and __heap_end:

/* kernel/main.c — one new line */

extern void rust_heap_init(void);   /* provided by libpurgatory_rs.a */

void kernel_main(void) {
    /* ... */
    rust_heap_init();                /* was: heap_init();  */
    kprint("Heap: ready\n");
    /* ... */
}

heap_init from kernel/heap.c no longer exists in the image, so we rename the call site to rust_heap_init. Everything else stays kmalloc, kfree, their prototypes in include/kernel/heap.h, the scheduler’s use of them. The C code cannot tell the difference, and neither can the user staring at the UART. The boot banner is byte-identical to that of the latest post.


Register Snapshot at the Boundary

Here is the CPU state immediately before and after the first C-to-Rust kmalloc(128), taken from a GDB session on QEMU’s -gdb tcp::1234. The scheduler is doing its very first task_create, so x0 holds 128, and the return address is the next C instruction after bl kmalloc in task_create:

bios@confessions ~/registers · Before bl kmalloc — C side, about to cross into Rust
x0 0x0000000000000080 First argument per AAPCS64 — size_t size = 128. Identical whether the callee is C or Rust.
lr 0x00000000400043A4 Return address: the instruction after `bl kmalloc` inside task_create. Rust will ret into this.
sp 0x000000004002BFE0 The boot stack set up in boot.S. Rust uses this same stack — it does not have its own.
pc 0x000000004000C1B0 The `bl kmalloc` itself. After the branch, pc will become the address of the Rust-emitted kmalloc symbol.
daif 0x0000000000000000 All interrupts enabled. The timer could fire at any instruction — this is precisely why Rust masks it on entry.
pstate.el 0x0000000000000001 EL1. Rust also runs at EL1 alongside the C kernel — no privilege change across the FFI.

The hand-off is invisible to the ISA. extern "C" on the Rust side means the only difference at the instruction level is which .text pages the PC jumps into.

bios@confessions ~/registers · Inside Rust, just after IrqGuard::new()
x0 0x0000000000000080 Still 128 — Rust received it as `size: usize`. No marshalling.
sp 0x000000004002BFB0 Rust grew the stack by 48 bytes for its local variables (the IrqGuard, the MutexGuard, scratch). Same stack as C.
pc 0x000000004000C2E8 Inside Rust's kmalloc, a few instructions past the function prologue.
daif 0x00000000000000C0 IRQ (I) and FIQ (F) bits set. IrqGuard masked both; timer interrupts will wait until the guard drops.
pstate.el 0x0000000000000001 Still EL1. Rust is the kernel now — same privilege, same address space, same page table.

DAIF flipping from 0x00 to 0xC0 is the only visible effect of the whole FFI crossing — and it would be undone the moment `_g` goes out of scope.


Booting It

The first time you build it, it also needs to create the cargo build, and that takes ~40 seconds to compile core and alloc. All the subsequent builds are a few hundred milliseconds:

That is deliberately identical, character-for-character, to the Post 10 boot. Nothing on the UART tells you the allocator has changed. The only way to confirm at runtime that the Rust allocator is servicing kmalloc is to run aarch64-elf-nm on the running kernel.elf | grep kmalloc shows the symbol resolved against the Rust .a, and objdump -d kernel.elf --disassemble=kmalloc shows the msr daifset, #2 that only IrqGuard::new() emits.

Behind that silence, the scheduler’s task_create runs three kmalloc calls per task: the PCB, the kernel stack, and the user stack. Six in total for task_a plus task_b, all serviced by Rust, with nobody above the allocator aware that anything has changed.


What Broke (And Why)

The longest-lasting failure mode had nothing to do with Rust or FFI. After cargo build succeeded and aarch64-elf-ld succeeded, the kernel booted as far as rust_heap_init, printed two characters, and froze. QEMU’s -d int,exec log showed no exceptions, but the CPU was just running instructions in a tight loop, somewhere.

The cause was the compiler-builtins-mem feature I hadn’t initially enabled. Rust’s Block::split emitted a memcpy for copying the header fields to the new s plit block. Our linker had no memcpy symbol, and because the libpurgatory_rs.a was built with lazy relocations resolved at static-link time, so the “undefined reference” slipped through. aarch64-elf-ld substituted address 0x0 for the missing symbol and reported nothing because of the allocator .a contained a memcpy weak-symbol from a transitive dep, just for a different architecture.

At runtime, the first kmalloc that hit a split path branched to 0x0, which, on virt is unmapped, and took a data-abort loop. The vector handler happily re-triggered the same data abort on the first ldr in the handler, faulting infinitely at EL1 with no progress to the UART.

I had to enable compiler-builtins-mem in .cargo/config.toml:

build-std-features = ["compiler-builtins-mem"]

linked in the right memcpy/memset/memmove/memcmp, the fault went away, and the boot reached the first task. The lesson for anyone porting this: if your Rust .a links cleanly but the kernel hangs inside the first Rust call, objdump -d the final ELF and grep for bl 0x0. That’s the shape of a missing intrinsic. The compiler_builtins crate is what your C toolchain’s libgcc and compiler-rt would provide; on an -unknown-none target, you have to opt into it explicitly.


What’s Next

The kernel now has a safety island under it. Every object the kernel allocates at runtime flows through code the compiler has statically proven cannot corrupt the free list. The C that sits on top of the island didn’t move an inch.

Two things this enables. First, we can keep migrating small modules one at a time without disturbing the working kernel. The syscall dispatcher is a good next candidate, and a Rust version can add validation of the access_ok pointer. When we build the shell in the future, that would be another natural target: parsing a command line in no_std Rust is much friendlier than parsing one in C.

Second, we now have alloc, Box, Vec, BTreeMap and String available inside any Rust module we write from here on, at zero extra cost. The next posts in-memory filesystem will take advantage of exactly that.

The next post is the filesystem. A ramdisk, some very simple FAT-style directory structures, and the first programs we can actually store and load instead of linking into the kernel image. The hardest decision of that post will be: filesystem in C, to match the rest of the kernel, or in Rust, to take advantage of the alloc crate’s data structures now that we have them. I won’t spoil which I picked. But the option exists because of this post.


Sources