Skip to content
Fangrui Song edited this page Nov 5, 2022 · 5 revisions

Clone the mainline kernel: git clone https://github.com/torvalds/linux.git --single-branch (usually faster than https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git)

Install packages needed for building a kernel.

sudo apt install libncurses-dev flex bison openssl libssl-dev \
                 dkms libelf-dev libudev-dev libpci-dev       \
                 libiberty-dev autoconf
make -j$(nproc) O=/tmp/linux/x86_64 defconfig
#scripts/config --file /tmp/linux/x86_64/.config -e KVM
scripts/config --file /tmp/linux/x86_64/.config -e XFS_FS -e DEBUG_INFO -e DEBUG_INFO_DWARF5 -e GDB_SCRIPTS
scripts/config --file /tmp/linux/x86_64/.config -e CRYPTO_LZ4 -e CRYPTO_LZO -e ZRAM

make -j$(nproc) O=/tmp/linux/x86_64 bzImage modules
scripts/clang-tools/gen_compile_commands.py -d /tmp/linux/x86_64  # For ccls
sudo sshfs -p 2222 -o allow_other root@localhost:/ /mnt/fedora   # Add `PermitRootLogin yes` to guest /etc/ssh/sshd_config
make -j$(nproc) O=/tmp/linux/x86_64 INSTALL_PATH=/mnt/fedora/boot install
make -j$(nproc) O=/tmp/linux/x86_64 INSTALL_MOD_PATH=/mnt/fedora modules_install

In the guest, create a new entry under /boot/loader/entries/ based on an existing one. Reuse initramfs but update vmlinuz.

vim /etc/default/grub  # Add nokaslr to GRUB_CMDLINE_LINUX
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
sudo grubby --set-default /boot/vmlinuz-6.*  # Select the new vmlinuz

(grubby doc) (If the guest is Debian, do sudo update-initramfs -k all -c; sudo update-grub)

Alternatively, append -kernel /tmp/linux/x86_64/arch/x86/boot/bzImage -initrd path/to/initramfs -append 'root=/dev/mapper/fedora_ju1f2mb4--trk--14--1-root ro rd.lvm.lv=fedora_ju1f2mb4-trk-14-1/root rhgb quiet nokaslr' to the qemu command line. Append -nographic -append 'tty=consoleS0 ... to avoid video output (a SDL window). (The virtual serial port and the QEMU monitor are multiplexed onto stdio.)

qemu-system-x86_64 -enable-kvm -nographic -m 16384 -smp 8 -cpu host -kernel /tmp/linux/x86_64/arch/x86/boot/bzImage -initrd /tmp/c/initramfs-5.19.7-300.fc37.x86_64.img -append 'console=ttyS0 root=/dev/mapper/fedora_ju1f2mb4--trk--14--1-root ro rd.lvm.lv=fedora_ju1f2mb4-trk-14-1/root rhgb quiet nokaslr' -drive file=fedora.img,if=virtio -net nic,model=virtio -nic user,hostfwd=tcp::2222-:22

Rerun qemu-system with -s -S. cgdb /tmp/linux/x86_64/vmlinuz; target remote :1234; hb start_kernel; c

Fedora uses /dev/zram0 as a swap device by default (https://fedoraproject.org/wiki/Changes/SwapOnZRAM). See man zranctl for other distributions. Run cat /proc/swaps (or swapon --show) to list swap devices.

LLVM

To build the kernel with Clang, lld, and LLVM binary utilities, build the tools in a llvm-project build directory.

ninja -C /tmp/RelA clang lld llvm-{ar,nm,objcopy,objdump,readelf,strings}

PATH=/tmp/RelA/bin:$PATH make -j$(nproc) LLVM=1 O=/tmp/linux/x86_64

initramfs

Use dracut to build an initramfs.

Use lsinitramfs path/to/initramfs (from initramfs-tools) to list files in an initramfs.

tools/perf

make -j$(nproc) O=/tmp/linux/perf ARCH=x86_64 -C tools/perf

Note: don't use a kernel build directory.

kbuild performance

https://lkml.org/lkml/2018/5/2/74 from Ingo Molnar:

First I make sure that cpufreq is set to 'performance':

for ((cpu=0; cpu<120; cpu++)); do
  G=/sys/devices/system/cpu/cpu$cpu/cpufreq/scaling_governor
  [ -f $G ] && echo performance > $G
done

Then I copy a kernel tree to /tmp (ramfs) as root:

cd /tmp
rm -rf linux
git clone ~/linux linux
cd linux
make defconfig >/dev/null

... and then we can build the kernel in such a loop (as root again):

perf stat --repeat 10 --null --pre		'\
  cp -a kernel ../kernel.copy.$(date +%s);	 \
  rm -rf *;					 \
  git checkout .;				 \
  echo 1 > /proc/sys/vm/drop_caches;		 \
  find ../kernel* -type f | xargs cat >/dev/null;  \
  make -j kernel >/dev/null;			 \
  make clean >/dev/null 2>&1;			 \
  sync						'\
  						 \
  make -j16 >/dev/null
Clone this wiki locally