neonvm: build and pass kernel tools to neonvm runner

[iddm]

This allows the compute instances to have the tools (for now just perf and the kernel headers) available in the guest VM, so that the VM can use those. These tools are required to be passed through only because they are tied to the kernel version they were built for. However, for the future, we are not limited to only such a use case and can basically include anything.

[Copilot]

Pull Request Overview

This PR enhances the neonvm project by building and passing the perf binary to the neonvm runner. The changes include adding a new tools path configuration in the runner and disks modules, updating Dockerfiles to copy the tools binary, and modifying the Makefile to handle the tools artifacts.

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
neonvm-runner/cmd/main.go	Introduced a new constant and flag for tools path; updated the call to setupVMDisks.
neonvm-runner/cmd/disks.go	Updated setupVMDisks to accept a Config object; added a block to generate an ISO for tools.
neonvm-runner/Dockerfile	Copied the tools directory and created a symlink for the perf binary.
neonvm-kernel/Dockerfile	Extended package installations and added build steps to compile and copy the perf binary.
neonvm-kernel/.gitignore	Updated to ignore the tools directory.
Makefile	Modified to remove previous tools and copy the updated tools from the Docker image.
.github/workflows/build-images.yaml	Updated to copy the tools directory from the image.

[petuhovskiy]

LGTM

Would be nice to try running this perf on staging, both on x86/ARM, for postgres 15/16/17, to confirm that it's working in all possible configurations.

[sharnoff]

Left some comments -- broadly looks ok, mostly just some specifics

[iddm]

Added kernel headers.

[sharnoff]

lgtm!

[iddm]

In the known place will require this as well:

-    ignore=["autoscaling/neonvm-kernel/vmlinuz"],
+    ignore=["autoscaling/neonvm-kernel/vmlinuz", "autoscaling/neonvm-kernel/tools"],

[iddm]

Please do not merge this PR without me. I need to test that the headers and the kheaders.ko.zst are properly placed in the directories which are looked for by the bcc-profile at runtime.

[iddm]

As we have changed in the compute to also allow using eBPFs, as well as there has been a long-awaiting request to support eBPFs fully, there are a bit more changes required now. In my last commit which I have just pushed, I have enabled the required kernel configuration options to enable us to use eBPFs at runtime successfully. The eBPFs now compile just fine, as some of those even work, but not all of them. This is due to the reason that some of the bpf headers require a kernel header file GEN-reg-for-each.h or something like that. This header is only generated for the x86 architecture and is not required for aarch64, so the latter should just work.

To pack the "tools", which now includes:

1. perf binary.
2. kernel headers and modules necessary for eBPFs and other kernel tools.

I had to make an adjustment in the ISO9660-generating code and change the version suffix of the kernel release name. First, the code for ISO9660 generation, which had existed before my changes, didn't recursively go through the contentPath and didn't package anything what isn't a file. In order to simplify the things, I made it work recursively over all the directories in the contentPath to include every single file from there, so to fully pack the contentPath into an iso-file. However, due to our naming of the kernel release, it wasn't possible to pack the things into the iso file as-is, as the ISO standard require no more than 31 characters per path entry, and our kernel release name for twice or even more than that. So I changed the version suffix to a much smaller one, still helpful, and also got approvals from the others from the autoscaling team.

I am going to leave for a couple of weeks now, so this is a list of the things i'll do once I come back:

1. Change the arm64 kernel configuration file accordingly, to enable the same features as I have enabled for x86.
2. Make sure the x86 required headers are generated and properly copied into the modules tree.

The testing can be done by tilt up full, and using the autoscaling changes and the compute changes.

[iddm]

The Iso9660, besides having character limits and other limits, also requires lower-casing the file names, for some reason. And our Go dependency for that did that as well. I thought about creating a vfat or ext4 image from the go code and it worked to some degree, but it was taking too much time, what would have influenced the startup time of the computes. So I opted in for another approach, which is to create the ext4 disk image file when building the kernel, and from the Go code simply passing it to qemu, so that we have it immediately and we require no additional work, almost not changing the startup performance of the computes at all.

The current approach allows the files to (obviously) have correct names and does not impose any other restrictions.

I tested the built kernel module and the headers with a compute, using bptrace, execsnoop-bpfcc and even our compute code, all seem to have worked. Will perform additional (proper) testing tomorrow, but we are almost there now.