Did you just drop Linux support?

[nextgenthemes]

Summary

I see you dropped support for Vagrant and seem to moved to Lima now. Turns out Lima does actually support Linux.

(AI tells me that it's optimized for Mac and that it adds unnecessary overhead on Linux and give me some command for Qemu/KVM stuff that points directly to an ubuntu.iso - did not try it, do not really care about overhead as long as it works)

trellis --version
1.15.0

trellis vm start
Error: No VM managers are supported on linux yet.

Is this error message coming from Trellis CLI or from Lima? (I just installed the latest alpha)

So I can no longer run a local VM on a Linux host with Trellis?

Additional context

No response

[retlehs]

Can you test this change? #501

[nextgenthemes]

How do I tests this?

brew install --HEAD roots/tap/trellis-cli-dev

Does not include that patch. I also tried

brew install --build-from-source https://github.com/roots/trellis-cli/commit/6202701b91625c66ca0d7918bd7bd7fc63d56eef
Warning: No available formula with the name "6202701b91625c66ca0d7918bd7bd7fc63d56eef".
==> Searching for similarly named formulae...
Error: No formulae found for 6202701b91625c66ca0d7918bd7bd7fc63d56eef.

[retlehs]

1. git clone --branch retlehs-patch-1 git@github.com:roots/trellis-cli.git
2. go build

Then run the built trellis-cli binary

[nextgenthemes]

The error has changed.

./trellis vm start
Error: unsupported OS or macOS version. The macOS Virtualization Framework requires macOS 13.0 (Ventura) or later.

[retlehs]

Just pushed another update - can you pull it down, rebuild, and test again?

[nextgenthemes]

Sadly, I am still the only one who actually uses Trellis on a Linux desktop?

... vmType "vz" is not a registered driver

It needs to be qemu on Linux. I changed it to qemu the pkg/lima/files/config.yml and now there is progress. Still breaks though. I got a network error, seems networking also needs to be changed on linux. AI made the suggestion to remove the networking config and let lima choose something. Did that.

Ended up with a broken vm that I could not stop or delete properly.

Needed to install virtiofsd on my host OS.

INFO[0018] READY. Run `limactl shell nextgenthemes.com` to open the shell. 
Error starting VM.
Could not determine IP address for VM instance: exit status 1
Cannot find device "lima0"

But I can ssh into the vm now. I can provision it but it fails at

TASK [wordpress-install : Create web root of sites] ****************************

IP address for the VM is supposed to be 192.168.5.15 now?

I can limactl shell nextgenthemes.com into the vm and ping google from inside it. But from my host I can not ping 192.168.5.15

[retlehs]

Doubt you’re the only one, unfortunately none of us on the team have access to a host Linux setup right now

Thanks for the issue and testing. Can you try adjusting network settings in ~/.lima/<instance_name>/lima.yaml?

[nextgenthemes]

OK, I just remember you releasing a version that broke Linux b4, and I was the only one reporting and fixing it. Seemed like it's not tested.

Is it not possible to write some GitHub actions to test this? I guess VM inception does not work.

I am not sure my network config is actually wrong now, I removed the networking part and recompiled trellis-cli. For completeness.

pkg/lima/files/config.yml

vmType: "qemu"

completely removed this:

networks:
- vzNAT: true

This actually fixed it so far that I could actually start the vm. And the file that is written to ~/.lima/<instance_name>/lima.yaml is overwritten by "trellis vm start" anyway. Also, not sure what I am supposed to adjust it to now.

./trellis vm start                                     
Running command => limactl start nextgenthemes.com
INFO[0000] Using the existing instance "nextgenthemes.com" 
INFO[0000] Starting the instance "nextgenthemes.com" with internal VM driver "qemu" 
INFO[0000] [hostagent] hostagent socket created at /home/user/.lima/nextgenthemes.com/ha.sock 
INFO[0001] [hostagent] Using system firmware ("/usr/share/OVMF/OVMF_CODE_4M.fd") 
INFO[0003] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/home/user/.lima/nextgenthemes.com/serial*.log") 
INFO[0004] SSH Local Port: 46379                        
INFO[0003] [hostagent] Waiting for the essential requirement 1 of 2: "ssh" 
INFO[0024] [hostagent] The essential requirement 1 of 2 is satisfied 
INFO[0024] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh" 
INFO[0025] [hostagent] The essential requirement 2 of 2 is satisfied 
INFO[0026] [hostagent] Waiting for the guest agent to be running 
INFO[0026] [hostagent] Forwarding "/run/lima-guestagent.sock" (guest) to "/home/user/.lima/nextgenthemes.com/ga.sock" (host) 
INFO[0026] [hostagent] Guest agent is running           
INFO[0026] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished" 
INFO[0026] [hostagent] Forwarding TCP from 127.0.0.1:6379 to 127.0.0.1:6379 
INFO[0026] [hostagent] Not forwarding TCP 127.0.0.53:53 
INFO[0027] [hostagent] Not forwarding TCP 127.0.0.54:53 
INFO[0027] [hostagent] The final requirement 1 of 1 is satisfied 
INFO[0027] [hostagent] Forwarding TCP from 127.0.0.1:3306 to 127.0.0.1:3306 
INFO[0028] READY. Run `limactl shell nextgenthemes.com` to open the shell. 
Error starting VM.
Could not determine IP address for VM instance: exit status 1
Cannot find device "lima0"

limactl shell nextgenthemes.com                        
bash: line 1: cd: /home/user/[censoired]/trellis: No such file or directory
bash: line 1: cd: /home/user: No such file or directory

ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:55:55:0c:99:f0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.15/24 metric 200 brd 192.168.5.255 scope global dynamic eth0
       valid_lft 85606sec preferred_lft 85606sec
    inet6 fec0::5055:55ff:fe0c:99f0/64 scope site dynamic mngtmpaddr noprefixroute 
       valid_lft 86102sec preferred_lft 14102sec
    inet6 fe80::5055:55ff:fe0c:99f0/64 scope link 
       valid_lft forever preferred_lft forever

No idea why it tries to cd into the directory where I have trellis on my host OS when I ssh into the VM

For vagrant, the IP default was 192.168.56.5

Not sure where 192.168.5.15 comes from. No idea how the limactl shell nextgenthemes.com actually successfully ssh's into the VM because it not with that IP.

[retlehs]

That IP address is coming from Lima’s defaults

https://lima-vm.io/docs/config/network/user/

[swalkinshaw]

The problem is that macOS offers vzNAT networking where the guest IP is reachable from the host. But on Linux, the guest IP is not reachable making this harder.

https://lima-vm.io/docs/config/network/vmnet/ vs https://lima-vm.io/docs/config/network/user/

[swalkinshaw]

Are you running Ubuntu @nextgenthemes? This is probably the kick I need to properly support Linux.

Two things:

1. I did a prototype of LXD support in Add LXD VM support #363 years ago. It got close but had a few issues. LXD would offer much better performance than QEMU I think
2. If we do just unify on Lima (and perf is acceptable), there's no way around the networking limitations I described above. This means we'll need a solution with a reverse HTTP proxy running on the host.

Now, it would be cool if Trellis handled as much automatically as it could, but in the short term, anyone could configure Nginx or Caddy manually. But regardless, the host needs the HTTP port on the guest forwarded. #600 will do exactly this.

So theoretically with that PR and #501, you could at least run the reverse proxy manually and access the site.

limactl list --json example.com | jq ".config.portForwards[] | select(.guestPort == 80) | .hostPort"

[swalkinshaw]

LXD uses QEMU I guess? TIL

https://canonical.com/lxd

[nextgenthemes]

Reverse proxy needs to set up on the host for Lima?

I got to the port forwarding part, but then I gave up on Lima. I am actually working in a libvirt role right now. What I am doing is pure libvirt QEMU KVM, but libvirt-lxc can also handle LXD.