Installation of Windows Server 2025 on KVM Hypervisors in bootloop

[sourceYan]

Hi CS Community,

on a fresh setup with CS 4.21.0.0 and Alma Linux 9 KVM (9.1.0), i can't boot into the Windows Setup.

The VM boot from iso with "Loading files..." and then appear for 1 sec. the Windows Logo and the VM boot over again.
I tried to F8 the system do on the bootloader "Don't restart on error" - but it don't work.

Checksum of the ISO is good (tested also the eval iso from MS)
Everything in CS is default, no UEFI or Secure boot in the moment.
Windows Server 2022 will run/install successfully.

Any ideas? =)

[sourceYan]

qemu-vm logfile on the host is:

2025-09-25 05:50:08.406+0000: starting up libvirt version: 10.10.0, package: 7.7.el9_6.alma.1 (AlmaLinux Packaging Team <[email protected]>, 2025-09-17-05:06:48, ), qemu version: 9.1.0qemu-kvm-9.1.0-15.el9_6.9.alma.1, kernel: 5.14.0-570.44.1.el9_6.x86_64, hostname: kvm01a.cloudstack.lab LC_ALL=C \ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin \ HOME=/var/lib/libvirt/qemu/domain-11-i-2-19-VM \ XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-11-i-2-19-VM/.local/share \ XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-11-i-2-19-VM/.cache \ XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-11-i-2-19-VM/.config \ /usr/libexec/qemu-kvm \ -name guest=i-2-19-VM,debug-threads=on \ -S \ -object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-11-i-2-19-VM/master-key.aes"}' \ -machine pc-i440fx-rhel7.6.0,usb=off,dump-guest-core=off,memory-backend=pc.ram,acpi=on \ -accel kvm \ -cpu qemu64,hv-time=on \ -m size=8388608k \ -object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":8589934592}' \ -overcommit mem-lock=off \ -smp 4,sockets=1,dies=1,clusters=1,cores=4,threads=1 \ -uuid d7621509-013d-41db-aed9-a200dd829032 \ -smbios 'type=1,manufacturer=Apache Software Foundation,product=CloudStack KVM Hypervisor,serial=d7621509-013d-41db-aed9-a200dd829032,uuid=d7621509-013d-41db-aed9-a200dd829032' \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=45,server=on,wait=off \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=localtime \ -no-shutdown \ -boot strict=on \ -device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ -device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.0","addr":"0x4"}' \ -blockdev '{"driver":"file","filename":"/mnt/797fe666-3e97-3ec5-be39-89ec2080f44f/f38b1695-5f18-4295-93d6-ae88bdbab295","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap","cache":{"direct":true,"no-flush":false}}' \ -blockdev '{"node-name":"libvirt-2-format","read-only":false,"cache":{"direct":true,"no-flush":false},"driver":"qcow2","file":"libvirt-2-storage","backing":null}' \ -device '{"driver":"ide-hd","bus":"ide.0","unit":0,"drive":"libvirt-2-format","id":"ide0-0-0","bootindex":2,"write-cache":"on","serial":"f38b16955f18429593d6"}' \ -blockdev '{"driver":"file","filename":"/mnt/9673ce91-6cd7-3d04-8999-9965c0156a5d/208-2-d0e38aa9-ad95-305a-a36d-90e8c56075fc.iso","node-name":"libvirt-1-storage","read-only":true}' \ -device '{"driver":"ide-cd","bus":"ide.1","unit":0,"drive":"libvirt-1-storage","id":"ide0-1-0","bootindex":1}' \ -netdev '{"type":"tap","fd":"46","id":"hostnet0"}' \ -device '{"driver":"e1000","netdev":"hostnet0","id":"net0","mac":"02:01:00:cd:00:09","bus":"pci.0","addr":"0x3"}' \ -chardev pty,id=charserial0 \ -device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \ -chardev socket,id=charchannel0,fd=44,server=on,wait=off \ -device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \ -device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"1"}' \ -audiodev '{"id":"audio1","driver":"none"}' \ -object '{"qom-type":"tls-creds-x509","id":"vnc-tls-creds0","dir":"/etc/pki/libvirt-vnc","endpoint":"server","verify-peer":true}' \ -vnc 10.20.10.11:2,password=on,tls-creds=vnc-tls-creds0,audiodev=audio1 \ -device '{"driver":"cirrus-vga","id":"video0","bus":"pci.0","addr":"0x2"}' \ -device '{"driver":"i6300esb","id":"watchdog0","bus":"pci.0","addr":"0x6"}' \ -watchdog-action none \ -device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.0","addr":"0x5"}' \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on 2025-09-25 05:50:08.407+0000: Domain id=11 is tainted: high-privileges 2025-09-25 05:50:08.408+0000: Domain id=11 is tainted: deprecated-config (machine type 'pc-i440fx-rhel7.6.0') 2025-09-25 05:50:08.408+0000: Domain id=11 is tainted: deprecated-config (CPU model 'qemu64') char device redirected to /dev/pts/0 (label charserial0) 2025-09-25T05:50:09.682433Z qemu-kvm: warning: Machine type 'pc-i440fx-rhel7.6.0' is deprecated: machines from the previous RHEL major release are subject to deletion in the next RHEL major release 2025-09-25T05:50:09.683656Z qemu-kvm: warning: CPU model qemu64-x86_64-cpu is deprecated -- use at least 'Nehalem' / 'Opteron_G4', or 'host' / 'max' 2025-09-25T05:50:09.699997Z qemu-kvm: -device {"driver":"cirrus-vga","id":"video0","bus":"pci.0","addr":"0x2"}: warning: 'cirrus-vga' is deprecated, please use a different VGA card instead

but no new log event if the reboot occurs.

[weizhouapache]

@sourceYan
you can try

• stop the vm
• add a vm setting guest.cpu.mode=host-passthrough
• start the vm again

[weizhouapache]

I think the default CPU (qemu64) may be not supported by Windows Server 2025

[sourceYan]

Can i adjust the "profile" Windows Server 2025 with this setting so every new VM Deployed get this option by default?

[weizhouapache]

Can i adjust the "profile" Windows Server 2025 with this setting so every new VM Deployed get this option by default?

yes, you can add the same setting to the template

[sourceYan]

thanks again!
But only with ISO install it's not possible? for example to set this in a offering or the guest-OS settings?
So i have to create a Template for that?

[weizhouapache]

you can configure it in agent.properties, but it will affect all vms

[daviftorres]

Dear @sourceYan , sorry for jumping on late to the discussion.

I believe the problem resides here "Everything in CS is default, no UEFI or Secure boot in the moment."

In our experience, the latest Windows only got to pass this "bootloop" after we set up UEFI.

Appended the following lines to /etc/cloudstack/agent/uefi.properties.

guest.nvram.template.secure=/usr/share/OVMF/OVMF_VARS_4M.ms.fd
guest.nvram.template.legacy=/usr/share/OVMF/OVMF_VARS_4M.fd
guest.nvram.path=/var/lib/libvirt/qemu/nvram/
guest.loader.secure=/usr/share/OVMF/OVMF_CODE_4M.secboot.fd
guest.loader.legacy=/usr/share/OVMF/OVMF_CODE_4M.fd

We also made other configuration to /etc/cloudstack/agent/agent.properties, such as:

guest.cpu.mode=host-passthrough
guest.cpu.features=vmx aes smx mmx ht
vm.memballoon.stats.period=5
vm.video.hardware=qxl

Hope that helps.

[weizhouapache]

thanks @daviftorres for sharing

[StepBee]

I agree with @daviftorres - UEFI is a must now.

Just an important hint from operational experience - if live migrations are required now and in future, never use
guest.cpu.mode=host-passthrough
if you plan to update your hosts OS and kernel from time to time.

Some bad suprises could arise breaking live migrations between identical hosts - some with newer kernel some with older kernel. Smooth rolling maintenance with live migrations will become impossible due to newer kernels suddenly supporting and detecting more performance counters on the identical CPU and not allowing live migrations between non-updated hosts and already updated hosts.
Use
guest.cpu.mode=host-model
instead, even for identical hosts.

[SviridoffA]

@StepBee Thank you for sharing. Could you please tell which OS version you are using on the hypervisor? Or, if this is about the kernel, is this behavior observed on all OS? Also, can you tell between which kernel versions this behavior was observed when transitioning?

[StepBee]

@SviridoffA we experienced the issue when upgrading from Ubuntu 20.04 Kernel 5.4.0-174-generic to Ubuntu 24.04 Kernel 6.8.0-60-generic. But observed this change in the kernel was happening already at version 5.15.0-153-generic as well.
I would assume this should have happened across all Linux OS - but have no other hosts to prove it.

The error appearing in our case whentrying to live migrate from non-updated hosts to an already updated host was
qemu-system-x86_64: error: failed to set MSR 0x38f to 0xf000000ff qemu-system-x86_64: target/i386/kvm/kvm.c:3210: kvm_buf_set_msrs: Assertion ret == cpu->kvm_msr_buf->nmsrs' failed.

The combination of the Intel Xeon Gold CPU and the new kernel version with new features surprisingly brought up more performance counters, visible for example executing

Old kernel:

# rdmsr 0x38f
f000000ff

New kernel:

# rdmsr 0x38f
1000f000000ff

Even after extensive research and testing, we have not been able to track this down to a specific CPU flag (which we potentially could have disabled) or any other feature to enable live migration again.

[SviridoffA]

@StepBee Thank you very much for information 🤝, we'll keep it in mind ✍