How I set up Windows 7 (or Vista) GPU passthrough with libvirt/QEMU/KVM

[MattKC]

Can anyone with an Nvidia GPU confirm if this guide works? I have one, and it didn't work for me, and it seems like that's probably why it also didn't work for gregory003.

EDIT: For more information, it just stays on a black screen, but I can hear audio via the passthrough audio.
UEFI virtual machines work for me, and I do have x-vga enabled, so it's probably something to do with Nvidia.

I can't confirm for sure right now, but just to be sure, have you set "kvm hidden" and "vendor id"? Those are necessary for NVIDIA cards to init on KVM, and I have a mini-guide here if you don't have those.

[PortalPlayer]

I can't confirm for sure right now, but just to be sure, have you set "kvm hidden" and "vendor id"? Those are necessary for NVIDIA cards to init on KVM, and I have a mini-guide here if you don't have those.

I have both, but it still doesn't work. Still a black screen. I know the VM is working, since I hear audio, but that's pretty much it.

EDIT: Fixed it. I had to pass a patched ROM over. Despite not needing one on other VMs, I guess it was required here.

To get your GPU BIOS on Linux, you'll need to run this as root:

Code: Select all

echo 1 > /sys/bus/pci/devices/0000:01:00.0/rom
cat /sys/bus/pci/devices/0000:01:00.0/rom > path/to/dump/vbios.rom
echo 0 > /sys/bus/pci/devices/0000:01:00.0/rom

Replace 0000:01:00.0 with your PCI ID.

Note that the Linux method didn't work for me, but the Windows method did. Try the Windows method if this didn't work.

You can also extract it using GPU-Z on Windows.

Then, open it inside of a hex editor, and search for the string "VIDEO". Look a couple bytes back until you find the hex values "0x55 0xAA". Delete everything before those bytes, so the file starts with those two hex values. Don't worry, you're not going to break your GPU doing this, none of these changes are being made to it. Save the patched ROM to a location (I saved mine to /etc/nvidia.rom).

Finally, edit your VM XML, and add

Code: Select all

<rom file="/etc/nvidia.rom"/>

inside the hostdev for your GPU, replace "/etc/nvidia.rom" with the path you have your patched ROM in.

It should work now.

I'm not that good at explaining stuff, so if this doesn't actually work for you, then you can look up another guide on Google.

[foxlet]

Note that the Linux method didn't work for me, but the Windows method did. Try the Windows method if this didn't work.

The Linux method won't work either way because (during boot) the kernel ends up modifying the shadow VBIOS that is resident in memory (to enable kernel modesetting, etc), and ultimately that's what you end up dumping from sysfs. It's a known "issue" that the NVIDIA Windows driver doesn't play well with the Linux-modified vbios, hence the fix being to dump it directly from the card's ROM using custom tools (either NVIDIA's own flash tool, or GPU-Z).

It's does help to use a copy of VBIOS by default though, you'll get a lot more consistency with QEMU's behavior.

[foxlet]

At first, SeaBIOS + passthrough doesn't appear to work at all, which seemed to reinforce all the people who claimed it was problematic. My passed through GPU displayed no image whatsoever. Regardless, after all the frustration I had with UEFI, the handful of people saying SeaBIOS worked fine for them made me want to try it myself again.

It turns out there was one thing I was missing when trying to passthrough with SeaBIOS. You have to enable x-vga. For some reason I'd never come across this before, but this was the magic flag that made my GPU get picked up by the BIOS and subsequently Windows 7.

To add extra context, the reason why "x-vga" is needed is because it enables support for the Linux VGA arbitrator. Since legacy VGA calls (used to initialize the card) are still handled by writing to a fixed address (one decided back in the ISA days), the only way for the host and guest to share low-level VGA access is through the arbitrator. UEFI Class 3 hardware, on the other hand, use protected-mode drivers that don't require legacy VGA services

[foxlet]

Just for fun... I've managed to get early Vista previews (and Longhorn) working in QEMU through the right combination of hardware. This card is just about old enough that the NVIDIA drivers don't have any "Code 43" VM protection lol

[acidiclight]

Hmmmm. I haven't read the full post yet since it's a lot of text but I'm very intrigued. I used to have a Windows 7 VM with full GPU passthrough working with an RTX 2060, but at the time I was also doing Single-GPU passthrough and needed to pass through USB controllers as well (so I could have keyboard/mouse support). Could never get it working though because my motherboard's chipset is too new for Win7 and thus it had no idea how to talk to any USB device I plugged in.

(I'm aware you can directly pass in USB devices instead of their controllers, and this is ultimately what I did, however it breaks hot-swap support which is fundamentally incompatible with things like external HDDs and thumbdrives.)

What I'm curious about now is how to get 8/8.1 working. Tried a few days ago, and it does natively support UEFI obviously....however I could NOT get nvidia drivers to install at all. I ended up giving up and upgrading the VM to Windows 10.

[MattKC]

What I'm curious about now is how to get 8/8.1 working. Tried a few days ago, and it does natively support UEFI obviously....however I could NOT get nvidia drivers to install at all. I ended up giving up and upgrading the VM to Windows 10.

I managed to get Windows 8.1 working with no real trouble, though I was using an AMD GPU which are obviously much less hostile to KVM than NVIDIA GPUs are. If you didn't before, definitely try the "kvm hidden" and "vendor ID" patches, those tend to get NVIDIA cards cooperating, though someone here had to load a custom VBIOS to make it work.

[Jellington]

Apparently, it might be possible to boot 7 in pure UEFI mode by disabling the default vga.sys driver that supposedly hangs the system on boot. I might try that myself once I figure out how to slipstream updates into base SP1 ISO to not waste time with broken Windows Update.
https://reboot.pro/index.php?showtopic=21108

[MattKC]

I have tried the VGA.SYS disabling but didn't have any luck. I did get Windows 7 booting in UEFI mode since writing this guide though so maybe it deserves an update at some point.

[Jellington]

I have tried the VGA.SYS disabling but didn't have any luck.

Same, I've tried injecting the QXL driver into the system image and the system just froze on black screen. Apparently, someone was trying to backport the driver from early Windows 8 beta builds but they haven't had much luck yet.

[Jellington]

Tested the legally acquired FlashBoot preloader (flashboot33n.efi) and it does allow me to properly boot into the vanilla Windows 7 system with full resolution, yet installing AMD display drivers results in the Code 12 error. Worth noting that I don't have any virtual graphics adapter connected at that point, only the passed through GPU. Adding x-vga seems to have no effect on UEFI guests.

Upd: after lurking for some libvirt documentation, I stumbled upon this page. Apparently, "pcie-root-port" controllers support hotplug while the "pcie-root" doesn't. After changing the passed GPU to use the pcie-root (changed bus to 0x00 in <address>), the system booted and started successfully installing drivers, but now the displau freezes on boot no matter what preloader (uefiseven, flashboot or lack thereof) I'm using. The "Unable to unlock VGA ROM memory at C0000" error with uefiseven has disappeared, so that might actually be some sort of progress. With QXL adapter attached in addtion I can still hear the Windows startup chime, but there's none if I leave only the real GPU connected.

[bluesin]

Hello, I got it to work through OP's guide with SeaBIOS and OVMF with MrLeRien's help. I created this account to help anyone having trouble setting this up.
If u can't see image after setting GPU, it's because your GPU driver is not recognized (essentially because it is not installed). You gotta make a VNC or SPICE (I guess it works equally) connection to watch through the VNC viewer from other device (I used my cellphone with RVNC Viewer). Then you will, using this device, install Nvidia Driver, restart pc and if you got it working, you will see a black screen for some seconds before it loads through your GPU. I had mine working with GPU and the KVM graphics, I had no problems with both working at the same time.
Sorry for bad english, hope I helped!

[cinnamonpancake]

hey! it seems like since qemu version 8.2, x-vga is bugged in some way and vga graphics look completely messed up and dont render some things until actual graphics drivers get loaded (at least for my RTX 2070 super)
it still works fine once drivers are installed but the biggest issue i have with this is it causes windows 7's boot animation to fall back to vista's, meaning i dont get to see the starting windows animation with the orbs :< (also f8 advanced options dont show up at all)
another issue is that windows 10's setup literally refuses to start with this. it just shows a blinking cursor for a couple seconds and then reboots not as big of an issue since windows 10 has full UEFI support but still
i kinda wanna look into qemu's source code and try and figure out whats causing it, but just letting everyone know if you have this issue and wanna fix this you have to build qemu 8.1 and use that for BIOS guests for now

[OrangeCat]

Glad I found this post!

I'd like to share my experience on this topic. I am passing through an Apple variant of Radeon HD 5770, on an intel 6700K + Z270 platform. The hypervisor I am using is not qemu itself but Proxmox (so close enough).

I recently purchased this card solely for Snow Leopard passthrough but seeing it working with XP, Vista and W7 is a bonus. Maybe I can share the Snow Leopard experience in another post of yours

This Mac Radeon HD 5770 is interesting: It's supposed to support EFI boot but I just got a black screen if I use OVMF. It posts just fine under SeaBIOS. I checked the rom file extracted under Windows 10 (bare-metal installation) and it looked like it supports UEFI but may be it just doesn't play well with OVMF?
Also tried several vbios rom files I find close enough to my extracted version, including Apple's and some XFX variant but none of them worked.

I eventually managed to get this card posting under UEFI using the following steps:
1. Installed a Windows 10 Guest (too lazy to pull the card out of an itx case).
2. After a successful automated driver installation, dump 5770's bios using GPU-Z. At this step, GPU-Z shows this card is not UEFI capable, that is, there is no check mark under ATI's logo; however, I do recall if I install Windows 10 on bare metal, it would show it's UEFI-capable; but it still won't post if I turn off CSM in BIOS. These indicates the card is somewhat not UEFI-capable, and maybe it only works in an old Mac.
3. Patch the video bios file using a tool called "gopupd", which will say this vbios does not support UEFI but it can add that capability and generate a patched vbios.
4. Using this patched vbios file can make the card post under OVMF.

However, I cannot install Vista or Windows 7 using OVMF. It will just stuck on "Windows is loading files" bar. Some posts said adding CPU flag "+kvm_pv_unhalt" can help, and sometimes it works if I don't passthrough a GPU but use only virtual display, but it does not work if I passthrough this 5770. Eventually, I chose to use SeaBIOS instead.

Installation processes were just fine, and for the GPU driver it may require some manual steps due to expired driver signature (use Hardware Manager or Computer Management interface in Windows and update driver by pointing to the extracted driver file folder, click continue anyways if it says the driver was not verified or something like that).

This particular card has one DVI and two miniDP ports, I am currently using a DVI to HDMI cable to connect it with a 4K display. I will output max 1920x1080 resolution (at least it's integer scaling). Didn't bother using DP which (I think) supports 1440P.
Another note on Vista's DreamScene: it shows a black desktop if any of the official DreamScene videos is selected, but it will play sample videos. The solution is to install a video codec pack (K-Lite_Codec_Pack, latest version support in Vista is 1676).

I also have a Nvidia Quadro 4000 (Fermi) card, and that too was originally purchased for Snow Leopard, but I haven't tested it as much. That card has even more problems and that's why I got this HD 5770 instead.