I have had success with PCI Passthrough and KVM under Debian Linux to pass a dedicated GPU to a Windows VM. Here are some notes!
- Debian Stretch or later. Debian Stretch has the latest qemu (2.6) and libvirt (2.1.0) at time of writing. I was unsuccessful with Jessie due to fairly old qemu and libvirt, and no backports of qemu or libvirt are available at this time.
- Recent motherboard and CPU which supports VT-d or AMD-Vi and IOMMU
- Two video cards (GPUs). One will be for the Linux desktop and one will be dedicated for the Windows VM.
- GPU brand: Get a recent AMD GPU. Nvidia disables their device in Windows with ‘Code 43‘ when virtualization is detected though there are some workarounds with recent versions of qemu. Intel passthrough is experimental at this point for Haswell or later and could be worth checking out if so inspired. AMD is the most straightforward for passthrough. Buy card, stick it in, and it works. One can even do some crazy stuff with AMD.
- Second monitor connected to the passthrough GPU
- Second USB keyboard and USB mouse to dedicate to the VM to make life easier. One could also use a KVM switch (the other KVM).
- Windows 8.1 or later. Windows 8.1 has better UEFI support than Windows 7 and will work more care free overall with GPU devices and passthrough.
Reference for more info
Good general reference for PCI Passthrough:
How is the performance?
With the aid of Hyper-V Enligtenments in KVM, performance is as native as bare metal.
OK Let’s Do This Bro!
Step 1: Install kvm and associated packages
sudo apt install virt-manager qemu-kvm ovmf bridge-utils uml-utilities libvirt-bin
It is a good idea to add your username to these groups for general kvm usage as a desktop user:
sudo gpasswd -a $(whoami) kvm sudo gpasswd -a $(whoami) libvirt sudo gpasswd -a $(whoami) libvirt-qemu
Step 2: Enable IOMMU
Edit /etc/default/grub and add the following to the GRUB_CMDLINE_LINUX_DEFAULT entry:
For Intel cpus, add:
For AMD cpus, add:
Step 3: Create new grub.cfg with that change
sudo grub-mkconfig -o /boot/grub/grub.cfg
Step 4: Blacklist the Linux driver of the passthrough device
Create / edit a file named, say, /etc/modprobe.d/blacklist.conf and add the following contents:
blacklist nouveau blacklist radeon blacklist amdgpu
Step 5: Reboot
Step 6: Locate the PCI IDs of the passthrough GPU device
The following command will locate the iommu groups on your rig:
for iommu_group in $(find /sys/kernel/iommu_groups/ -maxdepth 1 -mindepth 1 -type d); do echo "IOMMU group $(basename "$iommu_group")"; for device in $(ls -1 "$iommu_group"/devices/); do echo -n $'\t'; lspci -nns "$device"; done; done
In my case, the example output shows my AMD device to passthrough:
IOMMU group 11 05:00.0 VGA compatible controller : Advanced Micro Devices, Inc. [AMD/ATI] Venus LE [Radeon HD 8830M] [1002:682b] (rev 87) 05:00.1 Audio device : Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series] [1002:aab0]
Note the PCI IDs of the GPU and also the HDMI audio device. The IDs are the last indications in brackets at the end of each device 1002:682b and 1002:aab0 in my case.
Step 7: Add PCI IDs to /etc/modprobe.d/vfio.conf
Create the file /etc/modprobe.d/vfio.conf and add both PCI IDs of the device to passthrough.
In my example, the file looks like:
# pass thru AMD gpu options vfio-pci ids=1002:682b,1002:aab0
Step 8: Add vifo moudles to initrd
Edit the file /etc/initramfs-tools/modules
Add the following:
vfio vfio_iommu_type1 vfio_pci vfio_virqfd
Save and then create a new initrd image via:
sudo update-initramfs -u
Step 9: Enable passthrough in qemu’s config
nvram = [ "/usr/share/OVMF/OVMF_CODE.fd:/usr/share/OVMF/OVMF_VARS.fd" ]
Step 10: Reboot
Windows VM Setup
From this point, create a VM in virt-manager or select an existing Windows VM one wishes to pass through the PCI device.
For best performance, enable Hyper-V Enlightenments for the Windows VM. A how-to guide can be found here.
To select the passthrough device, from virt-manager click on Add Hardware > PCI Host Device and select the passthrough GPU and then also its HDMI audio device. It is recommended to pass all components of a GPU device, from what I have gathered.
Once inside Windows, be sure to install the GPU’s drivers as one would do with any bare metal Windows install. Then connect a second display to the passthrough GPU and you should be all set.
Note: It is also possible to perhaps use a single monitor which has multiple inputs and switch inputs to the Windows VM and back to Linux. However, I find it very satisfying to have a completely separate monitor to game and then have also my regular Linux desktop available side by side on the desk. The experience is as if there are two computers under one’s desk. I can pause a game, check something online / respond to a chat, and go back to the game. Or watch a video while gaming, etc.
If using an AMD GPU, congrats it just works! For NVIDIA, check out this reddit post for some more info.
NVIDIA: Code 43 🙁
Other general Windows performance tips:
- Don’t use any anti-virus or ‘security’ software
- Disable Windows Defender completely
- Disable System Restore
- Set Performance Visual Effects to: Adjust for best performance
- Set Windows Updates to manual check only
- Disable and remove any of the active start screen news or other widgets, or whatever that mess is called
- Use virtio drivers for everything (disk, network, etc).
Special shout out to #teamCO