Tools: Configuring GPU Pass-Through for AMD cards on openSUSE Kalpa - Guide

Tools: Configuring GPU Pass-Through for AMD cards on openSUSE Kalpa - Guide

Prerequisites

Configuring the host

Verify the host environment

Enable IOMMU

Configure VFIO and isolate the GPU used for pass-through

Load the VFIO driver

Including the driver in the initrd file

Configuring the guest

Requirements for the guest configuration Verify that the host support VT-d technology and that it is already enabled in the firmware settings: IOMMU is disabled by default. You need to enable it at boot time in the /etc/kernel/cmdline configuration file. 1) For Intel CPU add this line to file: 2) For AMD CPU add this line instead: 3) Finally, when you cat the file, you will see content like this: 1) Find the card vendor and model IDs: 2) Create the file /etc/modprobe.d/vfio.conf with the following content: 1) Create the file /etc/dracut.conf.d/gpu-passthrough.conf and add the following content (mind the leading whitespace): 2) Re-generate the initrd file: 3) Reboot the host machine For most of the changes in the above steps to take effect, you need to reboot the host machine: 4) Verify IOMMU is Active After your Kalpa host boots back up, open a terminal and check the kernel logs to confirm that the hardware isolation is working: This section describes how to configure the guest virtual machine so that it can use the host's GPU. This article was inspired by Configuring GPU Pass-Through for NVIDIA cards Templates let you quickly answer FAQs or store snippets for re-use. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse

Command

Copy

# dmesg | grep -e "Directed I/O" [ 0.283295] [ T1] DMAR: Intel(R) Virtualization Technology for Directed I/O # dmesg | grep -e "Directed I/O" [ 0.283295] [ T1] DMAR: Intel(R) Virtualization Technology for Directed I/O # dmesg | grep -e "Directed I/O" [ 0.283295] [ T1] DMAR: Intel(R) Virtualization Technology for Directed I/O intel_iommu=on iommu=pt rd.driver.pre=vfio-pci intel_iommu=on iommu=pt rd.driver.pre=vfio-pci intel_iommu=on iommu=pt rd.driver.pre=vfio-pci amd_iommu=on iommu=pt rd.driver.pre=vfio-pci amd_iommu=on iommu=pt rd.driver.pre=vfio-pci amd_iommu=on iommu=pt rd.driver.pre=vfio-pci # cat /etc/kernel/cmdline root=/dev/nvme0n1p2 splash=silent swapaccount=1 systemd.show_status=1 mitigations=auto quiet security=selinux selinux=1 intel_iommu=on iommu=pt rd.driver.pre=vfio-pci # cat /etc/kernel/cmdline root=/dev/nvme0n1p2 splash=silent swapaccount=1 systemd.show_status=1 mitigations=auto quiet security=selinux selinux=1 intel_iommu=on iommu=pt rd.driver.pre=vfio-pci # cat /etc/kernel/cmdline root=/dev/nvme0n1p2 splash=silent swapaccount=1 systemd.show_status=1 mitigations=auto quiet security=selinux selinux=1 intel_iommu=on iommu=pt rd.driver.pre=vfio-pci # lspci -nn | grep -i "AMD" 03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 33 [Radeon RX 7600/7600 XT/7600M XT/7600S/7700S / PRO W7600] [1002:7480] (rev cf) 03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 HDMI/DP Audio [1002:ab30] # lspci -nn | grep -i "AMD" 03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 33 [Radeon RX 7600/7600 XT/7600M XT/7600S/7700S / PRO W7600] [1002:7480] (rev cf) 03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 HDMI/DP Audio [1002:ab30] # lspci -nn | grep -i "AMD" 03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 33 [Radeon RX 7600/7600 XT/7600M XT/7600S/7700S / PRO W7600] [1002:7480] (rev cf) 03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 HDMI/DP Audio [1002:ab30] options vfio-pci ids=1002:7480,1002:ab30 options vfio-pci ids=1002:7480,1002:ab30 options vfio-pci ids=1002:7480,1002:ab30 add_drivers+=" vfio vfio_iommu_type1 vfio_pci " add_drivers+=" vfio vfio_iommu_type1 vfio_pci " add_drivers+=" vfio vfio_iommu_type1 vfio_pci " # sdbootutil list-snapshots 1 first root filesystem 2 Snapshot Update of #1 3 Snapshot Update of #2 4 Snapshot Update of #2 5 Snapshot Update of #4 6 Snapshot Update of #5 7 Snapshot Update of #4 # sdbootutil list-kernels 7 ok /lib/modules/6.19.5-2-default/vmlinuz -> opensuse-microos-6.19.5-2-default-7.conf # sdbootutil -weight: 500;">update-all-entries 7 # sdbootutil mkinitrd 7 # sdbootutil list-snapshots 1 first root filesystem 2 Snapshot Update of #1 3 Snapshot Update of #2 4 Snapshot Update of #2 5 Snapshot Update of #4 6 Snapshot Update of #5 7 Snapshot Update of #4 # sdbootutil list-kernels 7 ok /lib/modules/6.19.5-2-default/vmlinuz -> opensuse-microos-6.19.5-2-default-7.conf # sdbootutil -weight: 500;">update-all-entries 7 # sdbootutil mkinitrd 7 # sdbootutil list-snapshots 1 first root filesystem 2 Snapshot Update of #1 3 Snapshot Update of #2 4 Snapshot Update of #2 5 Snapshot Update of #4 6 Snapshot Update of #5 7 Snapshot Update of #4 # sdbootutil list-kernels 7 ok /lib/modules/6.19.5-2-default/vmlinuz -> opensuse-microos-6.19.5-2-default-7.conf # sdbootutil -weight: 500;">update-all-entries 7 # sdbootutil mkinitrd 7 dmesg | grep -i "Adding to iommu group" [ 0.283068] [ T1] pci 0000:00:00.0: Adding to iommu group 0 [ 0.283078] [ T1] pci 0000:00:01.0: Adding to iommu group 1 [ 0.283091] [ T1] pci 0000:00:14.0: Adding to iommu group 2 [ 0.283097] [ T1] pci 0000:00:14.2: Adding to iommu group 2 [ 0.283106] [ T1] pci 0000:00:15.0: Adding to iommu group 3 [ 0.283115] [ T1] pci 0000:00:16.0: Adding to iommu group 4 [ 0.283122] [ T1] pci 0000:00:17.0: Adding to iommu group 5 [ 0.283137] [ T1] pci 0000:00:1a.0: Adding to iommu group 6 [ 0.283147] [ T1] pci 0000:00:1c.0: Adding to iommu group 7 [ 0.283155] [ T1] pci 0000:00:1c.3: Adding to iommu group 8 [ 0.283163] [ T1] pci 0000:00:1c.4: Adding to iommu group 9 [ 0.283172] [ T1] pci 0000:00:1d.0: Adding to iommu group 10 [ 0.283188] [ T1] pci 0000:00:1f.0: Adding to iommu group 11 [ 0.283195] [ T1] pci 0000:00:1f.3: Adding to iommu group 11 [ 0.283201] [ T1] pci 0000:00:1f.4: Adding to iommu group 11 [ 0.283208] [ T1] pci 0000:00:1f.5: Adding to iommu group 11 [ 0.283216] [ T1] pci 0000:01:00.0: Adding to iommu group 12 [ 0.283224] [ T1] pci 0000:02:00.0: Adding to iommu group 13 [ 0.283235] [ T1] pci 0000:03:00.0: Adding to iommu group 14 [ 0.283245] [ T1] pci 0000:03:00.1: Adding to iommu group 15 [ 0.283260] [ T1] pci 0000:04:00.0: Adding to iommu group 16 [ 0.283268] [ T1] pci 0000:06:00.0: Adding to iommu group 17 [ 0.283275] [ T1] pci 0000:07:00.0: Adding to iommu group 18 dmesg | grep -i "Adding to iommu group" [ 0.283068] [ T1] pci 0000:00:00.0: Adding to iommu group 0 [ 0.283078] [ T1] pci 0000:00:01.0: Adding to iommu group 1 [ 0.283091] [ T1] pci 0000:00:14.0: Adding to iommu group 2 [ 0.283097] [ T1] pci 0000:00:14.2: Adding to iommu group 2 [ 0.283106] [ T1] pci 0000:00:15.0: Adding to iommu group 3 [ 0.283115] [ T1] pci 0000:00:16.0: Adding to iommu group 4 [ 0.283122] [ T1] pci 0000:00:17.0: Adding to iommu group 5 [ 0.283137] [ T1] pci 0000:00:1a.0: Adding to iommu group 6 [ 0.283147] [ T1] pci 0000:00:1c.0: Adding to iommu group 7 [ 0.283155] [ T1] pci 0000:00:1c.3: Adding to iommu group 8 [ 0.283163] [ T1] pci 0000:00:1c.4: Adding to iommu group 9 [ 0.283172] [ T1] pci 0000:00:1d.0: Adding to iommu group 10 [ 0.283188] [ T1] pci 0000:00:1f.0: Adding to iommu group 11 [ 0.283195] [ T1] pci 0000:00:1f.3: Adding to iommu group 11 [ 0.283201] [ T1] pci 0000:00:1f.4: Adding to iommu group 11 [ 0.283208] [ T1] pci 0000:00:1f.5: Adding to iommu group 11 [ 0.283216] [ T1] pci 0000:01:00.0: Adding to iommu group 12 [ 0.283224] [ T1] pci 0000:02:00.0: Adding to iommu group 13 [ 0.283235] [ T1] pci 0000:03:00.0: Adding to iommu group 14 [ 0.283245] [ T1] pci 0000:03:00.1: Adding to iommu group 15 [ 0.283260] [ T1] pci 0000:04:00.0: Adding to iommu group 16 [ 0.283268] [ T1] pci 0000:06:00.0: Adding to iommu group 17 [ 0.283275] [ T1] pci 0000:07:00.0: Adding to iommu group 18 dmesg | grep -i "Adding to iommu group" [ 0.283068] [ T1] pci 0000:00:00.0: Adding to iommu group 0 [ 0.283078] [ T1] pci 0000:00:01.0: Adding to iommu group 1 [ 0.283091] [ T1] pci 0000:00:14.0: Adding to iommu group 2 [ 0.283097] [ T1] pci 0000:00:14.2: Adding to iommu group 2 [ 0.283106] [ T1] pci 0000:00:15.0: Adding to iommu group 3 [ 0.283115] [ T1] pci 0000:00:16.0: Adding to iommu group 4 [ 0.283122] [ T1] pci 0000:00:17.0: Adding to iommu group 5 [ 0.283137] [ T1] pci 0000:00:1a.0: Adding to iommu group 6 [ 0.283147] [ T1] pci 0000:00:1c.0: Adding to iommu group 7 [ 0.283155] [ T1] pci 0000:00:1c.3: Adding to iommu group 8 [ 0.283163] [ T1] pci 0000:00:1c.4: Adding to iommu group 9 [ 0.283172] [ T1] pci 0000:00:1d.0: Adding to iommu group 10 [ 0.283188] [ T1] pci 0000:00:1f.0: Adding to iommu group 11 [ 0.283195] [ T1] pci 0000:00:1f.3: Adding to iommu group 11 [ 0.283201] [ T1] pci 0000:00:1f.4: Adding to iommu group 11 [ 0.283208] [ T1] pci 0000:00:1f.5: Adding to iommu group 11 [ 0.283216] [ T1] pci 0000:01:00.0: Adding to iommu group 12 [ 0.283224] [ T1] pci 0000:02:00.0: Adding to iommu group 13 [ 0.283235] [ T1] pci 0000:03:00.0: Adding to iommu group 14 [ 0.283245] [ T1] pci 0000:03:00.1: Adding to iommu group 15 [ 0.283260] [ T1] pci 0000:04:00.0: Adding to iommu group 16 [ 0.283268] [ T1] pci 0000:06:00.0: Adding to iommu group 17 [ 0.283275] [ T1] pci 0000:07:00.0: Adding to iommu group 18 - You need an additional display card on the host (eg. 1 igpu + 1 dedicate gpu or 2 dedicate gpu). - GPU pass-through is supported on the AMD64/Intel 64 architecture only. - Use Q35 chipset if possible. - Install the guest VM using UEFI firmware. - Add the following emulated Graphic: Spice or VNC - Add the following emulated Device: qxl, VGA or Virtio - Add the host PCI device (03:00.0 in our example) to the guest. - For the best performance, we recommend using virtio drivers for the network card and storage.