2022年5月4日星期三

Proxmox | PVE GPU passthrough

這邊介紹Proxmox PVE GPU passthrough，直接看下面的設定步驟吧。

步驟一：啟用IOMMU功能(Intel CPU)

編輯/etc/default/grub，新增intel_iommu=on參數，如下

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"

更新grub

root@pve03:~# update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.13.19-2-pve
Found initrd image: /boot/initrd.img-5.13.19-2-pve
Found memtest86+ image: /boot/memtest86+.bin
Found memtest86+ multiboot image: /boot/memtest86+_multiboot.bin
done

重開機，檢查IOMMU有沒有啟動，如果是disable那就是BIOS要去啟用這個功能。

root@pve04:~# dmesg | grep -e DMAR -e IOMMU
[    0.012537] ACPI: DMAR 0x0000000079577538 000148 (v01 SUPERM SMCI--MB 00000001 INTL 20091013)
[    0.012580] ACPI: Reserving DMAR table memory at [mem 0x79577538-0x7957767f]
[    0.551443] DMAR: IOMMU enabled

步驟二：新增vfio模組

新增下面參數到/etc/modules

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

步驟三：確認Remapping

確認Remapping功能有啟用，如果沒有顯示一樣是到BIOS裡面開啟。

root@pve04:~# dmesg | grep 'remapping'
[    1.253853] DMAR-IR: Enabled IRQ remapping in xapic mode
[    1.253855] x2apic: IRQ remapping doesn't support X2APIC mode

步驟三：確認IOMMU Isolation

只要有跑出類似下面訊息就代表有Isolation有成功。

root@pve04:~# find /sys/kernel/iommu_groups/ -type l
/sys/kernel/iommu_groups/55/devices/0000:7f:17.0
/sys/kernel/iommu_groups/55/devices/0000:7f:17.7
/sys/kernel/iommu_groups/55/devices/0000:7f:17.5
/sys/kernel/iommu_groups/55/devices/0000:7f:17.6
/sys/kernel/iommu_groups/55/devices/0000:7f:17.4
/sys/kernel/iommu_groups/83/devices/0000:00:1d.0
/sys/kernel/iommu_groups/17/devices/0000:ff:14.1
/sys/kernel/iommu_groups/45/devices/0000:7f:14.0
/sys/kernel/iommu_groups/73/devices/0000:00:05.4
/sys/kernel/iommu_groups/35/devices/0000:7f:0f.6
/sys/kernel/iommu_groups/35/devices/0000:7f:0f.4
/sys/kernel/iommu_groups/35/devices/0000:7f:0f.0
/sys/kernel/iommu_groups/35/devices/0000:7f:0f.5
/sys/kernel/iommu_groups/35/devices/0000:7f:0f.1
/sys/kernel/iommu_groups/7/devices/0000:ff:10.0
/sys/kernel/iommu_groups/7/devices/0000:ff:10.7
/sys/kernel/iommu_groups/7/devices/0000:ff:10.5
/sys/kernel/iommu_groups/7/devices/0000:ff:10.1
/sys/kernel/iommu_groups/7/devices/0000:ff:10.6