PVE enables the hardware graphics card pass through function

PVE Enable hardware graphics card pass through function

The first edit GRUB The configuration file ：

[email protected]:~# vim /etc/default/grub
[email protected]:~#
[email protected]:~# cat /etc/default/grub
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'


GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on video=efifb:off"
GRUB_CMDLINE_LINUX=""


# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"


# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console


# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480


# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true


# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"


# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"
[email protected]:~#


 Turn on IOMMU Support ：


GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on video=efifb:off"
 If it is AMD Of CPU：


GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on video=efifb:off"
 Copy code 

to update GRUB：

[email protected]:~# update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.11.22-5-pve
Found initrd image: /boot/initrd.img-5.11.22-5-pve
Found linux image: /boot/vmlinuz-5.11.22-4-pve
Found initrd image: /boot/initrd.img-5.11.22-4-pve
Found memtest86+ image: /boot/memtest86+.bin
Found memtest86+ multiboot image: /boot/memtest86+_multiboot.bin
done
[email protected]:~#
 Copy code 

Add the required system modules （ drive ）：

[email protected]:~#
[email protected]:~# echo "vfio" >> /etc/modules
[email protected]:~# echo "vfio_iommu_type1" >> /etc/modules
[email protected]:~# echo "vfio_pci" >> /etc/modules
[email protected]:~# echo "vfio_virqfd" >> /etc/modules
[email protected]:~#
[email protected]:~# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.



# Generated by sensors-detect on Fri Sep 24 17:22:44 2021
# Chip drivers
coretemp
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
 Copy code 

Then add the module （ drive ） The blacklist , That is to let GPU The device will not use these drivers after the next system startup , Free up the equipment for vfio For driving ：

Intel To show ：

echo "blacklist snd_hda_intel" >> /etc/modprobe.d/pve-blacklist.conf
echo "blacklist snd_hda_codec_hdmi" >> /etc/modprobe.d/pve-blacklist.conf
echo "blacklist i915" >> /etc/modprobe.d/pve-blacklist.conf
 Copy code 

N card /A card ：

echo "blacklist nouveau" >> /etc/modprobe.d/pve-blacklist.conf
echo "blacklist radeon" >> /etc/modprobe.d/pve-blacklist.conf
 Copy code 

If it is N The card also needs to add the following configuration to kvm.conf（ According to foreigners, it is to avoid some inexplicable mistakes ）：

echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
 Copy code 

[email protected]:~# echo "blacklist nouveau" >> /etc/modprobe.d/pve-blacklist.conf
[email protected]:~# echo "blacklist radeon" >> /etc/modprobe.d/pve-blacklist.conf
[email protected]:~#
[email protected]:~# cat /etc/modprobe.d/pve-blacklist.conf
# This file contains a list of modules which are not supported by Proxmox VE


# nidiafb see bugreport https://bugzilla.proxmox.com/show_bug.cgi?id=701
blacklist nvidiafb
blacklist nouveau
blacklist radeon
[email protected]:~#
[email protected]:~# echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
[email protected]:~#
[email protected]:~# cat /etc/modprobe.d/kvm.conf
options kvm ignore_msrs=1
 Copy code 

Update kernel ：

[email protected]:~# update-initramfs -u
update-initramfs: Generating /boot/initrd.img-5.11.22-5-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
No /etc/kernel/proxmox-boot-uuids found, skipping ESP sync.
[email protected]:~#
 Copy code 

Restart the machine ：

[email protected]:~# reboot

After restarting, check whether the module is loaded normally ：

[email protected]:~# lsmod | grep vfio
vfio_pci               57344  1
vfio_virqfd            16384  1 vfio_pci
irqbypass              16384  11 vfio_pci,kvm
vfio_iommu_type1       36864  1
vfio                   36864  5 vfio_iommu_type1,vfio_pci
[email protected]:~#
 Copy code 

View graphics card

[email protected]:~# lspci -nn | grep NV
86:00.0 3D controller [0302]: NVIDIA Corporation TU104GL [Tesla T4] [10de:1eb8] (rev a1)
[email protected]:~#
[email protected]:~#
 Copy code 

View graphics card ID, Write to configuration

[email protected]:~# lspci -n -s 86:00
86:00.0 0302: 10de:1eb8 (rev a1)
[email protected]:~#
[email protected]:~#
[email protected]:~# echo "options vfio-pci ids=10de:1eb8" > /etc/modprobe.d/vfio.conf
[email protected]:~#
 Copy code 

blog.csdn.net/qq_33921750

my.oschina.net/u/3981543

www.zhihu.com/people/chen…

segmentfault.com/u/hppyvyv6/…

juejin.cn/user/331578…

space.bilibili.com/352476552/a…

cloud.tencent.com/developer/c…

You know 、CSDN、 Open source in China 、 Think no 、 Nuggets 、 Bili, Bili 、 Tencent cloud