LookingGlass/doc/ivshmem_kvmfr.rst

240 lines
6.3 KiB
ReStructuredText
Raw Normal View History

:orphan:
.. _ivshmem_kvmfr:
IVSHMEM with the KVMFR module (Recommended)
###########################################
The kernel module implements a basic interface to the IVSHMEM device
for Looking Glass allowing DMA GPU transfers.
.. _ivshmem_kvmfr_prereq:
Prerequisites
-------------
The Linux kernel headers for your kernel version are required for building
along with `dkms` to manage the module when you upgrade your kernel.
.. code:: bash
apt-get install linux-headers-$(uname -r) dkms
Then switch to the ``module/`` directory
.. code:: bash
cd module/
.. _ivshmem_kvmfr_dkms:
Installing
~~~~~~~~~~
To install the module into DKMS, run
.. code:: bash
dkms install "."
.. _ivshmem_kvmfr_loading:
Loading
~~~~~~~
Using the value you should have already calculated as per
:ref:`Determining Memory <libvirt_determining_memory>`, simply use
``modprobe`` with the parameter ``static_size_mb``, for example:
.. code:: bash
modprobe kvmfr static_size_mb=32
2024-03-08 07:32:38 +00:00
Alternatively you can make this setting permanent by creating the file
``/etc/modprobe.d/kvmfr.conf`` with the following content.
.. code:: text
options kvmfr static_size_mb=32
After this has been done, simply running ``modprobe kvmfr`` is all that is
required.
.. note::
Don't forget to adjust ``static_size_mb`` to your needs.
.. _ivshmem_kvmfr_systemd:
systemd-modules-load
~~~~~~~~~~~~~~~~~~~~
For convenience, you may load the KVMFR module when starting your computer.
We can use the ``systemd-modules-load.service(8)`` service for this task.
Create the file ``/etc/modules-load.d/kvmfr.conf`` with the following
contents::
# KVMFR Looking Glass module
kvmfr
This will now run the next time you start your machine.
.. _ivshmem_kvmfr_verification:
Verification
~~~~~~~~~~~~
If everything has been done correctly you should see the following output in
dmesg:
.. code:: text
kvmfr: creating 1 static devices
You should now also have the character device ``/dev/kvmfr0``
.. code:: bash
$ ls -l /dev/kvmfr0
crw------- 1 root root 242, 0 Mar 5 05:53 /dev/kvmfr0
.. warning::
If you start the VM prior to loading the module, QEMU will create the file
``/dev/kvmfr0`` as a regular file. You can confirm if this has happened by
2024-03-08 07:34:22 +00:00
running ``ls -l /dev/kvmfr0`` and checking if the file size is greater then
2024-03-08 07:32:38 +00:00
zero, or the permissions do not start with ``c``. If this has occurred, you
must delete the file and reload the module.
.. _ivhsmem_kvmfr_permissions:
Permissions
~~~~~~~~~~~
The module will create the ``/dev/kvmfr0`` node, which represents the KVMFR
interface. To use the interface, you need permission to access it by either
creating a udev rule to ensure your user can read and write to it, or simply
change its ownership manually, i.e.:
.. code:: bash
sudo chown user:kvm /dev/kvmfr0
As an example, you can create a new file in ``/etc/udev/rules.d/99-kvmfr.rules``
with the following contents::
SUBSYSTEM=="kvmfr", OWNER="user", GROUP="kvm", MODE="0660"
(replace ``user`` with your username)
.. _ivshmem_kvmfr_libvirt:
libvirt
2024-03-09 02:36:02 +00:00
~~~~~~~
2024-03-20 16:58:17 +00:00
Using the module in libvirt requires adding a ``<qemu:commandline>`` block to
your libvirt XML configuration. That block, in turn, requires modifying the
XML domain namespace. To modify the namespace edit the ``<domain>`` tag at
the top of your XML config to:
.. code:: xml
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
then add one of the ``<qemu:commandline>`` blocks below based on your
QEMU/libvirt versions.
.. note::
Make sure to add both the block and the domain namespace change in a
single editing session prior to saving it. Failure to do so will cause
libvirt to reject the changes.
Starting with QEMU 6.2 and libvirt 7.9, JSON style QEMU configuration is the
default syntax. Users running QEMU 6.2 or later **and** libvirt 7.9 or later,
should use this XML block to configure their VM for kvmfr:
.. code:: xml
<qemu:commandline>
<qemu:arg value='-device'/>
<qemu:arg value='{"driver":"ivshmem-plain","id":"shmem0","memdev":"looking-glass"}'/>
<qemu:arg value='-object'/>
<qemu:arg value='{"qom-type":"memory-backend-file","id":"looking-glass","mem-path":"/dev/kvmfr0","size":33554432,"share":true}'/>
</qemu:commandline>
.. note::
- The ``"size"`` tag represents the size of the shared memory device in
bytes. Once you determine the proper size of the device as per
:ref:`Determining Memory <libvirt_determining_memory>`, use the figure you
got to calculate the size in bytes:
``size_in_MB x 1024 x 1024 = size_in_bytes``
If you are running QEMU older than 6.2 or libvirt older than 7.9, please use
legacy syntax for IVSHMEM setup:
.. code:: xml
<qemu:commandline>
<qemu:arg value='-device'/>
<qemu:arg value='ivshmem-plain,id=shmem0,memdev=looking-glass'/>
<qemu:arg value='-object'/>
<qemu:arg value='memory-backend-file,id=looking-glass,mem-path=/dev/kvmfr0,size=32M,share=yes'/>
</qemu:commandline>
.. note::
2024-03-20 16:58:17 +00:00
Using the legacy syntax on QEMU 6.2/libvirt 7.9 may cause QEMU to
abort with the following error message:
"``error: internal error: ... PCI: slot 1 function 0 not available for pcie-root-port, in use by ivshmem-plain``"
Running libvirt this way violates AppArmor and cgroups policies, which will
block the VM from running. These policies must be amended to allow the VM
to start.
.. tip::
If you are not sure, you likely have cgroups also as this is usually deployed
and configured by default by most distributions when you install libvirt.
AppArmor
2024-03-09 02:36:02 +00:00
^^^^^^^^
Create ``/etc/apparmor.d/local/abstractions/libvirt-qemu`` if it doesn't exist
and add the following:
.. code:: text
# Looking Glass
/dev/kvmfr0 rw,
cgroups
2024-03-09 02:36:02 +00:00
^^^^^^^
Edit the file ``/etc/libvirt/qemu.conf`` and uncomment the ``cgroup_device_acl``
block, adding ``/dev/kvmfr0`` to the list. To make this change active you then
must restart ``libvirtd``
.. code:: bash
sudo systemctl restart libvirtd.service
.. _ivshmem_kvmfr_qemu:
QEMU
2024-03-09 02:36:02 +00:00
~~~~
If you are using QEMU directly without libvirt, add the following arguments to your
``qemu`` command line::
-device ivshmem-plain,id=shmem0,memdev=looking-glass
-object memory-backend-file,id=looking-glass,mem-path=/dev/kvmfr0,size=32M,share=yes
.. note::
The ``size`` argument must be the same size you passed
to the ``static_size_mb`` argument when loading the kernel module.