LookingGlass/module
Quantum 7e58278858 [module] fault in pages one by one for ivshmem devices
It appears that the PCI BAR memory is slow to access with remap_pfn_range
and that it should instead be faulted in one page at a time.

The commit 5774e21965 implemented the former
behaviour and caused a performance regression in the VM->VM case.

This commit retores the old behaviour, but extends it to support mmaping
the kvmfr device directly, without going through a dmabuf.
2021-02-21 14:13:24 +11:00
..
dkms.conf [module] bump the version 2021-01-03 23:42:43 +11:00
kvmfr.c [module] fault in pages one by one for ivshmem devices 2021-02-21 14:13:24 +11:00
kvmfr.h [module] added missing kvmfr.h, fixes #253 2020-03-22 09:20:09 +11:00
Makefile [kvmfr] stop the module building the test application by default 2020-01-29 14:01:52 +11:00
README.md [doc] Add client ini example to kernel module README.md 2021-02-21 10:31:49 +11:00
test.c [module] test mmaping with offsets in test program 2021-02-21 10:31:48 +11:00
test.expected [module] test mmaping with offsets in test program 2021-02-21 10:31:48 +11:00

This kernel module implements a basic interface to the IVSHMEM device for LookingGlass when using LookingGlass in VM->VM mode.

Additionally, in VM->host mode, it can be used to generate a shared memory device on the host machine that supports dmabuf.

Compiling (Manual)

Make sure you have your kernel headers installed first, on Debian/Ubuntu use the following command.

apt-get install linux-headers-$(uname -r)

Then simply run make and you're done.

Loading

For VM->VM mode, simply run:

insmod kvmfr.ko

For VM->host mode with dmabuf, instead of creating a shared memory file, load this module with the parameter static_size_mb. For example, a 128 MB shared memory device can be created with:

insmod kvmfr.ko static_size_mb=128

Multiple devices can be created by separating the sizes with commas. For example, static_size_mb=128,64 would create two kvmfr devices: kvmfr0 would be 128 MB and kvmfr1 would be 64 MB.

Compiling & Installing (DKMS)

You can install this module into DKMS so that it persists across kernel upgrades. Simply run:

dkms install .

Loading

For VM->VM, simply modprobe the module:

modprobe kvmfr

For VM->host with dmabuf, modprobe with the parameter static_size_mb:

modprobe kvmfr static_size_mb=128

Just like above, multiple devices can be created by separating the sizes with commas.

Usage

This will create the /dev/kvmfr0 node that represents the KVMFR interface. To use the interface you need permission to access it by either creating a udev rule to ensure your user can read and write to it, or simply change its ownership manually, ie:

sudo chown user:user /dev/kvmfr0

An example udev rule, which you can put in /etc/udev/rules.d/99-kvmfr.rules, is (replace user with your username):

SUBSYSTEM=="kvmfr", OWNER="user", GROUP="kvm", MODE="0660"

Usage with looking glass is simple, you only need to specify the path to the device node, for example:

./looking-glass-client -f /dev/kvmfr0

You may also use a config file: ~/.looking-glass-client.ini, or /etc/looking-glass-client.ini.

[app]
shmFile=/dev/kvmfr0

VM->Host

In VM->host mode, use this device in place of the shared memory file.

For example, with qemu, you would use the following arguments:

-device ivshmem-plain,id=shmem0,memdev=looking-glass
-object memory-backend-file,id=looking-glass,mem-path=/dev/kvmfr0,size=128M,share=yes

Note that the size argument must be the same size as what you passed to static_size_mb argument for the kernel module.

libvirt

With libvirt, you can use the following XML block:

<qemu:commandline>
  <qemu:arg value='-device'/>
  <qemu:arg value='ivshmem-plain,id=shmem0,memdev=looking-glass'/>
  <qemu:arg value='-object'/>
  <qemu:arg value='memory-backend-file,id=looking-glass,mem-path=/dev/kvmfr0,size=128M,share=yes'/>
</qemu:commandline>

Remember to add xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0' to the <domain>.

On certain distros, running libvirt this way poses issues with apparmor and cgroups.

For apparmor, in /etc/apparmor.d/abstractions/libvirt-qemu, append:

# Looking Glass
/dev/kvmfr0 rw,

For cgroups, in /etc/libvirt/qemu.conf, uncomment the cgroup_device_acl block and add /dev/kvmfr0 to the list. Then restart libvirtd:

sudo systemctl restart libvirtd.service