mirror of
https://github.com/gnif/LookingGlass.git
synced 2024-11-22 05:27:20 +00:00
[misc] removed README.md before release as it needs rewriting
This commit is contained in:
Geoffrey McRae
parent
9f790ff51d
commit
6b5649a648
216
README.md
216
README.md
@ -1,216 +0,0 @@
|
|||||||
Looking Glass
|
|
||||||
=============
|
|
||||||
|
|
||||||
Purpose
|
|
||||||
-------
|
|
||||||
|
|
||||||
To provide a low latency KVM Guest client that relies on video capture rather
|
|
||||||
than the output of a virtual VGA device. The goal of this project is to allow
|
|
||||||
lossless zero latency display of a guest on the host on the local host.
|
|
||||||
|
|
||||||
The Problem
|
|
||||||
-----------
|
|
||||||
|
|
||||||
Exising solutions such as Spice are designed to work with a virtual VGA
|
|
||||||
adaptor, but when a physical VGA device is passed into the guest VM the
|
|
||||||
display can not be presented to the host, instead a physical monitor must be
|
|
||||||
attached to the physcal VGA device. While this may be acceptible to some it
|
|
||||||
requires a physical switch between the host and guest VM preventing the VM
|
|
||||||
from being inlined into the host window manager.
|
|
||||||
|
|
||||||
Existing Solutions
|
|
||||||
------------------
|
|
||||||
|
|
||||||
Some have reported that they are able to stream using Steam InHouse Streaming
|
|
||||||
to stream the desktop back to the host by launching a program such as notepad
|
|
||||||
in the guest, and then tabbing to desktop. While this works Steam is designed
|
|
||||||
to stream this over a local area network and as such employs technologies such
|
|
||||||
as compression and packetization for network transmission, there are several
|
|
||||||
issues with this solution:
|
|
||||||
|
|
||||||
* Latency from guest to client is generally no less then 50-80ms, this is
|
|
||||||
very noticable when using a cursor input or running games that demand high
|
|
||||||
precision and fast input.
|
|
||||||
|
|
||||||
* Compressing the stream adds additional CPU/GPU overhead that can degrade
|
|
||||||
performance.
|
|
||||||
|
|
||||||
* Compressing converts to the YUV422P colorspace which degrades quality
|
|
||||||
quite substantially.
|
|
||||||
|
|
||||||
* The stream is tuned for transmission over a network which not only adds
|
|
||||||
additional overheads, it also adds additional buffers that increase
|
|
||||||
latency.
|
|
||||||
|
|
||||||
Our Solution
|
|
||||||
------------
|
|
||||||
|
|
||||||
Qemu contains a virtual device that allows mapping of shared memory between
|
|
||||||
host and guest called 'ivshmem'. This device has been used for very specific
|
|
||||||
cases where extremely high bandwidth low latency networking/firewalls have
|
|
||||||
been required and as such a windows driver was never written for this device.
|
|
||||||
|
|
||||||
We developed a Windows driver for this device and have worked with Red Hat to
|
|
||||||
have it included into the official [VirtIO windows driver repository][1].
|
|
||||||
|
|
||||||
The shared memory segment is intialized by the guest with a strcture that
|
|
||||||
describes the format of the frame and the current mouse coordinates. Frame data
|
|
||||||
is appended to the buffer after this header.
|
|
||||||
|
|
||||||
We have decided upon he name "KVM FrameRelay", or "KVMFR" for this technology.
|
|
||||||
|
|
||||||
### Video Transfer
|
|
||||||
|
|
||||||
The guest runs a program that uses the ivshmem device as a direct transfer
|
|
||||||
between the guest and host. This application uses accelerated capture methods
|
|
||||||
such as NvFBC or DXGI to capture the guests framebuffer. This captured frame is
|
|
||||||
then copied into the shared memory region. After the frame has been copied a
|
|
||||||
doorbell is set in ivshmem which the host client application is waiting on,
|
|
||||||
after which the client application waits for an IRQ from the host to state
|
|
||||||
that the frame has been processed.
|
|
||||||
|
|
||||||
After the host client application is notified there is a new frame it maps the
|
|
||||||
frame to the host video card by means of a SDL streaming texture. Once a copy
|
|
||||||
of the frame has been completed the host client then notifies the guest by means
|
|
||||||
of raising the IRQ the guest is waiting on.
|
|
||||||
|
|
||||||
At this time the guest still requires a monitor or dummy device attached that
|
|
||||||
presents valid EDID information for the resolutions that are desired. There
|
|
||||||
may be a way around this but that is not the goal of this project at this time
|
|
||||||
and is thus out of scope. This author found it easiest to just use the
|
|
||||||
secondary input to one of his monitors to present the EDID information.
|
|
||||||
|
|
||||||
### Keyboard and Mouse Input
|
|
||||||
|
|
||||||
The host client implements the Spice protocol and is able to send mouse and
|
|
||||||
keyboard events over the existing communications channels provided by qemu.
|
|
||||||
|
|
||||||
If using libvirt the virtual tablet device must be removed from the host as
|
|
||||||
this application expects to and requires to use relative positioning for all
|
|
||||||
interactions. If the tablet device exists qemu will give priority to it for
|
|
||||||
click events which are unreliable when used in this way.
|
|
||||||
|
|
||||||
There are also some issues with the qemu PS2 controller that will cause mouse
|
|
||||||
input lockups when keyboard events and mouse events occur at the same time.
|
|
||||||
The virtual USB keyboard and mouse inputs while are somewhat more reliable also
|
|
||||||
suffer random dropouts at high rates due to USB hub emulation timing issues.
|
|
||||||
|
|
||||||
At the time of writing this we have found the best solution is to use the PS2
|
|
||||||
mouse and the VirtIO Keyboard device.
|
|
||||||
|
|
||||||
The guest program returns the current mouse position as part of each frame,
|
|
||||||
this is so that it is possible to syncronise the host mouse position in the
|
|
||||||
window to that reported by the guest.
|
|
||||||
|
|
||||||
We recommend that the guest have the [mouse acceleration disabled][2] so that
|
|
||||||
the guest's mouse position is 1:1 with the host. This makes for a seemless
|
|
||||||
experience when the cursor is moved inside and outside of the client
|
|
||||||
application on the host.
|
|
||||||
|
|
||||||
### Audio
|
|
||||||
|
|
||||||
This client doesn't implement any form of audio, the guest needs to be
|
|
||||||
configured to use the hosts for audio directly. At this time we recommend using
|
|
||||||
pulseaudio as the ALSA implementation seems to be problematic and requires
|
|
||||||
some attention.
|
|
||||||
|
|
||||||
Testing
|
|
||||||
-------
|
|
||||||
|
|
||||||
Our test scenario involved running a physical monitor along side the guest
|
|
||||||
client, at this time we have only tested NvFBC as the capture technology on the
|
|
||||||
guest.
|
|
||||||
|
|
||||||
### Hardware
|
|
||||||
|
|
||||||
| | |
|
|
||||||
|-----------|------------------------------------------------------------|
|
|
||||||
| MB | Asrock AB350 Pro4 |
|
|
||||||
| CPU | AMD Ryzen 7 1700 @ 3.8GHz |
|
|
||||||
| RAM | 16GB DDR4 |
|
|
||||||
| Host GPU | NVidia GTX 1080 Ti running 4 monitors @ 1920x1200 |
|
|
||||||
| Guest GPU | NVidia GTX 680 [modified to a Quadro K5000][3] @ 1920x1200 |
|
|
||||||
|
|
||||||
### Software
|
|
||||||
|
|
||||||
| | |
|
|
||||||
|-------|----------------------------------------------------|
|
|
||||||
| Host | Debian 9 + [NPT patch][4] |
|
|
||||||
| Guest | Windows 10 running in KVM on the i440 device tree. |
|
|
||||||
|
|
||||||
Results
|
|
||||||
-------
|
|
||||||
|
|
||||||
Latency has not yet been measured, but in practice to this author it is
|
|
||||||
extremely hard to notice any latency between the physical display and host
|
|
||||||
client output when streaming 24bit RGB. When streaming YUV422P there no
|
|
||||||
percevable latency at all, this indicates that the memory copy is the slowest
|
|
||||||
part of the process. To combat this multi-threaded memory copy has been
|
|
||||||
implemented on both the host and guest ends of the transfer.
|
|
||||||
|
|
||||||
At this time we are waiting for a signed ivshmem driver by Red Hat to be
|
|
||||||
released but since this may take quite some time we are currently in the
|
|
||||||
process of obtaining a driver signing certificate.
|
|
||||||
|
|
||||||
Status
|
|
||||||
------
|
|
||||||
|
|
||||||
The host software is considered 90% complete for devices that are capable of
|
|
||||||
NvFBC and has been written in such as way to allow for other capture interfaces
|
|
||||||
to be easily added to support other platforms.
|
|
||||||
|
|
||||||
The client software is considered 90% complete, features such as copy & paste
|
|
||||||
between client and host are missing and it would be nice if it could also
|
|
||||||
serve as the ivshmem-server. The client is able to support a variaty of pixel
|
|
||||||
formats types depending on the capability and performance of the capture API
|
|
||||||
that is in use.
|
|
||||||
|
|
||||||
Security
|
|
||||||
--------
|
|
||||||
|
|
||||||
This project has not been written with security of the shared memory in mind,
|
|
||||||
it is certainly possible at this time for the client to abuse the headers and
|
|
||||||
input invalid information that could be used to compromise the host system.
|
|
||||||
|
|
||||||
At this time this is not a concern as our use case of this application at this
|
|
||||||
time is in a completely trusted environment, but before it is more widely put
|
|
||||||
into use this will need to be addressed.
|
|
||||||
|
|
||||||
Improvements
|
|
||||||
------------
|
|
||||||
|
|
||||||
It may be possible to avoid the additional copy by handing the guest a block
|
|
||||||
of memory directly from the hosts graphics device. This would allow the guest
|
|
||||||
to stream directly into the texture on the host. This however could be another
|
|
||||||
avenue of escape for malicious code on the guest and would need very careful
|
|
||||||
consideration before any attempt is made to implement this.
|
|
||||||
|
|
||||||
To achieve the above it would be required to modify qemu integrating the
|
|
||||||
features of this client directly so that a SDL\_Texture's RAM could be mapped
|
|
||||||
directly into the guest.
|
|
||||||
|
|
||||||
License
|
|
||||||
-------
|
|
||||||
|
|
||||||
~~~
|
|
||||||
KVMGFX Client - A KVM Client for VGA Passthrough
|
|
||||||
Copyright (C) 2017 Geoffrey McRae <geoff@hostfission.com>
|
|
||||||
|
|
||||||
This program is free software; you can redistribute it and/or modify it under
|
|
||||||
the terms of the GNU General Public License as published by the Free Software
|
|
||||||
Foundation; either version 2 of the License, or (at your option) any later
|
|
||||||
version.
|
|
||||||
|
|
||||||
This program is distributed in the hope that it will be useful, but WITHOUT ANY
|
|
||||||
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
|
|
||||||
PARTICULAR PURPOSE. See the GNU General Public License for more details.
|
|
||||||
|
|
||||||
You should have received a copy of the GNU General Public License along with
|
|
||||||
this program; if not, write to the Free Software Foundation, Inc., 59 Temple
|
|
||||||
Place, Suite 330, Boston, MA 02111-1307 USA
|
|
||||||
~~~
|
|
||||||
|
|
||||||
[1]: https://github.com/virtio-win/kvm-guest-drivers-windows/tree/master/ivshmem
|
|
||||||
[2]: http://donewmouseaccel.blogspot.com.au/2010/03/markc-windows-7-mouse-acceleration-fix.html
|
|
||||||
[3]: http://www.eevblog.com/forum/chat/hacking-nvidia-cards-into-their-professional-counterparts/
|
|
||||||
[4]: https://patchwork.kernel.org/patch/10027525/
|
|
||||||
Loading…
Reference in New Issue
Block a user