mirror of
https://github.com/gnif/LookingGlass.git
synced 2024-11-10 00:28:20 +00:00
[misc] removed README.md before release as it needs rewriting
This commit is contained in:
parent
9f790ff51d
commit
6b5649a648
216
README.md
216
README.md
@ -1,216 +0,0 @@
|
||||
Looking Glass
|
||||
=============
|
||||
|
||||
Purpose
|
||||
-------
|
||||
|
||||
To provide a low latency KVM Guest client that relies on video capture rather
|
||||
than the output of a virtual VGA device. The goal of this project is to allow
|
||||
lossless zero latency display of a guest on the host on the local host.
|
||||
|
||||
The Problem
|
||||
-----------
|
||||
|
||||
Exising solutions such as Spice are designed to work with a virtual VGA
|
||||
adaptor, but when a physical VGA device is passed into the guest VM the
|
||||
display can not be presented to the host, instead a physical monitor must be
|
||||
attached to the physcal VGA device. While this may be acceptible to some it
|
||||
requires a physical switch between the host and guest VM preventing the VM
|
||||
from being inlined into the host window manager.
|
||||
|
||||
Existing Solutions
|
||||
------------------
|
||||
|
||||
Some have reported that they are able to stream using Steam InHouse Streaming
|
||||
to stream the desktop back to the host by launching a program such as notepad
|
||||
in the guest, and then tabbing to desktop. While this works Steam is designed
|
||||
to stream this over a local area network and as such employs technologies such
|
||||
as compression and packetization for network transmission, there are several
|
||||
issues with this solution:
|
||||
|
||||
* Latency from guest to client is generally no less then 50-80ms, this is
|
||||
very noticable when using a cursor input or running games that demand high
|
||||
precision and fast input.
|
||||
|
||||
* Compressing the stream adds additional CPU/GPU overhead that can degrade
|
||||
performance.
|
||||
|
||||
* Compressing converts to the YUV422P colorspace which degrades quality
|
||||
quite substantially.
|
||||
|
||||
* The stream is tuned for transmission over a network which not only adds
|
||||
additional overheads, it also adds additional buffers that increase
|
||||
latency.
|
||||
|
||||
Our Solution
|
||||
------------
|
||||
|
||||
Qemu contains a virtual device that allows mapping of shared memory between
|
||||
host and guest called 'ivshmem'. This device has been used for very specific
|
||||
cases where extremely high bandwidth low latency networking/firewalls have
|
||||
been required and as such a windows driver was never written for this device.
|
||||
|
||||
We developed a Windows driver for this device and have worked with Red Hat to
|
||||
have it included into the official [VirtIO windows driver repository][1].
|
||||
|
||||
The shared memory segment is intialized by the guest with a strcture that
|
||||
describes the format of the frame and the current mouse coordinates. Frame data
|
||||
is appended to the buffer after this header.
|
||||
|
||||
We have decided upon he name "KVM FrameRelay", or "KVMFR" for this technology.
|
||||
|
||||
### Video Transfer
|
||||
|
||||
The guest runs a program that uses the ivshmem device as a direct transfer
|
||||
between the guest and host. This application uses accelerated capture methods
|
||||
such as NvFBC or DXGI to capture the guests framebuffer. This captured frame is
|
||||
then copied into the shared memory region. After the frame has been copied a
|
||||
doorbell is set in ivshmem which the host client application is waiting on,
|
||||
after which the client application waits for an IRQ from the host to state
|
||||
that the frame has been processed.
|
||||
|
||||
After the host client application is notified there is a new frame it maps the
|
||||
frame to the host video card by means of a SDL streaming texture. Once a copy
|
||||
of the frame has been completed the host client then notifies the guest by means
|
||||
of raising the IRQ the guest is waiting on.
|
||||
|
||||
At this time the guest still requires a monitor or dummy device attached that
|
||||
presents valid EDID information for the resolutions that are desired. There
|
||||
may be a way around this but that is not the goal of this project at this time
|
||||
and is thus out of scope. This author found it easiest to just use the
|
||||
secondary input to one of his monitors to present the EDID information.
|
||||
|
||||
### Keyboard and Mouse Input
|
||||
|
||||
The host client implements the Spice protocol and is able to send mouse and
|
||||
keyboard events over the existing communications channels provided by qemu.
|
||||
|
||||
If using libvirt the virtual tablet device must be removed from the host as
|
||||
this application expects to and requires to use relative positioning for all
|
||||
interactions. If the tablet device exists qemu will give priority to it for
|
||||
click events which are unreliable when used in this way.
|
||||
|
||||
There are also some issues with the qemu PS2 controller that will cause mouse
|
||||
input lockups when keyboard events and mouse events occur at the same time.
|
||||
The virtual USB keyboard and mouse inputs while are somewhat more reliable also
|
||||
suffer random dropouts at high rates due to USB hub emulation timing issues.
|
||||
|
||||
At the time of writing this we have found the best solution is to use the PS2
|
||||
mouse and the VirtIO Keyboard device.
|
||||
|
||||
The guest program returns the current mouse position as part of each frame,
|
||||
this is so that it is possible to syncronise the host mouse position in the
|
||||
window to that reported by the guest.
|
||||
|
||||
We recommend that the guest have the [mouse acceleration disabled][2] so that
|
||||
the guest's mouse position is 1:1 with the host. This makes for a seemless
|
||||
experience when the cursor is moved inside and outside of the client
|
||||
application on the host.
|
||||
|
||||
### Audio
|
||||
|
||||
This client doesn't implement any form of audio, the guest needs to be
|
||||
configured to use the hosts for audio directly. At this time we recommend using
|
||||
pulseaudio as the ALSA implementation seems to be problematic and requires
|
||||
some attention.
|
||||
|
||||
Testing
|
||||
-------
|
||||
|
||||
Our test scenario involved running a physical monitor along side the guest
|
||||
client, at this time we have only tested NvFBC as the capture technology on the
|
||||
guest.
|
||||
|
||||
### Hardware
|
||||
|
||||
| | |
|
||||
|-----------|------------------------------------------------------------|
|
||||
| MB | Asrock AB350 Pro4 |
|
||||
| CPU | AMD Ryzen 7 1700 @ 3.8GHz |
|
||||
| RAM | 16GB DDR4 |
|
||||
| Host GPU | NVidia GTX 1080 Ti running 4 monitors @ 1920x1200 |
|
||||
| Guest GPU | NVidia GTX 680 [modified to a Quadro K5000][3] @ 1920x1200 |
|
||||
|
||||
### Software
|
||||
|
||||
| | |
|
||||
|-------|----------------------------------------------------|
|
||||
| Host | Debian 9 + [NPT patch][4] |
|
||||
| Guest | Windows 10 running in KVM on the i440 device tree. |
|
||||
|
||||
Results
|
||||
-------
|
||||
|
||||
Latency has not yet been measured, but in practice to this author it is
|
||||
extremely hard to notice any latency between the physical display and host
|
||||
client output when streaming 24bit RGB. When streaming YUV422P there no
|
||||
percevable latency at all, this indicates that the memory copy is the slowest
|
||||
part of the process. To combat this multi-threaded memory copy has been
|
||||
implemented on both the host and guest ends of the transfer.
|
||||
|
||||
At this time we are waiting for a signed ivshmem driver by Red Hat to be
|
||||
released but since this may take quite some time we are currently in the
|
||||
process of obtaining a driver signing certificate.
|
||||
|
||||
Status
|
||||
------
|
||||
|
||||
The host software is considered 90% complete for devices that are capable of
|
||||
NvFBC and has been written in such as way to allow for other capture interfaces
|
||||
to be easily added to support other platforms.
|
||||
|
||||
The client software is considered 90% complete, features such as copy & paste
|
||||
between client and host are missing and it would be nice if it could also
|
||||
serve as the ivshmem-server. The client is able to support a variaty of pixel
|
||||
formats types depending on the capability and performance of the capture API
|
||||
that is in use.
|
||||
|
||||
Security
|
||||
--------
|
||||
|
||||
This project has not been written with security of the shared memory in mind,
|
||||
it is certainly possible at this time for the client to abuse the headers and
|
||||
input invalid information that could be used to compromise the host system.
|
||||
|
||||
At this time this is not a concern as our use case of this application at this
|
||||
time is in a completely trusted environment, but before it is more widely put
|
||||
into use this will need to be addressed.
|
||||
|
||||
Improvements
|
||||
------------
|
||||
|
||||
It may be possible to avoid the additional copy by handing the guest a block
|
||||
of memory directly from the hosts graphics device. This would allow the guest
|
||||
to stream directly into the texture on the host. This however could be another
|
||||
avenue of escape for malicious code on the guest and would need very careful
|
||||
consideration before any attempt is made to implement this.
|
||||
|
||||
To achieve the above it would be required to modify qemu integrating the
|
||||
features of this client directly so that a SDL\_Texture's RAM could be mapped
|
||||
directly into the guest.
|
||||
|
||||
License
|
||||
-------
|
||||
|
||||
~~~
|
||||
KVMGFX Client - A KVM Client for VGA Passthrough
|
||||
Copyright (C) 2017 Geoffrey McRae <geoff@hostfission.com>
|
||||
|
||||
This program is free software; you can redistribute it and/or modify it under
|
||||
the terms of the GNU General Public License as published by the Free Software
|
||||
Foundation; either version 2 of the License, or (at your option) any later
|
||||
version.
|
||||
|
||||
This program is distributed in the hope that it will be useful, but WITHOUT ANY
|
||||
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. See the GNU General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU General Public License along with
|
||||
this program; if not, write to the Free Software Foundation, Inc., 59 Temple
|
||||
Place, Suite 330, Boston, MA 02111-1307 USA
|
||||
~~~
|
||||
|
||||
[1]: https://github.com/virtio-win/kvm-guest-drivers-windows/tree/master/ivshmem
|
||||
[2]: http://donewmouseaccel.blogspot.com.au/2010/03/markc-windows-7-mouse-acceleration-fix.html
|
||||
[3]: http://www.eevblog.com/forum/chat/hacking-nvidia-cards-into-their-professional-counterparts/
|
||||
[4]: https://patchwork.kernel.org/patch/10027525/
|
Loading…
Reference in New Issue
Block a user