Commit Graph

106 Commits

Author SHA1 Message Date
Geoffrey McRae
78b8e2a73c [host] windows: make D3DKMTSetProcessSchedulingPriorityClass global
Testing shows that `D3DKMTSetProcessSchedulingPriorityClass` has a
positive performance impact for NvFBC as well as DXGI, as such always
try to boost the priority for the windows host.
2021-07-10 12:27:30 +10:00
Geoffrey McRae
041b95507d [host] windows/nvfbc/common: strip out broken "enhanced" event logic
This so called "enhanced" event logic is completely flawed and can never
work correctly, better to strip it out and put our faith in windows to
handle the events for us.

And yes, I am fully aware I wrote the utter trash in the first place :)
2021-07-09 10:22:03 +10:00
Geoffrey McRae
1effd5fddc [common] debug: add color support to debug messages
People often miss the warnings about invalid arguments in their command
line, this last minute patch attempts to address this by making
warnings, errors, fixme's and fatal errors stand out if stdout is a TTY.
2021-07-07 23:05:46 +10:00
Geoffrey McRae
cd10e02862 Revert "[host] avoid manual windres command"
This reverts commit d82f2e510d.
While the proposed change is more correct, it breaks the generation of
the file due to failure to locate the resource files, such as
`resources/icon.ico`.
2021-07-04 19:30:06 +10:00
arcnmx
d82f2e510d [host] avoid manual windres command 2021-07-03 04:28:28 +10:00
Geoffrey McRae
2160dee23a [host] linux: implement new truncated frame support 2021-06-12 19:10:32 +10:00
Geoffrey McRae
d36c4f0e83 [host] kvmfr: allow the frame size to exceed the available memory
This change allows the host to still transmit a frame that is truncated
if the IVSHMEM size is too small to allow for a full frame.
2021-06-12 18:44:28 +10:00
Geoffrey McRae
f02d61d665 [host] dxgi: sleep until it's close to time to map
This change adds an average function to time how long it takes the GPU
to copy and map the texture, and then uses this average to sleep for 80%
of this average lowering CPU usage and potentially decreasing lock
contention.
2021-06-06 12:26:36 +10:00
Quantum
24d0aa0c18 [all] normalize copyright on all source files 2021-06-06 11:53:05 +10:00
Geoffrey McRae
fcf6abc7c6 [host] NvFBC/DXGI: make DXGI the default instead of the fallback
It has been detemined that a failure to init NvFBC causes a 20-30%
performance penalty on non NvFBC supported hardware (GeForce) when using
DXGI, as such reverse the order and default to using DXGI as our first
option.

If NvFBC is still desired, pr #500 added the option `app:capture` which
can be used to force NvFBC.
2021-06-06 06:14:24 +10:00
Geoffrey McRae
0d9b0bd367 [host] dxgi: increase maxTextures default to 4
Testing shows that at high frame rates the default of 3 is hampering
performance, increasing this to 4 yields a substantial performance
improvement.
2021-06-06 01:35:00 +10:00
Geoffrey McRae
87568f97eb [host] linux: add missing library to fix compilation 2021-06-04 12:44:21 +10:00
Geoffrey McRae
0b4e98881f [host] app: add os_showMessage for Linux
Linux doesn't have a generic way to show a dialog box, so we just use
DEBUG_INFO to print the message out to the console.
2021-06-04 12:39:11 +10:00
Geoffrey McRae
fb1b30b728 [host] app: calculate and report the required IVSHMEM size
One of the most common issues reported in the support channels is the
IVSHMEM size being too small. This change adds a calculation to
determine an optimal size and uses the new `os_showMessage` platform
method to display a message box to the user with the error.
2021-06-04 12:31:15 +10:00
Quantum
ec81a353c2 [host] nvfbc: fix null dereference in mouse hook handler
Since we now let the mouse hook linger until the process is killed, the
cursor event that the hook signals may now be null, as the capture could
have stopped. If the hook fires during this time, a crash occurs.
2021-04-29 11:53:19 +10:00
Quantum
6a1ec9420e [host] service: compare SIDs directly without string conversion
Instead of converting every SID to string with ConvertSidToStringSidA
and compare it with the magical SID string for local system with strcmp,
we could instead create the local system SID and compare directly with
EqualSid.
2021-04-29 11:52:23 +10:00
Quantum
93d97424df [host] service: disable handle inheritance when spawning host
We don't actually have any handles that should be inherited, so specifying
TRUE for bInheritHandles to CreateProcessAsUserA is pointless.

Furthermore, according to MSDN, "[y]ou cannot inherit handles across
sessions," and we are spawning the host in a different session, so this
is even more pointless.
2021-04-29 11:51:41 +10:00
Quantum
2afad4e1be [host] service: disable privileges as soon as they are not needed
It is usually considered good practice to enable elevated privileges
for the shortest duration possible.
2021-04-29 11:51:30 +10:00
Quantum
5bfb33c739 [host] windows: re-implement open log safely
Instead of doing ShellExecute from the service, we instead get the token
of the currently logged in user, and do CreateProcessAsUserA to run
notepad with that token. This should be safe.
2021-03-22 09:06:07 +11:00
Quantum
a089c4ea32 [host] service: introduce fatal errors for ivshmem failures
Also for failure to parse command line. For these errors, restarting
with exponential backoff will not help: no amount of restarting the
service could possibly make the ivshmem device exist or larger, so
we shouldn't try.
2021-03-22 08:54:58 +11:00
Quantum
c12c6ea3c7 [host] service: retry a few times when host fails to restart
Certain users of Radeon cards have observed that the host fails to start
at boot, with D3D11CreateDevice failing with HSTATUS 0x887a0004, which
translates to "The specified device interface or feature level is not
supported on this system."

This failure results in a LG_HOST_EXIT_FAILED exit code, which the service
does not attempt to restart. The user has to manually restart the service
for the host application to work.

These users reported that the host application started fine on
B2. This strongly suggests that the fix to enable capturing the login
screen made the host application start too early during the boot process,
and the graphics driver did not have time to initialize fully.

This PR allows the service to retry a few times on LG_HOST_EXIT_FAILED,
with exponential backoff, before giving up. This should cover this bug
and other similar bugs related to the early initialization which I do not
have logs for.
2021-03-22 08:54:58 +11:00
Quantum
a4290b290b [host] cmake: link correctly against libxcb-xfixes.so
We are using <xcb/xfixes.h>, so it makes sense to link against the
xcb library instead of libXfixes.so directly.
2021-03-15 12:12:48 +11:00
Quantum
f2c0b8c0b4 [host] allow user to select capture backend
This commit introduces a new option, app:capture, which can be set to
either DXGI or NvFBC to force the host application to use that backend.

This is very useful for testing DXGI on Quadro cards, which would default
to running with NvFBC.
2021-02-27 17:41:44 +11:00
Quantum
8e98f863b6 [host] windows: detect whether screensaver is disabled in the guest
This will allow us to add an option to disable the screensaver on the client
when an application in the guest requests it. This behaviour may be useful
when the guest is doing media playback.
2021-02-21 10:31:48 +11:00
Quantum
ff0a859ceb [host] nvfbc: avoid recreating mouse hook and 1x1 window
The mouse hook code is very fragile, and we would like to avoid unhooking
and re-hooking as much as possible.

After this commit, this is done only once, and the hook and 1x1 window is
only destroyed upon exit. This, of course, comes with the downside of
the slight performance penalty if the guest machine is used directly while
the host is running and the client is not running.
2021-01-31 12:17:14 +11:00
Quantum
1b48ac842a [host] nvfbc: fix resource leak when pointer thread creation fails
Moving NvFBCToSysSetup to nvfbc_init means that when the pointer thread
fails to be created, NvFBCToSysRelease needs to be called.

To resolve such cleanup issues in the future, we instead call nvfbc_deinit,
which should cleanup everything that needs to be cleaned up. fails.
2021-01-31 11:21:24 +11:00
Quantum
a702c912ae [host] nvfbc: move NvFBCToSysCreate into nvfbc_init
When NvFBCToSysCapture reports recreation is required, we return
CAPTURE_RESULT_REINIT, which eventually calls nvfbc_deinit and then
nvfbc_init.

However, the NvFBC object is actually created in nvfbc_create, which
means the NvFBC object is never actually recreated. The result is an
endless cycle of NvFBC asking for recreation. This commonly manifests
as the client waiting endlessly for the host when the guest machine
reboots.

In this commit, the NvFBC object creation is moved into nvfbc_init,
and when recreation is required, it will actually be recreated.
2021-01-31 10:57:51 +11:00
Quantum
acc3298344 [host] nvfbc: cleanup threads created by nvfbc_init on failure
mouseHook_install and dwmForceComposition both create threads, but these
are only freed in nvfbc_deinit which is not called if nvfbc_init fails.
These should be freed if the pointer thread fails to be created, as
nothing else could be cleaning it up.
2021-01-31 09:57:07 +11:00
Quantum
c91b7f647d [host] installer: create start menu shortcut to log directory
This commit makes the installer create a shortcut to the log directory
introduced by the previous commit.
2021-01-29 15:56:01 +11:00
Quantum
1761ea2b9b [host] windows: move log path to %ProgramData%\Looking Glass (host)
Instead of using %windir%\Temp, which is not accessible by default and
contains a lot of unrelated files, as the location for our log files,
this commit moves it to %ProgramData%\Looking Glass (host), which will
be a dedicated directory just for the LG host log files. This applies
to both the host application logs and the service logs.

Also, we now switched to using PathCombineA from shlwapi.dll instead
of using snprintf, which greatly simplifies the code. PathCombineA
guarantees that the path would not overflow a buffer of MAX_PATH.
2021-01-29 15:56:01 +11:00
Quantum
fb916cbac1 [host] nvfbc: always update cursor shape on startup
This ensures that the top-left position of the cursor sprite is correctly
computed.
2021-01-28 11:18:02 +11:00
Quantum
b97130cf20 [host] nvfbc: generate cursor position update on startup
Before this commit, the NvFBC backend only generated the first cursor
position update when the mouse moves. Therefore, if the user does
not move the mouse, the cursor will be shown at (0, 0), which is not
ideal.

This commit changes this behaviour to unconditionally generate a
cursor update when the mouse hook initializes.
2021-01-28 11:18:02 +11:00
Geoffrey McRae
e2e49bce13 [host] service: fix possible use of unitialized variable 2021-01-27 01:23:58 +11:00
Geoffrey McRae
0d7be70b56 [host] dxgi: fix maybe uninitialized warning 2021-01-27 01:21:06 +11:00
Geoffrey McRae
6b0699e664 [host] installer: include the debug PDB if it is available 2021-01-26 22:55:25 +11:00
Quantum
f9ec32b255 [host] service: disable buffering on the log file
Before this change, the log is buffered, so if the host application exits
for any reason, it usually would not show up in the log file immediately,
and the service has to be restarted for the logs to be flushed.

This commit disables the buffering so that any log entries shows up
immediately.
2021-01-25 09:35:03 +11:00
Quantum
d610aaf2cf [host] nvfbc: update cursor position on shape change
This is because we keep track of the top-left corner of the cursor, not
the location of the hotspot. When the cursor shape changes, the hotspot
location may also change. When it does, the position of the top-left
corner changes and requires an update.

In the case that we do not have the current cursor position, which
happens on startup, we do not generate this update.
2021-01-23 20:37:09 +11:00
Geoffrey McRae
04774d9cd6 [host] fix faults caused by improper startup/shudown/restart ordering 2021-01-21 17:05:30 +11:00
Geoffrey McRae
6b8161972d [host] nvfbc: prevent possible double free 2021-01-21 16:28:20 +11:00
Geoffrey McRae
98ea8b0bb8 [host] nvfbc: remove invalid close of the HMONITOR handle 2021-01-21 16:17:24 +11:00
Geoffrey McRae
536df254e0 [host] fix the return code for the platform specific init 2021-01-21 15:44:19 +11:00
Geoffrey McRae
e926bad759 [host] dont overwrite the exit code on shutdown 2021-01-21 15:14:50 +11:00
Geoffrey McRae
ad9e84eaaa [host] return a proper exit code 2021-01-21 15:07:19 +11:00
Quantum
ffa72c7992 [host] nvfbc: force composition to capture some full screen apps
NvFBC is unable to capture certain applications that bypasses the DWM
compositor, for example, Firefox playing video in full screen. This
has been a known issue for a long time with Nvidia's ShadowPlay, see:
* https://www.nvidia.com/en-us/geforce/forums/geforce-experience/14/233709/
* https://crbug.com/609857

Nvidia won't fix this, but there are workarounds. For example, we
create a transparent 1x1 layered window, which forces desktop composition
to be enabled.

Note that SetLayeredWindowAttributes also supports alpha-based transparency,
but setting transparency to 0 will cause DWM to skip composition. We could
use a transparency of 1, but this ruins the image by the slightest bit,
which is unacceptable. Therefore, we must use chroma key-based
transparency, which tricks DWM into compositing despite being fully
transparent.
2021-01-21 12:14:03 +11:00
Geoffrey McRae
cac454d9cf [host] dxgi: reverse the rotation angle.
This is undocumented however testing yields that DXGI DD reports the
inverse rotation. Research shows that this is because of a difference in
coordiate spaces.

Ref: https://docs.microsoft.com/en-us/windows/uwp/gaming/supporting-screen-rotation-directx-and-cpp
2021-01-18 15:15:36 +11:00
Geoffrey McRae
f5587b6b6b [host] all: pass back the desktop rotation to the client 2021-01-18 13:53:29 +11:00
Quantum
95e1b48f83 [host] windows: make mousehook.c work on secure desktop
Basically, this creates a separate thread for the mouse events, and this
thread detects that the desktop has changed (say to the secure desktop),
and unhooks, switches to the new desktop, and then rehooks.

This allows the cursor location to be updated while using NvFBC on secure
desktop and the login screen.
2021-01-17 13:23:29 +11:00
Quantum
fe835b98d5 [host] windows: sleep for 1 second instead of 1 millisecond
This is definitely supposed to sleep for 1 second.
1 ms is basically no throttling.
2021-01-17 02:58:27 +11:00
Quantum
c5c43d99f3 [host] windows: allow capture of login screen
WTSGetActiveConsoleSessionId will return a session even if it's not logged in,
unlike our old GetInteractiveSessionID function. Launching looking glass on
such a console session will allow the login screen to be captured.

Note that WTSGetActiveConsoleSessionId() will return 0xFFFFFFFF if there are
no sessions attached.
2021-01-17 02:56:29 +11:00
Quantum
4858bb5899 [host] windows: avoid quoting issues with CreateProcessAsUserA
To quote MSDN documentation:

> The lpApplicationName parameter can be NULL, in which case the executable
> name must be the first white space–delimited string in lpCommandLine. If
> the executable or path name has a space in it, there is a risk that a
> different executable could be run because of the way the function parses
> spaces. The following example is dangerous because the function will
> attempt to run "Program.exe", if it exists, instead of "MyApp.exe".
>
>   LPTSTR szCmdline[] = _tcsdup(TEXT("C:\\Program Files\\MyApp"));
>   CreateProcessAsUser(hToken, NULL, szCmdline, /*...*/ );
>
> If a malicious user were to create an application called "Program.exe" on
> a system, any program that incorrectly calls CreateProcessAsUser using the
> Program Files directory will run this application instead of the intended
> application.
>
> To avoid this problem, do not pass NULL for lpApplicationName.

So instead, we pass the executable to lpApplicationName instead, which avoids
the issue. MSDN says:

> The lpCommandLine parameter can be NULL. In that case, the function uses
> the string pointed to by lpApplicationName as the command line.

This also avoids the strdup since lpApplicationName is LPCSTR unlike
lpCommandLine which is LPSTR.
2021-01-16 11:48:03 +11:00