[host] nvfbc: force composition to capture some full screen apps

NvFBC is unable to capture certain applications that bypasses the DWM
compositor, for example, Firefox playing video in full screen. This
has been a known issue for a long time with Nvidia's ShadowPlay, see:
* https://www.nvidia.com/en-us/geforce/forums/geforce-experience/14/233709/
* https://crbug.com/609857

Nvidia won't fix this, but there are workarounds. For example, we
create a transparent 1x1 layered window, which forces desktop composition
to be enabled.

Note that SetLayeredWindowAttributes also supports alpha-based transparency,
but setting transparency to 0 will cause DWM to skip composition. We could
use a transparency of 1, but this ruins the image by the slightest bit,
which is unacceptable. Therefore, we must use chroma key-based
transparency, which tricks DWM into compositing despite being fully
transparent.
This commit is contained in:
Quantum
2021-01-20 02:11:49 -05:00
committed by Geoffrey McRae
parent 428b498cca
commit ffa72c7992
4 changed files with 165 additions and 0 deletions

View File

@@ -21,6 +21,7 @@ Place, Suite 330, Boston, MA 02111-1307 USA
#include "interface/platform.h"
#include "common/windebug.h"
#include "windows/mousehook.h"
#include "windows/force_compose.h"
#include "common/option.h"
#include "common/framebuffer.h"
#include "common/event.h"
@@ -191,6 +192,8 @@ static bool nvfbc_init(void)
if (this->seperateCursor)
this->cursorEvents[1] = lgWrapEvent(event);
dwmForceComposition();
DEBUG_INFO("Cursor mode : %s", this->seperateCursor ? "decoupled" : "integrated");
Sleep(100);
@@ -221,6 +224,7 @@ static void nvfbc_stop(void)
static bool nvfbc_deinit(void)
{
mouseHook_remove();
dwmUnforceComposition();
if (this->cursorEvents[0])
{