Commit Graph

2141 Commits

Author SHA1 Message Date
Geoffrey McRae
b8955fd012 [client] input: add display server behaviour tests
Extract input policy from the Wayland and X11 protocol handlers.

Exercise both implementations against literal Wayland event traces.
2026-08-10 17:15:35 +10:00
Geoffrey McRae
87aa61510c [client] audio: harden low-latency streaming
Rework audio provider and backend lifecycles so playback and capture
callbacks quiesce without blocking real-time threads. Move activation,
teardown, controls, retries, and diagnostics onto bounded workers.

Harden USB audio cadence, feedback, and capture recovery.
Preserve source clocks through recording and pace packets from the
device clock. Bound queues, waits, conversion buffers, and packet sizes.

Make PipeWire and PulseAudio stream control thread-safe and recoverable.
Correct latency clock domains, coalesce rate updates, preserve recent
capture under overload, and keep logging outside real-time callbacks.
2026-08-10 16:36:12 +10:00
Geoffrey McRae
4434985aa3 [client] usb audio: add microphone recording
Some checks failed
build / client (Debug, map[cc:clang cxx:clang++], libdecor) (push) Has been cancelled
build / client (Debug, map[cc:clang cxx:clang++], xdg-shell) (push) Has been cancelled
build / client (Debug, map[cc:gcc cxx:g++], libdecor) (push) Has been cancelled
build / client (Debug, map[cc:gcc cxx:g++], xdg-shell) (push) Has been cancelled
build / client (Release, map[cc:clang cxx:clang++], libdecor) (push) Has been cancelled
build / client (Release, map[cc:clang cxx:clang++], xdg-shell) (push) Has been cancelled
build / client (Release, map[cc:gcc cxx:g++], libdecor) (push) Has been cancelled
build / client (Release, map[cc:gcc cxx:g++], xdg-shell) (push) Has been cancelled
build / module (push) Has been cancelled
build / host-linux (push) Has been cancelled
build / host-windows-cross (push) Has been cancelled
build / host-windows-native (push) Has been cancelled
build / idd (push) Has been cancelled
build / obs (clang) (push) Has been cancelled
build / obs (gcc) (push) Has been cancelled
build / docs (push) Has been cancelled
Add a composite UAC2 microphone function with its own clock so
recording and playback can use independent sample rates.

Packetize asynchronous capture through usbredir, follow the host
capture clock, and preserve the newest frames across stalls.

Reconfigure active recording without another permission prompt and
fix lost PipeWire capture wakeups.
2026-08-10 06:32:05 +10:00
Geoffrey McRae
d660574a93 [client] audio: recover immediately from provider underruns
Treat a negative provider-controlled playback ring as a timeline
discontinuity rather than carrying its underrun debt.

Rebase to the normal low-water target and reset feedback correction so
guest stalls resume at low latency instead of discarding audio for
seconds.
2026-08-10 05:29:39 +10:00
Geoffrey McRae
73b15e4723 [client] usbredir: gracefully disconnect USB devices
Wait for the usbredir disconnect acknowledgement before closing SPICE.

Block replug while removal is pending and bound the shutdown wait.
2026-08-10 05:01:55 +10:00
Geoffrey McRae
2ca2553086 [client] audio: limit timing graph update rate
Sample the playback ring graph at its original 40 Hz rate so its
1,200 entries continue to represent 30 seconds of history.

Keep packet processing and asynchronous USB feedback at full rate.
2026-08-10 04:49:34 +10:00
Geoffrey McRae
0f3c1db261 [client] usb audio: add asynchronous clock feedback
Advertise an asynchronous UAC2 OUT endpoint with explicit feedback so
Windows paces USB audio from the host playback clock.

Drive feedback from measured backend consumption plus the existing
buffer phase controller. Preserve the local and backend resampler paths
for providers without active feedback.

Keep audio diagnostics in the correct clock domains, report ring and
backend latency separately, and avoid hot-path feedback wakeups.
2026-08-10 04:44:55 +10:00
Geoffrey McRae
fa8a95569a [client] audio: preserve native playback formats 2026-08-10 03:57:03 +10:00
Geoffrey McRae
ff665c2031 [client] usb audio: support up to 192khz 24-bit
We can not support 32-bit because Windows is buggy:

The exact Windows 11 mmsys.cpl from build 26100.8737 populates Speaker
Setup testing against hard-coded speaker masks and for each calls
IKsFormatSupport::IsFormatSupported() using 24 hard-coded PCM formats.

These formats do not include 32-bit in 32-bit, consequently, every one
of the wizard’s format probes fails and it inserts zero channel-list
entries.

It does however test for 24-bit in 32-bit which passes, however this
exposes another breakage due to an inconsistency inside Windows
usbaudio2.sys

For bSubslotSize = 4, bBitResolution = 24, Windows does this:

1. It publishes a KS range whose minimum and maximum bit depths are both
   taken from bBitResolution, so the range says 24-bit.
2. It correctly constructs the actual stream format as:
   wBitsPerSample = 32, wValidBitsPerSample = 24
3. KS range intersection rejects that 24-in-32 format because its
   container is 32 bits while the published range says 24.
4. A packed 24-bit candidate passes range intersection, but the
   subsequent exact stream check rejects it because it uses a three-byte
   container while USB advertises four bytes.

Therefore no format can pass both checks. That exactly explains:

* USB Device Tree Viewer seeing 24-bit device ranges.
* The MMDevice endpoint exposing no usable formats.
* Windows being unable to start playback.

I verified this in the exact Windows 10.0.26100.8737 driver:

* KSPin::AddDataRange stores bBitResolution into both KS bit-depth
  bounds.
* MiniportWave::DataRangeIntersection tests those bounds.
* WaveFormatInfo::IsMatchingStreamFormat separately compares both the
  container width and valid-bit count.
2026-08-10 03:47:55 +10:00
Geoffrey McRae
a1da8c14f3 [client] usb audio: connect USB redirection
Add an opt-in USB audio fallback and connect the emulator to an
available PureSpice USBREDIR channel.

Keep transport-provided audio in priority and serialize USB parser
processing and teardown on the PureSpice thread.
2026-08-09 22:29:11 +10:00
Geoffrey McRae
06f111dc90 [client] usb audio: add audio provider
Adapt the emulated UAC2 playback stream to LG_AudioOps.

Preserve borrowed packet delivery, publish reconstructed timing, and
synchronize provider availability and attachment transitions.
2026-08-09 22:12:39 +10:00
Geoffrey McRae
6979b93da5 [client] usb audio: emulate UAC2 playback
Add a fixed-format USB Audio Class 2 playback device for the
client-side USB redirection bridge.

Advertise stereo 48 kHz IEEE float audio and handle enumeration,
configuration, alternate settings, clock requests, and isochronous data.
2026-08-09 21:30:20 +10:00
Geoffrey McRae
6d1b0c3004 [client] usbredir: add parser bridge
Add a serialized bridge between PureSpice USB redirection channels and
libusbredirparser.

Claim one channel and stream borrowed fragments directly through the
parser without packet allocations. Provide an atomic device plug request
for later audio integration.
2026-08-09 21:09:52 +10:00
Geoffrey McRae
4c53f5b9f8 [client] clipboard: add transport provider interface
Some checks failed
build / client (Debug, map[cc:clang cxx:clang++], libdecor) (push) Has been cancelled
build / client (Debug, map[cc:clang cxx:clang++], xdg-shell) (push) Has been cancelled
build / client (Debug, map[cc:gcc cxx:g++], libdecor) (push) Has been cancelled
build / client (Debug, map[cc:gcc cxx:g++], xdg-shell) (push) Has been cancelled
build / client (Release, map[cc:clang cxx:clang++], libdecor) (push) Has been cancelled
build / client (Release, map[cc:clang cxx:clang++], xdg-shell) (push) Has been cancelled
build / client (Release, map[cc:gcc cxx:g++], libdecor) (push) Has been cancelled
build / client (Release, map[cc:gcc cxx:g++], xdg-shell) (push) Has been cancelled
build / module (push) Has been cancelled
build / host-linux (push) Has been cancelled
build / host-windows-cross (push) Has been cancelled
build / host-windows-native (push) Has been cancelled
build / idd (push) Has been cancelled
build / obs (clang) (push) Has been cancelled
build / obs (gcc) (push) Has been cancelled
build / docs (push) Has been cancelled
Route clipboard exchange through a provider interface. Connected
transports take priority while SPICE remains the fallback.

Track asynchronous X11 and Wayland transfers by request, preserving
clipboard ownership across provider changes.

Use PureSpice clipboard status notifications for capability and
lifecycle updates.
2026-08-09 20:10:46 +10:00
Geoffrey McRae
0da4bfa8dd [client] audio: add transport provider interface
Some checks failed
build / client (Debug, map[cc:clang cxx:clang++], libdecor) (push) Has been cancelled
build / client (Debug, map[cc:clang cxx:clang++], xdg-shell) (push) Has been cancelled
build / client (Debug, map[cc:gcc cxx:g++], libdecor) (push) Has been cancelled
build / client (Debug, map[cc:gcc cxx:g++], xdg-shell) (push) Has been cancelled
build / client (Release, map[cc:clang cxx:clang++], libdecor) (push) Has been cancelled
build / client (Release, map[cc:clang cxx:clang++], xdg-shell) (push) Has been cancelled
build / client (Release, map[cc:gcc cxx:g++], libdecor) (push) Has been cancelled
build / client (Release, map[cc:gcc cxx:g++], xdg-shell) (push) Has been cancelled
build / module (push) Has been cancelled
build / host-linux (push) Has been cancelled
build / host-windows-cross (push) Has been cancelled
build / host-windows-native (push) Has been cancelled
build / idd (push) Has been cancelled
build / obs (clang) (push) Has been cancelled
build / obs (gcc) (push) Has been cancelled
build / docs (push) Has been cancelled
Route audio through a provider interface. Connected transports take
priority while SPICE remains the fallback.

Carry explicit sample formats, rates, channel layouts, and source clocks
through playback and recording. Add optional presentation clock feedback
for active synchronization.
2026-08-09 17:57:09 +10:00
Geoffrey McRae
40d6342d46 [client] overlay: initialize ImGui frame timing
Some checks failed
build / client (Debug, map[cc:clang cxx:clang++], libdecor) (push) Has been cancelled
build / client (Debug, map[cc:clang cxx:clang++], xdg-shell) (push) Has been cancelled
build / client (Debug, map[cc:gcc cxx:g++], libdecor) (push) Has been cancelled
build / client (Debug, map[cc:gcc cxx:g++], xdg-shell) (push) Has been cancelled
build / client (Release, map[cc:clang cxx:clang++], libdecor) (push) Has been cancelled
build / client (Release, map[cc:clang cxx:clang++], xdg-shell) (push) Has been cancelled
build / client (Release, map[cc:gcc cxx:g++], libdecor) (push) Has been cancelled
build / client (Release, map[cc:gcc cxx:g++], xdg-shell) (push) Has been cancelled
build / module (push) Has been cancelled
build / host-linux (push) Has been cancelled
build / host-windows-cross (push) Has been cancelled
build / host-windows-native (push) Has been cancelled
build / idd (push) Has been cancelled
build / obs (clang) (push) Has been cancelled
build / obs (gcc) (push) Has been cancelled
build / docs (push) Has been cancelled
Initialize the previous ImGui frame timestamp before rendering
overlays. Without this, the first DeltaTime equals host uptime.

Recent ImGui versions track background and foreground draw-list
activity using float time. At long host uptimes its precision can span
multiple display frames, preventing those lists from being reset and
submitting translucent overlay commands more than once.
2026-08-09 15:08:55 +10:00
Geoffrey McRae
89f0311abc [client] tests: update mouse cases for direct warping
Remove confinement handshake expectations now that Wayland advertises
functional warp support. Verify immediate viewport transitions and cursor
alignment when capture is released.
2026-08-09 12:39:23 +10:00
Quantum
19a22ac8ca [client] core: always align mouse on uncapture
Previously, we only aligned the mouse in the client when the cursor is
visible. This wasn't a problem because the cursor position in the guest
took precedence and we only used relative input.

Since we now have absolute input, the absolute position of the cursor in
the client matters at all times when we exit capture, so we always call
`core_alignToGuest` now.
2026-08-09 10:32:13 +10:00
Quantum
62d2b37b30 [client] wayland: log pointer warp mechanism 2026-08-09 10:32:13 +10:00
Quantum
25328ec5ba [client] wayland: implement warping through pointer-warp-v1 protocol 2026-08-09 10:32:13 +10:00
Quantum
e2be90e264 [client] wayland: restore cursor warp support via pointer confining
This notably does not restore any of the cursed pointer confining logic.
2026-08-09 10:32:13 +10:00
Quantum
70ae98491d [client] core: simplify core_setCursorInView logic
We don't need any of that weird grabPointer stuff now that we have absolute
input. Deleting this will allow Wayland to advertise warp support without
doing any of the nasty confining stuff on grab/release.
2026-08-09 10:32:13 +10:00
Geoffrey McRae
9f2d6f6413 [client] wayland: constrain pointer only during capture
Some checks failed
build / client (Debug, map[cc:clang cxx:clang++], libdecor) (push) Has been cancelled
build / client (Debug, map[cc:clang cxx:clang++], xdg-shell) (push) Has been cancelled
build / client (Debug, map[cc:gcc cxx:g++], libdecor) (push) Has been cancelled
build / client (Debug, map[cc:gcc cxx:g++], xdg-shell) (push) Has been cancelled
build / client (Release, map[cc:clang cxx:clang++], libdecor) (push) Has been cancelled
build / client (Release, map[cc:clang cxx:clang++], xdg-shell) (push) Has been cancelled
build / client (Release, map[cc:gcc cxx:g++], libdecor) (push) Has been cancelled
build / client (Release, map[cc:gcc cxx:g++], xdg-shell) (push) Has been cancelled
build / module (push) Has been cancelled
build / host-linux (push) Has been cancelled
build / host-windows-cross (push) Has been cancelled
build / host-windows-native (push) Has been cancelled
build / idd (push) Has been cancelled
build / obs (clang) (push) Has been cancelled
build / obs (gcc) (push) Has been cancelled
build / docs (push) Has been cancelled
Remove the persistent confined-pointer workaround that emulated cursor
warping on Wayland. Normal pointer motion now uses absolute input only.

Create relative and locked pointer objects only during capture. Preserve
the request across pointer capability changes, and suppress relative
events until the compositor confirms the lock.

On uncapture, publish the current absolute position instead of trying to
warp the host cursor.
2026-08-09 03:32:58 +10:00
Geoffrey McRae
48f0b66a4f [client] input: forward extended mouse buttons
Map the remaining Linux mouse buttons through Wayland and evdev.

Use one X11 translation for cooked and raw press and release events.

Ignore unsupported horizontal scrolling and extended SPICE buttons.
2026-08-09 02:50:02 +10:00
Geoffrey McRae
4367fbc277 [client/idd] input: avoid replaying absolute position
Some checks failed
build / client (Debug, map[cc:clang cxx:clang++], libdecor) (push) Has been cancelled
build / client (Debug, map[cc:clang cxx:clang++], xdg-shell) (push) Has been cancelled
build / client (Debug, map[cc:gcc cxx:g++], libdecor) (push) Has been cancelled
build / client (Debug, map[cc:gcc cxx:g++], xdg-shell) (push) Has been cancelled
build / client (Release, map[cc:clang cxx:clang++], libdecor) (push) Has been cancelled
build / client (Release, map[cc:clang cxx:clang++], xdg-shell) (push) Has been cancelled
build / client (Release, map[cc:gcc cxx:g++], libdecor) (push) Has been cancelled
build / client (Release, map[cc:gcc cxx:g++], xdg-shell) (push) Has been cancelled
build / module (push) Has been cancelled
build / host-linux (push) Has been cancelled
build / host-windows-cross (push) Has been cancelled
build / host-windows-native (push) Has been cancelled
build / idd (push) Has been cancelled
build / obs (clang) (push) Has been cancelled
build / obs (gcc) (push) Has been cancelled
build / docs (push) Has been cancelled
2026-08-09 02:07:38 +10:00
Geoffrey McRae
008a4482d6 [client] tests: initialize debug for input manager 2026-08-09 01:57:36 +10:00
Geoffrey McRae
487920ada3 [client/idd] input: add transport diagnostics 2026-08-09 01:55:45 +10:00
Geoffrey McRae
5b90384123 [client/idd] input: report endpoint status 2026-08-09 01:36:54 +10:00
Geoffrey McRae
3e38c62050 [client] input: use absolute transport positioning 2026-08-09 01:23:30 +10:00
Geoffrey McRae
27a938e741 [client] input: send reports over LGMP
Use the LGMP input queue when the host advertises transport input,
while preserving SPICE fallback through the existing input abstraction.

Claim input lazily, publish ordered full-state keyboard and mouse
reports without blocking rendering, and keep the owner lease alive.
Reserve queue capacity for discrete transitions, coalesce only motion
under pressure, and release ownership safely during idle and disconnect.

Keep the Linux-to-HID map in the client and update LGMP to the
nonblocking data-send API required by the input worker.
2026-08-09 01:03:41 +10:00
Geoffrey McRae
f495e5b7ba [client] input: abstract guest input backends
Introduce LG_InputOps and route keyboard and mouse input through the
active video transport when it provides an input backend.

Keep SPICE as the fallback and force it while the SPICE display is
active. Add atomic shared/exclusive locking for safe backend changes.
2026-08-08 21:23:32 +10:00
Geoffrey McRae
e9d4b2f563 [client] tests: keep splash out of captures
Hide the startup splash immediately after overlay initialization when a
render-pipeline capture is requested. This occurs before the frame
thread is released, so no EGL surface buffer can contain splash pixels.
2026-08-08 01:26:42 +10:00
Geoffrey McRae
4e82af162d [client] tests: cover padded frame strides
Add a configurable stride to the synthetic transport and poison row
padding so incorrect row addressing is visible in captured output.

Cover full-frame and moving-damage BGRA uploads with 512-byte and
5120-byte pitches, and verify the advertised layout in the log.
2026-08-08 00:29:32 +10:00
Geoffrey McRae
05599ffd02 [client] cursor: restore Wayland pointer convergence
Keep the tracked Wayland hardware position sourced only from absolute
wl_pointer events. Relative motion remains unclipped at output edges,
so accumulating it made the client believe the hardware cursor had
reached the guest position and caused the corrective warp to be
skipped.

Keep cursor coordinates in shape-origin form after absolute
realignment. Preserve the logical hotspot when a shape-only update
changes its hotspot, preventing either path from leaving a permanent
offset.
2026-08-06 22:26:14 +10:00
Geoffrey McRae
5ab10614f2 [client] cursor: improve Wayland hotspot alignment
Center the Wayland diagnostic cursor on its actual pointer hotspot and
preserve it while cursor themes are reloaded for output-scale changes.

Round fractional guest positions to the nearest logical pixel instead of
biasing synchronization toward the top-left corner.

Resynchronize the hardware pointer when a cursor shape changes its
hotspot. Only perform the extra synchronization when the hotspot
actually changes, and update the basic-mode projection at the same time.
2026-08-06 22:26:14 +10:00
Geoffrey McRae
e4874ee32d [client] cursor: restore Wayland edge synchronization
Keep guest-to-host pointer synchronization active while an attempted
surface exit is blocked by the edge of a Wayland output. Continue
forwarding relative motion and wait for wl_pointer.leave before marking
the cursor outside the window.

Distinguish whole-surface releases from exits into letterbox space so
the expected asynchronous release callback cannot disable the feedback
loop before the pointer is confined again.

Treat Wayland pointer capability loss as a real leave so an unrelated
inactive callback cannot be mistaken for the intentional release.
2026-08-06 22:26:14 +10:00
Geoffrey McRae
5cc7f06185 [client] cursor: pace composited redraws
Coalesce pointer updates to the nominal output cadence instead of
waking the renderer for every transport packet. Keep request and
presented generations separate so updates racing a render receive a
trailing swap.

Let JIT consume cursor changes on display callbacks to preserve VRR
cadence, and use a deadline worker for non-JIT rendering. Give normal
source frames a short grace period so an on-rate feed absorbs the
cursor update instead of producing an extra swap.

Repaint visibility and cursor-content changes even when movement
redraws are disabled, including the transition that hides the cursor.
2026-08-06 22:26:10 +10:00
Geoffrey McRae
604b3082f3 [client] overlay: coalesce pointer redraws during splash fade
Pointer updates bypassed the paced overlay render and woke the renderer
for every cursor packet. On the default single-buffer EGL surface,
each full fade redraw could expose the desktop before the splash was
blended again.

Coalesce pointer redraws into the existing fade cadence while keeping
the latest cursor state. Immediate cursor redraw resumes when the fade
ends.
2026-08-06 18:17:34 +10:00
Geoffrey McRae
d8604ec144 [client] metrics: hide presentation timing 2026-08-06 18:04:10 +10:00
Geoffrey McRae
ea6e155bbf [client] metrics: correlate Wayland presentation timing 2026-08-06 18:04:10 +10:00
Geoffrey McRae
bdfb1c3ce0 [client] metrics: expose cadence pipeline stages 2026-08-06 18:04:10 +10:00
Geoffrey McRae
0b744d2286 [common] kvmfr: report cadence hold and lead timing 2026-08-06 18:04:10 +10:00
Geoffrey McRae
1b78384da3 [client] wayland: publish stable presentation cadence 2026-08-06 18:04:10 +10:00
Geoffrey McRae
988675e53a [client] x11: recover stalled present cadence
Reseed the synthetic Present target window when compositor MSC advances
outside the queued range. This prevents a long compositor stall from
permanently exhausting JIT cadence callbacks.

Mark the recovery sample as discontinuous so calibration does not treat the
MSC jump or completion burst as renderer skipping.
2026-08-06 18:02:08 +10:00
Geoffrey McRae
a0b7b12ffc [client] tree: restore unrelated working changes
Restore client and test files that were already staged when the OBS
cadence commit was created. Keep that work out of the scheduler series
while leaving its working-tree contents untouched.
2026-08-06 18:02:08 +10:00
Geoffrey McRae
8a04037c9b [obs] scheduler: serialize cadence controls
Track the LGMP serial for each accepted frame-schedule message and wait
for its acknowledgement before sending another. Preserve pending RESET,
IMMEDIATE, and feedback state after definite enqueue failures.

Use bounded retry polling and request one fresh immediate frame after an
ambiguous pointer-queue fault recovers.
2026-08-06 18:02:08 +10:00
Geoffrey McRae
1040683a3a [client] scheduler: serialize cadence controls
Keep one accepted frame-schedule control in flight until LGMP reports
that its serial has been consumed. Retain ambiguous tokens across pointer
queue recovery instead of retransmitting RESET or IMMEDIATE messages.

Back off acknowledgement polling and definite enqueue failures. Use modular
serial comparison so acknowledgement remains correct across rollover.
2026-08-06 18:02:08 +10:00
Geoffrey McRae
903a8efe75 [client] scheduler: release ownership without live cadence
Renew the frame-schedule lease only while the display server is
producing real cadence callbacks. Release ownership after the cadence
stalls, and reacquire with an immediate reset when it resumes.

For non-JIT rendering, treat only a successful presentation as live
cadence.
2026-08-06 18:02:08 +10:00
Geoffrey McRae
3d1064001d [client] scheduler: qualify presentation feedback
Track cadence, interruption, and calibration wake reasons independently
across X11 and Wayland. Feed phase correction back only when a pure
cadence wake renders the exact frame that was queued at that wake.

Propagate EGL swap results through the display-server boundary. Reject
failed presentations and keep pending Wayland HDR descriptions inactive
until the preceding surface commit succeeds.
2026-08-06 18:02:08 +10:00
Geoffrey McRae
515f4ba06f [client/host/idd/obs] scheduler: validate phase feedback
Bump KVMFR to version 28 and attach generation, epoch, and deadline
identities to cadence deliveries and their timing records.

Accept phase corrections only for the exact completed publication. Keep
VM and client clocks independent by exchanging relative phase errors.

Make submission, completion, and work-estimate accounting best effort
so scheduler state cannot delay GPU submission or frame publication.
2026-08-06 18:02:08 +10:00