180 lines
11 KiB
Markdown
180 lines
11 KiB
Markdown
## Context
|
|
|
|
Phase 3C freezes a gateway-only control/media baseline, but the Data Plane macOS target is
|
|
still a template and the shared Protocol has no requested display mode, absolute pointer, or
|
|
high-resolution scroll contract. The closed Server already owns broker persistence,
|
|
allocation policy, manifest disclosure, and provider-work. The GPLv3 Data Plane owns the
|
|
pure-Go gateway/Apollo translation plus the future Rust core and native client. The Protocol
|
|
repository remains the only wire authority.
|
|
|
|
The implementation must preserve strict legacy JSON decoding, exact immutable Protocol pins,
|
|
no direct client-to-provider path, encoded-media relay without gateway decode/transcode, and
|
|
bounded release-all behavior. UI references are retained in the Planning repository as
|
|
review evidence only.
|
|
|
|
## Goals / Non-Goals
|
|
|
|
**Goals:**
|
|
|
|
- Deliver one native Apple-Silicon SwiftUI/AppKit client for macOS 26 with macOS 15/14
|
|
compatibility, built against a deterministic fake core before real streaming integration.
|
|
- Carry a client-requested display mode through Server allocation to provider work, while the
|
|
Server remains the only policy/clamp authority and both requested/effective values remain
|
|
visible to the client.
|
|
- Add provider-neutral absolute pointer and high-resolution scroll events without changing
|
|
the gateway-only route or coupling the public contract to Apollo packet shapes.
|
|
- Keep lifecycle, input release, local preview storage, secrets, media, and diagnostics
|
|
bounded and testable at their trust boundaries.
|
|
- Preserve a small, sized/versioned Rust C ABI and a single Swift unsafe bridge.
|
|
|
|
**Non-Goals:**
|
|
|
|
- Remote apps, manual PC/direct provider connections, live display renegotiation,
|
|
multi-monitor remote topology, HDR, microphone, file/folder redirection, image clipboard,
|
|
macros/timed sequences, Intel, provider-specific UI, or a plugin framework.
|
|
- Gateway decode, encode, transcode, render, cgo, native sidecar, insecure retry, or silent
|
|
capability fallback.
|
|
- Public release, signing credentials, deployment, promotion, or agent access to the owner's
|
|
Apollo host. Live Apollo/SudoMaker evidence remains owner E2E.
|
|
|
|
## Decisions
|
|
|
|
### 1. Repository and compatibility ownership
|
|
|
|
- Protocol adds a reusable `DisplayMode` object and optional request/response fields while
|
|
retaining control wire version 1. Generated Go represents optional referenced objects as
|
|
pointers so `nil` is omitted; Rust/Swift use optionals. A legacy request receives the
|
|
legacy response shape, while a display-aware request requires `display.request.v1` and a
|
|
display-aware manifest contains the accepted value.
|
|
- Protocol registers `display.request.v1`, `input.absolute.v1`, and `input.scroll.v1` and
|
|
extends VGI1 only behind those negotiated features. Old gateways continue to reject
|
|
unknown kinds; clients therefore never send an unadvertised kind.
|
|
- Protobuf remains unchanged unless an observed consumer or verification gate proves that
|
|
the JSON control route actually depends on it. This avoids an unrelated second contract.
|
|
- A verified-unused immutable Phase 3D Protocol RC is frozen before Server/Data pins advance
|
|
together. No sibling `replace` or local wire fork is committed.
|
|
|
|
### 2. Server-owned display decision
|
|
|
|
- Broker sessions gain nullable requested/effective width, height, and FPS columns plus the
|
|
immutable policy-version identity needed to replay allocation/reconnect. Dedicated columns
|
|
keep the decision queryable and avoid overloading the existing bandwidth policy JSON.
|
|
- Request validation accepts width 320..16384, height 200..8640, and FPS 1..240. Requested
|
|
mode participates in every idempotency identity, including waiting sessions.
|
|
- Allocation selects the machine, resolves the immutable machine/pool/global policy, computes
|
|
the accepted mode, and atomically persists the allocation and accepted values. Reconnect
|
|
reuses the persisted result; it does not re-clamp against later policy.
|
|
- Clamp is proportional and deterministic:
|
|
`scale = min(policyWidth/requestWidth, policyHeight/requestHeight, 1)`;
|
|
`effectiveWidth = floorToEven(requestWidth * scale)`;
|
|
`effectiveHeight = floorToEven(requestHeight * scale)`;
|
|
`effectiveFPS = min(requestedFPS, policyFPS)`.
|
|
- Provider work uses the persisted effective dimensions/FPS and the selected immutable
|
|
policy's codec/bitrate/audio. Legacy sessions retain today's exact policy dimensions.
|
|
|
|
### 3. Provider-neutral input
|
|
|
|
- VGI1 kind `0x06` has exactly eight body bytes: big-endian unsigned `x`, `y`, viewport
|
|
width, and viewport height. Width/height must be nonzero; x < width and y < height.
|
|
- VGI1 kind `0x07` has exactly four body bytes: big-endian signed 16-bit vertical and
|
|
horizontal high-resolution scroll deltas.
|
|
- Protocol docs, fixtures, Go/Rust/Swift classifiers, Data encode/decode, fuzzing, transport,
|
|
and Apollo translation share the same vectors. Apollo packet details remain private to the
|
|
adapter and must be justified by pinned reference source before implementation.
|
|
- Absolute/scroll events do not enter the pressed-state ledger. Keys, buttons, and controllers
|
|
continue to release through the existing ledger on every authority/lifecycle exit.
|
|
|
|
### 4. Native workspace, display discovery, and input ownership
|
|
|
|
- The client uses `NavigationSplitView`, native controls/materials, and standard macOS 26
|
|
Liquid Glass APIs conditionally. macOS 15/14 retain identical hierarchy with native
|
|
materials; no custom glass framework is introduced.
|
|
- Desktop cards use a bounded adaptive `300...360 pt` width, `16:10` aspect ratio, and
|
|
`16 pt` spacing. Cards grow continuously within natural column bands, including three
|
|
columns; a new column appears only when every card can remain at least `300 pt` wide;
|
|
cards clamp at `360 pt` and retain residual row width. Screenshot pixels are
|
|
non-authoritative. Card actions remain accessible without hover, and no direct-connect
|
|
button or endpoint entry is added.
|
|
- Automatic mode reads physical pixels for the screen where the session window opens and
|
|
uses its maximum FPS. A defensible notch-safe built-in mode is preferred; otherwise Full
|
|
Native is disclosed. A display move offers a new-session reconnect and never mutates the
|
|
active stream.
|
|
- The renderer aspect-fits. Absolute coordinates originate only inside the rendered viewport;
|
|
letterbox bars produce no pointer event.
|
|
- Native `NSWindow` fullscreen preserves top-edge system UI in absolute mode. Relative mode
|
|
hides/captures the cursor. Reserved mode-toggle and emergency-release chords are handled
|
|
before mapping and never cross the core boundary. Mode switches release pressed input and
|
|
clamp cursor restoration to the rendered viewport.
|
|
|
|
### 5. State, Rust core, and platform boundaries
|
|
|
|
- Swift actors own authentication, resources, brokerage, streaming, and settings; immutable
|
|
projections reach `MainActor` views. One bridge owns every C pointer/callback/lifetime.
|
|
- The Rust core owns tunnel negotiation, authenticated framing, packet reconstruction,
|
|
encoded media delivery, input encoding, statistics, cancellation, and structured errors.
|
|
Apple frameworks own decode/render/audio/device/input capture and product state.
|
|
- The C ABI uses fixed-width values, explicit lengths, opaque handles, sized/versioned tables,
|
|
a declared callback queue, caught panics, cancellation, and late-callback/destroy rules.
|
|
- Fakes cover every external boundary before real core/media integration. No speculative
|
|
adapter/factory/plugin layer is added beyond interfaces already needed for deterministic
|
|
tests.
|
|
|
|
### 6. Preview and diagnostic privacy
|
|
|
|
- Stable desktop previews are downscaled to `656 x 410`, stored in the sandbox cache as mode
|
|
`0600`, and written only after an active session ends cleanly. Failed launch retains the
|
|
prior preview. Pool cards use generic art.
|
|
- Logout or entitlement loss clears affected previews. Hide/Clear settings are explicit.
|
|
Backup, telemetry, OSLog, crashes, support bundles, and default diagnostics exclude preview
|
|
and raw media bytes.
|
|
- Diagnostics are allowlisted structured state/correlation/performance summaries; tokens,
|
|
grants, provider endpoints, media, input, clipboard, and Windows secrets never enter them.
|
|
|
|
### 7. Delivery and evidence
|
|
|
|
- Land capability-sized commits in order: Planning/OpenSpec, Protocol, Server, gateway input,
|
|
fake-core native shell, Rust/core integration, platform media, then full qualification.
|
|
- Every nontrivial boundary starts with a focused failing test/vector and ends with the
|
|
narrow affected gate; one complete clean-source gate runs only after freeze.
|
|
- macOS 26/15/14 native behavior, accessibility, lifecycle, privacy, and platform media require
|
|
real target evidence. The owner later runs the only live Apollo/SudoMaker acceptance.
|
|
|
|
## Risks / Trade-offs
|
|
|
|
- [Strict legacy decoders reject additive response fields] -> Emit display fields only for a
|
|
negotiated display-aware request; retain legacy shapes and cross-version fixtures.
|
|
- [Requested dimensions exceed policy or change aspect] -> Apply one Server-owned
|
|
proportional/even clamp, persist it atomically, and disclose both values.
|
|
- [Absolute coordinates mismatch presentation] -> Normalize against the rendered viewport,
|
|
reject letterbox coordinates, and test edge/scale/display cases.
|
|
- [Relative capture strands local input] -> Keep an unremappable emergency release, release
|
|
before every mode/authority transition, and test focus/sleep/cancel/revocation paths.
|
|
- [Preview bytes leak user media] -> Minimize one local file, mode `0600`, clean-session-only
|
|
writes, lifecycle clearing, and explicit exclusion/secret-canary tests.
|
|
- [Apollo input semantics are guessed] -> Stop until the pinned provider source establishes
|
|
the exact packet; do not encode provider assumptions into VGI.
|
|
- [macOS API rendering differs by release/accessibility setting] -> Test deterministic
|
|
geometry and state automatically; use bounded manual material review without snapshots of
|
|
OS-owned pixels as the sole gate.
|
|
- [The full client scope is large] -> Keep serial capability commits and stop at the first
|
|
failed contract/environment gate rather than introducing fallbacks or partial claims.
|
|
|
|
## Migration Plan
|
|
|
|
1. Freeze Planning/OpenSpec and visual references.
|
|
2. Implement/freeze an immutable Protocol RC with legacy and new feature fixtures.
|
|
3. Apply the forward-only Server migration and update Server/Data Protocol pins together.
|
|
4. Deploy no schema or runtime automatically; validate clean install/upgrade/legacy rows and
|
|
gateway fixtures locally/private CI first.
|
|
5. Build the native shell against fakes, then integrate the frozen core/gateway surfaces.
|
|
6. Roll back application binaries only while schema compatibility permits; nullable additive
|
|
fields preserve legacy row behavior. Never down-migrate or rewrite applied migrations.
|
|
7. Keep signing, publication, deployment, promotion, and live owner E2E under separate
|
|
authorization.
|
|
|
|
## Open Questions
|
|
|
|
None at proposal time. Exact Apollo absolute-pointer/scroll packet vectors must be established
|
|
from the approved pinned source before the adapter step; absence of that evidence is an
|
|
implementation hard stop, not a design choice to guess.
|