VerseVDI-Data-Plane/openspec/changes/archive/2026-07-30-correct-gateway-qualification-evidence/design.md

Context

The qualification driver already reaches the production Apollo-to-QUIC path, but its source shaper reorders jitter even when reorder is disabled, its packet accounting cannot identify unexplained loss, and in-process resource counters include the provider/client driver.

Goals / Non-Goals

Goals:

• Attribute every source unit to one bounded production-path outcome.
• Keep impairment axes independently configured and observed.
• Sample CPU, heap, allocation, and goroutine use from the gateway process only.
• Record measured monotonic-clock overhead.

Non-Goals:

• No second simulator, profiling service, production dependency, or expanded impairment matrix.
• No larger queues or relaxed acceptance limits without measured need.

Decisions

• Reuse the existing source-boundary shaper, preserve source order unless explicit reorder is enabled, and limit catch-up to one media serialization interval. Record the fixed-seed applied-delay standard deviation separately from the jitter observed after ordered traversal.
• Drive processing sends at the configured source rate while a separate public-client receive loop validates ordered payload delivery. Use cooperative scheduling with a bounded high-resolution final wait in the parent driver so sub-millisecond packet spacing does not depend on host sleep granularity.
• Assign stable source sequence identifiers and retain per-stage counts so injected loss, provider/FEC drop, queue replacement, QUIC failure, and client miss are disjoint.
• Reuse the established gateway child-test pattern for the actual gateway server; the Apollo fixture and QUIC client remain in the parent driver. A token-protected loopback test control endpoint starts and stops bounded child-owned recording and returns aggregate stage state.
• Stream queue, processing, and pacing samples from the production sendMedia boundary to child-owned raw evidence. Sample child RUSAGE_SELF, Go heap, allocations, and goroutines once per second with independent per-run baselines.
• Buffer at most 4,096 child-owned timing samples before the gzip writer; encode rows into a reused byte buffer, drain every sample before recording stops, and backpressure on sustained writer overload instead of dropping evidence or formatting/compressing synchronously in the media loop.
• Bound native video at 256 packets (about 30 ms and less than 0.4 MiB per session at the largest fixture unit) after the sustained public-path regression observed a 141-packet scheduler/GC stall with the 64-packet bound; keep audio and events at 16 and retain latest-unit replacement. Sample heap objects, allocated objects/bytes, and live goroutines through runtime/metrics while retaining RUSAGE_SELF for CPU.
• Measure clock overhead as the median elapsed time per read across 1,000 batches of 100 monotonic reads and record that method.

Risks / Trade-offs

• [Ordered release suppresses some delivered jitter] → Retain both the applied fixed-seed delay distribution and the separately observed ordered-traversal jitter.
• [Stage attribution double-counts a unit] → Record one terminal outcome per source sequence and validate accounting equality.
• [Process sampling perturbs qualification] → Use bounded low-rate samples and include the sampling method in evidence.
• [The source driver consumes CPU for precise pacing] → Keep it in the parent process excluded by the gateway-only resource sampler, and yield cooperatively until the final 50 microseconds.
• [Shared private runners cannot sustain the reviewed 20/50/80 Mbps gates] → Run the complete verifier on the registered on-demand xhigh runner; the frozen qualification remains authoritative for the normative duration.