docs(openspec): archive gateway completion changes

openspec/specs/complete-encoded-frame-transport/spec.md

@@ -0,0 +1,38 @@
+# complete-encoded-frame-transport Specification
+
+## Purpose
+Define complete encoded-frame transport and the bounded native media queues required to preserve frame bytes and boundaries through the gateway.
+## Requirements
+### Requirement: Production transport preserves complete encoded frames
+The gateway SHALL carry each recovered Apollo encoded frame as one Protocol datagram-v2 sequence, preserve exact bytes and frame boundaries through the production media queue, pacer, QUIC transport, and independent reassembler, and reject frames outside Protocol bounds before forwarding.
+
+#### Scenario: Large source-shaped frame
+- **WHEN** Apollo UDP/FEC recovers a valid encoded frame above 18,864 bytes within the reviewed maximum
+- **THEN** the independent client receives one byte-identical frame with the same boundary
+
+#### Scenario: Invalid fragment stream
+- **WHEN** fragments are oversized, inconsistent, conflicting duplicates, outside the reorder/state/time bounds, or claim an oversized frame
+- **THEN** the client emits no partial payload and bounded state is released
+
+### Requirement: Native video queue has count byte and latency bounds
+The native provider video queue SHALL retain at most 16 complete frames, at
+most 4 MiB of encoded frame bytes, and no frame for more than 250 milliseconds.
+It SHALL replace the oldest entry when full, expire stale entries independently
+of queue activity, and increment truthful drop telemetry for every replacement
+or expiry. Cleanup and cancellation MUST stop expiry work and release all queued
+payload references.
+
+#### Scenario: Sustained realistic frames
+- **WHEN** a provider produces realistic variable-size complete frames faster than a slow Verse reader can forward them
+- **THEN** retained entries, bytes, and age remain within the reviewed per-session limits and newer frames continue to progress
+
+#### Scenario: Session cleanup
+- **WHEN** a session terminates, disconnects, or is cancelled with queued video
+- **THEN** queued frames are released, blocked readers wake, and no media crosses after quiescence
+
+### Requirement: Other provider queues remain independently bounded
+Audio and provider event queues SHALL retain independent count and payload bounds and MUST NOT share the video byte budget.
+
+#### Scenario: Video saturation
+- **WHEN** the video queue reaches its byte or age bound
+- **THEN** audio and terminal event delivery retain their existing independent bounded capacity

openspec/specs/gateway-qualification/spec.md

@@ -5,49 +5,64 @@ Define the deterministic processing, impairment, pacing, and evidence boundaries
 for qualifying a frozen Phase 3C gateway candidate.
 ## Requirements
 ### Requirement: Fixed media processing qualification
-The qualification harness SHALL drive pinned-mTLS Apollo management, encrypted
-RTSP, ENet, and provider UDP through native source validation, `readUDPMedia`,
-recovery/FEC, byte/count/latency-bounded production queues, the production fair
-pacer, Protocol complete-frame fragmentation, Verse framing/QUIC, and an
-independent bounded client reassembler for 1080p60 H.264 at 20 Mbps, 1440p120
-HEVC at 50 Mbps, and 4K60 HEVC at 80 Mbps. The source fixture SHALL emit
-deterministic variable-size complete encoded frames at the named 60/120 FPS
-rate, preserve exact target bytes over each fixed interval, and include bounded
-larger keyframes without codec operation. After a recorded warm-up, the frozen
-candidate SHALL run each profile for ten wall-clock minutes, preserve every
-frame's bytes and boundary, retain every monotonic processing sample plus
-bounded provider-queue observations, and report frame count, frame rate,
-bitrate, min, median, p90, p95, p99, max, mean, standard deviation, and measured
-batched monotonic-clock overhead and method. Processing begins at complete
-provider-frame receipt and ends at QUIC handoff, excluding client transit and
-pacing. Queue delay SHALL measure provider-queue residence, processing SHALL
-measure gateway work before pacing, and pacing delay SHALL measure scheduler
-waiting. Native video queues SHALL retain at most 16 complete frames, 4 MiB,
-and 250 milliseconds; audio and event queues SHALL remain independently bounded
-at 16 units. CPU, heap, allocations, and goroutines SHALL be measured from the
-isolated gateway process only; CPU SHALL be actual OS user plus system
-consumption and MUST NOT include idle wall capacity or unrelated parent
-fixture/client work. Successive profiles SHALL use independent resource-counter
-baselines. Any bypass, payload or frame-boundary mutation, frame-rate/count
-mismatch, wall-duration violation, bitrate outside both lower and upper bounds,
-unexplained clean-path loss, zero or unbounded clock overhead, or p95 above 5
-ms SHALL fail.
+The qualification harness SHALL drive pinned-mTLS Apollo management, encrypted RTSP, ENet, and provider UDP through native source validation, `readUDPMedia`, recovery/FEC, byte/count/latency-bounded production queues, the production fair pacer, Protocol complete-frame fragmentation, Verse framing/QUIC, and an independent bounded client reassembler for 1080p60 H.264 at 20 Mbps, 1440p120 HEVC at 50 Mbps, and 4K60 HEVC at 80 Mbps. The source fixture SHALL emit deterministic variable-size complete encoded frame units at the named 60/120 FPS rate, preserve exact target bytes over each fixed interval, and include bounded larger keyframes without codec operation. After a recorded warm-up, the frozen candidate SHALL run each profile for ten wall-clock minutes, preserve every frame's bytes and boundary, retain every monotonic processing sample plus bounded provider-queue observations, and report frame count, frame rate, bitrate, count, min, median, p90, p95, p99, max, mean, standard deviation, measured batched monotonic-clock overhead and method, and observed bitrate. Processing begins at complete provider-frame receipt and ends at QUIC handoff, excluding client transit and pacing. Queue delay SHALL measure provider-queue residence, processing SHALL measure gateway work before pacing, and pacing delay SHALL measure scheduler waiting. CPU, heap, allocations, and goroutines SHALL be measured from the isolated gateway process only; CPU SHALL be actual OS user plus system consumption and MUST NOT include idle wall capacity or unrelated parent fixture/client work. Successive profiles SHALL use independent resource-counter baselines. Any bypass, payload or boundary mutation, frame-rate/count mismatch, wall-duration violation, bitrate outside both lower and upper bounds, unexplained clean-path loss, zero or unbounded clock overhead, or p95 above 5 ms SHALL fail.
+
+Within each complete frame the source fixture SHALL reproduce pinned Apollo's source schedule by deriving packets per millisecond from the raw UDP block size at 80% of 1 Gbps, bounding each source batch to the smaller of 64 KiB or 64 packets, capturing the monotonic batch start immediately after the first successful shard write, scheduling the next batch no earlier than that start plus the current batch's raw-block serialization interval, carrying that schedule across frames, and making pacing waits context-cancellable. A delayed batch SHALL remain late rather than trigger an overdue catch-up burst.
+
+Native Apollo video ingress SHALL request a 2,195,456-byte socket receive buffer before media ping or worker startup and SHALL drain the connected video socket into a fixed FIFO pool of exactly 2,048 slots before the existing single decrypt/FEC processor. Each slot and the saturation scratch buffer SHALL be `apolloMediaMaximumPacket + 1` bytes so oversized datagrams remain rejected. A full pool SHALL NOT stop socket draining: each successfully read accepted-size discard SHALL increment both media-ingress and media-drop telemetry without allocation, while oversized reads SHALL retain the existing rejection accounting. Socket closure SHALL cancel the video read, and video/audio channels SHALL close only after the audio reader, video drain, and video processor exit. Audio and control ingress SHALL remain unchanged.
+
+The production fair pacer SHALL retain its 5 ms instantaneous catch-up ceiling. When a flow resumes later than that ceiling, it SHALL carry the remaining valid schedule debt only within the existing 250 ms provider-queue horizon and SHALL repay that debt using an interval no shorter than 20/21 of its nominal equal-tier fair-share interval. It SHALL return to the nominal interval when the debt is repaid. Simultaneous debt across eight equal-tier flows and the existing 25% and 50% capacity changes SHALL preserve the existing share-error contract and SHALL NOT exceed 105% of configured aggregate capacity in any rolling five-second window.
+
+Capacity-step convergence and rolling-cap evidence SHALL use the monotonic receive time and encoded length of every raw public QUIC datagram observed immediately after the independent client's `ReceiveDatagram` returns and before decode or reassembly. For each reduction, target bytes per second and the five-second cap SHALL derive from the configured public-wire rate, `qualificationMediaPacerKbps(profile, reduction) * 1000 / 8`. Completed logical-payload observations SHALL remain separate and SHALL continue to measure payload integrity, loss, reorder, latency, throughput, and queue behavior. An impairment capacity step SHALL include every delivery observation at or after its recorded transition through constrained-run completion; a later transition SHALL NOT truncate the earlier step's retained tail. It SHALL anchor measurement windows at the first such public delivery, require four consecutive 250 ms windows between 90% and 105% of its target, converge within ten seconds, and remain at or below 105% in every rolling five-second window.
+
+The constrained profile SHALL retain a bounded gzip CSV containing exactly two capacity-transition records and every observed raw public datagram delivery under one monotonic epoch captured before the constrained sequence. The schema SHALL distinguish transition and delivery records and SHALL contain `record_type`, `reduction_percent`, `transition_after_ns`, `received_after_ns`, and `encoded_bytes`; fields not applicable to a record type SHALL remain empty and SHALL be rejected when populated. The manifest SHALL bind the file name, SHA-256, compressed byte count, total row count, delivery row count, transition row count, monotonic timebase, exact 25% and 50% transition offsets, and each capacity summary's raw recomputation source. The row bound SHALL derive from the configured constrained-job and fragment bounds rather than a prior run's observed row count.
+
+The retained fairness manifest SHALL bind its 25% and 50% transition offsets to the monotonic epoch of `fairness.csv.gz`. Fairness recomputation SHALL remain bounded to each separately collected `runQualificationFleetStage` capacity slice. An independent parser SHALL be able to reconstruct each stage and reproduce the rolling-five-second maximum, configured cap, and exact two-second fairness convergence from the retained raw files and manifest alone. The parser SHALL select normative versus smoke validation only from trusted caller input; normative validation SHALL require exactly 10,000 sent logical units and SHALL enforce the ten-second convergence and 105% rolling-cap gates unconditionally. Compressed, decompressed, row, field, offset, flow, and encoded-length limits SHALL derive from canonical writer schemas, configured row limits, and canonical run horizons and SHALL be enforced before CSV parsing can allocate an unbounded record. Missing raw-wire evidence; a wrong file hash, size, or count; duplicate or missing transitions; negative or nonmonotonic offsets; invalid encoded lengths; completed-logical-frame substitution; populated not-applicable fields; oversized input; or a summary mismatch SHALL fail qualification evidence acceptance.
 
 #### Scenario: Healthy fixed profile
 - **WHEN** a frozen candidate runs one fixed profile for the normative duration in the isolated qualification command
-- **THEN** the harness emits compressed raw frame/path and gateway-process
-  resource samples plus a summary tied to the exact command, CPU scope,
-  timing-overhead method, topology, source commit, immutable Protocol version,
-  environment, and payload hash
+- **THEN** the harness emits compressed raw frame/path and gateway-process resource samples plus a summary tied to the exact command, CPU scope, timing-overhead method, topology, source commit, immutable Protocol version, environment, and payload hash
 
 #### Scenario: Processing gate failure
-- **WHEN** any production path stage lacks a per-frame observation, stage
-  accounting does not balance, payload or frame boundaries change, duration,
-  frame-rate, frame-count, or bitrate bounds fail, measured p95 exceeds 5 ms,
-  parent work changes gateway CPU, idle capacity is reported as consumed CPU, or
-  timing overhead is absent
+- **WHEN** any production path stage lacks a per-frame observation, stage accounting does not balance, payload or frame boundaries change, duration, frame-rate, frame-count, or bitrate bounds fail, measured p95 exceeds 5 ms, parent work changes gateway CPU, idle capacity is reported as consumed CPU, or timing overhead is absent
 - **THEN** the qualification command exits unsuccessfully without recording a passing candidate
 
+#### Scenario: Source-shaped Apollo pacing is preserved
+- **WHEN** the fixture emits 1,072-byte encrypted video shards with 1,040-byte raw blocks for consecutive complete frames
+- **THEN** it uses 96 packets per millisecond, batches at most 63 shards, records each batch after its first successful shard write, starts each later batch no earlier than the prior batch's raw serialization interval, carries the schedule into the following frame, and emits no shard after a cancelled pacing wait
+
+#### Scenario: Video crypto processing stalls
+- **WHEN** the first video AEAD operation is blocked while a 662-shard keyframe arrives
+- **THEN** all 662 successful connected-socket reads reach media-ingress accounting before processing resumes, and after release the exact complete frame traverses recovery, the bounded production queue, pacer, QUIC, and independent reassembly
+
+#### Scenario: Video ingress pool saturates
+- **WHEN** all 2,048 fixed video slots are occupied
+- **THEN** accepted-size datagrams are deliberately discarded through the fixed scratch buffer and counted as ingress plus drops, oversized datagrams remain rejected, and cancellation closes every media worker without a race or leak
+
+#### Scenario: Repeated media-loop host stalls
+- **WHEN** three approximately 95 ms scheduling debts are introduced at separated completed-public-frame barriers while source recovery continues
+- **THEN** the pacer limits instantaneous catch-up to 5 ms, repays each remaining debt at no more than 5% above nominal fair share, preserves every frame in exact order and bytes without provider or gateway drops, stays within the existing queue bounds, and closes cleanly on cancellation
+
+#### Scenario: Capacity measurement crosses a short transition phase
+- **WHEN** a constrained 1080p flow carries nonzero bounded debt through the approximately 1.572-second 25% phase before the 50% transition
+- **THEN** convergence and rolling-cap checks use the observed 1,200-byte and 25-byte public datagrams against the configured wire targets, while the separately retained completed-payload observations cannot substitute for transport delivery timing
+
+#### Scenario: Aggregate-only capacity evidence is retained
+- **WHEN** a qualification bundle contains logical-frame impairment rows and aggregate capacity summaries but omits raw public-wire rows or fairness transition offsets
+- **THEN** independent evidence validation rejects the bundle as incomplete even if its in-process runtime assertions passed
+
+#### Scenario: Raw capacity evidence is independently recomputed
+- **WHEN** v10 validation reads the retained constrained wire CSV, fairness CSV, and manifest transitions
+- **THEN** it validates bounded schema, hashes, sizes, counts, monotonic offsets, encoded datagram lengths, and transition uniqueness, then exactly reproduces the full-after-transition impairment targets, four consecutive 250 ms convergence windows, every rolling-five-second maximum, and stage-bounded fairness two-second alignment
+
+#### Scenario: Retained counts cannot weaken normative gates
+- **WHEN** a purported normative bundle retains a sent count other than 10,000 or retains an 11-second convergence or over-cap summary
+- **THEN** validation rejects it regardless of any retained field value, while explicitly selected smoke validation still requires exact raw-summary recomputation
+
+#### Scenario: Retained CSV exceeds bounded evidence grammar
+- **WHEN** a wire or fairness gzip exceeds its canonical compressed or decompressed limit or contains an overlong field, out-of-horizon offset, unknown flow, or out-of-range encoded length
+- **THEN** validation rejects it through the bounded standard-library reader before an unbounded CSV record can be allocated
+
 ### Requirement: Bounded impairment qualification
 The harness SHALL run exactly the baseline, latency, jitter, loss, reorder, and constrained Section 7.2 profiles once by applying fixed-seed impairment at the source-shaped provider network boundary while traffic concurrently traverses the production gateway path. Baseline SHALL cover all three media profiles and the other profiles SHALL cover 1080p60. The harness MUST NOT serialize a complete provider-to-client traversal per source unit. Reorder-off profiles SHALL preserve source order through an ordered delay queue whose catch-up is limited to one media serialization interval; the fixed-seed applied-delay distribution and jitter observed after ordered traversal SHALL be reported separately. Loss-only traffic SHALL NOT gain implicit reorder. Reorder-on profiles SHALL inject and record only the fixed bounded reorder pattern. Each source unit SHALL have one attributable outcome across source emission, injected drop, native provider/FEC handling, bounded queue replacement, gateway forwarding, QUIC send/receive, and public-client delivery. Each artifact SHALL retain raw impairment and queue observations and record tool version, exact command/configuration, environment, candidate commit, immutable Protocol version, direction, queue discipline, topology, fixed seed, observed one-way latency, acknowledged Apollo ENet RTT, applied and observed jitter, injected and unexplained loss, reorder, throughput, drops, and capacity-step statistics.
 
@@ -76,3 +91,10 @@ Qualification artifacts SHALL contain no provider endpoint, credential, clipboar
 #### Scenario: Deterministic evidence publication
 - **WHEN** qualification completes
 - **THEN** the manifest labels fake-provider, path impairment, and local processing evidence separately and leaves live interoperability deferred-owner-e2e
+
+### Requirement: Retained private Linux candidate artifact
+A retained private Linux candidate artifact SHALL have API metadata whose `expires_at - created_at` interval is at least 30 elapsed days (2,592,000 seconds). Workflow intent, cleanup lag, and a local copy SHALL NOT substitute for the recorded API interval. A shorter interval SHALL fail the artifact-retention gate even when execution and artifact bytes pass. The workflow request MAY exceed 30 calendar days only to compensate for verified platform rounding; the acceptance threshold remains at least 30 elapsed days.
+
+#### Scenario: Platform rounding shortens retention
+- **WHEN** a private Linux candidate run passes execution and artifact-byte checks but its artifact API metadata records less than 2,592,000 seconds between creation and expiry
+- **THEN** the artifact-retention gate remains failed until a separately authorized candidate run records an interval of at least 2,592,000 seconds