test(gateway): retain raw wire qualification evidence

openspec/changes/truthful-fixed-frame-qualification/design.md

@@ -8,7 +8,7 @@ Because the bounded fixture uses loopback rather than a physical 1 Gbps link, v8
 
 The retained v6 qualification run passed its then-current checks but is superseded because its tight-loop sender contradicted the pinned Apollo schedule. Private Linux runs 123 and 124 remain failed evidence. One local v8 sustained run passed on Darwin, but it is neither Linux proof nor normative Section 7 evidence.
 
-The later Darwin non-sustained pre-CI invocation was not green and was not retried. Its 1440p120 profile delivered the exact 6,250,000 bytes in 120 frames plus all 6,483 source and warm-up shards with zero drops, but measured 46,973.13 kbps over an implied approximately 1.0644383 seconds and failed the 5% throughput gate. Private Linux full verification/artifact retention and the replacement v9 normative run remain open.
+The later Darwin non-sustained pre-CI invocation was not green and was not retried. Its 1440p120 profile delivered the exact 6,250,000 bytes in 120 frames plus all 6,483 source and warm-up shards with zero drops, but measured 46,973.13 kbps over an implied approximately 1.0644383 seconds and failed the 5% throughput gate. Private Linux full verification/artifact retention and the replacement v10 normative run remain open.
 
 Private Linux run 125 at the frozen v8 harness head is retained as failed evidence. Its exact 33-datagram gap between successful fixture writes and production `MediaIngress` equaled the Linux socket's 33 measured kernel UDP drops. The complete-frame queue, fair pacer, QUIC fragmentation, and public decoder were downstream and did not account for the loss.
 
@@ -22,6 +22,12 @@ Qualification v9 observes every raw public QUIC datagram immediately after the i
 
 Private Linux run 128/job 482 was the single push-triggered attempt at exact source `22433e5c45c179e9d487b59e5d80f1dcf3b285ce`. Linux `make verify`, the sustained gate, strict OpenSpec, deterministic artifact generation, upload, and the clean-checkout step passed. Artifact 28's binaries and SPDX matched the frozen hashes and source metadata. The retained log `/private/tmp/versevdi-gitea-run-128-job-482.log` has SHA-256 `6f5852dce815ba87a55d61aae34cb2fce17a1a62c5ee50e18c0034259ad0a029`. The workflow requested 30 retention days, but Gitea 1.27 floored the positive request delay and the API recorded `2026-08-09T23:18:18+07:00` through `2026-09-07T23:18:18+07:00`, exactly 2,505,600 seconds or 29 elapsed days. The run was not retried. It is passing Linux execution and artifact-byte evidence but retention-nonconforming, so it neither satisfies the private Linux artifact gate nor authorizes normative Section 7. The local 31-day request compensates for verified platform rounding without changing the acceptance threshold; a separately authorized future run must prove an API interval of at least 2,592,000 seconds.
 
+At exact source `c0e362c0285d267f8af4087d07943822311f60e1`, the first Section 7 process created mode-0750 `gateway-rc10-c0e362c` and stopped before fixture startup because the sandbox denied its required loopback bind. The directory remains empty as environment-boundary evidence. The separately authorized escalated attempt retained `gateway-rc10-c0e362c-a2`, passed the v9 runtime checks, and wrote 16 files with manifest SHA-256 `61ea55140dfe6b37332de03879ac33206dd5d47763d09fb0fc2f65bb1f8e02b4`. Independent review denied normative acceptance: the constrained logical CSV and manifest aggregates did not persist every raw public QUIC datagram receive offset/encoded length, and `fairness.csv.gz` lacked manifest transition offsets. A2 therefore remains runtime-passing but normative-raw-evidence-incomplete, without mutation or relabeling.
+
+Qualification v10 keeps the logical impairment CSV unchanged and adds only `impairment-constrained-1080p60-h264-wire.csv.gz`. One monotonic epoch is captured before the constrained delivery and transition sequence. The bounded CSV interleaves two explicit transition records with every public datagram observation in monotonic order using `record_type,reduction_percent,transition_after_ns,received_after_ns,encoded_bytes`; transition-only and delivery-only fields remain empty and are validated as such. Its maximum row count is derived from the existing constrained job bound, the frame-fragment count, and the two configured transitions rather than a captured-run row constant. The manifest binds the file name, SHA-256, compressed bytes, total/delivery/transition row counts, timebase, exact transition offsets, and each capacity summary's recomputation source. Each impairment step is recomputed from every delivery at or after its transition through completion, preserving the original v9 classifier semantics even after the next transition. Fairness remains stage-bounded because `runQualificationFleetStage` records each capacity stage as a separate slice; its summaries retain their 25% and 50% offsets relative to the existing fairness CSV epoch.
+
+The independent parser's caller explicitly selects smoke or normative authority. Normative validation requires exactly 10,000 sent logical units and always enforces the ten-second convergence and 105% rolling-cap gates; retained `sent` data cannot weaken them. Both wire and fairness readers reject compressed input before hashing when it exceeds a writer-derived bound, feed gzip output through a bounded standard-library reader before CSV parsing, and bound fields, offsets, flows, and encoded lengths from the canonical row count, schema, time horizon, and writer values. Aggregate-only v9 data and malformed schema/hash/count/order/transition/length or oversized inputs remain rejected.
+
 The ingress repair follows reviewed behavior rather than copying implementation source:
 
 - Apollo `adc5c5a0bd80831ce495434bb16aee2cd4175fb8`, GPL-3.0, `src/stream.cpp:1463-1474,1573-1627`, supplies the 80%-of-1-Gbps raw-block pacing, 64-KiB/64-packet batch cap, and cross-frame send schedule used by the fixture.
@@ -49,6 +55,8 @@ The native provider therefore requests `2,048 * 1,072 = 2,195,456` bytes with `S
 - Keep video decrypt/FEC single-threaded; only the bounded connected-socket drain is separated so crypto stalls cannot become unexplained kernel loss.
 - Preserve valid scheduling debt after bounded host stalls instead of converting it into provider-queue residence; repay it within the existing fair pacer without a new queue, interface, or configured headroom.
 - Classify constrained-capacity evidence from actual public datagram observations and configured wire capacity; do not infer transport timing from completed logical frames.
+- Persist constrained public-wire observations and transition events in one bounded monotonic-offset CSV, recompute impairment summaries from each transition through completion, and bind both impairment and stage-bounded fairness capacity summaries to their retained raw sources.
+- Select smoke versus normative evidence validation through trusted caller input and bound compressed bytes, decompressed bytes, fields, offsets, flows, and encoded lengths before independent CSV parsing.
 
 ## Risks / Trade-offs
 

openspec/changes/truthful-fixed-frame-qualification/proposal.md

@@ -12,6 +12,8 @@ Private Linux run 127/job 481 at exact source `122080ab342d20585d9a45db0017337b9
 
 Private Linux run 128/job 482 was the single push-triggered attempt at exact source `22433e5c45c179e9d487b59e5d80f1dcf3b285ce`. Linux `make verify`, the sustained gate, strict OpenSpec, deterministic artifact generation, upload, and the clean-checkout step passed. Artifact 28's binaries and SPDX matched the frozen hashes and source metadata. The retained log `/private/tmp/versevdi-gitea-run-128-job-482.log` has SHA-256 `6f5852dce815ba87a55d61aae34cb2fce17a1a62c5ee50e18c0034259ad0a029`. The workflow requested 30 retention days, but Gitea 1.27 floored the positive request delay and its API scheduled exactly 29 elapsed days. The run was not retried: it is passing Linux execution and artifact-byte evidence but retention-nonconforming, so it does not satisfy the private Linux artifact gate or authorize normative Section 7. The local 31-day request preserves the at-least-30-elapsed-day requirement; a separately authorized future run must prove `expires_at - created_at >= 2,592,000` seconds.
 
+At exact source `c0e362c0285d267f8af4087d07943822311f60e1`, the first Section 7 attempt created `gateway-rc10-c0e362c` and stopped at the sandbox loopback-bind boundary, leaving that mode-0750 directory empty. The separately authorized escalated attempt `gateway-rc10-c0e362c-a2` then passed the v9 runtime gates and retained 16 files; its manifest has SHA-256 `61ea55140dfe6b37332de03879ac33206dd5d47763d09fb0fc2f65bb1f8e02b4`. Independent review denied normative acceptance because v9 retained only logical impairment rows and aggregate capacity summaries: it did not retain the raw public datagram timestamps/lengths or fairness transition offsets needed to recompute those summaries. Both attempts remain preserved without relabeling; a2 is runtime-passing but normative-raw-evidence-incomplete.
+
 ## What Changes
 
 - Generate deterministic variable-size encoded frame units at the named frame rates and target bitrates, including bounded keyframes.
@@ -20,6 +22,7 @@ Private Linux run 128/job 482 was the single push-triggered attempt at exact sou
 - Decouple native video socket draining from the single decrypt/FEC processor with a fixed provider-scoped receive pool and request the source-backed video receive-buffer size.
 - Retain bounded valid per-flow pacing debt after a host stall and repay it at no more than 5% above nominal fair share.
 - Measure capacity convergence and rolling caps from each raw public QUIC datagram's observed length and receive time while retaining completed logical-payload observations for integrity and traversal results.
+- Persist the constrained public datagram observations and capacity transitions under one monotonic epoch, bind their hash/counts into the v10 manifest, and retain fairness transition offsets for independent recomputation. Impairment summaries use the full delivery tail after each transition; trusted caller input selects normative validation, and bounded readers reject oversized compressed, decompressed, and field data.
 - Keep short smoke tests separate and leave all prior normative artifacts unchanged.
 
 ## Capabilities
@@ -34,4 +37,4 @@ None.
 
 ## Impact
 
-The qualification harness, its canonical specification, native Apollo video ingress in `gateway/apollo_native.go`, and the existing production fair pacer in `gateway/telemetry.go`. Downstream complete-frame queues, audio/control ingress, codec/FEC formats, QUIC, dependencies, and public interfaces remain unchanged. No codec operation or normative rerun is included. Requirements: P3C-002, P3C-008, P3C-029, P3C-030, P3C-033, VER-009, VER-010, OPS-015.
+The qualification harness and its canonical specification, plus the already-reviewed native Apollo video ingress in `gateway/apollo_native.go` and production fair pacer in `gateway/telemetry.go`. This v10 correction changes test-only retained evidence, not production behavior. Downstream complete-frame queues, audio/control ingress, codec/FEC formats, QUIC, dependencies, and public interfaces remain unchanged. No codec operation or normative rerun is included. Requirements: P3C-002, P3C-008, P3C-029, P3C-030, P3C-033, VER-009, VER-010, OPS-015.

openspec/changes/truthful-fixed-frame-qualification/specs/gateway-qualification/spec.md

@@ -9,7 +9,11 @@ Native Apollo video ingress SHALL request a 2,195,456-byte socket receive buffer
 
 The production fair pacer SHALL retain its 5 ms instantaneous catch-up ceiling. When a flow resumes later than that ceiling, it SHALL carry the remaining valid schedule debt only within the existing 250 ms provider-queue horizon and SHALL repay that debt using an interval no shorter than 20/21 of its nominal equal-tier fair-share interval. It SHALL return to the nominal interval when the debt is repaid. Simultaneous debt across eight equal-tier flows and the existing 25% and 50% capacity changes SHALL preserve the existing share-error contract and SHALL NOT exceed 105% of configured aggregate capacity in any rolling five-second window.
 
-Capacity-step convergence and rolling-cap evidence SHALL use the monotonic receive time and encoded length of every raw public QUIC datagram observed immediately after the independent client's `ReceiveDatagram` returns and before decode or reassembly. For each reduction, target bytes per second and the five-second cap SHALL derive from the configured public-wire rate, `qualificationMediaPacerKbps(profile, reduction) * 1000 / 8`. Completed logical-payload observations SHALL remain separate and SHALL continue to measure payload integrity, loss, reorder, latency, throughput, and queue behavior. A capacity step SHALL include only delivery observations at or after its recorded transition, anchor measurement windows at the first such public delivery, require four consecutive 250 ms windows between 90% and 105% of its target, converge within ten seconds, and remain at or below 105% in every rolling five-second window.
+Capacity-step convergence and rolling-cap evidence SHALL use the monotonic receive time and encoded length of every raw public QUIC datagram observed immediately after the independent client's `ReceiveDatagram` returns and before decode or reassembly. For each reduction, target bytes per second and the five-second cap SHALL derive from the configured public-wire rate, `qualificationMediaPacerKbps(profile, reduction) * 1000 / 8`. Completed logical-payload observations SHALL remain separate and SHALL continue to measure payload integrity, loss, reorder, latency, throughput, and queue behavior. An impairment capacity step SHALL include every delivery observation at or after its recorded transition through constrained-run completion; a later transition SHALL NOT truncate the earlier step's retained tail. It SHALL anchor measurement windows at the first such public delivery, require four consecutive 250 ms windows between 90% and 105% of its target, converge within ten seconds, and remain at or below 105% in every rolling five-second window.
+
+The constrained profile SHALL retain a bounded gzip CSV containing exactly two capacity-transition records and every observed raw public datagram delivery under one monotonic epoch captured before the constrained sequence. The schema SHALL distinguish transition and delivery records and SHALL contain `record_type`, `reduction_percent`, `transition_after_ns`, `received_after_ns`, and `encoded_bytes`; fields not applicable to a record type SHALL remain empty and SHALL be rejected when populated. The manifest SHALL bind the file name, SHA-256, compressed byte count, total row count, delivery row count, transition row count, monotonic timebase, exact 25% and 50% transition offsets, and each capacity summary's raw recomputation source. The row bound SHALL derive from the configured constrained-job and fragment bounds rather than a prior run's observed row count.
+
+The retained fairness manifest SHALL bind its 25% and 50% transition offsets to the monotonic epoch of `fairness.csv.gz`. Fairness recomputation SHALL remain bounded to each separately collected `runQualificationFleetStage` capacity slice. An independent parser SHALL be able to reconstruct each stage and reproduce the rolling-five-second maximum, configured cap, and exact two-second fairness convergence from the retained raw files and manifest alone. The parser SHALL select normative versus smoke validation only from trusted caller input; normative validation SHALL require exactly 10,000 sent logical units and SHALL enforce the ten-second convergence and 105% rolling-cap gates unconditionally. Compressed, decompressed, row, field, offset, flow, and encoded-length limits SHALL derive from canonical writer schemas, configured row limits, and canonical run horizons and SHALL be enforced before CSV parsing can allocate an unbounded record. Missing raw-wire evidence; a wrong file hash, size, or count; duplicate or missing transitions; negative or nonmonotonic offsets; invalid encoded lengths; completed-logical-frame substitution; populated not-applicable fields; oversized input; or a summary mismatch SHALL fail qualification evidence acceptance.
 
 #### Scenario: Healthy fixed profile
 - **WHEN** a frozen candidate runs one fixed profile for the normative duration in the isolated qualification command
@@ -39,6 +43,22 @@ Capacity-step convergence and rolling-cap evidence SHALL use the monotonic recei
 - **WHEN** a constrained 1080p flow carries nonzero bounded debt through the approximately 1.572-second 25% phase before the 50% transition
 - **THEN** convergence and rolling-cap checks use the observed 1,200-byte and 25-byte public datagrams against the configured wire targets, while the separately retained completed-payload observations cannot substitute for transport delivery timing
 
+#### Scenario: Aggregate-only capacity evidence is retained
+- **WHEN** a qualification bundle contains logical-frame impairment rows and aggregate capacity summaries but omits raw public-wire rows or fairness transition offsets
+- **THEN** independent evidence validation rejects the bundle as incomplete even if its in-process runtime assertions passed
+
+#### Scenario: Raw capacity evidence is independently recomputed
+- **WHEN** v10 validation reads the retained constrained wire CSV, fairness CSV, and manifest transitions
+- **THEN** it validates bounded schema, hashes, sizes, counts, monotonic offsets, encoded datagram lengths, and transition uniqueness, then exactly reproduces the full-after-transition impairment targets, four consecutive 250 ms convergence windows, every rolling-five-second maximum, and stage-bounded fairness two-second alignment
+
+#### Scenario: Retained counts cannot weaken normative gates
+- **WHEN** a purported normative bundle retains a sent count other than 10,000 or retains an 11-second convergence or over-cap summary
+- **THEN** validation rejects it regardless of any retained field value, while explicitly selected smoke validation still requires exact raw-summary recomputation
+
+#### Scenario: Retained CSV exceeds bounded evidence grammar
+- **WHEN** a wire or fairness gzip exceeds its canonical compressed or decompressed limit or contains an overlong field, out-of-horizon offset, unknown flow, or out-of-range encoded length
+- **THEN** validation rejects it through the bounded standard-library reader before an unbounded CSV record can be allocated
+
 ### Requirement: Retained private Linux candidate artifact
 A retained private Linux candidate artifact SHALL have API metadata whose `expires_at - created_at` interval is at least 30 elapsed days (2,592,000 seconds). Workflow intent, cleanup lag, and a local copy SHALL NOT substitute for the recorded API interval. A shorter interval SHALL fail the artifact-retention gate even when execution and artifact bytes pass. The workflow request MAY exceed 30 calendar days only to compensate for verified platform rounding; the acceptance threshold remains at least 30 elapsed days.
 

openspec/changes/truthful-fixed-frame-qualification/tasks.md

@@ -23,8 +23,8 @@
 - [x] 5.1 Retain the v6 qualification attempt and mark its passing result superseded by the tight-loop source defect
 - [x] 5.2 Implement v8 post-first-write, non-collapsing complete-frame UDP pacing with persistent cross-frame carry and verify the focused, race, short-resource, and cross-platform compile checks that passed
 - [x] 5.3 Preserve private Linux runs 123 and 124 as failed evidence, the passing local v8 Darwin sustained run as non-Linux and non-normative, and the un-retried failed Darwin non-sustained pre-CI invocation with its exact throughput evidence
-- [ ] 5.4 Run private Linux full verification and retain deterministic Linux artifacts for the frozen v9 measurement descendant
-- [ ] 5.5 Run one separately approved replacement v9 normative Section 7 qualification
+- [ ] 5.4 Run private Linux full verification and retain deterministic Linux artifacts for the frozen v10 evidence descendant
+- [ ] 5.5 Run one separately approved replacement v10 normative Section 7 qualification
 
 ## 6. Native video ingress remediation
 
@@ -42,6 +42,11 @@
 ## 8. Public-wire capacity measurement correction
 
 - [x] 8.1 Preserve run 127/job 481 at `122080ab` as failed evidence with its exact log hash, no artifact, and no retry
-- [x] 8.2 Record raw independent-client QUIC datagram lengths/times and derive v9 capacity convergence and caps from configured wire rate while keeping logical payload observations separate
+- [x] 8.2 Observe raw independent-client QUIC datagram lengths/times in-process and derive v9 capacity convergence and caps from configured wire rate while keeping logical payload observations separate; this did not persist sufficient raw evidence for independent proof
 - [x] 8.3 Preserve run 128/job 482 as passing Linux execution and artifact-byte evidence but retention-nonconforming, with no retry
 - [ ] 8.4 Freeze the 31-day-request descendant and prove a future private artifact records at least 30 elapsed days before tasks 5.4, 5.5, 6.3, or 7.4 can close
+
+## 9. Retained public-wire evidence correction
+
+- [x] 9.1 Preserve the first empty `gateway-rc10-c0e362c` environment-boundary attempt and the runtime-passing but normative-raw-evidence-incomplete `gateway-rc10-c0e362c-a2` bundle without mutation or relabeling
+- [x] 9.2 Persist bounded v10 constrained public-wire rows and fairness transition offsets; recompute each impairment step from its transition through completion and fairness from its separate stage; select normative authority explicitly; and bound compressed, decompressed, row, field, time, flow, and encoded-length parsing