test(gateway): bind qualification to immutable Protocol

This commit is contained in:
sechmachine
2026-07-29 23:02:28 +07:00
parent f37f53eab2
commit 040ca73ce9
8 changed files with 100 additions and 15 deletions
@@ -18,7 +18,7 @@
- [x] 3.1 Run focused red/green checks, strict OpenSpec validation, `make
verify`, race/fuzz/resource checks, and freeze the harness commit.
- [ ] 3.2 Run the opt-in ten-minute and six-profile command exactly once
- [x] 3.2 Run the opt-in ten-minute and six-profile command exactly once
against the frozen candidate and archive raw artifacts and hashes.
- [ ] 3.3 Sync the canonical specification, archive the completed change, and
- [x] 3.3 Sync the canonical specification, archive the completed change, and
revalidate strictly without claiming live Apollo/macOS/firewall evidence.
@@ -0,0 +1,67 @@
# gateway-qualification Specification
## Purpose
Define the deterministic processing, impairment, pacing, and evidence boundaries
for qualifying a frozen Phase 3C gateway candidate.
## Requirements
### Requirement: Fixed media processing qualification
The qualification harness SHALL run 1080p60 H.264 at 20 Mbps, 1440p120 HEVC
at 50 Mbps, and 4K60 HEVC at 80 Mbps for ten wall-clock minutes each after a
recorded warm-up. It SHALL preserve encoded payload bytes, record every
monotonic processing sample, report count, min, median, p90, p95, p99, max,
mean, standard deviation, timing overhead, and observed bitrate, and fail when
any p95 exceeds 5 ms.
#### Scenario: Healthy fixed profile
- **WHEN** a frozen candidate runs one fixed profile for the normative duration
- **THEN** the harness emits compressed raw samples and a summary tied to the
exact source commit, Protocol version, environment, and payload hash.
#### Scenario: Processing gate failure
- **WHEN** payload integrity fails or measured p95 exceeds 5 ms
- **THEN** the qualification command exits unsuccessfully without recording a
passing candidate.
### Requirement: Bounded impairment qualification
The harness SHALL run exactly the baseline, latency, jitter, loss, reorder,
and constrained Section 7.2 profiles once. Baseline SHALL cover all three
media profiles and the other profiles SHALL cover 1080p60. Each artifact SHALL
record tool version, exact command/configuration, direction, queue discipline,
topology, fixed seed, and observed RTT, jitter, loss, reorder, throughput,
drops, and capacity-step statistics.
#### Scenario: Complete six-profile run
- **WHEN** the frozen candidate runs impairment qualification
- **THEN** one result exists for each named profile, with no Cartesian
expansion and with observed rather than configured statistics.
#### Scenario: Unsupported or unbounded configuration
- **WHEN** a profile name, packet count, queue bound, loss, reorder, or
bandwidth step falls outside the fixed catalog
- **THEN** the harness rejects it before allocating or running the simulation.
### Requirement: Fairness and cap qualification
The harness SHALL exercise the production fair pacer with eight equal-tier
synthetic sessions for the required 60-second virtual interval, report every
share error and Jain's fairness index, and fail above 10% share error. It SHALL
apply 25% and 50% capacity steps, fail convergence beyond ten virtual seconds,
and fail aggregate egress above 105% of the cap over any rolling five-second
window.
#### Scenario: Equal-tier and capacity-step evidence
- **WHEN** the frozen candidate runs scheduler qualification
- **THEN** the artifact contains per-flow bytes, share errors, Jain's index,
step convergence, and rolling cap observations derived from the production
pacer.
### Requirement: Honest qualification boundary
Qualification artifacts SHALL contain no provider endpoint, credential,
clipboard text, input payload, secret, raw media content, or claim of live
Apollo/macOS/firewall interoperability. The harness SHALL add no codec
operation, production dependency, cgo, sidecar, or direct provider route.
#### Scenario: Deterministic evidence publication
- **WHEN** qualification completes
- **THEN** the manifest labels fake-provider, virtual impairment, and local
processing evidence separately and leaves live interoperability
deferred-owner-e2e.