test(gateway): bind qualification to immutable Protocol
This commit is contained in:
+2
-2
@@ -18,7 +18,7 @@
|
||||
|
||||
- [x] 3.1 Run focused red/green checks, strict OpenSpec validation, `make
|
||||
verify`, race/fuzz/resource checks, and freeze the harness commit.
|
||||
- [ ] 3.2 Run the opt-in ten-minute and six-profile command exactly once
|
||||
- [x] 3.2 Run the opt-in ten-minute and six-profile command exactly once
|
||||
against the frozen candidate and archive raw artifacts and hashes.
|
||||
- [ ] 3.3 Sync the canonical specification, archive the completed change, and
|
||||
- [x] 3.3 Sync the canonical specification, archive the completed change, and
|
||||
revalidate strictly without claiming live Apollo/macOS/firewall evidence.
|
||||
@@ -0,0 +1,67 @@
|
||||
# gateway-qualification Specification
|
||||
|
||||
## Purpose
|
||||
Define the deterministic processing, impairment, pacing, and evidence boundaries
|
||||
for qualifying a frozen Phase 3C gateway candidate.
|
||||
## Requirements
|
||||
### Requirement: Fixed media processing qualification
|
||||
The qualification harness SHALL run 1080p60 H.264 at 20 Mbps, 1440p120 HEVC
|
||||
at 50 Mbps, and 4K60 HEVC at 80 Mbps for ten wall-clock minutes each after a
|
||||
recorded warm-up. It SHALL preserve encoded payload bytes, record every
|
||||
monotonic processing sample, report count, min, median, p90, p95, p99, max,
|
||||
mean, standard deviation, timing overhead, and observed bitrate, and fail when
|
||||
any p95 exceeds 5 ms.
|
||||
|
||||
#### Scenario: Healthy fixed profile
|
||||
- **WHEN** a frozen candidate runs one fixed profile for the normative duration
|
||||
- **THEN** the harness emits compressed raw samples and a summary tied to the
|
||||
exact source commit, Protocol version, environment, and payload hash.
|
||||
|
||||
#### Scenario: Processing gate failure
|
||||
- **WHEN** payload integrity fails or measured p95 exceeds 5 ms
|
||||
- **THEN** the qualification command exits unsuccessfully without recording a
|
||||
passing candidate.
|
||||
|
||||
### Requirement: Bounded impairment qualification
|
||||
The harness SHALL run exactly the baseline, latency, jitter, loss, reorder,
|
||||
and constrained Section 7.2 profiles once. Baseline SHALL cover all three
|
||||
media profiles and the other profiles SHALL cover 1080p60. Each artifact SHALL
|
||||
record tool version, exact command/configuration, direction, queue discipline,
|
||||
topology, fixed seed, and observed RTT, jitter, loss, reorder, throughput,
|
||||
drops, and capacity-step statistics.
|
||||
|
||||
#### Scenario: Complete six-profile run
|
||||
- **WHEN** the frozen candidate runs impairment qualification
|
||||
- **THEN** one result exists for each named profile, with no Cartesian
|
||||
expansion and with observed rather than configured statistics.
|
||||
|
||||
#### Scenario: Unsupported or unbounded configuration
|
||||
- **WHEN** a profile name, packet count, queue bound, loss, reorder, or
|
||||
bandwidth step falls outside the fixed catalog
|
||||
- **THEN** the harness rejects it before allocating or running the simulation.
|
||||
|
||||
### Requirement: Fairness and cap qualification
|
||||
The harness SHALL exercise the production fair pacer with eight equal-tier
|
||||
synthetic sessions for the required 60-second virtual interval, report every
|
||||
share error and Jain's fairness index, and fail above 10% share error. It SHALL
|
||||
apply 25% and 50% capacity steps, fail convergence beyond ten virtual seconds,
|
||||
and fail aggregate egress above 105% of the cap over any rolling five-second
|
||||
window.
|
||||
|
||||
#### Scenario: Equal-tier and capacity-step evidence
|
||||
- **WHEN** the frozen candidate runs scheduler qualification
|
||||
- **THEN** the artifact contains per-flow bytes, share errors, Jain's index,
|
||||
step convergence, and rolling cap observations derived from the production
|
||||
pacer.
|
||||
|
||||
### Requirement: Honest qualification boundary
|
||||
Qualification artifacts SHALL contain no provider endpoint, credential,
|
||||
clipboard text, input payload, secret, raw media content, or claim of live
|
||||
Apollo/macOS/firewall interoperability. The harness SHALL add no codec
|
||||
operation, production dependency, cgo, sidecar, or direct provider route.
|
||||
|
||||
#### Scenario: Deterministic evidence publication
|
||||
- **WHEN** qualification completes
|
||||
- **THEN** the manifest labels fake-provider, virtual impairment, and local
|
||||
processing evidence separately and leaves live interoperability
|
||||
deferred-owner-e2e.
|
||||
Reference in New Issue
Block a user