19 Commits
Author SHA1 Message Date
sechmachine 2c8023edf9 ci(gateway): retain Linux candidate artifacts
Verify Data Plane / gateway (push) Failing after 13s
2026-07-30 11:17:10 +07:00
sechmachine 0c4d87406e test(gateway): qualify production path artifacts 2026-07-30 11:15:55 +07:00
sechmachine baf4073f68 fix(gateway): secure control and terminal ownership 2026-07-30 11:15:48 +07:00
sechmachine df75b1d250 style(openspec): normalize canonical spec eof 2026-07-30 07:40:27 +07:00
sechmachine f292bd3599 docs(openspec): archive gateway audit remediation 2026-07-30 07:39:18 +07:00
sechmachine 9d627413a0 fix(gateway): qualify stepped impairment throughput 2026-07-30 06:59:32 +07:00
sechmachine b7a42dc652 fix(gateway): qualify concurrent fair allocation 2026-07-30 06:19:00 +07:00
sechmachine ff23fc9876 fix(gateway): settle qualification enqueue evidence 2026-07-30 05:38:51 +07:00
sechmachine 99d9d7f85a fix(gateway): accept positive traversal counter deltas 2026-07-30 05:31:17 +07:00
sechmachine a2f8e32d4e fix(gateway): synchronize qualification stage evidence 2026-07-30 05:12:48 +07:00
sechmachine 8904753c16 build(gateway): pin immutable Protocol RC7 2026-07-30 05:06:22 +07:00
sechmachine 70667d5aee fix(gateway): enforce audited production traversal 2026-07-30 04:48:02 +07:00
sechmachine d3852d15f3 fix(gateway): close Phase 3C audit gaps 2026-07-30 01:46:00 +07:00
sechmachine 040ca73ce9 test(gateway): bind qualification to immutable Protocol 2026-07-29 23:02:28 +07:00
sechmachine f37f53eab2 test(gateway): add Section 7 qualification harness 2026-07-29 22:13:48 +07:00
sechmachine 8826f5c804 fix(gateway): bound fake media shutdown 2026-07-29 22:13:32 +07:00
sechmachine 994d1f00d1 chore(gateway): pin Protocol RC6 2026-07-29 21:47:54 +07:00
sechmachine 45be5ccea9 fix(gateway): use registered control flows 2026-07-29 21:16:58 +07:00
sechmachine 236c7c2973 feat(gateway): repair native Apollo provider path 2026-07-29 17:24:34 +07:00
66 changed files with 10164 additions and 443 deletions
+13
View File
@@ -28,5 +28,18 @@ jobs:
bun-version: "1.3.13"
- name: Verify Go and OpenSpec baseline
run: make OPENSPEC='bunx --bun @fission-ai/openspec@1.5.0' verify
- name: Build and inspect reproducible Linux gateway artifacts
run: |
make gateway-linux DIST_DIR=dist
sha256sum dist/verse-gateway-linux-* | tee dist/SHA256SUMS
file dist/verse-gateway-linux-* | tee dist/file.txt
go version -m dist/verse-gateway-linux-amd64 > dist/go-version-amd64.txt
go version -m dist/verse-gateway-linux-arm64 > dist/go-version-arm64.txt
- uses: christopherhx/gitea-upload-artifact@v4
with:
name: verse-gateway-linux-${{ gitea.sha }}
path: dist/*
if-no-files-found: error
retention-days: 30
- name: Verify clean checkout
run: git diff --exit-code
+7 -1
View File
@@ -1,7 +1,8 @@
.PHONY: format-check module-verify build vet test openspec verify
.PHONY: format-check module-verify build vet test openspec gateway-linux verify
GO ?= go
OPENSPEC ?= openspec
DIST_DIR ?= dist
format-check:
@test -z "$$(gofmt -l $$(find gateway -type f -name '*.go' -print))"
@@ -21,4 +22,9 @@ test:
openspec:
$(OPENSPEC) validate --all --strict --no-interactive
gateway-linux:
mkdir -p "$(DIST_DIR)"
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GOWORK=off $(GO) build -mod=readonly -trimpath -buildvcs=false -ldflags=-buildid= -o "$(DIST_DIR)/verse-gateway-linux-amd64" ./cmd/verse-gateway
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 GOWORK=off $(GO) build -mod=readonly -trimpath -buildvcs=false -ldflags=-buildid= -o "$(DIST_DIR)/verse-gateway-linux-arm64" ./cmd/verse-gateway
verify: format-check module-verify build vet test openspec
+21 -8
View File
@@ -9,15 +9,28 @@ The gateway uses the following exact third-party dependency:
The existing Apple Xcode project was generated by the platform tool and remains
reserved for a later native-client phase.
Apollo, Moonlight, and related repositories are external research references
only. Before any source is copied, adapted, linked, embedded, or used to create
fixtures, update this file and the Phase 3C provenance record with:
The gateway's Apollo fixtures and provider-scoped protocol implementation were
independently implemented after consulting wire behavior in these exact
external research references. No implementation source from them is copied,
linked, or embedded:
- upstream repository and exact commit;
- source and destination paths;
- license and retained notices;
- whether the work is copied, derived, or independently implemented; and
- modifications made by VerseVDI.
- Apollo `adc5c5a0bd80831ce495434bb16aee2cd4175fb8`, GPLv3:
`src/rtsp.cpp`, `src/stream.cpp`, `src/audio.cpp`, `src/audio.h`,
`src/nvhttp.cpp`, `LICENSE`, and `NOTICE`.
- Moonlight Qt `c0c4d6056569bba40ac4458a3c225c05ff86df6d` with common-c
pin `2ea47752c3051d72a64bcca190024e8b354fa1ef`, GPLv3:
`src/ControlStream.c`, `src/Video.h`, `src/RtpAudioQueue.h`,
`src/RtpAudioQueue.c`, `src/SdpGenerator.c`, `src/AudioStream.c`, and
`LICENSE.txt`. The consulted files were verified byte-identical to the
recorded local standalone common-c checkout `703a0694`.
- cgutman/enet `aca87840b57f045a1f7f9299e4b1b9b8e2a5e2f1`, MIT:
`protocol.c`, `peer.c`, `include/enet/protocol.h`, and `LICENSE`.
- nanors `17fc7d61afb2fdd9a9aff38fbd7d4f2ff73a4508`, MIT:
`rs.c`, `deps/obl/gf2_8_tables.h`, and `LICENSE`.
Any future copying, adaptation, linking, or embedding requires an updated
source/destination provenance record and retained license notices before the
change is accepted.
The VerseVDI Protocol is maintained in a separate repository and must be
consumed only through an exact immutable release.
+75 -2
View File
@@ -57,7 +57,7 @@ func run() error {
controlPlaneClient := gateway.NewControlPlaneClient(controlPlane, &http.Client{Transport: transport, Timeout: 5 * time.Second})
provider := gateway.NewApolloAdapter(gateway.NewNativeApolloBackend(), gateway.ProviderIdentity{})
capabilities := gateway.DefaultCapabilities()
server, err := gateway.NewServer(gateway.ServerConfig{ListenAddress: listen, TLSConfig: serverTLS, GatewayID: gatewayID, Capabilities: capabilities, ProviderCapabilities: capabilities, Admission: controlPlaneClient, ProviderStateReporter: controlPlaneClient, Provider: provider, PacerKbps: 100000})
server, err := gateway.NewServer(gateway.ServerConfig{ListenAddress: listen, TLSConfig: serverTLS, GatewayID: gatewayID, Capabilities: capabilities, ProviderCapabilities: capabilities, Admission: controlPlaneClient, ProviderStateReporter: controlPlaneClient, ClipboardAuditReporter: controlPlaneClient, Provider: provider, PacerKbps: 100000})
if err != nil {
return err
}
@@ -90,6 +90,8 @@ func heartbeatLoop(ctx context.Context, client *gateway.ControlPlaneClient, serv
ticker := time.NewTicker(2 * time.Second)
defer ticker.Stop()
var sequence int64
var sampler heartbeatSampler
_, _ = sampler.sample(time.Now(), server.Metrics())
for {
select {
case <-ctx.Done():
@@ -104,11 +106,82 @@ func heartbeatLoop(ctx context.Context, client *gateway.ControlPlaneClient, serv
state = "draining"
}
metrics := server.Metrics()
_ = client.Heartbeat(ctx, protocol.GatewayHeartbeat{Version: "1", GatewayID: registration.GatewayID, Sequence: sequence, ObservedAt: time.Now().UTC().Format(time.RFC3339Nano), ActiveConnections: metrics.ActiveSessions, EgressKbps: registration.BandwidthCapacityKbps, State: state})
observedAt := time.Now()
egressKbps, telemetry := sampler.sample(observedAt, metrics)
_ = client.Heartbeat(ctx, protocol.GatewayHeartbeat{
Version: "1", GatewayID: registration.GatewayID, Sequence: sequence,
ObservedAt: observedAt.UTC().Format(time.RFC3339Nano), ActiveConnections: metrics.ActiveSessions,
EgressKbps: egressKbps, State: state, Telemetry: telemetry,
})
}
}
}
type heartbeatSampler struct {
observedAt time.Time
mediaBytes uint64
}
func (s *heartbeatSampler) sample(observedAt time.Time, metrics gateway.MetricsSnapshot) (int64, protocol.GatewayTelemetry) {
egressKbps := int64(0)
elapsedMillis := observedAt.Sub(s.observedAt).Milliseconds()
if !s.observedAt.IsZero() && elapsedMillis > 0 && metrics.MediaBytes >= s.mediaBytes {
delta := metrics.MediaBytes - s.mediaBytes
milliseconds := uint64(elapsedMillis)
whole, remainder := delta/milliseconds, delta%milliseconds
if whole > 125_000_000 {
egressKbps = 1_000_000_000
} else {
rate := whole*8 + remainder*8/milliseconds
if rate > 1_000_000_000 {
rate = 1_000_000_000
}
egressKbps = int64(rate)
}
}
s.observedAt, s.mediaBytes = observedAt, metrics.MediaBytes
return egressKbps, protocol.GatewayTelemetry{
AdmittedSessions: boundedMetric(metrics.AdmittedSessions), AdmissionRejects: boundedMetric(metrics.AdmissionRejects),
Reconnects: boundedMetric(metrics.Reconnects), DrainTransitions: boundedMetric(metrics.DrainTransitions),
MediaDrops: boundedMetric(metrics.MediaDrops), MediaPackets: boundedMetric(metrics.MediaPackets),
MediaBytes: boundedMetric(metrics.MediaBytes), QueueDelayMicros: boundedMetric(metrics.QueueDelayNanos / 1000),
ProcessingDelayMicros: boundedMetric(metrics.ProcessingDelayNanos / 1000), ProcessingSamples: boundedMetric(metrics.ProcessingSamples),
PacingDelayMicros: boundedMetric(metrics.PacingDelayNanos / 1000), ProviderErrors: boundedMetric(metrics.ProviderErrors),
InputRejected: boundedMetric(metrics.InputRejected), ControlRttMicros: boundedMetric(metrics.ControlRTTNanos / 1000),
ControlJitterMicros: boundedMetric(metrics.ControlJitterNanos / 1000), ControlLossPpm: boundedMetric(metrics.ControlLossPPM),
PendingReliable: boundedMetric(metrics.PendingReliable), ProviderState: providerStateName(metrics.ProviderState),
}
}
func boundedMetric(value uint64) int64 {
const maximum = uint64(^uint64(0) >> 1)
if value > maximum {
return int64(maximum)
}
return int64(value)
}
func providerStateName(value uint64) string {
switch value {
case 1:
return gateway.ProviderStateStarting
case 2:
return gateway.ProviderStateReady
case 3:
return gateway.ProviderStateDisconnected
case 4:
return gateway.ProviderStateTerminating
case 5:
return gateway.ProviderStateTerminated
case 6:
return gateway.ProviderStateCleanup
case 7:
return gateway.ProviderStateFailed
default:
return "unknown"
}
}
func loadTLS(certFile, keyFile, clientCAFile string) (*tls.Config, *tls.Config, error) {
certificate, err := tls.LoadX509KeyPair(certFile, keyFile)
if err != nil {
+116
View File
@@ -0,0 +1,116 @@
package main
import (
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"io"
"math/big"
"net/http"
"net/http/httptest"
"testing"
"time"
"git.sechmachine.io.vn/sechmachine/VerseVDI-Data-Plane/gateway"
protocol "git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol/gen/go/protocol"
)
func TestHeartbeatReportsMeasuredEgressInsteadOfConfiguredCapacity(t *testing.T) {
heartbeats := make(chan protocol.GatewayHeartbeat, 1)
control := httptest.NewTLSServer(http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
if request.URL.Path == "/api/v1/gateway/heartbeat" {
heartbeat, err := protocol.DecodeGatewayHeartbeat(mustReadBody(t, request))
if err != nil {
t.Errorf("decode heartbeat: %v", err)
} else {
heartbeats <- heartbeat
}
}
response.WriteHeader(http.StatusNoContent)
}))
defer control.Close()
server, err := gateway.NewServer(gateway.ServerConfig{
ListenAddress: "127.0.0.1:0", TLSConfig: heartbeatTestTLS(t), GatewayID: "gateway-1",
Admission: gateway.AdmissionFunc(func(context.Context, protocol.TunnelAdmissionRequest) (protocol.SessionAuthority, error) {
return protocol.SessionAuthority{}, gateway.ErrAdmissionRejected
}),
Provider: gateway.NewFakeApollo(gateway.FakeApolloConfig{}),
})
if err != nil {
t.Fatal(err)
}
defer server.Close()
registration := protocol.GatewayRegistration{GatewayID: "gateway-1", BandwidthCapacityKbps: 100000}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go heartbeatLoop(ctx, gateway.NewControlPlaneClient(control.URL, control.Client()), server, registration)
select {
case heartbeat := <-heartbeats:
cancel()
if heartbeat.EgressKbps != 0 {
t.Fatalf("idle measured egress = %d Kbps, want 0; configured capacity is not traffic", heartbeat.EgressKbps)
}
case <-time.After(3 * time.Second):
t.Fatal("heartbeat was not published")
}
}
func TestHeartbeatSamplerUsesByteDeltaAndMonotonicElapsed(t *testing.T) {
var sampler heartbeatSampler
start := time.Now()
if egress, _ := sampler.sample(start, gateway.MetricsSnapshot{MediaBytes: 1000}); egress != 0 {
t.Fatalf("first sample egress = %d, want baseline 0", egress)
}
egress, telemetry := sampler.sample(start.Add(2*time.Second), gateway.MetricsSnapshot{
AdmittedSessions: 2, AdmissionRejects: 3, Reconnects: 4, DrainTransitions: 5,
MediaDrops: 6, MediaPackets: 7, MediaBytes: 17000, QueueDelayNanos: 9000,
ProcessingDelayNanos: 10000, ProcessingSamples: 11, PacingDelayNanos: 12000,
ProviderErrors: 13, InputRejected: 14, ControlRTTNanos: 15000,
ControlJitterNanos: 16000, ControlLossPPM: 17, PendingReliable: 18, ProviderState: 2,
})
if egress != 64 || telemetry.MediaBytes != 17000 || telemetry.MediaPackets != 7 ||
telemetry.QueueDelayMicros != 9 || telemetry.ProviderState != gateway.ProviderStateReady {
t.Fatalf("sample = egress:%d telemetry:%#v", egress, telemetry)
}
}
func mustReadBody(t *testing.T, request *http.Request) []byte {
t.Helper()
defer request.Body.Close()
data, err := io.ReadAll(request.Body)
if err != nil {
t.Fatal(err)
}
return data
}
func heartbeatTestTLS(t *testing.T) *tls.Config {
t.Helper()
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
template := &x509.Certificate{
SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "gateway.test"},
NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour),
IsCA: true, BasicConstraintsValid: true, KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
}
der, err := x509.CreateCertificate(rand.Reader, template, template, &key.PublicKey, key)
if err != nil {
t.Fatal(err)
}
certificate := tls.Certificate{Certificate: [][]byte{der}, PrivateKey: key}
pool := x509.NewCertPool()
pool.AddCert(template)
return &tls.Config{
MinVersion: tls.VersionTLS13, Certificates: []tls.Certificate{certificate},
ClientAuth: tls.RequireAndVerifyClientCert, ClientCAs: pool,
}
}
+160
View File
@@ -0,0 +1,160 @@
package gateway
const (
apolloAudioDataShards = 4
apolloAudioParityShards = 2
apolloAudioTotalShards = apolloAudioDataShards + apolloAudioParityShards
apolloAudioMaximumBlocks = 4
)
type apolloAudioFECBlock struct {
base uint16
timestamp uint32
ssrc uint32
haveFEC bool
size int
shards [apolloAudioTotalShards][]byte
received [apolloAudioTotalShards]bool
count int
}
type apolloAudioAssembler struct {
blocks map[uint16]*apolloAudioFECBlock
}
func (a *apolloAudioAssembler) Add(codec *apolloMediaCodec, shard apolloAudioShard) ([][]byte, bool, error) {
if codec == nil || len(shard.payload) == 0 || len(shard.payload) > 1408 || len(shard.payload)%16 != 0 {
return nil, false, errApolloMedia
}
if a.blocks == nil {
a.blocks = make(map[uint16]*apolloAudioFECBlock)
}
base := shard.base
if base&3 != 0 {
return nil, false, errApolloMedia
}
index := 0
if shard.parity {
if shard.parityIndex >= apolloAudioParityShards {
return nil, false, errApolloMedia
}
index = apolloAudioDataShards + int(shard.parityIndex)
} else {
index = int(uint16(shard.sequence - base))
if index >= apolloAudioDataShards {
return nil, false, errApolloMedia
}
}
evicted := false
block := a.blocks[base]
if block == nil {
if len(a.blocks) >= apolloAudioMaximumBlocks {
var oldest uint16
var maximumAge uint16
for candidate := range a.blocks {
age := base - candidate
if age > maximumAge && age < 1<<15 {
oldest, maximumAge = candidate, age
}
}
if maximumAge == 0 {
return nil, false, errApolloMedia
}
delete(a.blocks, oldest)
evicted = true
}
block = &apolloAudioFECBlock{base: base}
a.blocks[base] = block
}
if block.size == 0 {
block.size = len(shard.payload)
} else if block.size != len(shard.payload) {
return nil, evicted, errApolloMedia
}
if shard.parity {
if block.haveFEC && (block.timestamp != shard.timestamp || block.ssrc != shard.ssrc) {
return nil, evicted, errApolloMedia
}
block.timestamp, block.ssrc, block.haveFEC = shard.timestamp, shard.ssrc, true
} else {
if block.haveFEC && (shard.timestamp != block.timestamp+uint32(index*5) || shard.ssrc != block.ssrc) {
return nil, evicted, errApolloMedia
}
}
if block.received[index] {
return nil, evicted, errApolloMedia
}
block.shards[index] = append([]byte(nil), shard.payload...)
block.received[index] = true
block.count++
if block.count < apolloAudioDataShards {
return nil, evicted, nil
}
if err := reconstructApolloAudioBlock(block); err != nil {
return nil, evicted, err
}
output := make([][]byte, apolloAudioDataShards)
for index := range output {
payload, err := codec.openApolloAudioCipher(base+uint16(index), block.shards[index])
if err != nil {
return nil, evicted, err
}
output[index] = payload
}
delete(a.blocks, base)
return output, evicted, nil
}
func reconstructApolloAudioBlock(block *apolloAudioFECBlock) error {
if block == nil || block.count < apolloAudioDataShards || block.size == 0 {
return errApolloMedia
}
missing := false
for index := 0; index < apolloAudioDataShards; index++ {
if !block.received[index] {
missing = true
block.shards[index] = make([]byte, block.size)
}
}
if !missing {
return nil
}
if !block.haveFEC {
return errApolloMedia
}
rows := make([][]byte, 0, apolloAudioDataShards)
shards := make([][]byte, 0, apolloAudioDataShards)
for index, received := range block.received {
if !received {
continue
}
rows = append(rows, apolloAudioFECRow(index))
shards = append(shards, block.shards[index])
if len(rows) == apolloAudioDataShards {
break
}
}
inverse, ok := apolloGFInvert(rows)
if !ok {
return errApolloMedia
}
for index := 0; index < apolloAudioDataShards; index++ {
if block.received[index] {
continue
}
for source, coefficient := range inverse[index] {
apolloGFAXPY(block.shards[index], shards[source], coefficient)
}
}
return nil
}
func apolloAudioFECRow(index int) []byte {
if index < apolloAudioDataShards {
row := make([]byte, apolloAudioDataShards)
row[index] = 1
return row
}
parity := [8]byte{0x77, 0x40, 0x38, 0x0e, 0xc7, 0xa7, 0x0d, 0x6c}
return append([]byte(nil), parity[(index-apolloAudioDataShards)*apolloAudioDataShards:(index-apolloAudioDataShards+1)*apolloAudioDataShards]...)
}
+106
View File
@@ -0,0 +1,106 @@
package gateway
import (
"crypto/aes"
"crypto/cipher"
"encoding/binary"
"errors"
)
const (
apolloControlOuterType = 0x0001
apolloControlHeaderSize = 8
apolloControlTagSize = 16
apolloControlInnerSize = 4
apolloControlMaximumPlain = 2048
apolloControlTypeStart = 0x0307
apolloControlTypeIDR = 0x0302
apolloControlTypePing = 0x0200
apolloControlTypeInput = 0x0206
apolloControlTypeFEC = 0x5502
apolloControlTypeRumble = 0x010b
apolloControlTypeHDR = 0x010e
apolloControlTypeTerm = 0x0109
)
var errApolloControl = errors.New("apollo control malformed")
type apolloControlMessage struct {
typeID uint16
payload []byte
}
type apolloControlCodec struct {
aead cipher.AEAD
nextClient uint32
lastHost uint32
hostSeen bool
}
func newApolloControlCodec(key []byte) (*apolloControlCodec, error) {
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
aead, err := cipher.NewGCM(block)
if err != nil {
return nil, err
}
return &apolloControlCodec{aead: aead}, nil
}
func (c *apolloControlCodec) SealClient(typeID uint16, payload []byte) ([]byte, error) {
if c == nil || c.aead == nil || typeID == 0 || len(payload) > apolloControlMaximumPlain || c.nextClient == ^uint32(0) {
return nil, errApolloControl
}
sequence := c.nextClient
c.nextClient++
inner := make([]byte, apolloControlInnerSize+len(payload))
binary.LittleEndian.PutUint16(inner[:2], typeID)
binary.LittleEndian.PutUint16(inner[2:4], uint16(len(payload)))
copy(inner[4:], payload)
nonce := apolloControlNonce(sequence, 'C')
sealed := c.aead.Seal(nil, nonce[:], inner, nil)
packet := make([]byte, apolloControlHeaderSize+len(sealed))
binary.LittleEndian.PutUint16(packet[:2], apolloControlOuterType)
binary.LittleEndian.PutUint16(packet[2:4], uint16(4+len(sealed)))
binary.LittleEndian.PutUint32(packet[4:8], sequence)
copy(packet[8:24], sealed[len(inner):])
copy(packet[24:], sealed[:len(inner)])
return packet, nil
}
func (c *apolloControlCodec) OpenHost(packet []byte) (apolloControlMessage, error) {
if c == nil || c.aead == nil || len(packet) < apolloControlHeaderSize+apolloControlTagSize+apolloControlInnerSize || len(packet) > apolloControlHeaderSize+apolloControlTagSize+apolloControlMaximumPlain {
return apolloControlMessage{}, errApolloControl
}
if binary.LittleEndian.Uint16(packet[:2]) != apolloControlOuterType || int(binary.LittleEndian.Uint16(packet[2:4])) != len(packet)-4 {
return apolloControlMessage{}, errApolloControl
}
sequence := binary.LittleEndian.Uint32(packet[4:8])
if c.hostSeen && sequence <= c.lastHost {
return apolloControlMessage{}, errApolloControl
}
nonce := apolloControlNonce(sequence, 'H')
sealed := make([]byte, len(packet)-apolloControlHeaderSize)
copy(sealed, packet[24:])
copy(sealed[len(packet)-24:], packet[8:24])
plaintext, err := c.aead.Open(nil, nonce[:], sealed, nil)
if err != nil || len(plaintext) < apolloControlInnerSize {
return apolloControlMessage{}, errApolloControl
}
length := int(binary.LittleEndian.Uint16(plaintext[2:4]))
if length != len(plaintext)-apolloControlInnerSize || length > apolloControlMaximumPlain {
return apolloControlMessage{}, errApolloControl
}
c.lastHost, c.hostSeen = sequence, true
return apolloControlMessage{typeID: binary.LittleEndian.Uint16(plaintext[:2]), payload: append([]byte(nil), plaintext[4:]...)}, nil
}
func apolloControlNonce(sequence uint32, origin byte) [12]byte {
var nonce [12]byte
binary.LittleEndian.PutUint32(nonce[:4], sequence)
nonce[10], nonce[11] = origin, 'C'
return nonce
}
+709
View File
@@ -0,0 +1,709 @@
package gateway
import (
"context"
"crypto/rand"
"encoding/binary"
"errors"
"net"
"sync"
"time"
)
const (
apolloENetChannels = 48
apolloENetMaximumPacket = 4096
apolloENetMaximumPayload = 2048
apolloENetMaximumCommands = 32
apolloENetMaximumPending = 128
apolloENetMaximumReorder = 64
// These are the pinned ENet fork defaults. The connection remains subject
// to the stricter ten-second no-receive peer deadline below.
apolloENetTimeoutLimit = 32
apolloENetTimeoutMinimum = 5 * time.Second
apolloENetTimeoutMaximum = 30 * time.Second
apolloENetPeerTimeout = 10 * time.Second
apolloENetPeerIDMask = 0x0fff
apolloENetSentTimeFlag = 0x8000
apolloENetCompressedFlag = 0x4000
apolloENetSessionMask = 0x3000
apolloENetSessionShift = 12
apolloENetCommandMask = 0x0f
apolloENetAcknowledged = 0x80
apolloENetUnsequenced = 0x40
apolloENetConnect = 2
apolloENetVerifyConnect = 3
apolloENetDisconnect = 4
apolloENetPing = 5
apolloENetSendReliable = 6
apolloENetSendUnsequenced = 9
apolloENetBandwidthLimit = 10
apolloENetThrottleConfig = 11
)
var errApolloENet = errors.New("apollo ENet malformed")
type apolloENetState uint8
const (
apolloENetConnecting apolloENetState = iota
apolloENetConnected
apolloENetDisconnecting
apolloENetClosed
)
type apolloENetChannel struct {
nextOutgoing uint16
lastIncoming uint16
hasIncoming bool
incoming map[uint16][]byte
}
type apolloENetPendingKey struct {
channel uint8
sequence uint16
}
type apolloENetPending struct {
packet []byte
firstSent time.Time
sentTime time.Time
timeout time.Duration
attempts uint8
}
// apolloENetPeer is deliberately scoped to the Apollo adapter. It implements
// one negotiated ENet peer over one connected UDP socket and exposes no
// reusable transport abstraction.
type apolloENetPeer struct {
conn *net.UDPConn
now func() time.Time
mu sync.Mutex
state apolloENetState
peerID uint16
inboundSession uint8
outboundSession uint8
connectID uint32
channels [apolloENetChannels]apolloENetChannel
pending map[apolloENetPendingKey]*apolloENetPending
unsequenced uint16
rtt time.Duration
variance time.Duration
reliableSent uint64
retransmits uint64
lastReceive time.Time
lastSend time.Time
lastPing time.Time
disconnectAck chan struct{}
disconnectSeq uint16
onPayload func(uint8, bool, []byte)
onDisconnect func(error)
done chan struct{}
closeOnce sync.Once
}
func newApolloENetPeer(conn *net.UDPConn, now func() time.Time) (*apolloENetPeer, error) {
if conn == nil {
return nil, ErrProviderMalformed
}
if now == nil {
now = time.Now
}
return &apolloENetPeer{
conn: conn, now: now, state: apolloENetConnecting, pending: make(map[apolloENetPendingKey]*apolloENetPending),
rtt: 500 * time.Millisecond, variance: time.Millisecond, done: make(chan struct{}),
}, nil
}
func (p *apolloENetPeer) Connect(ctx context.Context, connectData uint32) error {
if p == nil || connectData == 0 {
return ErrProviderMalformed
}
var id [4]byte
if _, err := rand.Read(id[:]); err != nil {
return err
}
p.mu.Lock()
if p.state != apolloENetConnecting {
p.mu.Unlock()
return ErrProviderMalformed
}
p.connectID = binary.BigEndian.Uint32(id[:])
now := p.now()
packet := apolloENetConnectPacket(now, p.connectID, connectData)
p.pending[apolloENetPendingKey{channel: 0xff, sequence: 1}] = &apolloENetPending{packet: append([]byte(nil), packet...), firstSent: now, sentTime: now, timeout: apolloENetRetransmitTimeout(p.rtt, p.variance, 1), attempts: 1}
err := p.writeLocked(packet)
p.mu.Unlock()
if err != nil {
p.close(err)
return err
}
for {
if err := p.readOnce(ctx); err != nil {
p.close(err)
return err
}
p.mu.Lock()
connected := p.state == apolloENetConnected
p.mu.Unlock()
if connected {
go p.run()
return nil
}
}
}
func apolloENetConnectPacket(now time.Time, connectID, data uint32) []byte {
packet := make([]byte, 52)
apolloENetHeader(packet, apolloENetPeerIDMask, 0, now)
packet[4] = apolloENetConnect | apolloENetAcknowledged
packet[5] = 0xff
binary.BigEndian.PutUint16(packet[6:8], 1)
binary.BigEndian.PutUint16(packet[8:10], 0)
packet[10], packet[11] = 0xff, 0xff
binary.BigEndian.PutUint32(packet[12:16], 1400)
binary.BigEndian.PutUint32(packet[16:20], 32768)
binary.BigEndian.PutUint32(packet[20:24], apolloENetChannels)
binary.BigEndian.PutUint32(packet[24:28], 0)
binary.BigEndian.PutUint32(packet[28:32], 0)
binary.BigEndian.PutUint32(packet[32:36], 5000)
binary.BigEndian.PutUint32(packet[36:40], 2)
binary.BigEndian.PutUint32(packet[40:44], 2)
binary.BigEndian.PutUint32(packet[44:48], connectID)
binary.BigEndian.PutUint32(packet[48:52], data)
return packet
}
func apolloENetHeader(packet []byte, peerID uint16, session uint8, now time.Time) {
value := peerID&apolloENetPeerIDMask | (uint16(session&3) << apolloENetSessionShift) | apolloENetSentTimeFlag
binary.BigEndian.PutUint16(packet[:2], value)
binary.BigEndian.PutUint16(packet[2:4], uint16(now.UnixMilli()))
}
func (p *apolloENetPeer) run() {
ticker := time.NewTicker(25 * time.Millisecond)
defer ticker.Stop()
for {
select {
case <-p.done:
return
case <-ticker.C:
if err := p.maintain(); err != nil {
p.close(err)
return
}
default:
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Millisecond)
err := p.readOnce(ctx)
cancel()
if err != nil && !errors.Is(err, context.DeadlineExceeded) && !isApolloENetTimeout(err) {
p.close(err)
return
}
}
}
}
func (p *apolloENetPeer) readOnce(ctx context.Context) error {
if p == nil || p.conn == nil {
return ErrProviderDisconnected
}
deadline := time.Now().Add(100 * time.Millisecond)
if contextDeadline, ok := ctx.Deadline(); ok && contextDeadline.Before(deadline) {
deadline = contextDeadline
}
if err := p.conn.SetReadDeadline(deadline); err != nil {
return err
}
buffer := make([]byte, apolloENetMaximumPacket+1)
count, err := p.conn.Read(buffer)
if err != nil {
if networkErr, ok := err.(net.Error); ok && networkErr.Timeout() {
return context.DeadlineExceeded
}
return err
}
if count < 4 || count > apolloENetMaximumPacket {
return errApolloENet
}
return p.handleDatagram(buffer[:count])
}
func isApolloENetTimeout(err error) bool {
return errors.Is(err, context.DeadlineExceeded)
}
func (p *apolloENetPeer) handleDatagram(packet []byte) error {
if len(packet) < 4 || len(packet) > apolloENetMaximumPacket {
return errApolloENet
}
header := binary.BigEndian.Uint16(packet[:2])
if header&apolloENetCompressedFlag != 0 {
return errApolloENet
}
peerID := header & apolloENetPeerIDMask
session := uint8((header & apolloENetSessionMask) >> apolloENetSessionShift)
offset := 2
sentTime := uint16(0)
if header&apolloENetSentTimeFlag != 0 {
if len(packet) < 4 {
return errApolloENet
}
sentTime = binary.BigEndian.Uint16(packet[2:4])
offset = 4
}
p.mu.Lock()
state := p.state
if state == apolloENetClosed || (state == apolloENetConnected && (peerID != p.peerID || session != p.inboundSession)) {
p.mu.Unlock()
return errApolloENet
}
p.lastReceive = p.now()
p.mu.Unlock()
commands := 0
for offset < len(packet) {
commands++
if commands > apolloENetMaximumCommands || len(packet)-offset < 4 {
return errApolloENet
}
command := packet[offset] & apolloENetCommandMask
flags := packet[offset]
channel := packet[offset+1]
sequence := binary.BigEndian.Uint16(packet[offset+2 : offset+4])
consumed, err := p.handleCommand(command, flags, channel, sequence, sentTime, packet[offset:])
if err != nil || consumed < 4 || consumed > len(packet)-offset {
return errApolloENet
}
offset += consumed
}
return nil
}
func (p *apolloENetPeer) handleCommand(command, flags, channel uint8, sequence, sentTime uint16, data []byte) (int, error) {
switch command {
case 1:
if len(data) < 8 {
return 0, errApolloENet
}
return 8, p.acknowledge(channel, binary.BigEndian.Uint16(data[4:6]), binary.BigEndian.Uint16(data[6:8]))
case apolloENetVerifyConnect:
if len(data) < 44 {
return 0, errApolloENet
}
return 44, p.verifyConnect(sequence, sentTime, data[:44])
case apolloENetDisconnect:
if len(data) < 8 {
return 0, errApolloENet
}
if flags&apolloENetAcknowledged != 0 {
if err := p.sendAcknowledge(channel, sequence, sentTime); err != nil {
return 0, err
}
}
return 8, ErrProviderDisconnected
case apolloENetPing:
if flags&apolloENetAcknowledged != 0 {
if err := p.sendAcknowledge(channel, sequence, sentTime); err != nil {
return 0, err
}
}
return 4, nil
case apolloENetSendReliable:
if len(data) < 6 {
return 0, errApolloENet
}
length := int(binary.BigEndian.Uint16(data[4:6]))
if length > apolloENetMaximumPayload || len(data) < 6+length {
return 0, errApolloENet
}
if flags&apolloENetAcknowledged == 0 || channel >= apolloENetChannels {
return 0, errApolloENet
}
if err := p.sendAcknowledge(channel, sequence, sentTime); err != nil {
return 0, err
}
deliver, err := p.acceptReliable(channel, sequence, data[6:6+length])
if err != nil {
return 0, err
}
for _, message := range deliver {
p.deliver(channel, true, message)
}
return 6 + length, nil
case apolloENetSendUnsequenced:
if len(data) < 8 {
return 0, errApolloENet
}
length := int(binary.BigEndian.Uint16(data[6:8]))
if flags&apolloENetUnsequenced == 0 || channel >= apolloENetChannels || length > apolloENetMaximumPayload || len(data) < 8+length {
return 0, errApolloENet
}
p.deliver(channel, false, data[8:8+length])
return 8 + length, nil
case apolloENetBandwidthLimit:
if len(data) < 12 {
return 0, errApolloENet
}
return 12, nil
case apolloENetThrottleConfig:
if len(data) < 16 {
return 0, errApolloENet
}
return 16, nil
case apolloENetConnect, 7, 8, 12:
return 0, errApolloENet
default:
return 0, errApolloENet
}
}
func (p *apolloENetPeer) verifyConnect(sequence, sentTime uint16, data []byte) error {
p.mu.Lock()
defer p.mu.Unlock()
if p.state != apolloENetConnecting || binary.BigEndian.Uint32(data[40:44]) != p.connectID || binary.BigEndian.Uint32(data[16:20]) != apolloENetChannels {
return errApolloENet
}
p.peerID = binary.BigEndian.Uint16(data[4:6])
p.inboundSession = data[6]
p.outboundSession = data[7]
p.state = apolloENetConnected
delete(p.pending, apolloENetPendingKey{channel: 0xff, sequence: 1})
return p.sendAcknowledgeLocked(0xff, sequence, sentTime)
}
func (p *apolloENetPeer) acknowledge(channel uint8, sequence, sentTime uint16) error {
p.mu.Lock()
if p.state == apolloENetDisconnecting && channel == 0xff && sequence == p.disconnectSeq && p.disconnectAck != nil {
close(p.disconnectAck)
p.disconnectAck = nil
p.mu.Unlock()
return nil
}
key := apolloENetPendingKey{channel: channel, sequence: sequence}
pending, ok := p.pending[key]
if !ok {
p.mu.Unlock()
return nil
}
delete(p.pending, key)
measured := p.now().Sub(pending.sentTime)
if measured < 0 || measured > 30*time.Second {
p.mu.Unlock()
return errApolloENet
}
delta := durationAbs(p.rtt - measured)
p.variance += (delta - p.variance) / 4
p.rtt += (measured - p.rtt) / 8
p.mu.Unlock()
_ = sentTime
return nil
}
func durationAbs(value time.Duration) time.Duration {
if value < 0 {
return -value
}
return value
}
func (p *apolloENetPeer) acceptReliable(channel uint8, sequence uint16, payload []byte) ([][]byte, error) {
p.mu.Lock()
defer p.mu.Unlock()
state := &p.channels[channel]
if !state.hasIncoming {
state.hasIncoming = true
state.lastIncoming = sequence
return [][]byte{append([]byte(nil), payload...)}, nil
}
if sequence == state.lastIncoming+1 {
state.lastIncoming = sequence
deliver := [][]byte{append([]byte(nil), payload...)}
for {
next := state.lastIncoming + 1
queued, ok := state.incoming[next]
if !ok {
return deliver, nil
}
delete(state.incoming, next)
state.lastIncoming = next
deliver = append(deliver, queued)
}
}
if apolloENetSequenceGreater(sequence, state.lastIncoming) {
if uint16(sequence-state.lastIncoming) > 1024 || len(state.incoming) >= apolloENetMaximumReorder {
return nil, errApolloENet
}
if state.incoming == nil {
state.incoming = make(map[uint16][]byte)
}
if _, duplicate := state.incoming[sequence]; !duplicate {
state.incoming[sequence] = append([]byte(nil), payload...)
}
}
return nil, nil
}
func apolloENetSequenceGreater(first, second uint16) bool {
return (first > second && first-second <= 32768) || (first < second && second-first > 32768)
}
func (p *apolloENetPeer) deliver(channel uint8, reliable bool, payload []byte) {
p.mu.Lock()
callback := p.onPayload
p.mu.Unlock()
if callback != nil {
callback(channel, reliable, append([]byte(nil), payload...))
}
}
func (p *apolloENetPeer) SendReliable(channel uint8, payload []byte) error {
if channel >= apolloENetChannels || len(payload) == 0 || len(payload) > apolloENetMaximumPayload {
return ErrProviderMalformed
}
p.mu.Lock()
defer p.mu.Unlock()
if p.state != apolloENetConnected || len(p.pending) >= apolloENetMaximumPending {
return ErrProviderDisconnected
}
state := &p.channels[channel]
state.nextOutgoing++
if state.nextOutgoing == 0 {
state.nextOutgoing++
}
now := p.now()
packet := make([]byte, 10+len(payload))
apolloENetHeader(packet, p.peerID, p.outboundSession, now)
packet[4] = apolloENetSendReliable | apolloENetAcknowledged
packet[5] = channel
binary.BigEndian.PutUint16(packet[6:8], state.nextOutgoing)
binary.BigEndian.PutUint16(packet[8:10], uint16(len(payload)))
copy(packet[10:], payload)
key := apolloENetPendingKey{channel: channel, sequence: state.nextOutgoing}
p.pending[key] = &apolloENetPending{packet: append([]byte(nil), packet...), firstSent: now, sentTime: now, timeout: apolloENetRetransmitTimeout(p.rtt, p.variance, 1), attempts: 1}
if err := p.writeLocked(packet); err != nil {
delete(p.pending, key)
return err
}
p.reliableSent++
return nil
}
func (p *apolloENetPeer) SendUnsequenced(channel uint8, payload []byte) error {
if channel >= apolloENetChannels || len(payload) == 0 || len(payload) > apolloENetMaximumPayload {
return ErrProviderMalformed
}
p.mu.Lock()
defer p.mu.Unlock()
if p.state != apolloENetConnected {
return ErrProviderDisconnected
}
p.unsequenced++
packet := make([]byte, 12+len(payload))
apolloENetHeader(packet, p.peerID, p.outboundSession, p.now())
packet[4] = apolloENetSendUnsequenced | apolloENetUnsequenced
packet[5] = channel
binary.BigEndian.PutUint16(packet[6:8], 0)
binary.BigEndian.PutUint16(packet[8:10], p.unsequenced)
binary.BigEndian.PutUint16(packet[10:12], uint16(len(payload)))
copy(packet[12:], payload)
return p.writeLocked(packet)
}
func (p *apolloENetPeer) sendAcknowledge(channel uint8, sequence, sentTime uint16) error {
p.mu.Lock()
defer p.mu.Unlock()
return p.sendAcknowledgeLocked(channel, sequence, sentTime)
}
func (p *apolloENetPeer) sendAcknowledgeLocked(channel uint8, sequence, sentTime uint16) error {
if p.state == apolloENetClosed {
return ErrProviderDisconnected
}
packet := make([]byte, 12)
peerID := p.peerID
session := p.outboundSession
if p.state == apolloENetConnecting {
peerID, session = apolloENetPeerIDMask, 0
}
apolloENetHeader(packet, peerID, session, p.now())
packet[4] = 1
packet[5] = channel
binary.BigEndian.PutUint16(packet[6:8], 0)
binary.BigEndian.PutUint16(packet[8:10], sequence)
binary.BigEndian.PutUint16(packet[10:12], sentTime)
return p.writeLocked(packet)
}
func (p *apolloENetPeer) maintain() error {
p.mu.Lock()
defer p.mu.Unlock()
if p.state != apolloENetConnected && p.state != apolloENetDisconnecting {
return nil
}
now := p.now()
if !p.lastReceive.IsZero() && now.Sub(p.lastReceive) > apolloENetPeerTimeout {
return ErrProviderTimeout
}
for key, pending := range p.pending {
if now.Sub(pending.sentTime) < pending.timeout {
continue
}
if now.Sub(pending.firstSent) >= apolloENetTimeoutMaximum || (apolloENetExceededTimeoutLimit(pending.attempts) && now.Sub(pending.firstSent) >= apolloENetTimeoutMinimum) {
return ErrProviderTimeout
}
pending.timeout = apolloENetRetransmitTimeout(p.rtt, p.variance, pending.attempts)
pending.attempts++
p.retransmits++
pending.sentTime = now
binary.BigEndian.PutUint16(pending.packet[2:4], uint16(now.UnixMilli()))
if err := p.writeLocked(pending.packet); err != nil {
return err
}
p.pending[key] = pending
}
if now.Sub(p.lastPing) >= 500*time.Millisecond {
p.lastPing = now
return p.sendPingLocked()
}
return nil
}
func apolloENetRetransmitTimeout(rtt, variance time.Duration, attempts uint8) time.Duration {
if rtt < time.Millisecond {
rtt = time.Millisecond
}
if variance < time.Millisecond {
variance = time.Millisecond
}
base := rtt + minDuration(rtt, 4*variance)
if base > apolloENetTimeoutMaximum/5 {
base = apolloENetTimeoutMaximum / 5
}
if attempts == 0 {
attempts = 1
}
if attempts > apolloENetTimeoutLimit {
attempts = apolloENetTimeoutLimit
}
return base * time.Duration(attempts)
}
func apolloENetExceededTimeoutLimit(attempts uint8) bool {
return attempts >= 6 // 1 << (attempts - 1) reaches the fork's limit of 32.
}
func minDuration(first, second time.Duration) time.Duration {
if first < second {
return first
}
return second
}
func (p *apolloENetPeer) sendPingLocked() error {
if len(p.pending) >= apolloENetMaximumPending {
return ErrProviderTimeout
}
state := &p.channels[0]
state.nextOutgoing++
if state.nextOutgoing == 0 {
state.nextOutgoing++
}
now := p.now()
packet := make([]byte, 8)
apolloENetHeader(packet, p.peerID, p.outboundSession, now)
packet[4] = apolloENetPing | apolloENetAcknowledged
packet[5] = 0
binary.BigEndian.PutUint16(packet[6:8], state.nextOutgoing)
p.pending[apolloENetPendingKey{channel: 0, sequence: state.nextOutgoing}] = &apolloENetPending{packet: append([]byte(nil), packet...), firstSent: now, sentTime: now, timeout: apolloENetRetransmitTimeout(p.rtt, p.variance, 1), attempts: 1}
if err := p.writeLocked(packet); err != nil {
delete(p.pending, apolloENetPendingKey{channel: 0, sequence: state.nextOutgoing})
return err
}
p.reliableSent++
return nil
}
func (p *apolloENetPeer) telemetry() ProviderTelemetry {
if p == nil {
return ProviderTelemetry{}
}
p.mu.Lock()
defer p.mu.Unlock()
return ProviderTelemetry{
ControlRTT: p.rtt, ControlJitter: p.variance, ReliableSent: p.reliableSent,
ReliableRetransmits: p.retransmits, PendingReliable: uint64(len(p.pending)),
}
}
func (p *apolloENetPeer) writeLocked(packet []byte) error {
if len(packet) < 4 || len(packet) > apolloENetMaximumPacket {
return ErrProviderMalformed
}
count, err := p.conn.Write(packet)
if err != nil || count != len(packet) {
if err != nil {
return err
}
return ErrProviderDisconnected
}
p.lastSend = p.now()
return nil
}
func (p *apolloENetPeer) Disconnect(ctx context.Context) error {
if p == nil {
return nil
}
p.mu.Lock()
if p.state == apolloENetClosed {
p.mu.Unlock()
return nil
}
p.state = apolloENetDisconnecting
ack := make(chan struct{})
p.disconnectAck, p.disconnectSeq = ack, 1
packet := make([]byte, 12)
apolloENetHeader(packet, p.peerID, p.outboundSession, p.now())
packet[4] = apolloENetDisconnect | apolloENetAcknowledged
packet[5] = 0xff
binary.BigEndian.PutUint16(packet[6:8], 1)
if err := p.writeLocked(packet); err != nil {
p.mu.Unlock()
p.close(err)
return err
}
p.mu.Unlock()
deadline := time.NewTimer(2 * time.Second)
defer deadline.Stop()
select {
case <-ack:
p.close(nil)
return nil
case <-p.done:
return nil
case <-ctx.Done():
p.close(ctx.Err())
return ctx.Err()
case <-deadline.C:
p.close(ErrProviderTimeout)
return ErrProviderTimeout
}
}
func (p *apolloENetPeer) close(err error) {
if p == nil {
return
}
p.closeOnce.Do(func() {
p.mu.Lock()
p.state = apolloENetClosed
callback := p.onDisconnect
p.mu.Unlock()
close(p.done)
_ = p.conn.Close()
if callback != nil && err != nil {
callback(err)
}
})
}
+348
View File
@@ -0,0 +1,348 @@
package gateway
import (
"context"
"encoding/binary"
"encoding/hex"
"net"
"testing"
"time"
)
func TestApolloENetConnectWireVectorAndRTO(t *testing.T) {
now := time.UnixMilli(0x1234)
packet := apolloENetConnectPacket(now, 0x01020304, 0x12345678)
expected, err := hex.DecodeString("8fff123482ff00010000ffff00000578000080000000003000000000000000000000138800000002000000020102030412345678")
if err != nil {
t.Fatal(err)
}
if string(packet) != string(expected) {
t.Fatalf("CONNECT wire bytes = %x, want %x", packet, expected)
}
if got := apolloENetRetransmitTimeout(100*time.Millisecond, 30*time.Millisecond, 1); got != 200*time.Millisecond {
t.Fatalf("initial RTO = %s", got)
}
if got := apolloENetRetransmitTimeout(100*time.Millisecond, 30*time.Millisecond, 6); got != 1200*time.Millisecond {
t.Fatalf("bounded retry RTO = %s", got)
}
}
func TestApolloENetConnectVerifies48ChannelsAndFlushesAck(t *testing.T) {
server, err := net.ListenUDP("udp", &net.UDPAddr{IP: net.ParseIP("127.0.0.1")})
if err != nil {
t.Fatal(err)
}
defer server.Close()
client, err := net.DialUDP("udp", nil, server.LocalAddr().(*net.UDPAddr))
if err != nil {
t.Fatal(err)
}
peer, err := newApolloENetPeer(client, time.Now)
if err != nil {
t.Fatal(err)
}
defer peer.close(nil)
received := make(chan string, 3)
peer.onPayload = func(_ uint8, _ bool, payload []byte) { received <- string(payload) }
serverDone := make(chan error, 1)
go func() {
buffer := make([]byte, apolloENetMaximumPacket)
count, remote, readErr := server.ReadFromUDP(buffer)
if readErr != nil {
serverDone <- readErr
return
}
packet := buffer[:count]
if len(packet) != 52 || packet[4] != apolloENetConnect|apolloENetAcknowledged || packet[5] != 0xff || binary.BigEndian.Uint32(packet[20:24]) != apolloENetChannels || binary.BigEndian.Uint32(packet[48:52]) != 0x12345678 {
serverDone <- ErrProviderMalformed
return
}
verify := make([]byte, 48)
apolloENetHeader(verify, 0, 0, time.Now())
verify[4] = apolloENetVerifyConnect | apolloENetAcknowledged
verify[5] = 0xff
binary.BigEndian.PutUint16(verify[6:8], 1)
binary.BigEndian.PutUint16(verify[8:10], 7)
verify[10], verify[11] = 2, 3
binary.BigEndian.PutUint32(verify[12:16], 1400)
binary.BigEndian.PutUint32(verify[16:20], 32768)
binary.BigEndian.PutUint32(verify[20:24], apolloENetChannels)
binary.BigEndian.PutUint32(verify[44:48], binary.BigEndian.Uint32(packet[44:48]))
if _, writeErr := server.WriteToUDP(verify, remote); writeErr != nil {
serverDone <- writeErr
return
}
count, _, readErr = server.ReadFromUDP(buffer)
if readErr != nil {
serverDone <- readErr
return
}
ack := buffer[:count]
if len(ack) != 12 || ack[4]&apolloENetCommandMask != 1 || binary.BigEndian.Uint16(ack[8:10]) != 1 {
serverDone <- ErrProviderMalformed
return
}
serverDone <- nil
}()
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
if err := peer.Connect(ctx, 0x12345678); err != nil {
t.Fatalf("Connect() error = %v", err)
}
for _, command := range []struct {
sequence uint16
payload string
}{{1, "A"}, {3, "C"}, {2, "B"}} {
if err := peer.handleDatagram(sourceShapedENetReliablePacket(peer.peerID, peer.inboundSession, command.sequence, []byte(command.payload))); err != nil {
t.Fatalf("handleDatagram() error = %v", err)
}
}
if err := <-serverDone; err != nil {
t.Fatalf("ENet server error = %v", err)
}
ordered := ""
for range 3 {
select {
case payload := <-received:
ordered += payload
case <-time.After(time.Second):
t.Fatalf("reliable payload order = %q, want ABC", ordered)
}
}
if ordered != "ABC" {
t.Fatalf("reliable payload order = %q, want ABC", ordered)
}
}
func TestApolloENetFakeClockRetransmitsLostReliablePacketAtForkRTO(t *testing.T) {
server, err := net.ListenUDP("udp", &net.UDPAddr{IP: net.ParseIP("127.0.0.1")})
if err != nil {
t.Fatal(err)
}
defer server.Close()
client, err := net.DialUDP("udp", nil, server.LocalAddr().(*net.UDPAddr))
if err != nil {
t.Fatal(err)
}
now := time.Unix(1, 0)
peer, err := newApolloENetPeer(client, func() time.Time { return now })
if err != nil {
t.Fatal(err)
}
defer peer.close(nil)
peer.state, peer.peerID, peer.outboundSession, peer.lastPing = apolloENetConnected, 1, 2, now
peer.rtt, peer.variance = 100*time.Millisecond, 30*time.Millisecond
if err := peer.SendReliable(apolloChannelKeyboard, []byte{0x01}); err != nil {
t.Fatal(err)
}
assertApolloENetReliablePacket(t, server, 1)
now = now.Add(199 * time.Millisecond)
if err := peer.maintain(); err != nil {
t.Fatal(err)
}
assertApolloENetPending(t, peer, 1, 200*time.Millisecond)
now = now.Add(time.Millisecond)
if err := peer.maintain(); err != nil {
t.Fatal(err)
}
assertApolloENetReliablePacket(t, server, 1)
assertApolloENetPending(t, peer, 2, 200*time.Millisecond)
now = now.Add(199 * time.Millisecond)
if err := peer.maintain(); err != nil {
t.Fatal(err)
}
assertApolloENetPending(t, peer, 2, 200*time.Millisecond)
now = now.Add(time.Millisecond)
if err := peer.maintain(); err != nil {
t.Fatal(err)
}
assertApolloENetReliablePacket(t, server, 1)
assertApolloENetPending(t, peer, 3, 400*time.Millisecond)
telemetry := peer.telemetry()
if telemetry.ControlRTT != 100*time.Millisecond || telemetry.ControlJitter != 30*time.Millisecond || telemetry.ReliableSent != 1 || telemetry.ReliableRetransmits != 2 || telemetry.PendingReliable != 1 {
t.Fatalf("ENet telemetry = %#v", telemetry)
}
}
func TestApolloENetFakeClockKeepsSessionAlivePastElevenVirtualSeconds(t *testing.T) {
server, err := net.ListenUDP("udp", &net.UDPAddr{IP: net.ParseIP("127.0.0.1")})
if err != nil {
t.Fatal(err)
}
defer server.Close()
client, err := net.DialUDP("udp", nil, server.LocalAddr().(*net.UDPAddr))
if err != nil {
t.Fatal(err)
}
now := time.Unix(1, 0)
peer, err := newApolloENetPeer(client, func() time.Time { return now })
if err != nil {
t.Fatal(err)
}
defer peer.close(nil)
peer.state, peer.peerID, peer.inboundSession, peer.outboundSession, peer.lastPing, peer.lastReceive = apolloENetConnected, 1, 1, 2, now, now
for virtual := 500 * time.Millisecond; virtual <= 11*time.Second; virtual += 500 * time.Millisecond {
now = time.Unix(1, 0).Add(virtual)
if err := peer.maintain(); err != nil {
t.Fatalf("maintain() at virtual %s = %v", virtual, err)
}
assertApolloENetPingPacket(t, server)
peer.mu.Lock()
sequence := peer.channels[0].nextOutgoing
peer.mu.Unlock()
if err := peer.handleDatagram(sourceShapedENetAcknowledgePacket(peer.peerID, peer.inboundSession, 0, sequence)); err != nil {
t.Fatalf("handleDatagram() ACK at virtual %s = %v", virtual, err)
}
}
if peer.state != apolloENetConnected || len(peer.pending) != 0 {
t.Fatalf("virtual ENet session state = %v pending=%d", peer.state, len(peer.pending))
}
}
func TestApolloENetDisconnectCompletesOnProviderAcknowledgement(t *testing.T) {
server, err := net.ListenUDP("udp", &net.UDPAddr{IP: net.ParseIP("127.0.0.1")})
if err != nil {
t.Fatal(err)
}
defer server.Close()
client, err := net.DialUDP("udp", nil, server.LocalAddr().(*net.UDPAddr))
if err != nil {
t.Fatal(err)
}
peer, err := newApolloENetPeer(client, time.Now)
if err != nil {
t.Fatal(err)
}
defer peer.close(nil)
peer.state, peer.peerID, peer.inboundSession, peer.outboundSession = apolloENetConnected, 1, 1, 2
go peer.run()
serverDone := make(chan error, 1)
go func() {
buffer := make([]byte, apolloENetMaximumPacket)
count, remote, readErr := server.ReadFromUDP(buffer)
if readErr != nil {
serverDone <- readErr
return
}
packet := buffer[:count]
if count != 12 || packet[4]&apolloENetCommandMask != apolloENetDisconnect || packet[5] != 0xff {
serverDone <- ErrProviderMalformed
return
}
_, writeErr := server.WriteToUDP(sourceShapedENetAcknowledgePacket(1, 1, 0xff, binary.BigEndian.Uint16(packet[6:8])), remote)
serverDone <- writeErr
}()
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
if err := peer.Disconnect(ctx); err != nil {
t.Fatalf("Disconnect() = %v after provider ACK", err)
}
if err := <-serverDone; err != nil {
t.Fatalf("provider ACK = %v", err)
}
}
func assertApolloENetReliablePacket(t *testing.T, server *net.UDPConn, sequence uint16) {
t.Helper()
buffer := make([]byte, apolloENetMaximumPacket)
if err := server.SetReadDeadline(time.Now().Add(time.Second)); err != nil {
t.Fatal(err)
}
count, _, err := server.ReadFromUDP(buffer)
if err != nil {
t.Fatal(err)
}
if count != 11 || buffer[4]&apolloENetCommandMask != apolloENetSendReliable || binary.BigEndian.Uint16(buffer[6:8]) != sequence {
t.Fatalf("retransmitted packet = %x", buffer[:count])
}
}
func assertApolloENetPingPacket(t *testing.T, server *net.UDPConn) {
t.Helper()
buffer := make([]byte, apolloENetMaximumPacket)
if err := server.SetReadDeadline(time.Now().Add(time.Second)); err != nil {
t.Fatal(err)
}
count, _, err := server.ReadFromUDP(buffer)
if err != nil {
t.Fatal(err)
}
if count != 8 || buffer[4]&apolloENetCommandMask != apolloENetPing || buffer[4]&apolloENetAcknowledged == 0 {
t.Fatalf("ping packet = %x", buffer[:count])
}
}
func assertApolloENetPending(t *testing.T, peer *apolloENetPeer, attempts uint8, timeout time.Duration) {
t.Helper()
pending := peer.pending[apolloENetPendingKey{channel: apolloChannelKeyboard, sequence: 1}]
if pending == nil || pending.attempts != attempts || pending.timeout != timeout {
t.Fatalf("pending reliable state = %#v, want attempts=%d timeout=%s", pending, attempts, timeout)
}
}
func sourceShapedENetReliablePacket(peerID uint16, session uint8, sequence uint16, payload []byte) []byte {
return sourceShapedENetReliablePacketOn(peerID, session, apolloChannelKeyboard, sequence, payload)
}
func sourceShapedENetReliablePacketOn(peerID uint16, session, channel uint8, sequence uint16, payload []byte) []byte {
packet := make([]byte, 10+len(payload))
binary.BigEndian.PutUint16(packet[:2], peerID|(uint16(session&3)<<apolloENetSessionShift)|apolloENetSentTimeFlag)
binary.BigEndian.PutUint16(packet[2:4], 0x1234)
packet[4] = apolloENetSendReliable | apolloENetAcknowledged
packet[5] = channel
binary.BigEndian.PutUint16(packet[6:8], sequence)
binary.BigEndian.PutUint16(packet[8:10], uint16(len(payload)))
copy(packet[10:], payload)
return packet
}
func sourceShapedENetAcknowledgePacket(peerID uint16, session, channel uint8, sequence uint16) []byte {
packet := make([]byte, 12)
binary.BigEndian.PutUint16(packet[:2], peerID|(uint16(session&3)<<apolloENetSessionShift)|apolloENetSentTimeFlag)
packet[4] = 1
packet[5] = channel
binary.BigEndian.PutUint16(packet[8:10], sequence)
return packet
}
func TestApolloControlWireVectorAndTagFailure(t *testing.T) {
codec, err := newApolloControlCodec([]byte("0123456789abcdef"))
if err != nil {
t.Fatal(err)
}
packet, err := codec.SealClient(apolloControlTypePing, []byte{4, 0, 0, 0, 0, 0})
if err != nil {
t.Fatal(err)
}
const expected = "01001e0000000000c688b2e3cb8a5869293f01293a19486c692d0799eb8e220a74b4"
if string(packet) != string(mustDecodeHex(t, expected)) {
t.Fatalf("control wire bytes = %x", packet)
}
tampered := append([]byte(nil), packet...)
tampered[len(tampered)-1] ^= 1
if _, err := codec.OpenHost(tampered); err == nil {
t.Fatal("OpenHost() accepted a tag failure")
}
}
func TestApolloKeyboardInputWireVector(t *testing.T) {
packet, err := encodeApolloInputEvent(InputEvent{Device: "keyboard", Code: 30, Pressed: true, Payload: []byte{2}})
if err != nil {
t.Fatal(err)
}
const expected = "0000000a03000000001e00020000"
if string(packet.payload) != string(mustDecodeHex(t, expected)) {
t.Fatalf("keyboard packet = %x, want %s", packet.payload, expected)
}
}
func mustDecodeHex(t *testing.T, value string) []byte {
t.Helper()
decoded, err := hex.DecodeString(value)
if err != nil {
t.Fatal(err)
}
return decoded
}
+90
View File
@@ -0,0 +1,90 @@
package gateway
import (
"encoding/binary"
"unicode/utf8"
)
const (
apolloChannelGeneric = 0
apolloChannelUrgent = 1
apolloChannelKeyboard = 2
apolloChannelMouse = 3
apolloChannelUTF8 = 6
apolloChannelGamepad = 16
)
type apolloInputPacket struct {
channel uint8
payload []byte
}
func encodeApolloInputEvent(event InputEvent) (apolloInputPacket, error) {
switch event.Device {
case "keyboard":
if event.Code < 0 || event.Code > 0xffff || len(event.Payload) > 1 {
return apolloInputPacket{}, ErrInputMalformed
}
packet := make([]byte, 14)
binary.BigEndian.PutUint32(packet[:4], 10)
if event.Pressed {
binary.LittleEndian.PutUint32(packet[4:8], 3)
} else {
binary.LittleEndian.PutUint32(packet[4:8], 4)
}
binary.LittleEndian.PutUint16(packet[9:11], uint16(event.Code))
if len(event.Payload) == 1 {
packet[11] = event.Payload[0]
}
return apolloInputPacket{channel: apolloChannelKeyboard, payload: packet}, nil
case "mouse-button":
if event.Code < 1 || event.Code > 8 || len(event.Payload) != 0 {
return apolloInputPacket{}, ErrInputMalformed
}
packet := make([]byte, 9)
binary.BigEndian.PutUint32(packet[:4], 5)
magic := uint32(8)
if !event.Pressed {
magic = 9
}
binary.LittleEndian.PutUint32(packet[4:8], magic)
packet[8] = byte(event.Code)
return apolloInputPacket{channel: apolloChannelMouse, payload: packet}, nil
case "mouse-relative":
if event.Pressed || len(event.Payload) != 4 {
return apolloInputPacket{}, ErrInputMalformed
}
packet := make([]byte, 12)
binary.BigEndian.PutUint32(packet[:4], 8)
binary.LittleEndian.PutUint32(packet[4:8], 7)
copy(packet[8:], event.Payload)
return apolloInputPacket{channel: apolloChannelMouse, payload: packet}, nil
case "utf8":
if event.Pressed || len(event.Payload) == 0 || len(event.Payload) > utf8.UTFMax || !utf8.Valid(event.Payload) || utf8.RuneCount(event.Payload) != 1 {
return apolloInputPacket{}, ErrInputMalformed
}
packet := make([]byte, 8+len(event.Payload))
binary.BigEndian.PutUint32(packet[:4], uint32(4+len(event.Payload)))
binary.LittleEndian.PutUint32(packet[4:8], 0x17)
copy(packet[8:], event.Payload)
return apolloInputPacket{channel: apolloChannelUTF8, payload: packet}, nil
case "controller":
if event.Code < 0 || event.Code > 15 || len(event.Payload) != 16 {
return apolloInputPacket{}, ErrInputMalformed
}
packet := make([]byte, 34)
binary.BigEndian.PutUint32(packet[:4], 30)
binary.LittleEndian.PutUint32(packet[4:8], 0x0c)
binary.LittleEndian.PutUint16(packet[8:10], 0x1a)
binary.LittleEndian.PutUint16(packet[10:12], uint16(event.Code))
copy(packet[12:14], event.Payload[:2])
binary.LittleEndian.PutUint16(packet[14:16], 0x14)
copy(packet[16:28], event.Payload[2:14])
binary.LittleEndian.PutUint16(packet[28:30], 0x9c)
copy(packet[30:32], event.Payload[14:16])
binary.LittleEndian.PutUint16(packet[32:34], 0x55)
return apolloInputPacket{channel: apolloChannelGamepad + uint8(event.Code), payload: packet}, nil
default:
return apolloInputPacket{}, ErrInputMalformed
}
}
+181
View File
@@ -0,0 +1,181 @@
package gateway
import (
"crypto/aes"
"crypto/cipher"
"encoding/binary"
"errors"
)
const (
// Apollo limits clear audio payloads to 1400 bytes and uses AES-CBC with
// PKCS#7 padding. FEC adds its fixed 12-byte header outside that ciphertext.
// This is the largest accepted UDP datagram, not an allocation hint.
apolloMediaMaximumPacket = 12 + 12 + 1408
apolloVideoHeaderSize = 32
apolloRTPHeaderSize = 12
apolloVideoNVHeaderSize = 16
apolloVideoRawPacketSize = 1024 + 16
)
var (
errApolloMedia = errors.New("apollo media malformed")
errApolloMediaParity = errors.New("apollo media parity packet")
)
type apolloMediaCodec struct {
block cipher.Block
aead cipher.AEAD
keyID uint32
}
type apolloRTPPacket struct {
extension bool
payloadType byte
sequence uint16
payload []byte
}
type apolloAudioShard struct {
sequence uint16
timestamp uint32
ssrc uint32
base uint16
parityIndex uint8
parity bool
payload []byte
}
func newApolloMediaCodec(key []byte, keyID uint32) (*apolloMediaCodec, error) {
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
aead, err := cipher.NewGCM(block)
if err != nil {
return nil, err
}
return &apolloMediaCodec{block: block, aead: aead, keyID: keyID}, nil
}
func apolloMediaPing(payload []byte, sequence uint32) []byte {
if len(payload) != 16 {
return nil
}
ping := make([]byte, 20)
copy(ping, payload)
binary.BigEndian.PutUint32(ping[16:], sequence)
return ping
}
func (c *apolloMediaCodec) OpenVideo(packet []byte) (apolloVideoShard, error) {
if c == nil || c.aead == nil || len(packet) != apolloVideoHeaderSize+apolloVideoRawPacketSize {
return apolloVideoShard{}, errApolloMedia
}
sealed := make([]byte, len(packet)-apolloVideoHeaderSize+apolloControlTagSize)
copy(sealed, packet[apolloVideoHeaderSize:])
copy(sealed[len(packet)-apolloVideoHeaderSize:], packet[16:apolloVideoHeaderSize])
plaintext, err := c.aead.Open(nil, packet[:12], sealed, nil)
if err != nil {
return apolloVideoShard{}, errApolloMedia
}
rtp, err := parseApolloRTP(plaintext)
if err != nil || !rtp.extension || rtp.payloadType != 0 || len(rtp.payload) != 1024 {
return apolloVideoShard{}, errApolloMedia
}
nv := rtp.payload[:apolloVideoNVHeaderSize]
flags := nv[8]
fecInfo := binary.LittleEndian.Uint32(nv[12:16])
dataPackets := int(fecInfo >> 22)
fecIndex := int((fecInfo >> 12) & 0x03ff)
fecPercent := int((fecInfo >> 4) & 0xff)
if dataPackets < 1 || dataPackets > apolloVideoMaximumDataShards {
return apolloVideoShard{}, errApolloMedia
}
parityPackets := (dataPackets*fecPercent + 99) / 100
if dataPackets+parityPackets > 255 || fecIndex >= dataPackets+parityPackets {
return apolloVideoShard{}, errApolloMedia
}
block := (nv[11] >> 4) & 0x03
lastBlock := (nv[11] >> 6) & 0x03
if block > lastBlock {
return apolloVideoShard{}, errApolloMedia
}
return apolloVideoShard{
frame: binary.LittleEndian.Uint32(nv[4:8]),
block: block,
lastBlock: lastBlock,
dataPackets: dataPackets,
parity: parityPackets,
index: fecIndex,
sequence: rtp.sequence,
streamIndex: binary.LittleEndian.Uint32(nv[:4]) >> 8,
flags: flags,
payload: append([]byte(nil), rtp.payload[apolloVideoNVHeaderSize:]...),
}, nil
}
func (c *apolloMediaCodec) OpenAudio(packet []byte) (apolloAudioShard, error) {
if c == nil || c.block == nil || len(packet) <= apolloRTPHeaderSize || len(packet) > apolloMediaMaximumPacket {
return apolloAudioShard{}, errApolloMedia
}
rtp, err := parseApolloRTP(packet)
if err != nil || rtp.extension || len(rtp.payload) == 0 {
return apolloAudioShard{}, errApolloMedia
}
if rtp.payloadType == 97 && len(rtp.payload)%aes.BlockSize == 0 {
return apolloAudioShard{
sequence: rtp.sequence, timestamp: binary.BigEndian.Uint32(packet[4:8]), ssrc: binary.BigEndian.Uint32(packet[8:12]),
base: rtp.sequence &^ 3, payload: append([]byte(nil), rtp.payload...),
}, nil
}
if rtp.payloadType != 127 || len(rtp.payload) <= 12 || len(rtp.payload)-12 > 1408 || rtp.payload[0] > 1 || rtp.payload[1] != 97 {
return apolloAudioShard{}, errApolloMedia
}
base := binary.BigEndian.Uint16(rtp.payload[2:4])
if base&3 != 0 || len(rtp.payload[12:])%aes.BlockSize != 0 {
return apolloAudioShard{}, errApolloMedia
}
return apolloAudioShard{
sequence: rtp.sequence, timestamp: binary.BigEndian.Uint32(rtp.payload[4:8]), ssrc: binary.BigEndian.Uint32(rtp.payload[8:12]),
base: base, parityIndex: rtp.payload[0], parity: true, payload: append([]byte(nil), rtp.payload[12:]...),
}, nil
}
func (c *apolloMediaCodec) openApolloAudioCipher(sequence uint16, payload []byte) ([]byte, error) {
if c == nil || c.block == nil || len(payload) == 0 || len(payload) > 1408 || len(payload)%aes.BlockSize != 0 {
return nil, errApolloMedia
}
plaintext := append([]byte(nil), payload...)
iv := make([]byte, aes.BlockSize)
binary.BigEndian.PutUint32(iv, c.keyID+uint32(sequence))
cipher.NewCBCDecrypter(c.block, iv).CryptBlocks(plaintext, plaintext)
padding := int(plaintext[len(plaintext)-1])
if padding == 0 || padding > aes.BlockSize || padding > len(plaintext) {
return nil, errApolloMedia
}
for _, value := range plaintext[len(plaintext)-padding:] {
if int(value) != padding {
return nil, errApolloMedia
}
}
return append([]byte(nil), plaintext[:len(plaintext)-padding]...), nil
}
func parseApolloRTP(packet []byte) (apolloRTPPacket, error) {
if len(packet) < apolloRTPHeaderSize || packet[0]>>6 != 2 || packet[0]&0x2f != 0 {
return apolloRTPPacket{}, errApolloMedia
}
offset := apolloRTPHeaderSize
extension := packet[0]&0x10 != 0
if extension {
if len(packet) < offset+4 || binary.BigEndian.Uint16(packet[14:16]) != 0 {
return apolloRTPPacket{}, errApolloMedia
}
offset += 4
}
if offset >= len(packet) {
return apolloRTPPacket{}, errApolloMedia
}
return apolloRTPPacket{extension: extension, payloadType: packet[1] & 0x7f, sequence: binary.BigEndian.Uint16(packet[2:4]), payload: packet[offset:]}, nil
}
+626 -121
View File
@@ -9,15 +9,20 @@ import (
"crypto/x509"
"encoding/binary"
"encoding/hex"
"encoding/xml"
"errors"
"fmt"
"io"
"net"
"net/http"
"net/url"
"sort"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"unicode/utf8"
protocol "git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol/gen/go/protocol"
)
@@ -29,16 +34,17 @@ type NativeApolloBackend struct {
Dialer *net.Dialer
mu sync.Mutex
pending map[string]net.Conn
pending map[string]*apolloRTSPSetup
}
func NewNativeApolloBackend() *NativeApolloBackend {
return &NativeApolloBackend{Dialer: &net.Dialer{Timeout: 5 * time.Second}, pending: make(map[string]net.Conn)}
return &NativeApolloBackend{Dialer: &net.Dialer{Timeout: 5 * time.Second}, pending: make(map[string]*apolloRTSPSetup)}
}
func (b *NativeApolloBackend) Management(ctx context.Context, request LaunchRequest) ([]byte, error) {
work := request.ProviderWork
if err := work.Validate(); err != nil || request.SessionID == "" || request.SessionID != work.SessionID || work.ProviderProfile != ProviderProfileApollo {
if err := work.Validate(); err != nil || validateApolloStreamPolicy(work.StreamPolicy) != nil ||
request.SessionID == "" || request.SessionID != work.SessionID || work.ProviderProfile != ProviderProfileApollo {
return nil, ErrProviderMalformed
}
client, err := newPinnedApolloHTTPClient(work)
@@ -53,7 +59,11 @@ func newPinnedApolloHTTPClient(work protocol.ProviderSessionWork) (*http.Client,
if err != nil {
return nil, err
}
return &http.Client{Transport: &http.Transport{TLSClientConfig: tlsConfig}, Timeout: 5 * time.Second}, nil
return &http.Client{
Transport: &http.Transport{TLSClientConfig: tlsConfig},
Timeout: 5 * time.Second,
CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse },
}, nil
}
func apolloGet(ctx context.Context, client *http.Client, work protocol.ProviderSessionWork, path string, values url.Values) ([]byte, error) {
@@ -74,6 +84,88 @@ func apolloGet(ctx context.Context, client *http.Client, work protocol.ProviderS
return readBounded(response.Body, 64*1024)
}
func apolloSessionRequest(work protocol.ProviderSessionWork, key []byte, keyID uint32) (string, url.Values) {
values := url.Values{
"rikey": {hex.EncodeToString(key)},
"rikeyid": {strconv.FormatUint(uint64(keyID), 10)},
"localAudioPlayMode": {"0"},
}
if work.ReconnectSequence > 0 {
return "/resume", values
}
values.Set("uniqueid", work.ClientID)
values.Set("appid", work.ApplicationID)
values.Set("corever", "1")
return "/launch", values
}
func apolloClipboardRequest(ctx context.Context, client *http.Client, host string, port int64, method, text string) ([]byte, error) {
if client == nil || host == "" || port < 1 || port > maxApolloRTSPPort || (method != http.MethodGet && method != http.MethodPost) {
return nil, ErrProviderMalformed
}
endpoint := url.URL{Scheme: "https", Host: net.JoinHostPort(host, strconv.FormatInt(port, 10)), Path: "/actions/clipboard", RawQuery: url.Values{"type": {"text"}}.Encode()}
var body io.Reader
if method == http.MethodPost {
if !utf8.ValidString(text) || len(text) > 65536 {
return nil, ErrProviderMalformed
}
body = bytes.NewReader([]byte(text))
}
request, err := http.NewRequestWithContext(ctx, method, endpoint.String(), body)
if err != nil {
return nil, ErrProviderMalformed
}
if method == http.MethodPost {
request.Header.Set("Content-Type", "text/plain; charset=utf-8")
}
response, err := client.Do(request)
if err != nil {
return nil, err
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
return nil, fmt.Errorf("provider clipboard status %d", response.StatusCode)
}
return readBounded(response.Body, 65536)
}
func apolloCancelRequest(ctx context.Context, client *http.Client, host string, port int64) error {
if client == nil || host == "" || port < 1 || port > maxApolloRTSPPort {
return ErrProviderMalformed
}
endpoint := url.URL{Scheme: "https", Host: net.JoinHostPort(host, strconv.FormatInt(port, 10)), Path: "/cancel"}
request, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint.String(), nil)
if err != nil {
return ErrProviderMalformed
}
response, err := client.Do(request)
if err != nil {
return err
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
return fmt.Errorf("provider cancel status %d", response.StatusCode)
}
body, err := readBounded(response.Body, 64*1024)
if err != nil {
return err
}
var result struct {
XMLName xml.Name `xml:"root"`
StatusCode int `xml:"status_code,attr"`
Cancel int `xml:"cancel"`
}
decoder := xml.NewDecoder(bytes.NewReader(body))
decoder.Strict = true
if err := decoder.Decode(&result); err != nil || result.XMLName.Local != "root" || result.StatusCode != http.StatusOK || result.Cancel != 1 {
return ErrProviderMalformed
}
if err := decoder.Decode(&struct{}{}); !errors.Is(err, io.EOF) {
return ErrProviderMalformed
}
return nil
}
func pinnedApolloTLSConfig(work protocol.ProviderSessionWork) (*tls.Config, error) {
identity, ok := providerIdentityFromKey(work.ProviderIdentity)
if !ok || !strings.HasPrefix(identity.Fingerprint, "sha256:") {
@@ -106,9 +198,14 @@ func pinnedApolloTLSConfig(work protocol.ProviderSessionWork) (*tls.Config, erro
}, nil
}
func (b *NativeApolloBackend) Setup(ctx context.Context, request LaunchRequest) ([]byte, error) {
func (b *NativeApolloBackend) Setup(ctx context.Context, request LaunchRequest, management []byte) ([]byte, error) {
work := request.ProviderWork
if err := work.Validate(); err != nil || request.SessionID == "" || request.SessionID != work.SessionID || work.ProviderProfile != ProviderProfileApollo {
if err := work.Validate(); err != nil || validateApolloStreamPolicy(work.StreamPolicy) != nil ||
request.SessionID == "" || request.SessionID != work.SessionID || work.ProviderProfile != ProviderProfileApollo {
return nil, ErrProviderMalformed
}
info, err := ParseManagementXML(management)
if err != nil || validateApolloProviderStreamPolicy(info, work.StreamPolicy) != nil {
return nil, ErrProviderMalformed
}
client, err := newPinnedApolloHTTPClient(work)
@@ -123,15 +220,13 @@ func (b *NativeApolloBackend) Setup(ctx context.Context, request LaunchRequest)
if _, err := rand.Read(key); err != nil {
return nil, err
}
defer zeroApolloSecret(key)
var keyID [4]byte
if _, err := rand.Read(keyID[:]); err != nil {
return nil, err
}
launch, err := apolloGet(ctx, client, work, "/launch", url.Values{
"uniqueid": {work.ClientID}, "appid": {work.ApplicationID}, "rikey": {hex.EncodeToString(key)},
"rikeyid": {strconv.FormatUint(uint64(binary.BigEndian.Uint32(keyID[:])), 10)}, "localAudioPlayMode": {"0"},
"corever": {"1"},
})
sessionPath, sessionValues := apolloSessionRequest(work, key, binary.BigEndian.Uint32(keyID[:]))
launch, err := apolloGet(ctx, client, work, sessionPath, sessionValues)
if err != nil {
return nil, err
}
@@ -147,50 +242,25 @@ func (b *NativeApolloBackend) Setup(ctx context.Context, request LaunchRequest)
if err != nil || streamPort != work.StreamPort {
return nil, ErrProviderMalformed
}
conn, err := b.Dialer.DialContext(ctx, "tcp", net.JoinHostPort(work.StreamHost, strconv.FormatInt(work.StreamPort, 10)))
setup, response, err := b.performRTSPHandshake(ctx, work, key, binary.BigEndian.Uint32(keyID[:]), streamURL)
if err != nil {
return nil, err
}
if deadline, ok := ctx.Deadline(); ok {
_ = conn.SetDeadline(deadline)
}
codec, err := newEncryptedRTSPCodec(key)
if err != nil {
_ = conn.Close()
return nil, err
}
requestText := "SETUP rtsp://" + work.StreamHost + "/streamid=video/0/0 RTSP/1.0\r\nCSeq: 1\r\nTransport: RTP/AVP/TCP;interleaved=0-1\r\n\r\n"
encoded, err := codec.SealClient([]byte(requestText))
if err != nil {
_ = conn.Close()
return nil, err
}
if _, err := conn.Write(encoded); err != nil {
_ = conn.Close()
return nil, err
}
response, err := readEncryptedRTSPHeaders(conn, codec)
if err != nil {
_ = conn.Close()
return nil, err
}
b.mu.Lock()
b.pending[request.SessionID] = conn
b.pending[request.SessionID] = setup
b.mu.Unlock()
return response, nil
}
func (b *NativeApolloBackend) Open(_ context.Context, request LaunchRequest, _ RTSPResponse) (ProviderSession, error) {
func (b *NativeApolloBackend) Open(ctx context.Context, request LaunchRequest, _ RTSPResponse) (ProviderSession, error) {
b.mu.Lock()
conn, ok := b.pending[request.SessionID]
setup, ok := b.pending[request.SessionID]
delete(b.pending, request.SessionID)
b.mu.Unlock()
if !ok || conn == nil {
if !ok || setup == nil {
return nil, ErrProviderDisconnected
}
session := newNativeApolloSession(conn, request.SessionID)
go session.readMedia()
return session, nil
return newNativeApolloProviderSession(ctx, setup)
}
func readBounded(reader io.Reader, max int) ([]byte, error) {
@@ -204,152 +274,585 @@ func readBounded(reader io.Reader, max int) ([]byte, error) {
return data, nil
}
func readEncryptedRTSPHeaders(conn net.Conn, codec *encryptedRTSPCodec) ([]byte, error) {
header := make([]byte, encryptedRTSPHeaderSize)
if _, err := io.ReadFull(conn, header); err != nil {
return nil, err
}
length := binary.BigEndian.Uint32(header[:4]) & 0x7fffffff
if length == 0 || length > encryptedRTSPMaxPayload {
return nil, ErrProviderMalformed
}
frame := make([]byte, encryptedRTSPHeaderSize+int(length))
copy(frame, header)
if _, err := io.ReadFull(conn, frame[encryptedRTSPHeaderSize:]); err != nil {
return nil, err
}
plaintext, err := codec.OpenHost(frame)
if err != nil || len(plaintext) > 16*1024 || !strings.HasSuffix(string(plaintext), "\r\n\r\n") {
return nil, ErrProviderMalformed
}
return plaintext, nil
}
type nativeApolloSession struct {
conn net.Conn
sessionID string
video chan []byte
audio chan []byte
mu sync.Mutex
state protocol.ProviderState
closeOnce sync.Once
done chan struct{}
readDone chan struct{}
enet *apolloENetPeer
control *apolloControlCodec
audioConn *net.UDPConn
videoConn *net.UDPConn
media *apolloMediaCodec
videoFEC apolloVideoAssembler
audioFEC apolloAudioAssembler
audioPing []byte
videoPing []byte
sessionID string
video chan ProviderMedia
audio chan ProviderMedia
events chan ProviderEvent
mu sync.Mutex
eventMu sync.Mutex
mediaMu sync.Mutex
controlMu sync.Mutex
state protocol.ProviderState
pressed map[string]InputEvent
closeOnce sync.Once
disconnectOnce sync.Once
channelsOnce sync.Once
done chan struct{}
readDone chan struct{}
managementClient *http.Client
managementHost string
managementPort int64
allowApplicationTermination bool
terminationErr error
mediaDrops atomic.Uint64
mediaQuiesced atomic.Bool
mediaIngress atomic.Uint64
mediaRecovered atomic.Uint64
mediaEnqueued atomic.Uint64
mediaQueueMaximum atomic.Uint64
}
func newNativeApolloSession(conn net.Conn, sessionID string) *nativeApolloSession {
return &nativeApolloSession{conn: conn, sessionID: sessionID, video: make(chan []byte, 16), audio: make(chan []byte, 16), state: protocol.ProviderState{Version: "1", SessionID: sessionID, State: ProviderStateStarting, Channels: []string{"video", "audio", "input", "feedback"}}, done: make(chan struct{}), readDone: make(chan struct{})}
func newNativeApolloSession(sessionID string) *nativeApolloSession {
return &nativeApolloSession{sessionID: sessionID, video: make(chan ProviderMedia, 16), audio: make(chan ProviderMedia, 16), events: make(chan ProviderEvent, 16), state: protocol.ProviderState{Version: "1", SessionID: sessionID, State: ProviderStateStarting, Channels: []string{"video", "audio", "input", "feedback"}}, pressed: make(map[string]InputEvent), done: make(chan struct{}), readDone: make(chan struct{})}
}
func newNativeApolloProviderSession(ctx context.Context, setup *apolloRTSPSetup) (*nativeApolloSession, error) {
if setup == nil || len(setup.streamKey) != 16 || setup.controlPort == 0 || setup.audioPort == 0 || setup.videoPort == 0 {
return nil, ErrProviderMalformed
}
defer zeroApolloSecret(setup.streamKey)
controlConn, err := dialApolloUDP(setup.streamHost, setup.controlPort)
if err != nil {
return nil, err
}
peer, err := newApolloENetPeer(controlConn, time.Now)
if err != nil {
_ = controlConn.Close()
return nil, err
}
codec, err := newApolloControlCodec(setup.streamKey)
if err != nil {
peer.close(err)
return nil, err
}
media, err := newApolloMediaCodec(setup.streamKey, setup.streamKeyID)
if err != nil {
peer.close(err)
return nil, err
}
session := newNativeApolloSession(setup.sessionID)
managementClient, err := newPinnedApolloHTTPClient(setup.providerWork)
if err != nil {
peer.close(err)
return nil, err
}
session.enet, session.control, session.media = peer, codec, media
session.managementClient = managementClient
session.managementHost, session.managementPort = setup.providerWork.ManagementHost, setup.providerWork.ManagementPort
session.allowApplicationTermination = setup.providerWork.ProviderApplicationTerminationAllowed
session.audioPing = append([]byte(nil), setup.audioPing...)
session.videoPing = append([]byte(nil), setup.videoPing...)
peer.onPayload = session.handleApolloControlPayload
peer.onDisconnect = session.handleApolloDisconnect
connectCtx, cancel := context.WithTimeout(ctx, 10*time.Second)
defer cancel()
if err := peer.Connect(connectCtx, setup.controlConnect); err != nil {
peer.close(err)
return nil, err
}
audioConn, err := dialApolloUDP(setup.streamHost, setup.audioPort)
if err != nil {
peer.close(err)
return nil, err
}
videoConn, err := dialApolloUDP(setup.streamHost, setup.videoPort)
if err != nil {
_ = audioConn.Close()
peer.close(err)
return nil, err
}
if _, err := audioConn.Write(apolloMediaPing(setup.audioPing, 1)); err != nil {
_ = audioConn.Close()
_ = videoConn.Close()
peer.close(err)
return nil, err
}
if _, err := videoConn.Write(apolloMediaPing(setup.videoPing, 1)); err != nil {
_ = audioConn.Close()
_ = videoConn.Close()
peer.close(err)
return nil, err
}
session.audioConn, session.videoConn = audioConn, videoConn
go session.readUDPMedia()
go session.periodicApolloMediaPing()
return session, nil
}
func dialApolloUDP(host string, port int) (*net.UDPConn, error) {
if host == "" || port < 1 || port > maxApolloRTSPPort {
return nil, ErrProviderMalformed
}
remote, err := net.ResolveUDPAddr("udp", net.JoinHostPort(host, strconv.Itoa(port)))
if err != nil {
return nil, err
}
return net.DialUDP("udp", nil, remote)
}
func (s *nativeApolloSession) Ready(context.Context) error {
if s.enet != nil {
if err := s.writeApolloControl(apolloChannelGeneric, true, apolloControlTypeIDR, []byte{0, 0}); err != nil {
return err
}
if err := s.writeApolloControl(apolloChannelGeneric, true, apolloControlTypeStart, []byte{0}); err != nil {
return err
}
go s.periodicApolloPing()
}
s.mu.Lock()
s.state.State = ProviderStateReady
s.mu.Unlock()
return nil
}
func (s *nativeApolloSession) Video() <-chan []byte { return s.video }
func (s *nativeApolloSession) Audio() <-chan []byte { return s.audio }
func (s *nativeApolloSession) Video() <-chan ProviderMedia { return s.video }
func (s *nativeApolloSession) Audio() <-chan ProviderMedia { return s.audio }
func (s *nativeApolloSession) Events() <-chan ProviderEvent { return s.events }
func (s *nativeApolloSession) Input(ctx context.Context, event InputEvent) error {
payload, err := EncodeInputEvent(event)
packet, err := encodeApolloInputEvent(event)
if err != nil {
return err
}
return s.writeControl(ctx, ControlPacket{Kind: 1, Sequence: event.Sequence, Payload: payload})
if err := ctx.Err(); err != nil {
return err
}
if err := s.writeApolloControl(packet.channel, true, apolloControlTypeInput, packet.payload); err != nil {
return err
}
if event.Device == "keyboard" || event.Device == "mouse-button" || event.Device == "controller" {
key := fmt.Sprintf("%s:%d", event.Device, event.Code)
s.mu.Lock()
if event.Pressed {
s.pressed[key] = event
} else {
delete(s.pressed, key)
}
s.mu.Unlock()
}
return nil
}
func (s *nativeApolloSession) Feedback(ctx context.Context, feedback Feedback) error {
return s.writeControl(ctx, ControlPacket{Kind: 3, Sequence: feedback.Sequence, Payload: feedback.Payload})
if err := ctx.Err(); err != nil {
return err
}
switch feedback.Kind {
case FeedbackIDR:
if len(feedback.Payload) != 0 {
return ErrProviderMalformed
}
return s.writeApolloControl(apolloChannelUrgent, true, apolloControlTypeIDR, []byte{0, 0})
case FeedbackFEC:
if !validGatewayFECStatus(feedback.Payload) {
return ErrProviderMalformed
}
return s.writeApolloControl(apolloChannelGeneric, false, apolloControlTypeFEC, feedback.Payload)
default:
return ErrProviderMalformed
}
}
func (s *nativeApolloSession) Reconnect(ctx context.Context) error {
return s.writeControl(ctx, ControlPacket{Kind: 4, Payload: []byte("RECN")})
func (s *nativeApolloSession) ReadClipboard(ctx context.Context) (string, error) {
response, err := apolloClipboardRequest(ctx, s.managementClient, s.managementHost, s.managementPort, http.MethodGet, "")
if err != nil {
return "", err
}
if !utf8.Valid(response) {
return "", ErrProviderMalformed
}
return string(response), nil
}
func (s *nativeApolloSession) WriteClipboard(ctx context.Context, text string) error {
_, err := apolloClipboardRequest(ctx, s.managementClient, s.managementHost, s.managementPort, http.MethodPost, text)
return err
}
func (s *nativeApolloSession) ReleaseAll(ctx context.Context) error {
return s.writeControl(ctx, ControlPacket{Kind: 2, Payload: []byte("RELEASE_ALL")})
s.mu.Lock()
pressed := make([]InputEvent, 0, len(s.pressed))
for _, event := range s.pressed {
pressed = append(pressed, event)
}
s.mu.Unlock()
sort.Slice(pressed, func(first, second int) bool {
if pressed[first].Device != pressed[second].Device {
return pressed[first].Device < pressed[second].Device
}
return pressed[first].Code < pressed[second].Code
})
for _, event := range pressed {
event.Pressed = false
if event.Device == "controller" {
event.Payload = make([]byte, len(event.Payload))
}
if err := s.Input(ctx, event); err != nil {
return err
}
}
return nil
}
func (s *nativeApolloSession) Terminate(ctx context.Context) error {
_ = s.writeControl(ctx, ControlPacket{Kind: 5, Payload: []byte("TEAR")})
var timedOut bool
var cleanupErr error
s.mu.Lock()
disconnected := s.state.State == ProviderStateDisconnected
s.mu.Unlock()
s.closeOnce.Do(func() {
if err := s.ReleaseAll(ctx); err != nil {
cleanupErr = err
}
close(s.done)
_ = s.conn.Close()
if s.enet != nil {
if err := s.enet.Disconnect(ctx); err != nil && cleanupErr == nil {
cleanupErr = err
}
}
if s.audioConn != nil {
_ = s.audioConn.Close()
}
if s.videoConn != nil {
_ = s.videoConn.Close()
}
select {
case <-s.readDone:
case <-ctx.Done():
timedOut = true
if cleanupErr == nil {
cleanupErr = ctx.Err()
}
}
if !timedOut {
close(s.video)
close(s.audio)
if cleanupErr == nil {
s.closeMediaChannels()
if s.allowApplicationTermination && !disconnected {
if err := apolloCancelRequest(ctx, s.managementClient, s.managementHost, s.managementPort); err != nil {
cleanupErr = err
}
}
}
if s.managementClient != nil {
s.managementClient.CloseIdleConnections()
}
s.mu.Lock()
s.terminationErr = cleanupErr
s.mu.Unlock()
})
s.mu.Lock()
if timedOut {
cleanupErr = s.terminationErr
if cleanupErr != nil {
s.state.State = ProviderStateCleanup
s.state.CleanupPending = true
} else if disconnected {
s.state.State = ProviderStateDisconnected
} else {
s.state.State = ProviderStateTerminated
}
s.mu.Unlock()
if timedOut {
return ctx.Err()
if cleanupErr != nil {
return cleanupErr
}
s.control, s.media = nil, nil
return nil
}
func zeroApolloSecret(secret []byte) {
for index := range secret {
secret[index] = 0
}
}
func (s *nativeApolloSession) State() protocol.ProviderState {
s.mu.Lock()
defer s.mu.Unlock()
return s.state
}
func (s *nativeApolloSession) writeControl(ctx context.Context, packet ControlPacket) error {
encoded, err := EncodeControlPacket(packet)
func (s *nativeApolloSession) Telemetry() ProviderTelemetry {
telemetry := s.enet.telemetry()
telemetry.State = s.State().State
telemetry.MediaDrops = s.mediaDrops.Load()
return telemetry
}
func (s *nativeApolloSession) writeApolloControl(channel uint8, reliable bool, typeID uint16, payload []byte) error {
if s.enet == nil || s.control == nil {
return ErrProviderDisconnected
}
s.controlMu.Lock()
encoded, err := s.control.SealClient(typeID, payload)
s.controlMu.Unlock()
if err != nil {
return err
}
if deadline, ok := ctx.Deadline(); ok {
_ = s.conn.SetWriteDeadline(deadline)
if reliable {
return s.enet.SendReliable(channel, encoded)
}
if _, err := s.conn.Write(encoded); err != nil {
return err
}
return nil
return s.enet.SendUnsequenced(channel, encoded)
}
func (s *nativeApolloSession) readMedia() {
defer close(s.readDone)
header := make([]byte, 4)
func (s *nativeApolloSession) periodicApolloPing() {
ticker := time.NewTicker(100 * time.Millisecond)
defer ticker.Stop()
for {
if _, err := io.ReadFull(s.conn, header); err != nil {
select {
case <-s.done:
return
}
if header[0] != '$' || (header[1] != 0 && header[1] != 1) {
return
}
length := int(header[2])<<8 | int(header[3])
if length > 65536 {
return
}
payload := make([]byte, length)
if _, err := io.ReadFull(s.conn, payload); err != nil {
return
}
if header[1] == 0 {
pushLatest(s.video, payload)
} else {
pushLatest(s.audio, payload)
case <-ticker.C:
if err := s.writeApolloControl(apolloChannelGeneric, true, apolloControlTypePing, []byte{4, 0, 0, 0, 0, 0, 0, 0}); err != nil {
s.handleApolloDisconnect(err)
return
}
}
}
}
func pushLatest(channel chan []byte, payload []byte) {
func (s *nativeApolloSession) periodicApolloMediaPing() {
ticker := time.NewTicker(500 * time.Millisecond)
defer ticker.Stop()
sequence := uint32(2)
for {
select {
case <-s.done:
return
case <-ticker.C:
if s.audioConn == nil || s.videoConn == nil {
s.handleApolloDisconnect(ErrProviderDisconnected)
return
}
if _, err := s.audioConn.Write(apolloMediaPing(s.audioPing, sequence)); err != nil {
s.handleApolloDisconnect(err)
return
}
if _, err := s.videoConn.Write(apolloMediaPing(s.videoPing, sequence)); err != nil {
s.handleApolloDisconnect(err)
return
}
sequence++
}
}
}
func (s *nativeApolloSession) handleApolloControlPayload(_ uint8, _ bool, payload []byte) {
s.controlMu.Lock()
message, err := s.control.OpenHost(payload)
s.controlMu.Unlock()
if err != nil {
s.handleApolloDisconnect(err)
return
}
switch message.typeID {
case apolloControlTypeTerm:
if len(message.payload) != 4 {
s.handleApolloDisconnect(ErrProviderMalformed)
return
}
s.quiesceMedia()
s.mu.Lock()
s.state.State = ProviderStateTerminated
s.mu.Unlock()
s.emitProviderEvent(ProviderEvent{Kind: ProviderEventTerminated, Payload: message.payload})
case apolloControlTypeRumble:
if len(message.payload) != 10 {
s.handleApolloDisconnect(ErrProviderMalformed)
return
}
controller := binary.LittleEndian.Uint16(message.payload[4:6])
if controller > 15 {
s.handleApolloDisconnect(ErrProviderMalformed)
return
}
payload := make([]byte, 5)
payload[0] = byte(controller)
binary.BigEndian.PutUint16(payload[1:3], binary.LittleEndian.Uint16(message.payload[6:8]))
binary.BigEndian.PutUint16(payload[3:5], binary.LittleEndian.Uint16(message.payload[8:10]))
s.emitProviderEvent(ProviderEvent{Kind: ProviderEventRumble, Payload: payload})
case apolloControlTypeHDR:
if len(message.payload) != 27 || message.payload[0] > 1 {
s.handleApolloDisconnect(ErrProviderMalformed)
return
}
s.emitProviderEvent(ProviderEvent{Kind: ProviderEventHDR, Payload: []byte{message.payload[0]}})
}
}
func (s *nativeApolloSession) emitProviderEvent(event ProviderEvent) {
if !s.enqueueProviderEvent(event) {
s.handleApolloDisconnect(ErrProviderMalformed)
}
}
func (s *nativeApolloSession) enqueueProviderEvent(event ProviderEvent) bool {
s.eventMu.Lock()
defer s.eventMu.Unlock()
select {
case s.events <- event:
return true
default:
}
if event.Kind != ProviderEventTerminated && event.Kind != ProviderEventDisconnected {
return false
}
select {
case <-s.events:
default:
return false
}
select {
case s.events <- event:
return true
default:
return false
}
}
func (s *nativeApolloSession) handleApolloDisconnect(err error) {
if err == nil {
return
}
s.quiesceMedia()
s.disconnectOnce.Do(func() {
s.mu.Lock()
if s.state.State == ProviderStateTerminated {
s.mu.Unlock()
return
}
s.state.State = ProviderStateDisconnected
s.mu.Unlock()
s.enqueueProviderEvent(ProviderEvent{Kind: ProviderEventDisconnected})
})
}
func (s *nativeApolloSession) quiesceMedia() {
if !s.mediaQuiesced.CompareAndSwap(false, true) {
return
}
if s.audioConn != nil {
_ = s.audioConn.Close()
}
if s.videoConn != nil {
_ = s.videoConn.Close()
}
}
func (s *nativeApolloSession) closeMediaChannels() {
s.channelsOnce.Do(func() {
s.mediaMu.Lock()
defer s.mediaMu.Unlock()
close(s.video)
close(s.audio)
})
}
func (s *nativeApolloSession) enqueueMedia(output chan ProviderMedia, payload []byte, receivedAt time.Time) bool {
s.mediaMu.Lock()
defer s.mediaMu.Unlock()
if len(payload) == 0 || s.mediaQuiesced.Load() {
return false
}
s.mediaRecovered.Add(1)
media := ProviderMedia{Payload: payload, ReceivedAt: receivedAt, EnqueuedAt: time.Now()}
if pushLatest(output, media) {
s.mediaDrops.Add(1)
}
s.mediaEnqueued.Add(1)
depth := uint64(len(output))
for maximum := s.mediaQueueMaximum.Load(); depth > maximum && !s.mediaQueueMaximum.CompareAndSwap(maximum, depth); maximum = s.mediaQueueMaximum.Load() {
}
return true
}
func (s *nativeApolloSession) readUDPMedia() {
if s.media == nil {
close(s.readDone)
s.closeMediaChannels()
return
}
var readers sync.WaitGroup
readers.Add(2)
read := func(conn *net.UDPConn, output chan ProviderMedia, video bool) {
defer readers.Done()
buffer := make([]byte, apolloMediaMaximumPacket+1)
for {
if s.mediaQuiesced.Load() {
return
}
if err := conn.SetReadDeadline(time.Now().Add(250 * time.Millisecond)); err != nil {
return
}
count, err := conn.Read(buffer)
receivedAt := time.Now()
if err != nil {
if networkErr, ok := err.(net.Error); ok && networkErr.Timeout() {
select {
case <-s.done:
return
default:
continue
}
}
return
}
if count > apolloMediaMaximumPacket {
continue
}
if s.mediaQuiesced.Load() {
return
}
s.mediaIngress.Add(1)
var payloads [][]byte
if video {
shard, openErr := s.media.OpenVideo(buffer[:count])
if openErr != nil {
continue
}
payload, err := s.videoFEC.Add(shard)
if err != nil || len(payload) == 0 {
continue
}
payloads = [][]byte{payload}
} else {
shard, openErr := s.media.OpenAudio(buffer[:count])
if openErr != nil {
continue
}
var evicted bool
payloads, evicted, err = s.audioFEC.Add(s.media, shard)
if evicted {
s.mediaDrops.Add(1)
}
}
if err != nil {
continue
}
for _, payload := range payloads {
s.enqueueMedia(output, payload, receivedAt)
}
}
}
go read(s.audioConn, s.audio, false)
go read(s.videoConn, s.video, true)
go func() {
readers.Wait()
close(s.readDone)
s.closeMediaChannels()
}()
}
func pushLatest[T any](channel chan T, payload T) bool {
select {
case channel <- payload:
return false
default:
select {
case <-channel:
@@ -357,7 +860,9 @@ func pushLatest(channel chan []byte, payload []byte) {
}
select {
case channel <- payload:
return true
default:
return true
}
}
}
File diff suppressed because it is too large Load Diff
+36
View File
@@ -0,0 +1,36 @@
package gateway
import (
"net"
"testing"
"time"
)
func FuzzApolloENetAndRTSPParsersStayBounded(f *testing.F) {
server, err := net.ListenUDP("udp", &net.UDPAddr{IP: net.ParseIP("127.0.0.1")})
if err != nil {
f.Fatal(err)
}
defer server.Close()
client, err := net.DialUDP("udp", nil, server.LocalAddr().(*net.UDPAddr))
if err != nil {
f.Fatal(err)
}
defer client.Close()
f.Add([]byte{0x80, 0x01, 0, 0, apolloENetPing | apolloENetAcknowledged, 0, 0, 1})
f.Add([]byte("RTSP/1.0 200 OK\r\nCSeq: 1\r\n\r\n"))
f.Fuzz(func(t *testing.T, data []byte) {
if len(data) > apolloENetMaximumPacket+1 {
data = data[:apolloENetMaximumPacket+1]
}
peer := &apolloENetPeer{
conn: client, now: time.Now, state: apolloENetConnected, peerID: 1, inboundSession: 0,
pending: make(map[apolloENetPendingKey]*apolloENetPending), rtt: time.Millisecond, variance: time.Millisecond,
}
_ = peer.handleDatagram(data)
_, _ = parseApolloRTSPMessage(data)
_, _ = ParseRTSPResponse(data)
_ = validateApolloDescribe(apolloRTSPMessage{headers: map[string]string{"content-type": "application/sdp"}, body: data})
_, _ = parseApolloRTP(data)
})
}
+559
View File
@@ -0,0 +1,559 @@
package gateway
import (
"context"
"fmt"
"io"
"net"
"net/url"
"strconv"
"strings"
"time"
protocol "git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol/gen/go/protocol"
)
const (
maxApolloRTSPHeaders = 16 << 10
maxApolloRTSPBody = 48 << 10
maxApolloRTSPPort = 65535
apolloEncryptionAll = 0x07
)
type apolloRTSPMessage struct {
status int
cseq uint32
headers map[string]string
body []byte
raw []byte
}
type apolloRTSPSetup struct {
sessionID string
audioPort int
videoPort int
controlPort int
audioPing []byte
videoPing []byte
controlConnect uint32
streamHost string
streamPort int64
providerWork protocol.ProviderSessionWork
streamKey []byte
streamKeyID uint32
}
func (b *NativeApolloBackend) performRTSPHandshake(ctx context.Context, work protocol.ProviderSessionWork, key []byte, keyID uint32, streamURL *url.URL) (*apolloRTSPSetup, []byte, error) {
if streamURL == nil || streamURL.Scheme != "rtspenc" || streamURL.Hostname() != work.StreamHost || streamURL.Port() != strconv.FormatInt(work.StreamPort, 10) || streamURL.User != nil || streamURL.RawQuery != "" || streamURL.Fragment != "" {
return nil, nil, ErrProviderMalformed
}
codec, err := newEncryptedRTSPCodec(key)
if err != nil {
return nil, nil, err
}
request := func(method, target, session string, headers []apolloRTSPHeader, body []byte, sequence uint32) (apolloRTSPMessage, error) {
return b.encryptedRTSPRequest(ctx, work, codec, method, target, session, headers, body, sequence)
}
options, err := request("OPTIONS", streamURL.String(), "", nil, nil, 1)
if err != nil {
return nil, nil, err
}
describe, err := request("DESCRIBE", streamURL.String(), "", []apolloRTSPHeader{{"Accept", "application/sdp"}, {"If-Modified-Since", "Thu, 01 Jan 1970 00:00:00 GMT"}}, nil, 2)
if err != nil {
return nil, nil, err
}
if err := validateApolloDescribe(describe); err != nil {
return nil, nil, err
}
setupHeaders := []apolloRTSPHeader{{"Transport", "unicast;X-GS-ClientPort=50000-50001"}, {"If-Modified-Since", "Thu, 01 Jan 1970 00:00:00 GMT"}}
audio, err := request("SETUP", "streamid=audio/0/0", "", setupHeaders, nil, 3)
if err != nil {
return nil, nil, err
}
sessionID, err := apolloRTSPSession(audio)
if err != nil {
return nil, nil, err
}
audioPort, err := apolloRTSPServerPort(audio)
if err != nil {
return nil, nil, err
}
audioPing, err := apolloRTSPPingPayload(audio)
if err != nil {
return nil, nil, err
}
video, err := request("SETUP", "streamid=video/0/0", sessionID, setupHeaders, nil, 4)
if err != nil {
return nil, nil, err
}
if err := apolloRTSPMatchSession(video, sessionID); err != nil {
return nil, nil, err
}
videoPort, err := apolloRTSPServerPort(video)
if err != nil {
return nil, nil, err
}
videoPing, err := apolloRTSPPingPayload(video)
if err != nil {
return nil, nil, err
}
control, err := request("SETUP", "streamid=control/13/0", sessionID, setupHeaders, nil, 5)
if err != nil {
return nil, nil, err
}
if err := apolloRTSPMatchSession(control, sessionID); err != nil {
return nil, nil, err
}
controlPort, err := apolloRTSPServerPort(control)
if err != nil {
return nil, nil, err
}
connectData, err := apolloRTSPConnectData(control)
if err != nil {
return nil, nil, err
}
announceBody, err := apolloAnnounceProfile(work.StreamPolicy)
if err != nil {
return nil, nil, err
}
announce, err := request("ANNOUNCE", "streamid=control/13/0", sessionID, []apolloRTSPHeader{{"Content-Type", "application/sdp"}}, announceBody, 6)
if err != nil {
return nil, nil, err
}
if err := apolloRTSPMatchSession(announce, sessionID); err != nil {
return nil, nil, err
}
play, err := request("PLAY", "/", sessionID, nil, nil, 7)
if err != nil {
return nil, nil, err
}
if err := apolloRTSPMatchSession(play, sessionID); err != nil {
return nil, nil, err
}
_ = options
setup := &apolloRTSPSetup{
sessionID: sessionID, audioPort: audioPort, videoPort: videoPort, controlPort: controlPort,
audioPing: audioPing, videoPing: videoPing, controlConnect: connectData, streamHost: work.StreamHost,
streamPort: work.StreamPort, providerWork: work, streamKey: append([]byte(nil), key...), streamKeyID: keyID,
}
return setup, append([]byte(nil), control.raw...), nil
}
type apolloRTSPHeader struct{ key, value string }
func (b *NativeApolloBackend) encryptedRTSPRequest(ctx context.Context, work protocol.ProviderSessionWork, codec *encryptedRTSPCodec, method, target, session string, headers []apolloRTSPHeader, body []byte, sequence uint32) (apolloRTSPMessage, error) {
if b == nil || b.Dialer == nil || codec == nil || sequence == 0 || len(body) > maxApolloRTSPBody || method == "" || target == "" {
return apolloRTSPMessage{}, ErrProviderMalformed
}
conn, err := b.Dialer.DialContext(ctx, "tcp", net.JoinHostPort(work.StreamHost, strconv.FormatInt(work.StreamPort, 10)))
if err != nil {
return apolloRTSPMessage{}, err
}
defer conn.Close()
deadline := time.Now().Add(5 * time.Second)
if contextDeadline, ok := ctx.Deadline(); ok && contextDeadline.Before(deadline) {
deadline = contextDeadline
}
if err := conn.SetDeadline(deadline); err != nil {
return apolloRTSPMessage{}, err
}
plaintext, err := buildApolloRTSPRequest(method, target, session, headers, body, sequence)
if err != nil {
return apolloRTSPMessage{}, err
}
frame, err := codec.SealClient(plaintext)
if err != nil {
return apolloRTSPMessage{}, err
}
if _, err := conn.Write(frame); err != nil {
return apolloRTSPMessage{}, err
}
response, err := readEncryptedRTSPMessage(conn, codec)
if err != nil {
return apolloRTSPMessage{}, err
}
if response.status != 200 || response.cseq != sequence {
return apolloRTSPMessage{}, ErrProviderMalformed
}
return response, nil
}
func buildApolloRTSPRequest(method, target, session string, headers []apolloRTSPHeader, body []byte, sequence uint32) ([]byte, error) {
if method == "" || target == "" || strings.ContainsAny(method, "\r\n ") || strings.ContainsAny(target, "\r\n") || sequence == 0 {
return nil, ErrProviderMalformed
}
var builder strings.Builder
builder.Grow(256 + len(body))
fmt.Fprintf(&builder, "%s %s RTSP/1.0\r\nCSeq: %d\r\n", method, target, sequence)
if session != "" {
if !validApolloRTSPToken(session) {
return nil, ErrProviderMalformed
}
fmt.Fprintf(&builder, "Session: %s\r\n", session)
}
seen := map[string]struct{}{"cseq": {}, "session": {}}
for _, header := range headers {
key := strings.ToLower(header.key)
if !validApolloRTSPToken(header.key) || header.value == "" || len(header.value) > 1024 || strings.ContainsAny(header.value, "\r\n") {
return nil, ErrProviderMalformed
}
if _, ok := seen[key]; ok {
return nil, ErrProviderMalformed
}
seen[key] = struct{}{}
fmt.Fprintf(&builder, "%s: %s\r\n", header.key, header.value)
}
if len(body) != 0 {
if _, ok := seen["content-length"]; ok {
return nil, ErrProviderMalformed
}
fmt.Fprintf(&builder, "Content-Length: %d\r\n", len(body))
}
builder.WriteString("\r\n")
builder.Write(body)
return []byte(builder.String()), nil
}
func readEncryptedRTSPMessage(conn net.Conn, codec *encryptedRTSPCodec) (apolloRTSPMessage, error) {
header := make([]byte, encryptedRTSPHeaderSize)
if _, err := io.ReadFull(conn, header); err != nil {
return apolloRTSPMessage{}, err
}
length := uint64(header[0]&0x7f)<<24 | uint64(header[1])<<16 | uint64(header[2])<<8 | uint64(header[3])
if header[0]&0x80 == 0 || length == 0 || length > encryptedRTSPMaxPayload {
return apolloRTSPMessage{}, ErrProviderMalformed
}
frame := make([]byte, encryptedRTSPHeaderSize+int(length))
copy(frame, header)
if _, err := io.ReadFull(conn, frame[encryptedRTSPHeaderSize:]); err != nil {
return apolloRTSPMessage{}, err
}
plaintext, err := codec.OpenHost(frame)
if err != nil {
return apolloRTSPMessage{}, ErrProviderMalformed
}
return parseApolloRTSPMessage(plaintext)
}
func parseApolloRTSPMessage(data []byte) (apolloRTSPMessage, error) {
if len(data) == 0 || len(data) > encryptedRTSPMaxPayload {
return apolloRTSPMessage{}, ErrProviderMalformed
}
headerEnd := strings.Index(string(data), "\r\n\r\n")
if headerEnd < 0 || headerEnd+4 > maxApolloRTSPHeaders {
return apolloRTSPMessage{}, ErrProviderMalformed
}
lines := strings.Split(string(data[:headerEnd]), "\r\n")
if len(lines) < 1 {
return apolloRTSPMessage{}, ErrProviderMalformed
}
parts := strings.SplitN(lines[0], " ", 3)
if len(parts) != 3 || parts[0] != "RTSP/1.0" || len(parts[2]) == 0 || len(parts[2]) > 128 {
return apolloRTSPMessage{}, ErrProviderMalformed
}
status, err := strconv.Atoi(parts[1])
if err != nil || status < 100 || status > 599 {
return apolloRTSPMessage{}, ErrProviderMalformed
}
message := apolloRTSPMessage{status: status, headers: make(map[string]string), raw: append([]byte(nil), data...)}
for _, line := range lines[1:] {
key, value, ok := strings.Cut(line, ":")
key = strings.ToLower(strings.TrimSpace(key))
value = strings.TrimSpace(value)
if !ok || !validApolloRTSPToken(key) || value == "" || len(value) > 1024 {
return apolloRTSPMessage{}, ErrProviderMalformed
}
if _, duplicate := message.headers[key]; duplicate {
return apolloRTSPMessage{}, ErrProviderMalformed
}
message.headers[key] = value
}
cseq, ok := message.headers["cseq"]
if !ok {
return apolloRTSPMessage{}, ErrProviderMalformed
}
parsedCSeq, err := strconv.ParseUint(cseq, 10, 32)
if err != nil || parsedCSeq == 0 {
return apolloRTSPMessage{}, ErrProviderMalformed
}
message.cseq = uint32(parsedCSeq)
message.body = append([]byte(nil), data[headerEnd+4:]...)
if len(message.body) > maxApolloRTSPBody {
return apolloRTSPMessage{}, ErrProviderMalformed
}
if length, hasLength := message.headers["content-length"]; hasLength {
declared, err := strconv.ParseUint(length, 10, 16)
if err != nil || int(declared) != len(message.body) {
return apolloRTSPMessage{}, ErrProviderMalformed
}
} else if len(message.body) != 0 {
return apolloRTSPMessage{}, ErrProviderMalformed
}
return message, nil
}
func validateApolloDescribe(message apolloRTSPMessage) error {
if message.headers["content-type"] != "application/sdp" || len(message.body) == 0 {
return ErrProviderMalformed
}
body := string(message.body)
lineEnding := "\n"
if strings.Contains(body, "\r\n") {
if strings.Contains(strings.ReplaceAll(body, "\r\n", ""), "\n") {
return ErrProviderMalformed
}
lineEnding = "\r\n"
}
if !strings.HasSuffix(body, lineEnding) {
return ErrProviderMalformed
}
attributes := map[string]string{}
seen := map[string]struct{}{}
stage := 0
stereo := false
for _, line := range strings.Split(strings.TrimSuffix(body, lineEnding), lineEnding) {
if line == "" {
return ErrProviderMalformed
}
if !strings.HasPrefix(line, "a=") {
if line == "sprop-parameter-sets=AAAAAU" && stage >= 3 && stage <= 4 {
stage = 5
continue
}
return ErrProviderMalformed
}
key, value, ok := strings.Cut(strings.TrimPrefix(line, "a="), ":")
if !ok || key == "" || value == "" || len(key) > 128 || len(value) > 256 {
return ErrProviderMalformed
}
if key == "fmtp" {
if stage < 3 || !validApolloSurroundParameters(value) {
return ErrProviderMalformed
}
if _, duplicate := seen["fmtp:"+value]; duplicate {
return ErrProviderMalformed
}
seen["fmtp:"+value] = struct{}{}
if value == "97 surround-params=21101" {
stereo = true
}
stage = 6
continue
}
if key == "rtpmap" && value == "98 AV1/90000" && stage >= 3 && stage <= 5 {
if _, duplicate := seen[key]; duplicate {
return ErrProviderMalformed
}
seen[key] = struct{}{}
stage = 6
continue
}
if key == "x-nv-video[0].refPicInvalidation" && value == "1" && stage == 3 {
if _, duplicate := seen[key]; duplicate {
return ErrProviderMalformed
}
seen[key] = struct{}{}
stage = 4
continue
}
if key != "x-ss-general.featureFlags" && key != "x-ss-general.encryptionSupported" && key != "x-ss-general.encryptionRequested" {
return ErrProviderMalformed
}
if _, duplicate := attributes[key]; duplicate || (key == "x-ss-general.featureFlags" && stage != 0) || (key == "x-ss-general.encryptionSupported" && stage != 1) || (key == "x-ss-general.encryptionRequested" && stage != 2) {
return ErrProviderMalformed
}
attributes[key] = value
stage++
}
featureFlags, featureFlagsOK := attributes["x-ss-general.featureFlags"]
supported, supportedOK := attributes["x-ss-general.encryptionSupported"]
requested, requestedOK := attributes["x-ss-general.encryptionRequested"]
if !featureFlagsOK || !supportedOK || !requestedOK || !stereo {
return ErrProviderMalformed
}
if _, err := strconv.ParseUint(featureFlags, 10, 32); err != nil {
return ErrProviderMalformed
}
supportedFlags, err := strconv.ParseUint(supported, 10, 32)
if err != nil || supportedFlags&apolloEncryptionAll != apolloEncryptionAll {
return ErrProviderMalformed
}
requestedFlags, err := strconv.ParseUint(requested, 10, 32)
if err != nil || requestedFlags&^supportedFlags != 0 || requestedFlags&1 == 0 {
return ErrProviderMalformed
}
return nil
}
func validApolloSurroundParameters(value string) bool {
const prefix = "97 surround-params="
if !strings.HasPrefix(value, prefix) {
return false
}
parameters := strings.TrimPrefix(value, prefix)
if len(parameters) < 5 || len(parameters) > 11 {
return false
}
channels := int(parameters[0] - '0')
streams := int(parameters[1] - '0')
coupled := int(parameters[2] - '0')
if (channels != 2 && channels != 6 && channels != 8) || len(parameters) != channels+3 || streams+coupled != channels || streams == 0 {
return false
}
used := [8]bool{}
for _, character := range parameters[3:] {
if character < '0' || int(character-'0') >= channels || used[character-'0'] {
return false
}
used[character-'0'] = true
}
return true
}
func apolloRTSPSession(message apolloRTSPMessage) (string, error) {
value, ok := message.headers["session"]
if !ok {
return "", ErrProviderMalformed
}
token, _, _ := strings.Cut(value, ";")
token = strings.TrimSpace(token)
if !validApolloRTSPToken(token) || len(token) > 256 {
return "", ErrProviderMalformed
}
return token, nil
}
func apolloRTSPMatchSession(message apolloRTSPMessage, expected string) error {
actual, err := apolloRTSPSession(message)
if err != nil || actual != expected {
return ErrProviderMalformed
}
return nil
}
func apolloRTSPServerPort(message apolloRTSPMessage) (int, error) {
transport, ok := message.headers["transport"]
if !ok {
return 0, ErrProviderMalformed
}
parts := strings.Split(transport, ";")
if len(parts) != 2 || parts[0] != "unicast" {
return 0, ErrProviderMalformed
}
key, value, ok := strings.Cut(parts[1], "=")
port, err := strconv.Atoi(value)
if !ok || key != "server_port" || err != nil || port < 1 || port > maxApolloRTSPPort {
return 0, ErrProviderMalformed
}
return port, nil
}
func apolloRTSPPingPayload(message apolloRTSPMessage) ([]byte, error) {
payload, ok := message.headers["x-ss-ping-payload"]
if !ok || len(payload) != 16 || !validApolloRTSPToken(payload) {
return nil, ErrProviderMalformed
}
return []byte(payload), nil
}
func apolloRTSPConnectData(message apolloRTSPMessage) (uint32, error) {
value, ok := message.headers["x-ss-connect-data"]
if !ok {
return 0, ErrProviderMalformed
}
if value == "" || strings.Trim(value, "0123456789") != "" {
return 0, ErrProviderMalformed
}
parsed, err := strconv.ParseUint(value, 10, 32)
if err != nil || parsed == 0 {
return 0, ErrProviderMalformed
}
return uint32(parsed), nil
}
func apolloAnnounceProfile(policy protocol.ProviderStreamPolicy) ([]byte, error) {
if err := validateApolloStreamPolicy(policy); err != nil {
return nil, err
}
format, supportsHEVC := int64(0), int64(0)
if policy.Codec == "HEVC" {
format, supportsHEVC = 1, 1
}
maximumBitrate := policy.BitrateKbps * 80 / 100
if maximumBitrate > 100000 {
maximumBitrate = 100000
}
return []byte("v=0\r\n" +
"o=android 0 0 IN IP4 0.0.0.0\r\n" +
"s=NVIDIA Streaming Client\r\n" +
fmt.Sprintf("a=x-nv-video[0].clientViewportWd:%d\r\n", policy.ResolutionWidth) +
fmt.Sprintf("a=x-nv-video[0].clientViewportHt:%d\r\n", policy.ResolutionHeight) +
fmt.Sprintf("a=x-nv-video[0].maxFPS:%d\r\n", policy.Fps) +
"a=x-nv-video[0].packetSize:1024\r\n" +
"a=x-nv-video[0].videoEncoderSlicesPerFrame:1\r\n" +
"a=x-nv-video[0].maxNumReferenceFrames:0\r\n" +
fmt.Sprintf("a=x-nv-clientSupportHevc:%d\r\n", supportsHEVC) +
fmt.Sprintf("a=x-nv-vqos[0].bitStreamFormat:%d\r\n", format) +
fmt.Sprintf("a=x-nv-video[0].initialBitrateKbps:%d\r\n", maximumBitrate) +
fmt.Sprintf("a=x-nv-video[0].initialPeakBitrateKbps:%d\r\n", maximumBitrate) +
fmt.Sprintf("a=x-nv-vqos[0].bw.minimumBitrateKbps:%d\r\n", maximumBitrate) +
fmt.Sprintf("a=x-nv-vqos[0].bw.maximumBitrateKbps:%d\r\n", maximumBitrate) +
"a=x-nv-vqos[0].fec.minRequiredFecPackets:2\r\n" +
"a=x-nv-vqos[0].qosTrafficType:5\r\n" +
"a=x-nv-audio.surround.numChannels:2\r\n" +
"a=x-nv-audio.surround.channelMask:3\r\n" +
"a=x-nv-audio.surround.AudioQuality:0\r\n" +
"a=x-nv-aqos.packetDuration:5\r\n" +
"a=x-nv-aqos.qosTrafficType:4\r\n" +
"a=x-nv-general.useReliableUdp:13\r\n" +
"a=x-nv-general.featureFlags:167\r\n" +
"a=x-ml-general.featureFlags:0\r\n" +
fmt.Sprintf("a=x-ml-video.configuredBitrateKbps:%d\r\n", policy.BitrateKbps) +
"a=x-ss-general.encryptionEnabled:7\r\n" +
"a=x-ss-video[0].chromaSamplingType:0\r\n" +
"a=x-ss-video[0].intraRefresh:0\r\n"), nil
}
func validateApolloStreamPolicy(policy protocol.ProviderStreamPolicy) error {
if err := policy.Validate(); err != nil || !policy.AudioEnabled || policy.BitrateKbps > 125000 ||
(policy.Codec != "H264" && policy.Codec != "HEVC") {
return ErrProviderMalformed
}
if (policy.Codec == "H264" && (policy.ResolutionWidth > 4096 || policy.ResolutionHeight > 4096)) ||
(policy.Codec == "HEVC" && (policy.ResolutionWidth > 8192 || policy.ResolutionHeight > 8192)) {
return ErrProviderMalformed
}
return nil
}
func validateApolloProviderStreamPolicy(info ManagementInfo, policy protocol.ProviderStreamPolicy) error {
if validateApolloStreamPolicy(policy) != nil || !info.HasServerCodecModeSupport || !info.HasMaxLumaPixelsHEVC {
return ErrProviderMalformed
}
switch policy.Codec {
case "H264":
if info.ServerCodecModeSupport&0x1 == 0 {
return ErrProviderMalformed
}
case "HEVC":
luma := uint64(policy.ResolutionWidth) * uint64(policy.ResolutionHeight)
if info.ServerCodecModeSupport&0x100 == 0 || info.MaxLumaPixelsHEVC == 0 || luma > info.MaxLumaPixelsHEVC {
return ErrProviderMalformed
}
default:
return ErrProviderMalformed
}
return nil
}
func validApolloRTSPToken(value string) bool {
if value == "" || len(value) > 128 {
return false
}
for _, character := range value {
if character <= 0x20 || character >= 0x7f || strings.ContainsRune("()<>@,;:\\\"/[]?={}", character) {
return false
}
}
return true
}
+279
View File
@@ -0,0 +1,279 @@
package gateway
import (
"bytes"
"encoding/binary"
)
const (
apolloVideoMaximumDataShards = 255
apolloVideoMaximumBlocks = 4
apolloVideoShardPayloadSize = apolloVideoRawPacketSize - apolloRTPHeaderSize - 4 - apolloVideoNVHeaderSize
)
type apolloVideoShard struct {
frame uint32
block uint8
lastBlock uint8
dataPackets int
parity int
index int
sequence uint16
streamIndex uint32
flags byte
payload []byte
}
type apolloVideoFECBlock struct {
dataPackets int
parity int
firstSeq uint16
streamBase uint32
haveBase bool
shards [][]byte
received []bool
count int
complete bool
}
type apolloVideoAssembler struct {
haveFrame bool
frame uint32
lastBlock uint8
blocks [apolloVideoMaximumBlocks]*apolloVideoFECBlock
}
func (a *apolloVideoAssembler) Add(shard apolloVideoShard) ([]byte, error) {
if len(shard.payload) != apolloVideoShardPayloadSize || shard.dataPackets < 1 || shard.dataPackets > apolloVideoMaximumDataShards || shard.parity < 0 || shard.dataPackets+shard.parity > 255 || shard.block > shard.lastBlock || shard.lastBlock >= apolloVideoMaximumBlocks || shard.index >= shard.dataPackets+shard.parity {
return nil, errApolloMedia
}
if !a.haveFrame || apolloFrameNewer(shard.frame, a.frame) {
*a = apolloVideoAssembler{haveFrame: true, frame: shard.frame, lastBlock: shard.lastBlock}
} else if shard.frame != a.frame {
return nil, nil
}
if shard.lastBlock != a.lastBlock {
return nil, errApolloMedia
}
block := a.blocks[shard.block]
if block == nil {
block = &apolloVideoFECBlock{
dataPackets: shard.dataPackets,
parity: shard.parity,
shards: make([][]byte, shard.dataPackets+shard.parity),
received: make([]bool, shard.dataPackets+shard.parity),
}
a.blocks[shard.block] = block
} else if block.dataPackets != shard.dataPackets || block.parity != shard.parity {
return nil, errApolloMedia
}
if shard.index < shard.dataPackets {
if shard.index == 0 && shard.flags&0x04 == 0 || shard.index == shard.dataPackets-1 && shard.flags&0x02 == 0 || shard.flags&^byte(0x07) != 0 {
return nil, errApolloMedia
}
base := shard.sequence - uint16(shard.index)
streamBase := shard.streamIndex - uint32(shard.index)
if !block.haveBase {
block.firstSeq, block.streamBase, block.haveBase = base, streamBase, true
} else if block.firstSeq != base || block.streamBase != streamBase {
return nil, errApolloMedia
}
}
if block.received[shard.index] {
if !bytes.Equal(block.shards[shard.index], shard.payload) {
return nil, errApolloMedia
}
return nil, nil
}
block.shards[shard.index] = append([]byte(nil), shard.payload...)
block.received[shard.index] = true
block.count++
if block.count < block.dataPackets {
return nil, nil
}
if !block.complete {
if err := reconstructApolloVideoBlock(block); err != nil {
return nil, err
}
block.complete = true
}
for index := uint8(0); index <= a.lastBlock; index++ {
if a.blocks[index] == nil || !a.blocks[index].complete {
return nil, nil
}
}
capacity := 0
for blockIndex := uint8(0); blockIndex <= a.lastBlock; blockIndex++ {
capacity += a.blocks[blockIndex].dataPackets * apolloVideoShardPayloadSize
}
frame := make([]byte, 0, capacity)
for blockIndex := uint8(0); blockIndex <= a.lastBlock; blockIndex++ {
for shardIndex := 0; shardIndex < a.blocks[blockIndex].dataPackets; shardIndex++ {
frame = append(frame, a.blocks[blockIndex].shards[shardIndex]...)
}
}
if len(frame) < 8 || frame[0] != 0x01 {
return nil, errApolloMedia
}
lastPayloadLength := int(binary.LittleEndian.Uint16(frame[4:6]))
end := len(frame) - apolloVideoShardPayloadSize + lastPayloadLength
if lastPayloadLength == 0 || lastPayloadLength > apolloVideoShardPayloadSize || end <= 8 || end > len(frame) {
return nil, errApolloMedia
}
*a = apolloVideoAssembler{}
return append([]byte(nil), frame[8:end]...), nil
}
func apolloFrameNewer(first, second uint32) bool {
return int32(first-second) > 0
}
func reconstructApolloVideoBlock(block *apolloVideoFECBlock) error {
if block == nil || block.count < block.dataPackets {
return errApolloMedia
}
missing := false
for index := 0; index < block.dataPackets; index++ {
if !block.received[index] {
missing = true
block.shards[index] = make([]byte, apolloVideoShardPayloadSize)
}
}
if !missing {
return nil
}
selectedRows := make([][]byte, 0, block.dataPackets)
selectedShards := make([][]byte, 0, block.dataPackets)
for index, received := range block.received {
if !received {
continue
}
selectedRows = append(selectedRows, apolloVideoFECRow(index, block.dataPackets, block.parity))
selectedShards = append(selectedShards, block.shards[index])
if len(selectedRows) == block.dataPackets {
break
}
}
if len(selectedRows) != block.dataPackets {
return errApolloMedia
}
inverse, ok := apolloGFInvert(selectedRows)
if !ok {
return errApolloMedia
}
for index := 0; index < block.dataPackets; index++ {
if block.received[index] {
continue
}
for source, coefficient := range inverse[index] {
apolloGFAXPY(block.shards[index], selectedShards[source], coefficient)
}
block.received[index] = true
}
return nil
}
func apolloVideoFECRow(index, dataPackets, parity int) []byte {
row := make([]byte, dataPackets)
if index < dataPackets {
row[index] = 1
return row
}
parityIndex := index - dataPackets
for dataIndex := range row {
row[dataIndex] = apolloGFInverse(byte((parity + dataIndex) ^ parityIndex))
}
return row
}
func apolloGFInvert(matrix [][]byte) ([][]byte, bool) {
size := len(matrix)
if size == 0 {
return nil, false
}
work := make([][]byte, size)
inverse := make([][]byte, size)
for row := range matrix {
if len(matrix[row]) != size {
return nil, false
}
work[row] = append([]byte(nil), matrix[row]...)
inverse[row] = make([]byte, size)
inverse[row][row] = 1
}
for column := 0; column < size; column++ {
pivot := column
for pivot < size && work[pivot][column] == 0 {
pivot++
}
if pivot == size {
return nil, false
}
work[column], work[pivot] = work[pivot], work[column]
inverse[column], inverse[pivot] = inverse[pivot], inverse[column]
factor := apolloGFInverse(work[column][column])
for index := column; index < size; index++ {
work[column][index] = apolloGFMultiply(work[column][index], factor)
}
for index := range inverse[column] {
inverse[column][index] = apolloGFMultiply(inverse[column][index], factor)
}
for row := 0; row < size; row++ {
if row == column || work[row][column] == 0 {
continue
}
factor = work[row][column]
for index := column; index < size; index++ {
work[row][index] ^= apolloGFMultiply(work[column][index], factor)
}
for index := range inverse[row] {
inverse[row][index] ^= apolloGFMultiply(inverse[column][index], factor)
}
}
}
return inverse, true
}
func apolloGFAXPY(destination, source []byte, coefficient byte) {
if coefficient == 0 {
return
}
for index := range destination {
destination[index] ^= apolloGFMultiply(source[index], coefficient)
}
}
func apolloGFInverse(value byte) byte {
if value == 0 {
return 0
}
return apolloGFPow(value, 254)
}
func apolloGFPow(value byte, exponent uint8) byte {
result := byte(1)
for exponent != 0 {
if exponent&1 != 0 {
result = apolloGFMultiply(result, value)
}
value = apolloGFMultiply(value, value)
exponent >>= 1
}
return result
}
func apolloGFMultiply(first, second byte) byte {
var product byte
for second != 0 {
if second&1 != 0 {
product ^= first
}
high := first & 0x80
first <<= 1
if high != 0 {
first ^= 0x1d
}
second >>= 1
}
return product
}
+8 -18
View File
@@ -15,29 +15,19 @@ func DefaultCapabilities() protocol.CapabilityProfile {
Media: "encoded",
Audio: "encoded",
SourceRateControl: "server",
ClientDecode: "h264-opus",
ClientDecode: []string{"hevc-opus", "h264-opus"},
}
}
func capabilityProfileUnset(profile protocol.CapabilityProfile) bool {
return profile.Transport == "" && profile.Framing == "" && profile.Media == "" &&
profile.Audio == "" && profile.SourceRateControl == "" && len(profile.ClientDecode) == 0
}
func IntersectCapabilities(profiles ...protocol.CapabilityProfile) (protocol.CapabilityProfile, error) {
if len(profiles) == 0 {
selected, err := protocol.IntersectCapabilityProfiles(profiles...)
if err != nil {
return protocol.CapabilityProfile{}, ErrNoCapabilityOverlap
}
for _, profile := range profiles {
if err := profile.Validate(); err != nil {
return protocol.CapabilityProfile{}, ErrNoCapabilityOverlap
}
}
selected := profiles[0]
for _, profile := range profiles[1:] {
if selected.Transport != profile.Transport ||
selected.Framing != profile.Framing ||
selected.Media != profile.Media ||
selected.Audio != profile.Audio ||
selected.SourceRateControl != profile.SourceRateControl ||
selected.ClientDecode != profile.ClientDecode {
return protocol.CapabilityProfile{}, ErrNoCapabilityOverlap
}
}
return selected, nil
}
+163
View File
@@ -0,0 +1,163 @@
package gateway
import (
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"errors"
"sync"
"time"
"unicode/utf8"
protocol "git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol/gen/go/protocol"
)
var (
ErrClipboardDenied = errors.New("clipboard policy denied")
ErrClipboardRate = errors.New("clipboard rate limited")
)
const clipboardRetention = time.Minute
type clipboardRecord struct {
digest [sha256.Size]byte
direction string
at time.Time
}
type clipboardGate struct {
policy protocol.ClipboardPolicy
now func() time.Time
mu sync.Mutex
updates []time.Time
seen map[string]clipboardRecord
}
func newClipboardGate(policy protocol.ClipboardPolicy, now func() time.Time) (*clipboardGate, error) {
if err := policy.Validate(); err != nil || now == nil {
return nil, ErrProviderMalformed
}
return &clipboardGate{policy: policy, now: now, seen: make(map[string]clipboardRecord, policy.MaxUpdatesPerMinute)}, nil
}
// ValidateGatewayClipboard applies the authenticated Server policy before any
// clipboard value can reach a provider or Verse client.
func ValidateGatewayClipboard(policy protocol.ClipboardPolicy, value protocol.GatewayClipboardText) error {
if err := policy.Validate(); err != nil || value.Validate() != nil || !utf8.ValidString(value.Text) {
return ErrProviderMalformed
}
if len(value.Text) > int(policy.MaxTextBytes) {
return ErrProviderMalformed
}
switch value.Direction {
case "client_to_provider":
if !policy.ClientToProviderEnabled {
return ErrClipboardDenied
}
case "provider_to_client":
if !policy.ProviderToClientEnabled {
return ErrClipboardDenied
}
default:
return ErrProviderMalformed
}
return nil
}
func (g *clipboardGate) fromClient(value protocol.GatewayClipboardText) (bool, error) {
if g == nil {
return false, ErrClipboardDenied
}
if err := ValidateGatewayClipboard(g.policy, value); err != nil {
return false, err
}
now := g.now()
digest := sha256.Sum256([]byte(value.Text))
g.mu.Lock()
defer g.mu.Unlock()
g.pruneLocked(now)
if record, ok := g.seen[value.LoopToken]; ok {
if record.direction == "provider_to_client" && record.digest == digest {
return true, nil
}
return false, ErrClipboardDenied
}
if !g.allowUpdateLocked(now) {
return false, ErrClipboardRate
}
g.seen[value.LoopToken] = clipboardRecord{digest: digest, direction: value.Direction, at: now}
return false, nil
}
func (g *clipboardGate) fromProvider(text string) (protocol.GatewayClipboardText, bool, error) {
if g == nil || !g.policy.ProviderToClientEnabled {
return protocol.GatewayClipboardText{}, false, ErrClipboardDenied
}
if !utf8.ValidString(text) || len(text) > int(g.policy.MaxTextBytes) {
return protocol.GatewayClipboardText{}, false, ErrProviderMalformed
}
now := g.now()
digest := sha256.Sum256([]byte(text))
g.mu.Lock()
defer g.mu.Unlock()
g.pruneLocked(now)
for _, record := range g.seen {
if record.digest == digest {
return protocol.GatewayClipboardText{}, true, nil
}
}
if !g.allowUpdateLocked(now) {
return protocol.GatewayClipboardText{}, false, ErrClipboardRate
}
for attempts := 0; attempts < 3; attempts++ {
var raw [24]byte
if _, err := rand.Read(raw[:]); err != nil {
return protocol.GatewayClipboardText{}, false, err
}
token := base64.RawURLEncoding.EncodeToString(raw[:])
if _, exists := g.seen[token]; exists {
continue
}
value := protocol.GatewayClipboardText{Direction: "provider_to_client", Text: text, Encoding: "utf-8", LoopToken: token}
g.seen[token] = clipboardRecord{digest: digest, direction: value.Direction, at: now}
return value, false, nil
}
return protocol.GatewayClipboardText{}, false, ErrClipboardDenied
}
func (g *clipboardGate) retractClient(value protocol.GatewayClipboardText) {
if g == nil {
return
}
digest := sha256.Sum256([]byte(value.Text))
g.mu.Lock()
defer g.mu.Unlock()
if record, ok := g.seen[value.LoopToken]; ok && record.direction == "client_to_provider" && record.digest == digest {
delete(g.seen, value.LoopToken)
}
}
func (g *clipboardGate) pruneLocked(now time.Time) {
minimum := now.Add(-clipboardRetention)
index := 0
for _, update := range g.updates {
if update.After(minimum) {
g.updates[index] = update
index++
}
}
g.updates = g.updates[:index]
for token, record := range g.seen {
if !record.at.After(minimum) {
delete(g.seen, token)
}
}
}
func (g *clipboardGate) allowUpdateLocked(now time.Time) bool {
if len(g.updates) >= int(g.policy.MaxUpdatesPerMinute) {
return false
}
g.updates = append(g.updates, now)
return true
}
+75
View File
@@ -0,0 +1,75 @@
package gateway
import (
"errors"
"strings"
"testing"
"time"
protocol "git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol/gen/go/protocol"
)
func TestValidateGatewayClipboardEnforcesServerOwnedPolicy(t *testing.T) {
policy := protocol.ClipboardPolicy{ClientToProviderEnabled: true, ProviderToClientEnabled: true, MaxTextBytes: 5, MaxUpdatesPerMinute: 2}
valid := protocol.GatewayClipboardText{Direction: "client_to_provider", Text: "hello", Encoding: "utf-8", LoopToken: "abcdefghijklmnop"}
if err := ValidateGatewayClipboard(policy, valid); err != nil {
t.Fatalf("ValidateGatewayClipboard() valid text = %v", err)
}
if err := ValidateGatewayClipboard(policy, protocol.GatewayClipboardText{Direction: "client_to_provider", Text: "hello", Encoding: "utf-8", LoopToken: "!!!!!!!!!!!!!!!!"}); err == nil {
t.Fatal("ValidateGatewayClipboard() accepted malformed loop token")
}
if err := ValidateGatewayClipboard(policy, protocol.GatewayClipboardText{Direction: "client_to_provider", Text: strings.Repeat("x", 6), Encoding: "utf-8", LoopToken: "abcdefghijklmnop"}); err == nil {
t.Fatal("ValidateGatewayClipboard() accepted oversized text")
}
disabled := policy
disabled.ClientToProviderEnabled = false
if err := ValidateGatewayClipboard(disabled, valid); err == nil {
t.Fatal("ValidateGatewayClipboard() accepted disabled direction")
}
}
func TestClipboardGateSuppressesReflectionsAndBoundsRate(t *testing.T) {
now := time.Date(2026, time.January, 1, 0, 0, 0, 0, time.UTC)
policy := protocol.ClipboardPolicy{ClientToProviderEnabled: true, ProviderToClientEnabled: true, MaxTextBytes: 64, MaxUpdatesPerMinute: 2}
gate, err := newClipboardGate(policy, func() time.Time { return now })
if err != nil {
t.Fatal(err)
}
client := protocol.GatewayClipboardText{Direction: "client_to_provider", Text: "client", Encoding: "utf-8", LoopToken: "abcdefghijklmnop"}
if suppress, err := gate.fromClient(client); err != nil || suppress {
t.Fatalf("fromClient() = suppress %t, err %v", suppress, err)
}
if _, suppress, err := gate.fromProvider("client"); err != nil || !suppress {
t.Fatalf("fromProvider() reflection = suppress %t, err %v", suppress, err)
}
host, suppress, err := gate.fromProvider("host")
if err != nil || suppress || host.Direction != "provider_to_client" {
t.Fatalf("fromProvider() host = %#v, suppress %t, err %v", host, suppress, err)
}
if suppress, err := gate.fromClient(host); err != nil || !suppress {
t.Fatalf("fromClient() host reflection = suppress %t, err %v", suppress, err)
}
if _, err := gate.fromClient(protocol.GatewayClipboardText{Direction: "client_to_provider", Text: "third", Encoding: "utf-8", LoopToken: "qrstuvwxyzABCDEF"}); !errors.Is(err, ErrClipboardRate) {
t.Fatalf("fromClient() rate error = %v, want ErrClipboardRate", err)
}
now = now.Add(time.Minute)
if suppress, err := gate.fromClient(protocol.GatewayClipboardText{Direction: "client_to_provider", Text: "after-window", Encoding: "utf-8", LoopToken: "0123456789abcdef"}); err != nil || suppress {
t.Fatalf("fromClient() after window = suppress %t, err %v", suppress, err)
}
}
func TestClipboardGatePermitsRetryAfterProviderWriteFailure(t *testing.T) {
policy := protocol.ClipboardPolicy{ClientToProviderEnabled: true, MaxTextBytes: 64, MaxUpdatesPerMinute: 2}
gate, err := newClipboardGate(policy, time.Now)
if err != nil {
t.Fatal(err)
}
value := protocol.GatewayClipboardText{Direction: "client_to_provider", Text: "retry", Encoding: "utf-8", LoopToken: "abcdefghijklmnop"}
if suppress, err := gate.fromClient(value); err != nil || suppress {
t.Fatalf("fromClient() = suppress %t, err %v", suppress, err)
}
gate.retractClient(value)
if suppress, err := gate.fromClient(value); err != nil || suppress {
t.Fatalf("fromClient() retry = suppress %t, err %v", suppress, err)
}
}
+26 -2
View File
@@ -8,6 +8,7 @@ import (
"fmt"
"io"
"net/http"
"net/url"
"strings"
protocol "git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol/gen/go/protocol"
@@ -97,16 +98,38 @@ func (c *ControlPlaneClient) ReportProviderState(ctx context.Context, state prot
return err
}
func (c *ControlPlaneClient) ReportClipboardAudit(ctx context.Context, audit protocol.GatewayClipboardAudit) error {
payload, err := protocol.EncodeGatewayClipboardAudit(audit)
if err != nil {
return err
}
_, err = c.post(ctx, "/api/v1/gateway/clipboard-audit", payload)
return err
}
func (c *ControlPlaneClient) post(ctx context.Context, path string, payload []byte) ([]byte, error) {
if c == nil || c.HTTPClient == nil || c.BaseURL == "" {
return nil, errors.New("control-plane client is not configured")
}
request, err := http.NewRequestWithContext(ctx, http.MethodPost, c.BaseURL+path, bytes.NewReader(payload))
baseURL, err := url.Parse(c.BaseURL)
if err != nil || baseURL.Scheme != "https" || baseURL.Host == "" {
return nil, errors.New("control-plane base URL must be absolute HTTPS")
}
requestURL := *baseURL
requestURL.Path = strings.TrimRight(baseURL.Path, "/") + path
requestURL.RawPath = ""
requestURL.RawQuery = ""
requestURL.Fragment = ""
request, err := http.NewRequestWithContext(ctx, http.MethodPost, requestURL.String(), bytes.NewReader(payload))
if err != nil {
return nil, err
}
request.Header.Set("Content-Type", "application/json")
response, err := c.HTTPClient.Do(request)
client := *c.HTTPClient
client.CheckRedirect = func(*http.Request, []*http.Request) error {
return errors.New("control-plane redirects are not permitted")
}
response, err := client.Do(request)
if err != nil {
return nil, err
}
@@ -129,3 +152,4 @@ func (c *ControlPlaneClient) post(ctx context.Context, path string, payload []by
}
var _ Admission = (*ControlPlaneClient)(nil)
var _ ClipboardAuditReporter = (*ControlPlaneClient)(nil)
+94
View File
@@ -0,0 +1,94 @@
package gateway
import (
"context"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"time"
protocol "git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol/gen/go/protocol"
)
func TestControlPlaneRejectsHTTPBeforeSending(t *testing.T) {
var requests atomic.Int64
server := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
requests.Add(1)
}))
defer server.Close()
err := NewControlPlaneClient(server.URL, server.Client()).Heartbeat(
context.Background(),
validControlPlaneHeartbeat("gateway-1"),
)
if err == nil {
t.Fatal("HTTP control-plane URL was accepted")
}
if got := requests.Load(); got != 0 {
t.Fatalf("HTTP control-plane received %d requests, want 0", got)
}
}
func TestControlPlaneRejectsHTTPSRedirectToHTTPWithoutDisclosure(t *testing.T) {
var downgradeRequests atomic.Int64
downgrade := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
downgradeRequests.Add(1)
}))
defer downgrade.Close()
serverTLS, clientTLS := testTLS(t)
source := httptest.NewUnstartedServer(http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
if len(request.TLS.PeerCertificates) == 0 {
t.Error("source did not authenticate the client certificate")
}
http.Redirect(response, request, downgrade.URL+"/capture", http.StatusTemporaryRedirect)
}))
source.TLS = serverTLS
source.StartTLS()
defer source.Close()
client := &http.Client{Transport: &http.Transport{TLSClientConfig: clientTLS}}
err := NewControlPlaneClient(source.URL, client).Heartbeat(
context.Background(),
validControlPlaneHeartbeat("gateway-secret"),
)
if err == nil {
t.Fatal("HTTPS-to-HTTP redirect was accepted")
}
if got := downgradeRequests.Load(); got != 0 {
t.Fatalf("downgrade target received %d requests, want 0", got)
}
}
func TestControlPlanePinnedMTLSRemainsFunctional(t *testing.T) {
serverTLS, clientTLS := testTLS(t)
var authenticated atomic.Bool
server := httptest.NewUnstartedServer(http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
authenticated.Store(request.TLS != nil && len(request.TLS.PeerCertificates) > 0)
response.WriteHeader(http.StatusNoContent)
}))
server.TLS = serverTLS
server.StartTLS()
defer server.Close()
client := &http.Client{Transport: &http.Transport{TLSClientConfig: clientTLS}}
if err := NewControlPlaneClient(server.URL, client).Heartbeat(
context.Background(),
validControlPlaneHeartbeat("gateway-1"),
); err != nil {
t.Fatalf("pinned mTLS heartbeat: %v", err)
}
if !authenticated.Load() {
t.Fatal("server did not authenticate the client certificate")
}
}
func validControlPlaneHeartbeat(gatewayID string) protocol.GatewayHeartbeat {
return protocol.GatewayHeartbeat{
Version: "1", GatewayID: gatewayID, Sequence: 1,
ObservedAt: time.Unix(1, 0).UTC().Format(time.RFC3339Nano),
State: "ready",
Telemetry: protocol.GatewayTelemetry{MediaPackets: 1, ProviderState: "ready"},
}
}
+105
View File
@@ -0,0 +1,105 @@
package gateway
import (
"sort"
"testing"
"time"
)
type syntheticPacerDelivery struct {
at time.Time
flow string
bytes int64
}
func TestFairPacerEightFlowSharesAndCapacitySteps(t *testing.T) {
start := time.Date(2026, time.January, 1, 0, 0, 0, 0, time.UTC)
flows := []string{"one", "two", "three", "four", "five", "six", "seven", "eight"}
pacer := newFairPacer(8000)
next := make(map[string]time.Time, len(flows))
baseline := runSyntheticPacer(pacer, start, start.Add(60*time.Second), flows, next)
assertSyntheticFairness(t, baseline, flows)
assertSyntheticCap(t, baseline, 1_000_000)
pacer.setKbps(6000)
quarter := runSyntheticPacer(pacer, start.Add(60*time.Second), start.Add(70*time.Second), flows, next)
assertSyntheticFairness(t, quarter, flows)
assertSyntheticCap(t, quarter, 750_000)
pacer.setKbps(4000)
half := runSyntheticPacer(pacer, start.Add(70*time.Second), start.Add(80*time.Second), flows, next)
assertSyntheticFairness(t, half, flows)
assertSyntheticCap(t, half, 500_000)
}
func TestFairPacerBoundsCatchupAfterHostStall(t *testing.T) {
start := time.Date(2026, time.January, 1, 0, 0, 0, 0, time.UTC)
pacer := newFairPacer(8000)
_ = pacer.reserveAt(start, "one", 1000)
resumed := start.Add(100 * time.Millisecond)
next := pacer.reserveAt(resumed, "one", 1000)
if next.Before(resumed.Add(-fairPacerMaximumCatchup)) || next.After(resumed.Add(10*time.Millisecond)) {
t.Fatalf("post-stall reservation = %s, want bounded catchup near %s", next, resumed)
}
}
func runSyntheticPacer(pacer *fairPacer, start, end time.Time, flows []string, next map[string]time.Time) []syntheticPacerDelivery {
const packetBytes = 1000
for _, flow := range flows {
if next[flow].IsZero() {
next[flow] = pacer.reserveAt(start, flow, packetBytes)
}
}
var deliveries []syntheticPacerDelivery
for {
flow := ""
at := end.Add(time.Nanosecond)
for _, candidate := range flows {
if next[candidate].Before(at) {
flow, at = candidate, next[candidate]
}
}
if at.After(end) {
return deliveries
}
deliveries = append(deliveries, syntheticPacerDelivery{at: at, flow: flow, bytes: packetBytes})
next[flow] = pacer.reserveAt(at, flow, packetBytes)
}
}
func assertSyntheticFairness(t *testing.T, deliveries []syntheticPacerDelivery, flows []string) {
t.Helper()
counts := make(map[string]int64, len(flows))
for _, delivery := range deliveries {
counts[delivery.flow] += delivery.bytes
}
total := int64(0)
for _, flow := range flows {
total += counts[flow]
}
target := total / int64(len(flows))
for _, flow := range flows {
delta := counts[flow] - target
if delta < 0 {
delta = -delta
}
if target == 0 || float64(delta)/float64(target) > 0.10 {
t.Fatalf("flow %s share=%d target=%d", flow, counts[flow], target)
}
}
}
func assertSyntheticCap(t *testing.T, deliveries []syntheticPacerDelivery, bytesPerSecond int64) {
t.Helper()
sort.Slice(deliveries, func(first, second int) bool { return deliveries[first].at.Before(deliveries[second].at) })
for first, total, last := 0, int64(0), 0; first < len(deliveries); first++ {
for last < len(deliveries) && deliveries[last].at.Sub(deliveries[first].at) <= 5*time.Second {
total += deliveries[last].bytes
last++
}
if total > bytesPerSecond*5*105/100 {
t.Fatalf("five-second egress=%d exceeds cap=%d", total, bytesPerSecond*5)
}
total -= deliveries[first].bytes
}
}
+172
View File
@@ -0,0 +1,172 @@
package gateway
import "encoding/binary"
type FeedbackKind uint8
const (
FeedbackIDR FeedbackKind = iota + 1
FeedbackFEC
FeedbackTerminalReceipt
)
const (
gatewayFeedbackHeaderSize = 8
gatewayFeedbackClient = 0
gatewayFeedbackGateway = 1
gatewayFeedbackIDR = 1
gatewayFeedbackFEC = 2
gatewayFeedbackTerminalAck = 3
gatewayFeedbackTerminated = 0x10
gatewayFeedbackRumble = 0x11
gatewayFeedbackHDR = 0x12
gatewayFeedbackDisconnected = 0x13
)
type gatewayFeedbackMessage struct {
direction byte
kind byte
payload []byte
}
func EncodeProviderEvent(event ProviderEvent) ([]byte, error) {
switch event.Kind {
case ProviderEventTerminated:
if len(event.Payload) != 4 {
return nil, ErrProviderMalformed
}
return encodeGatewayFeedback(gatewayFeedbackGateway, gatewayFeedbackTerminated, event.Payload)
case ProviderEventRumble:
if len(event.Payload) != 5 || event.Payload[0] > 15 {
return nil, ErrProviderMalformed
}
return encodeGatewayFeedback(gatewayFeedbackGateway, gatewayFeedbackRumble, event.Payload)
case ProviderEventHDR:
if len(event.Payload) != 1 || event.Payload[0] > 1 {
return nil, ErrProviderMalformed
}
return encodeGatewayFeedback(gatewayFeedbackGateway, gatewayFeedbackHDR, event.Payload)
case ProviderEventDisconnected:
if len(event.Payload) != 0 {
return nil, ErrProviderMalformed
}
return encodeGatewayFeedback(gatewayFeedbackGateway, gatewayFeedbackDisconnected, nil)
default:
return nil, ErrProviderMalformed
}
}
func EncodeClientFeedback(feedback Feedback) ([]byte, error) {
var kind byte
switch feedback.Kind {
case FeedbackIDR:
kind = gatewayFeedbackIDR
if len(feedback.Payload) != 0 {
return nil, ErrProviderMalformed
}
case FeedbackFEC:
kind = gatewayFeedbackFEC
if !validGatewayFECStatus(feedback.Payload) {
return nil, ErrProviderMalformed
}
case FeedbackTerminalReceipt:
kind = gatewayFeedbackTerminalAck
if len(feedback.Payload) != 0 {
return nil, ErrProviderMalformed
}
default:
return nil, ErrProviderMalformed
}
return encodeGatewayFeedback(gatewayFeedbackClient, kind, feedback.Payload)
}
func DecodeClientFeedback(data []byte) (Feedback, error) {
message, err := decodeGatewayFeedback(data)
if err != nil || message.direction != gatewayFeedbackClient {
return Feedback{}, ErrProviderMalformed
}
switch message.kind {
case gatewayFeedbackIDR:
if len(message.payload) != 0 {
return Feedback{}, ErrProviderMalformed
}
return Feedback{Kind: FeedbackIDR}, nil
case gatewayFeedbackFEC:
if !validGatewayFECStatus(message.payload) {
return Feedback{}, ErrProviderMalformed
}
return Feedback{Kind: FeedbackFEC, Payload: message.payload}, nil
case gatewayFeedbackTerminalAck:
if len(message.payload) != 0 {
return Feedback{}, ErrProviderMalformed
}
return Feedback{Kind: FeedbackTerminalReceipt}, nil
default:
return Feedback{}, ErrProviderMalformed
}
}
func validGatewayFECStatus(payload []byte) bool {
if len(payload) != 21 {
return false
}
totalData := binary.BigEndian.Uint16(payload[10:12])
totalParity := binary.BigEndian.Uint16(payload[12:14])
receivedData := binary.BigEndian.Uint16(payload[14:16])
receivedParity := binary.BigEndian.Uint16(payload[16:18])
return totalData > 0 && receivedData <= totalData && receivedParity <= totalParity && payload[18] <= 100 && payload[20] > 0 && payload[19] < payload[20]
}
func DecodeProviderEvent(data []byte) (ProviderEvent, error) {
message, err := decodeGatewayFeedback(data)
if err != nil || message.direction != gatewayFeedbackGateway {
return ProviderEvent{}, ErrProviderMalformed
}
switch message.kind {
case gatewayFeedbackTerminated:
if len(message.payload) != 4 {
return ProviderEvent{}, ErrProviderMalformed
}
return ProviderEvent{Kind: ProviderEventTerminated, Payload: message.payload}, nil
case gatewayFeedbackRumble:
if len(message.payload) != 5 || message.payload[0] > 15 {
return ProviderEvent{}, ErrProviderMalformed
}
return ProviderEvent{Kind: ProviderEventRumble, Payload: message.payload}, nil
case gatewayFeedbackHDR:
if len(message.payload) != 1 || message.payload[0] > 1 {
return ProviderEvent{}, ErrProviderMalformed
}
return ProviderEvent{Kind: ProviderEventHDR, Payload: message.payload}, nil
case gatewayFeedbackDisconnected:
if len(message.payload) != 0 {
return ProviderEvent{}, ErrProviderMalformed
}
return ProviderEvent{Kind: ProviderEventDisconnected}, nil
default:
return ProviderEvent{}, ErrProviderMalformed
}
}
func encodeGatewayFeedback(direction, kind byte, payload []byte) ([]byte, error) {
if len(payload) > 1016 {
return nil, ErrProviderMalformed
}
encoded := make([]byte, gatewayFeedbackHeaderSize+len(payload))
copy(encoded, "VGF1")
encoded[4], encoded[5] = direction, kind
binary.BigEndian.PutUint16(encoded[6:8], uint16(len(payload)))
copy(encoded[8:], payload)
return encoded, nil
}
func decodeGatewayFeedback(data []byte) (gatewayFeedbackMessage, error) {
if len(data) < gatewayFeedbackHeaderSize || len(data) > 1024 || string(data[:4]) != "VGF1" || len(data) != gatewayFeedbackHeaderSize+int(binary.BigEndian.Uint16(data[6:8])) {
return gatewayFeedbackMessage{}, ErrProviderMalformed
}
message := gatewayFeedbackMessage{direction: data[4], kind: data[5], payload: append([]byte(nil), data[8:]...)}
if message.direction != gatewayFeedbackClient && message.direction != gatewayFeedbackGateway {
return gatewayFeedbackMessage{}, ErrProviderMalformed
}
return message, nil
}
+911 -58
View File
File diff suppressed because it is too large Load Diff
+112 -18
View File
@@ -3,36 +3,130 @@ package gateway
import (
"encoding/binary"
"errors"
"unicode/utf8"
)
var ErrInputMalformed = errors.New("input event malformed")
const (
gatewayInputHeaderSize = 6
gatewayInputKeyboard = 1
gatewayInputMouse = 2
gatewayInputRelative = 3
gatewayInputUTF8 = 4
gatewayInputController = 5
)
func EncodeInputEvent(event InputEvent) ([]byte, error) {
if len(event.Device) == 0 || len(event.Device) > 64 || len(event.Payload) > 1024 {
switch event.Device {
case "keyboard":
if event.Code < 1 || event.Code > 0xffff || len(event.Payload) > 1 {
return nil, ErrInputMalformed
}
encoded := make([]byte, gatewayInputHeaderSize+4)
copy(encoded, "VGI1")
encoded[4], encoded[5], encoded[6] = gatewayInputKeyboard, 4, 0
if event.Pressed {
encoded[6] = 1
}
if len(event.Payload) == 1 {
encoded[7] = event.Payload[0]
}
binary.BigEndian.PutUint16(encoded[8:10], uint16(event.Code))
return encoded, nil
case "mouse-button":
if event.Code < 1 || event.Code > 5 || len(event.Payload) != 0 {
return nil, ErrInputMalformed
}
encoded := make([]byte, gatewayInputHeaderSize+3)
copy(encoded, "VGI1")
encoded[4], encoded[5], encoded[7] = gatewayInputMouse, 3, byte(event.Code)
if event.Pressed {
encoded[6] = 1
}
return encoded, nil
case "mouse-relative":
if event.Pressed || event.Code != 0 || len(event.Payload) != 4 {
return nil, ErrInputMalformed
}
encoded := make([]byte, gatewayInputHeaderSize+4)
copy(encoded, "VGI1")
encoded[4], encoded[5] = gatewayInputRelative, 4
copy(encoded[6:], event.Payload)
return encoded, nil
case "utf8":
if event.Pressed || event.Code != 0 || len(event.Payload) == 0 || len(event.Payload) > utf8.UTFMax || !utf8.Valid(event.Payload) || utf8.RuneCount(event.Payload) != 1 {
return nil, ErrInputMalformed
}
encoded := make([]byte, gatewayInputHeaderSize+len(event.Payload))
copy(encoded, "VGI1")
encoded[4], encoded[5] = gatewayInputUTF8, byte(len(event.Payload))
copy(encoded[6:], event.Payload)
return encoded, nil
case "controller":
if event.Code < 0 || event.Code > 15 || len(event.Payload) != 16 {
return nil, ErrInputMalformed
}
active := binary.BigEndian.Uint16(event.Payload[:2])
if (!event.Pressed && anyNonzero(event.Payload)) || (event.Pressed && active == 0) {
return nil, ErrInputMalformed
}
encoded := make([]byte, gatewayInputHeaderSize+17)
copy(encoded, "VGI1")
encoded[4], encoded[5], encoded[6] = gatewayInputController, 17, byte(event.Code)
copy(encoded[7:], event.Payload)
return encoded, nil
default:
return nil, ErrInputMalformed
}
encoded := make([]byte, 16+len(event.Device)+len(event.Payload))
copy(encoded[:4], "INP1")
binary.BigEndian.PutUint32(encoded[4:8], event.Sequence)
binary.BigEndian.PutUint32(encoded[8:12], uint32(event.Code))
if event.Pressed {
encoded[12] = 1
}
encoded[13] = byte(len(event.Device))
binary.BigEndian.PutUint16(encoded[14:16], uint16(len(event.Payload)))
copy(encoded[16:16+len(event.Device)], event.Device)
copy(encoded[16+len(event.Device):], event.Payload)
return encoded, nil
}
func DecodeInputEvent(data []byte) (InputEvent, error) {
if len(data) < 16 || len(data) > 1179 || string(data[:4]) != "INP1" || (data[12] != 0 && data[12] != 1) {
if len(data) < gatewayInputHeaderSize || len(data) > 1179 || string(data[:4]) != "VGI1" || len(data) != gatewayInputHeaderSize+int(data[5]) {
return InputEvent{}, ErrInputMalformed
}
deviceLength := int(data[13])
payloadLength := int(binary.BigEndian.Uint16(data[14:16]))
if deviceLength == 0 || deviceLength > 64 || payloadLength > 1024 || len(data) != 16+deviceLength+payloadLength {
kind, body := data[4], data[gatewayInputHeaderSize:]
switch kind {
case gatewayInputKeyboard:
if len(body) != 4 || body[0] > 1 || binary.BigEndian.Uint16(body[2:4]) == 0 {
return InputEvent{}, ErrInputMalformed
}
return InputEvent{Device: "keyboard", Code: int32(binary.BigEndian.Uint16(body[2:4])), Pressed: body[0] == 1, Payload: []byte{body[1]}}, nil
case gatewayInputMouse:
if len(body) != 3 || body[0] > 1 || body[1] < 1 || body[1] > 5 || body[2] != 0 {
return InputEvent{}, ErrInputMalformed
}
return InputEvent{Device: "mouse-button", Code: int32(body[1]), Pressed: body[0] == 1}, nil
case gatewayInputRelative:
if len(body) != 4 {
return InputEvent{}, ErrInputMalformed
}
return InputEvent{Device: "mouse-relative", Payload: append([]byte(nil), body...)}, nil
case gatewayInputUTF8:
if len(body) == 0 || len(body) > utf8.UTFMax || !utf8.Valid(body) || utf8.RuneCount(body) != 1 {
return InputEvent{}, ErrInputMalformed
}
return InputEvent{Device: "utf8", Payload: append([]byte(nil), body...)}, nil
case gatewayInputController:
if len(body) != 17 || body[0] > 15 {
return InputEvent{}, ErrInputMalformed
}
payload := append([]byte(nil), body[1:]...)
active := binary.BigEndian.Uint16(payload[:2])
if active == 0 && anyNonzero(payload[2:]) {
return InputEvent{}, ErrInputMalformed
}
return InputEvent{Device: "controller", Code: int32(body[0]), Pressed: active != 0, Payload: payload}, nil
default:
return InputEvent{}, ErrInputMalformed
}
return InputEvent{Sequence: binary.BigEndian.Uint32(data[4:8]), Code: int32(binary.BigEndian.Uint32(data[8:12])), Pressed: data[12] == 1, Device: string(data[16 : 16+deviceLength]), Payload: append([]byte(nil), data[16+deviceLength:]...)}, nil
}
func anyNonzero(data []byte) bool {
for _, value := range data {
if value != 0 {
return true
}
}
return false
}
+66
View File
@@ -0,0 +1,66 @@
package gateway
import (
"bytes"
"debug/buildinfo"
"debug/elf"
"os"
"os/exec"
"path/filepath"
"testing"
)
func TestGatewayLinuxArtifactsAreReproduciblePureGoELF(t *testing.T) {
first, second := t.TempDir(), t.TempDir()
for _, output := range []string{first, second} {
command := exec.Command("make", "-C", "..", "gateway-linux", "DIST_DIR="+output)
command.Env = append(os.Environ(), "GOCACHE="+filepath.Join(t.TempDir(), "go-cache"))
if result, err := command.CombinedOutput(); err != nil {
t.Fatalf("gateway-linux: %v\n%s", err, result)
}
}
for _, architecture := range []struct {
name string
machine elf.Machine
}{
{name: "amd64", machine: elf.EM_X86_64},
{name: "arm64", machine: elf.EM_AARCH64},
} {
firstPath := filepath.Join(first, "verse-gateway-linux-"+architecture.name)
secondPath := filepath.Join(second, "verse-gateway-linux-"+architecture.name)
firstBytes, err := os.ReadFile(firstPath)
if err != nil {
t.Fatal(err)
}
secondBytes, err := os.ReadFile(secondPath)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(firstBytes, secondBytes) {
t.Fatalf("linux/%s gateway build is not byte reproducible", architecture.name)
}
executable, err := elf.Open(firstPath)
if err != nil {
t.Fatalf("linux/%s ELF: %v", architecture.name, err)
}
if executable.FileHeader.Machine != architecture.machine {
_ = executable.Close()
t.Fatalf("linux/%s machine = %s", architecture.name, executable.FileHeader.Machine)
}
if err := executable.Close(); err != nil {
t.Fatal(err)
}
info, err := buildinfo.ReadFile(firstPath)
if err != nil {
t.Fatalf("linux/%s Go build info: %v", architecture.name, err)
}
settings := make(map[string]string, len(info.Settings))
for _, setting := range info.Settings {
settings[setting.Key] = setting.Value
}
if settings["GOOS"] != "linux" || settings["GOARCH"] != architecture.name || settings["CGO_ENABLED"] != "0" {
t.Fatalf("linux/%s build settings = %#v", architecture.name, settings)
}
}
}
+173 -73
View File
@@ -2,10 +2,10 @@ package gateway
import (
"context"
"encoding/binary"
"encoding/xml"
"errors"
"fmt"
"strconv"
"strings"
"sync"
"time"
@@ -67,8 +67,12 @@ func (i ProviderIdentity) Validate(now time.Time, expected ProviderIdentity) err
}
type ManagementInfo struct {
Identity ProviderIdentity
Name string
Identity ProviderIdentity
Name string
ServerCodecModeSupport uint32
MaxLumaPixelsHEVC uint64
HasServerCodecModeSupport bool
HasMaxLumaPixelsHEVC bool
}
func ParseManagementXML(data []byte) (ManagementInfo, error) {
@@ -83,6 +87,8 @@ func ParseManagementXML(data []byte) (ManagementInfo, error) {
NotBefore string `xml:"not_before"`
NotAfter string `xml:"not_after"`
Name string `xml:"name"`
CodecModes string `xml:"ServerCodecModeSupport"`
MaxHEVCLuma string `xml:"MaxLumaPixelsHEVC"`
}
decoder := xml.NewDecoder(strings.NewReader(string(data)))
decoder.Strict = true
@@ -109,7 +115,24 @@ func ParseManagementXML(data []byte) (ManagementInfo, error) {
if identity.UniqueID == "" || len(identity.UniqueID) > 128 || len(identity.Fingerprint) > 256 {
return ManagementInfo{}, ErrProviderMalformed
}
return ManagementInfo{Identity: identity, Name: document.Name}, nil
info := ManagementInfo{Identity: identity, Name: document.Name}
if document.CodecModes != "" {
value, parseErr := strconv.ParseUint(document.CodecModes, 10, 32)
if parseErr != nil {
return ManagementInfo{}, ErrProviderMalformed
}
info.ServerCodecModeSupport = uint32(value)
info.HasServerCodecModeSupport = true
}
if document.MaxHEVCLuma != "" {
value, parseErr := strconv.ParseUint(document.MaxHEVCLuma, 10, 64)
if parseErr != nil {
return ManagementInfo{}, ErrProviderMalformed
}
info.MaxLumaPixelsHEVC = value
info.HasMaxLumaPixelsHEVC = true
}
return info, nil
}
type RTSPResponse struct {
@@ -159,36 +182,6 @@ func ParseRTSPResponse(data []byte) (RTSPResponse, error) {
return response, nil
}
type ControlPacket struct {
Kind byte
Sequence uint32
Payload []byte
}
func EncodeControlPacket(packet ControlPacket) ([]byte, error) {
if len(packet.Payload) > 4096 {
return nil, ErrProviderMalformed
}
encoded := make([]byte, 11+len(packet.Payload))
copy(encoded[:4], "APC1")
encoded[4] = packet.Kind
binary.BigEndian.PutUint32(encoded[5:9], packet.Sequence)
binary.BigEndian.PutUint16(encoded[9:11], uint16(len(packet.Payload)))
copy(encoded[11:], packet.Payload)
return encoded, nil
}
func DecodeControlPacket(data []byte) (ControlPacket, error) {
if len(data) < 11 || len(data) > 4107 || string(data[:4]) != "APC1" {
return ControlPacket{}, ErrProviderMalformed
}
length := int(binary.BigEndian.Uint16(data[9:11]))
if length > 4096 || len(data) != 11+length {
return ControlPacket{}, ErrProviderMalformed
}
return ControlPacket{Kind: data[4], Sequence: binary.BigEndian.Uint32(data[5:9]), Payload: append([]byte(nil), data[11:]...)}, nil
}
type LaunchRequest struct {
SessionID string
Capabilities protocol.CapabilityProfile
@@ -207,20 +200,56 @@ type InputEvent struct {
type Feedback struct {
Sequence uint32
Kind FeedbackKind
Payload []byte
}
type ProviderEventKind uint8
const (
ProviderEventTerminated ProviderEventKind = iota + 1
ProviderEventRumble
ProviderEventHDR
ProviderEventDisconnected
)
type ProviderEvent struct {
Kind ProviderEventKind
Payload []byte
}
// ProviderTelemetry holds measured provider-channel state only; it never
// contains provider routes, credentials, or payload bytes.
type ProviderTelemetry struct {
State string
ControlRTT time.Duration
ControlJitter time.Duration
ReliableSent uint64
ReliableRetransmits uint64
PendingReliable uint64
MediaDrops uint64
}
type ProviderMedia struct {
Payload []byte
ReceivedAt time.Time
EnqueuedAt time.Time
}
type Provider interface {
Start(context.Context, LaunchRequest) (ProviderSession, error)
}
type ProviderSession interface {
Ready(context.Context) error
Video() <-chan []byte
Audio() <-chan []byte
Video() <-chan ProviderMedia
Audio() <-chan ProviderMedia
Events() <-chan ProviderEvent
Input(context.Context, InputEvent) error
Feedback(context.Context, Feedback) error
Reconnect(context.Context) error
ReadClipboard(context.Context) (string, error)
WriteClipboard(context.Context, string) error
Telemetry() ProviderTelemetry
ReleaseAll(context.Context) error
Terminate(context.Context) error
State() protocol.ProviderState
@@ -228,7 +257,7 @@ type ProviderSession interface {
type ApolloBackend interface {
Management(context.Context, LaunchRequest) ([]byte, error)
Setup(context.Context, LaunchRequest) ([]byte, error)
Setup(context.Context, LaunchRequest, []byte) ([]byte, error)
Open(context.Context, LaunchRequest, RTSPResponse) (ProviderSession, error)
}
@@ -269,7 +298,7 @@ func (a *ApolloAdapter) Start(ctx context.Context, request LaunchRequest) (Provi
if request.ProviderIdentity != "" && info.Identity.UniqueID != expected.UniqueID {
return nil, ErrProviderIdentity
}
rawRTSP, err := a.backend.Setup(ctx, request)
rawRTSP, err := a.backend.Setup(ctx, request, management)
if err != nil {
return nil, err
}
@@ -353,20 +382,22 @@ func (f *FakeApollo) Management(context.Context, LaunchRequest) ([]byte, error)
return []byte(fmt.Sprintf("<root><unique_id>%s</unique_id><fingerprint>%s</fingerprint><not_before>%s</not_before><not_after>%s</not_after><name>fixture-apollo</name></root>", identity.UniqueID, identity.Fingerprint, f.config.Now.Add(-time.Hour).Format(time.RFC3339), f.config.Now.Add(time.Hour).Format(time.RFC3339))), nil
}
func (f *FakeApollo) Setup(context.Context, LaunchRequest) ([]byte, error) {
func (f *FakeApollo) Setup(context.Context, LaunchRequest, []byte) ([]byte, error) {
if f.config.Failure == FakeFailureMalformed {
return []byte("RTSP/1.0 200 OK\r\n\r\n"), nil
}
return []byte("RTSP/1.0 200 OK\r\nSession: fixture-session\r\nTransport: RTP/AVP/TCP;interleaved=0-1\r\n\r\n"), nil
return []byte("RTSP/1.0 200 OK\r\nSession: fixture-session\r\nTransport: unicast;server_port=43000\r\n\r\n"), nil
}
func (f *FakeApollo) Open(_ context.Context, request LaunchRequest, _ RTSPResponse) (ProviderSession, error) {
session := &fakeSession{
failure: f.config.Failure,
video: make(chan []byte, 16),
audio: make(chan []byte, 16),
state: protocol.ProviderState{Version: "1", SessionID: request.SessionID, State: ProviderStateStarting, Channels: []string{"video", "audio", "input", "feedback"}},
pressed: make(map[string]struct{}),
failure: f.config.Failure,
video: make(chan ProviderMedia, 16),
audio: make(chan ProviderMedia, 16),
events: make(chan ProviderEvent, 16),
clipboardWrites: make(chan string, 1),
state: protocol.ProviderState{Version: "1", SessionID: request.SessionID, State: ProviderStateStarting, Channels: []string{"video", "audio", "input", "feedback"}},
pressed: make(map[string]struct{}),
}
for _, payload := range f.config.Video {
session.EmitVideo(payload)
@@ -402,24 +433,28 @@ func (f *FakeApollo) DisconnectProvider() {
}
type fakeSession struct {
mu sync.Mutex
failure FakeFailure
video chan []byte
audio chan []byte
state protocol.ProviderState
pressed map[string]struct{}
inputs []InputEvent
feedback []Feedback
releaseAll int
closeOnce sync.Once
mu sync.Mutex
failure FakeFailure
video chan ProviderMedia
audio chan ProviderMedia
events chan ProviderEvent
state protocol.ProviderState
pressed map[string]struct{}
inputs []InputEvent
feedback []Feedback
clipboard string
clipboardWrites chan string
releaseAll int
closeOnce sync.Once
}
func (s *fakeSession) Ready(ctx context.Context) error {
s.mu.Lock()
if s.failure == FakeFailureReadinessTimeout {
s.mu.Unlock()
<-ctx.Done()
return ctx.Err()
}
s.mu.Lock()
defer s.mu.Unlock()
if s.state.State == ProviderStateDisconnected {
return ErrProviderDisconnected
@@ -428,24 +463,58 @@ func (s *fakeSession) Ready(ctx context.Context) error {
return nil
}
func (s *fakeSession) Video() <-chan []byte { return s.video }
func (s *fakeSession) Audio() <-chan []byte { return s.audio }
func (s *fakeSession) Video() <-chan ProviderMedia { return s.video }
func (s *fakeSession) Audio() <-chan ProviderMedia { return s.audio }
func (s *fakeSession) Events() <-chan ProviderEvent { return s.events }
func (s *fakeSession) EmitEvent(event ProviderEvent) {
select {
case s.events <- ProviderEvent{Kind: event.Kind, Payload: append([]byte(nil), event.Payload...)}:
default:
}
}
func (s *fakeSession) EmitVideo(payload []byte) {
s.mu.Lock()
defer s.mu.Unlock()
if s.state.State == ProviderStateTerminating || s.state.State == ProviderStateTerminated || s.state.State == ProviderStateDisconnected {
return
}
now := time.Now()
media := ProviderMedia{Payload: append([]byte(nil), payload...), ReceivedAt: now, EnqueuedAt: now}
select {
case s.video <- append([]byte(nil), payload...):
case s.video <- media:
default:
<-s.video
s.video <- append([]byte(nil), payload...)
select {
case <-s.video:
default:
}
select {
case s.video <- media:
default:
}
}
}
func (s *fakeSession) EmitAudio(payload []byte) {
s.mu.Lock()
defer s.mu.Unlock()
if s.state.State == ProviderStateTerminating || s.state.State == ProviderStateTerminated || s.state.State == ProviderStateDisconnected {
return
}
now := time.Now()
media := ProviderMedia{Payload: append([]byte(nil), payload...), ReceivedAt: now, EnqueuedAt: now}
select {
case s.audio <- append([]byte(nil), payload...):
case s.audio <- media:
default:
<-s.audio
s.audio <- append([]byte(nil), payload...)
select {
case <-s.audio:
default:
}
select {
case s.audio <- media:
default:
}
}
}
@@ -475,13 +544,33 @@ func (s *fakeSession) Feedback(_ context.Context, feedback Feedback) error {
return nil
}
func (s *fakeSession) Reconnect(_ context.Context) error {
func (s *fakeSession) ReadClipboard(ctx context.Context) (string, error) {
if err := ctx.Err(); err != nil {
return "", err
}
s.mu.Lock()
defer s.mu.Unlock()
if s.state.State == ProviderStateTerminated {
return ErrProviderTerminated
if s.state.State != ProviderStateReady {
return "", ErrProviderDisconnected
}
return s.clipboard, nil
}
func (s *fakeSession) WriteClipboard(ctx context.Context, value string) error {
if err := ctx.Err(); err != nil {
return err
}
s.mu.Lock()
if s.state.State != ProviderStateReady {
s.mu.Unlock()
return ErrProviderDisconnected
}
s.clipboard = value
s.mu.Unlock()
select {
case s.clipboardWrites <- value:
default:
}
s.state.State = ProviderStateReady
return nil
}
@@ -497,7 +586,10 @@ func (s *fakeSession) ReleaseAll(_ context.Context) error {
}
func (s *fakeSession) Terminate(ctx context.Context) error {
if s.failure == FakeFailureTerminationTimeout {
s.mu.Lock()
terminationTimeout := s.failure == FakeFailureTerminationTimeout
s.mu.Unlock()
if terminationTimeout {
<-ctx.Done()
s.mu.Lock()
s.state.State = ProviderStateCleanup
@@ -510,14 +602,17 @@ func (s *fakeSession) Terminate(ctx context.Context) error {
s.mu.Unlock()
return nil
}
disconnected := s.state.State == ProviderStateDisconnected
s.state.State = ProviderStateTerminating
s.mu.Unlock()
s.closeOnce.Do(func() {
close(s.video)
close(s.audio)
})
s.mu.Lock()
s.state.State = ProviderStateTerminated
if disconnected {
s.state.State = ProviderStateDisconnected
} else {
s.state.State = ProviderStateTerminated
}
s.mu.Unlock()
return nil
}
@@ -528,10 +623,15 @@ func (s *fakeSession) State() protocol.ProviderState {
return s.state
}
func (s *fakeSession) Telemetry() ProviderTelemetry {
return ProviderTelemetry{State: s.State().State}
}
func (s *fakeSession) Disconnect() {
s.mu.Lock()
s.state.State = ProviderStateDisconnected
s.mu.Unlock()
s.EmitEvent(ProviderEvent{Kind: ProviderEventDisconnected})
}
func (s *fakeSession) ReleaseCount() int {
+412
View File
@@ -0,0 +1,412 @@
package gateway
import (
"compress/gzip"
"encoding/binary"
"io"
"os"
"os/exec"
"path/filepath"
"reflect"
"strconv"
"strings"
"testing"
"time"
)
func TestQualificationCatalogMatchesSection7(t *testing.T) {
media := qualificationMediaProfiles()
if len(media) != 3 {
t.Fatalf("media profile count = %d, want 3", len(media))
}
wantMedia := []qualificationMediaProfile{
{Name: "1080p60-h264", Codec: "h264", BitrateKbps: 20000, Duration: 10 * time.Minute, Warmup: time.Second, PacketBytes: 1179},
{Name: "1440p120-hevc", Codec: "hevc", BitrateKbps: 50000, Duration: 10 * time.Minute, Warmup: time.Second, PacketBytes: 1179},
{Name: "4k60-hevc", Codec: "hevc", BitrateKbps: 80000, Duration: 10 * time.Minute, Warmup: time.Second, PacketBytes: 1179},
}
if !reflect.DeepEqual(media, wantMedia) {
t.Fatalf("media profiles = %#v, want %#v", media, wantMedia)
}
impairments := qualificationImpairmentProfiles()
wantImpairments := []qualificationImpairmentProfile{
{Name: "baseline", RTT: 20 * time.Millisecond},
{Name: "latency", RTT: 150 * time.Millisecond},
{Name: "jitter", RTT: 50 * time.Millisecond, Jitter: 30 * time.Millisecond},
{Name: "loss", RTT: 50 * time.Millisecond, Jitter: 10 * time.Millisecond, LossPercent: 5},
{Name: "reorder", RTT: 100 * time.Millisecond, Jitter: 10 * time.Millisecond, LossPercent: 1, Reorder: true},
{Name: "constrained", RTT: 50 * time.Millisecond, Jitter: 10 * time.Millisecond, LossPercent: 2, Reorder: true, CapacitySteps: []int{25, 50}},
}
if !reflect.DeepEqual(impairments, wantImpairments) {
t.Fatalf("impairment profiles = %#v, want %#v", impairments, wantImpairments)
}
}
func TestQualificationProtocolVersionIsExplicitAndImmutable(t *testing.T) {
const version = "v1.0.0-phase3c-gateway-rc.8"
t.Setenv("VERSEVDI_QUALIFICATION_PROTOCOL_VERSION", version)
got, err := qualificationProtocolVersion()
if err != nil || got != version {
t.Fatalf("qualificationProtocolVersion() = %q, want %q", got, version)
}
for _, invalid := range []string{"", "unknown", "v1", " v1.0.0", "v1.0.0+mutable"} {
t.Setenv("VERSEVDI_QUALIFICATION_PROTOCOL_VERSION", invalid)
if _, err := qualificationProtocolVersion(); err == nil {
t.Fatalf("qualificationProtocolVersion() accepted %q", invalid)
}
}
}
func TestQualificationRecordsLinkedToolVersions(t *testing.T) {
versions, err := qualificationToolVersions()
if err != nil || versions["qualification"] != qualificationToolVersion ||
versions["go"] == "" || versions["quic-go"] == "" {
t.Fatalf("qualification tool versions = %#v, %v", versions, err)
}
}
func TestQualificationOutputAndStatisticsFailClosed(t *testing.T) {
if err := validateQualificationOutputDir("relative/evidence"); err == nil {
t.Fatal("relative evidence directory was accepted")
}
if err := validateQualificationOutputDir(filepath.Join(t.TempDir(), "evidence")); err != nil {
t.Fatalf("absolute evidence directory rejected: %v", err)
}
summary, err := summarizeQualificationSamples([]time.Duration{
time.Millisecond, 2 * time.Millisecond, 3 * time.Millisecond,
4 * time.Millisecond, 5 * time.Millisecond,
})
if err != nil {
t.Fatal(err)
}
if summary.Count != 5 || summary.Min != time.Millisecond || summary.Median != 3*time.Millisecond ||
summary.P90 != 5*time.Millisecond || summary.P95 != 5*time.Millisecond ||
summary.P99 != 5*time.Millisecond || summary.Max != 5*time.Millisecond ||
summary.Mean != 3*time.Millisecond || summary.Histogram["le_5ms"] != 5 {
t.Fatalf("summary = %#v", summary)
}
if err := enforceQualificationProcessingGate(summary); err != nil {
t.Fatalf("5 ms p95 rejected: %v", err)
}
summary.P95++
if err := enforceQualificationProcessingGate(summary); err == nil {
t.Fatal("p95 above 5 ms was accepted")
}
}
func TestQualificationShortProcessingWritesRawArtifact(t *testing.T) {
profile := qualificationMediaProfile{
Name: "smoke", Codec: "h264", BitrateKbps: 20000,
Duration: time.Second, Warmup: time.Millisecond, PacketBytes: 1000,
}
rawPath := filepath.Join(t.TempDir(), "processing.csv.gz")
summary, err := runQualificationProcessing(t, profile, rawPath)
if err != nil {
t.Fatal(err)
}
if summary.Count < 1 || summary.RawSamplesSHA256 == "" || summary.RawSamplesBytes < 1 ||
summary.ResourceSamples < 2 || summary.RawResourcesSHA256 == "" || summary.RawResourcesBytes < 1 ||
summary.CPUScope != "isolated gateway qualification process (gateway plus bounded fixture/client driver)" {
t.Fatalf("processing summary = %#v", summary)
}
file, err := os.Open(rawPath)
if err != nil {
t.Fatal(err)
}
defer file.Close()
reader, err := gzip.NewReader(file)
if err != nil {
t.Fatal(err)
}
raw, err := io.ReadAll(reader)
if err != nil {
t.Fatal(err)
}
if err := reader.Close(); err != nil {
t.Fatal(err)
}
if !strings.HasPrefix(string(raw), "elapsed_ns,processing_ns\n") || strings.Count(string(raw), "\n") != int(summary.Count)+1 {
t.Fatalf("raw sample rows do not match summary count: %q", raw)
}
}
func TestQualificationProcessingPreservesPayload(t *testing.T) {
profile := qualificationMediaProfiles()[0]
payload := qualificationPayload(profile)
trace, elapsed, err := newQualificationPath(t, profile, profile.BitrateKbps).traverse(t, payload)
if err != nil {
t.Fatal(err)
}
if !trace.PayloadPreserved || !trace.ApolloRecovered || !trace.VerseQUIC {
t.Fatalf("production path trace = %#v", trace)
}
if elapsed <= 0 {
t.Fatalf("processing duration = %s", elapsed)
}
}
func TestQualificationRepeatedTraversalTracksEveryProductionStage(t *testing.T) {
profile := qualificationMediaProfiles()[1]
pacerKbps := (profile.BitrateKbps*int64(profile.PacketBytes+frameHeaderSize) + int64(profile.PacketBytes) - 1) / int64(profile.PacketBytes)
path := newQualificationPath(t, profile, pacerKbps)
payload := qualificationPayload(profile)
if err := runQualificationWarmup(t, path, profile, payload); err != nil {
t.Fatal(err)
}
for index := 0; index < 25_000; index++ {
current := append([]byte(nil), payload...)
binary.BigEndian.PutUint32(current[len(current)-4:], uint32(index))
trace, _, err := path.traverse(t, current)
if err != nil {
t.Fatal(err)
}
if !trace.NativeUDPIngress || !trace.ApolloRecovered || !trace.ProductionQueue ||
!trace.ProductionMediaLoop || !trace.ProductionPacer || !trace.VerseQUIC ||
!trace.PublicClientDecode || !trace.PayloadPreserved {
t.Fatalf("traversal %d missed a production stage: %#v", index, trace)
}
}
}
func TestQualificationImpairmentIsDeterministicAndBounded(t *testing.T) {
profile := qualificationImpairmentProfiles()[3]
first, err := runQualificationImpairment(t, profile, qualificationMediaProfiles()[0], 1000, filepath.Join(t.TempDir(), "first.csv.gz"))
if err != nil {
t.Fatal(err)
}
second, err := runQualificationImpairment(t, profile, qualificationMediaProfiles()[0], 1000, filepath.Join(t.TempDir(), "second.csv.gz"))
if err != nil {
t.Fatal(err)
}
if first.Dropped != second.Dropped || first.InjectedReordered != second.InjectedReordered {
t.Fatalf("deterministic impairment selection differs:\n%#v\n%#v", first, second)
}
if first.Sent != 1000 || first.Delivered+first.Dropped != first.Sent ||
first.ObservedLossPercent < 3.5 || first.ObservedLossPercent > 6.5 ||
first.MaxQueuePackets > qualificationImpairmentQueuePackets || first.RawSamplesSHA256 == "" {
t.Fatalf("impairment observation = %#v", first)
}
if _, err := runQualificationImpairment(t, profile, qualificationMediaProfiles()[0], qualificationImpairmentMaxPackets+1, filepath.Join(t.TempDir(), "invalid.csv.gz")); err == nil {
t.Fatal("unbounded impairment packet count was accepted")
}
unknown := profile
unknown.Name = "private-simulator"
if _, err := runQualificationImpairment(t, unknown, qualificationMediaProfiles()[0], 1, filepath.Join(t.TempDir(), "unknown.csv.gz")); err == nil {
t.Fatal("unregistered impairment profile was accepted")
}
}
func TestQualificationRTTIsNotSyntheticDoubleOneWayCompletion(t *testing.T) {
rawPath := filepath.Join(t.TempDir(), "latency.csv.gz")
observation, err := runQualificationImpairment(
t,
qualificationImpairmentProfiles()[1],
qualificationMediaProfiles()[0],
40,
rawPath,
)
if err != nil {
t.Fatal(err)
}
meanLatency := qualificationRawMeanLatency(t, rawPath)
delta := observation.ObservedRTT - 2*meanLatency
if delta < 0 {
delta = -delta
}
if delta < 5*time.Millisecond {
t.Fatalf("RTT %s was synthesized as twice one-way completion %s", observation.ObservedRTT, meanLatency)
}
}
func TestQualificationFixedSeedJitterIsObservableOnTraversedTraffic(t *testing.T) {
profile := qualificationImpairmentProfiles()[2]
observation, err := runQualificationImpairment(
t,
profile,
qualificationMediaProfiles()[0],
200,
filepath.Join(t.TempDir(), "jitter.csv.gz"),
)
if err != nil {
t.Fatal(err)
}
if observation.RTTSource != "apollo_enet_acknowledge" || observation.ObservedRTT <= 0 {
t.Fatalf("RTT observation is not transport-acknowledged: %#v", observation)
}
if observation.ObservedLatency < 5*time.Millisecond || observation.ObservedLatency > 100*time.Millisecond {
t.Fatalf("observed one-way latency %s does not reflect configured traversal", observation.ObservedLatency)
}
if observation.ObservedJitter < 5*time.Millisecond || observation.ObservedJitter > 80*time.Millisecond {
t.Fatalf("observed jitter %s is outside reviewed fixed-seed tolerance", observation.ObservedJitter)
}
if observation.ObservedOutOfOrder == 0 {
t.Fatal("fixed-seed jitter was serialized away before production traversal")
}
}
func TestQualificationCPUTracksConsumedWorkNotIdleCapacity(t *testing.T) {
started := time.Now()
before := qualificationRuntimeSample(started)
time.Sleep(100 * time.Millisecond)
idle := qualificationRuntimeSample(started).CPUSeconds - before.CPUSeconds
if idle > 50*time.Millisecond.Seconds() {
t.Fatalf("idle CPU consumption = %.6fs, want at most 0.05s", idle)
}
workBefore := qualificationRuntimeSample(started).CPUSeconds
deadline := time.Now().Add(75 * time.Millisecond)
var value uint64 = 1
for time.Now().Before(deadline) {
value = value*6364136223846793005 + 1
}
if value == 0 {
t.Fatal("bounded CPU work was optimized away")
}
work := qualificationRuntimeSample(started).CPUSeconds - workBefore
if work <= idle || work < 20*time.Millisecond.Seconds() {
t.Fatalf("bounded work CPU = %.6fs, idle = %.6fs", work, idle)
}
}
func TestQualificationCPUIsolationExcludesParentTestWork(t *testing.T) {
output := filepath.Join(t.TempDir(), "cpu.txt")
command := exec.Command(os.Args[0], "-test.run=^TestQualificationCPUChild$", "-test.count=1")
command.Env = append(os.Environ(), "VERSEVDI_QUALIFICATION_CPU_CHILD="+output)
if err := command.Start(); err != nil {
t.Fatal(err)
}
deadline := time.Now().Add(150 * time.Millisecond)
var value uint64 = 1
for time.Now().Before(deadline) {
value = value*2862933555777941757 + 3037000493
}
if value == 0 {
t.Fatal("parent CPU work was optimized away")
}
if err := command.Wait(); err != nil {
t.Fatalf("CPU child: %v", err)
}
raw, err := os.ReadFile(output)
if err != nil {
t.Fatal(err)
}
consumed, err := strconv.ParseFloat(string(raw), 64)
if err != nil {
t.Fatal(err)
}
if consumed > 50*time.Millisecond.Seconds() {
t.Fatalf("isolated idle qualification process consumed %.6fs while parent test was busy", consumed)
}
}
func TestQualificationCPUChild(t *testing.T) {
output := os.Getenv("VERSEVDI_QUALIFICATION_CPU_CHILD")
if output == "" {
return
}
started := time.Now()
before := qualificationRuntimeSample(started)
time.Sleep(200 * time.Millisecond)
consumed := qualificationRuntimeSample(started).CPUSeconds - before.CPUSeconds
if err := os.WriteFile(output, []byte(strconv.FormatFloat(consumed, 'f', 9, 64)), 0o600); err != nil {
t.Fatal(err)
}
}
func qualificationRawMeanLatency(t *testing.T, path string) time.Duration {
t.Helper()
file, err := os.Open(path)
if err != nil {
t.Fatal(err)
}
defer file.Close()
compressed, err := gzip.NewReader(file)
if err != nil {
t.Fatal(err)
}
raw, err := io.ReadAll(compressed)
if err != nil {
t.Fatal(err)
}
if err := compressed.Close(); err != nil {
t.Fatal(err)
}
var total time.Duration
var count int
for _, line := range strings.Split(string(raw), "\n")[1:] {
fields := strings.Split(line, ",")
if len(fields) < 5 || fields[4] != "delivered" {
continue
}
sent, sentErr := strconv.ParseInt(fields[1], 10, 64)
delivered, deliveredErr := strconv.ParseInt(fields[2], 10, 64)
if sentErr != nil || deliveredErr != nil || delivered < sent {
t.Fatalf("invalid raw latency row %q", line)
}
total += time.Duration(delivered - sent)
count++
}
if count == 0 {
t.Fatal("no delivered raw latency rows")
}
return total / time.Duration(count)
}
func TestQualificationSixImpairmentProfilesTraverseProductionPath(t *testing.T) {
profiles := qualificationImpairmentProfiles()
if len(profiles) != 6 {
t.Fatalf("impairment profile count = %d, want exactly 6", len(profiles))
}
for _, profile := range profiles {
observation, err := runQualificationImpairment(t, profile, qualificationMediaProfiles()[0], 40,
filepath.Join(t.TempDir(), profile.Name+".csv.gz"))
if err != nil {
t.Fatalf("%s: %v", profile.Name, err)
}
if observation.Profile != profile.Name || observation.Delivered+observation.Dropped != 40 ||
observation.RawSamplesSHA256 == "" || observation.MaxQueuePackets > qualificationImpairmentQueuePackets {
t.Fatalf("%s observation = %#v", profile.Name, observation)
}
}
}
func TestQualificationLossAndSteppedThroughputBounds(t *testing.T) {
profiles := qualificationImpairmentProfiles()
for _, profile := range profiles[3:] {
observation, err := runQualificationImpairment(t, profile, qualificationMediaProfiles()[0],
qualificationImpairmentPacketCount, filepath.Join(t.TempDir(), profile.Name+".csv.gz"))
if err != nil {
t.Fatalf("%s: %v", profile.Name, err)
}
if observation.ObservedThroughputKbps <= 0 ||
profile.Name == "constrained" && len(observation.CapacityStepObservations) != 2 {
t.Fatalf("%s observation = %#v", profile.Name, observation)
}
}
}
func TestQualificationUsesPublicQUICAndProductionPacer(t *testing.T) {
qualificationTraverseProfiles(t, qualificationMediaProfiles())
evidence, err := qualificationPacerEvidence(t, filepath.Join(t.TempDir(), "fairness.csv.gz"), 2*time.Second, 4*time.Second)
if err != nil {
t.Fatal(err)
}
if len(evidence.PerFlowBytes) != 8 || len(evidence.CapacitySteps) != 2 ||
len(evidence.Series) != 10 || evidence.RawSamplesSHA256 == "" || evidence.JainIndex < 0.99 {
t.Fatalf("pacer evidence = %#v", evidence)
}
for _, step := range evidence.CapacitySteps {
if step.Convergence > 10*time.Second || step.MaximumFiveSecond > step.FiveSecondCap*105/100 {
t.Fatalf("capacity step = %#v", step)
}
}
}
func TestQualificationSmokeTraversesNativeApolloRecoveryQueuePacerAndQUIC(t *testing.T) {
trace := qualificationProductionPathSmoke(t, qualificationMediaProfiles()[0])
if !trace.ApolloRecovered || !trace.ProductionQueue || !trace.ProductionPacer ||
!trace.VerseQUIC || !trace.PayloadPreserved {
t.Fatalf("qualification production-path trace = %#v", trace)
}
}
File diff suppressed because it is too large Load Diff
+45
View File
@@ -0,0 +1,45 @@
package gateway
import (
"net"
"testing"
"time"
)
func TestGatewaySlowReaderStillCleansUpWithinBound(t *testing.T) {
harness := newGatewayTransportHarness(t)
for sequence := 0; sequence < 10_000; sequence++ {
harness.session.EmitVideo([]byte{byte(sequence)})
}
if err := harness.client.Close(); err != nil {
t.Fatal(err)
}
harness.waitReleased(t)
}
func TestGatewayMalformedUDPDoesNotAmplify(t *testing.T) {
harness := newGatewayTransportHarness(t)
connection, err := net.DialUDP("udp", nil, harness.server.Addr().(*net.UDPAddr))
if err != nil {
t.Fatal(err)
}
defer connection.Close()
request := []byte("invalid")
if _, err := connection.Write(request); err != nil {
t.Fatal(err)
}
if err := connection.SetReadDeadline(time.Now().Add(100 * time.Millisecond)); err != nil {
t.Fatal(err)
}
response := make([]byte, len(request)*3+1)
count, _, err := connection.ReadFromUDP(response)
if err != nil {
if timeout, ok := err.(net.Error); ok && timeout.Timeout() {
return
}
t.Fatal(err)
}
if count > len(request)*3 {
t.Fatalf("malformed UDP amplified %d bytes to %d", len(request), count)
}
}
+169 -32
View File
@@ -2,60 +2,197 @@ package gateway
import (
"context"
"sync"
"sync/atomic"
"time"
)
type Metrics struct {
ActiveSessions atomic.Int64
AdmissionRejects atomic.Uint64
MediaDrops atomic.Uint64
ProviderErrors atomic.Uint64
InputRejected atomic.Uint64
ActiveSessions atomic.Int64
AdmittedSessions atomic.Uint64
AdmissionRejects atomic.Uint64
Reconnects atomic.Uint64
DrainTransitions atomic.Uint64
MediaDrops atomic.Uint64
MediaPackets atomic.Uint64
MediaBytes atomic.Uint64
QueueDelayNanos atomic.Uint64
ProcessingDelayNanos atomic.Uint64
ProcessingSamples atomic.Uint64
PacingDelayNanos atomic.Uint64
ProviderErrors atomic.Uint64
InputRejected atomic.Uint64
ControlRTTNanos atomic.Uint64
ControlJitterNanos atomic.Uint64
ControlLossPPM atomic.Uint64
PendingReliable atomic.Uint64
ProviderState atomic.Uint64
}
type MetricsSnapshot struct {
ActiveSessions int64
AdmissionRejects uint64
MediaDrops uint64
ProviderErrors uint64
InputRejected uint64
ActiveSessions int64
AdmittedSessions uint64
AdmissionRejects uint64
Reconnects uint64
DrainTransitions uint64
MediaDrops uint64
MediaPackets uint64
MediaBytes uint64
QueueDelayNanos uint64
ProcessingDelayNanos uint64
ProcessingSamples uint64
PacingDelayNanos uint64
ProviderErrors uint64
InputRejected uint64
ControlRTTNanos uint64
ControlJitterNanos uint64
ControlLossPPM uint64
PendingReliable uint64
ProviderState uint64
}
func (m *Metrics) Snapshot() MetricsSnapshot {
return MetricsSnapshot{
ActiveSessions: m.ActiveSessions.Load(),
AdmissionRejects: m.AdmissionRejects.Load(),
MediaDrops: m.MediaDrops.Load(),
ProviderErrors: m.ProviderErrors.Load(),
InputRejected: m.InputRejected.Load(),
ActiveSessions: m.ActiveSessions.Load(),
AdmittedSessions: m.AdmittedSessions.Load(),
AdmissionRejects: m.AdmissionRejects.Load(),
Reconnects: m.Reconnects.Load(),
DrainTransitions: m.DrainTransitions.Load(),
MediaDrops: m.MediaDrops.Load(),
MediaPackets: m.MediaPackets.Load(),
MediaBytes: m.MediaBytes.Load(),
QueueDelayNanos: m.QueueDelayNanos.Load(),
ProcessingDelayNanos: m.ProcessingDelayNanos.Load(),
ProcessingSamples: m.ProcessingSamples.Load(),
PacingDelayNanos: m.PacingDelayNanos.Load(),
ProviderErrors: m.ProviderErrors.Load(),
InputRejected: m.InputRejected.Load(),
ControlRTTNanos: m.ControlRTTNanos.Load(),
ControlJitterNanos: m.ControlJitterNanos.Load(),
ControlLossPPM: m.ControlLossPPM.Load(),
PendingReliable: m.PendingReliable.Load(),
ProviderState: m.ProviderState.Load(),
}
}
type Pacer struct {
func (m *Metrics) observeProviderTelemetry(telemetry ProviderTelemetry) {
if m == nil {
return
}
m.ControlRTTNanos.Store(uint64(telemetry.ControlRTT))
m.ControlJitterNanos.Store(uint64(telemetry.ControlJitter))
m.PendingReliable.Store(telemetry.PendingReliable)
if telemetry.ReliableSent == 0 {
m.ControlLossPPM.Store(0)
} else {
m.ControlLossPPM.Store(telemetry.ReliableRetransmits * 1_000_000 / telemetry.ReliableSent)
}
}
func (m *Metrics) observeProviderState(state string) {
if m == nil {
return
}
switch state {
case ProviderStateStarting:
m.ProviderState.Store(1)
case ProviderStateReady:
m.ProviderState.Store(2)
case ProviderStateDisconnected:
m.ProviderState.Store(3)
case ProviderStateTerminating:
m.ProviderState.Store(4)
case ProviderStateTerminated:
m.ProviderState.Store(5)
case ProviderStateCleanup:
m.ProviderState.Store(6)
case ProviderStateFailed:
m.ProviderState.Store(7)
default:
m.ProviderState.Store(0)
}
}
// fairPacer is the gateway's one shared, equal-tier media scheduler. Each
// session can hold only its existing bounded provider media channel while it
// waits for the next reservation, so a slow client cannot grow a global queue.
type fairPacer struct {
mu sync.Mutex
bytesPerSecond int64
last time.Time
flows map[string]fairPacerFlow
reservations atomic.Uint64
}
func NewPacer(kbps int64) *Pacer {
if kbps < 1 {
return &Pacer{}
}
return &Pacer{bytesPerSecond: kbps * 1000 / 8}
type fairPacerFlow struct {
next time.Time
lastSeen time.Time
}
func (p *Pacer) Wait(ctx context.Context, bytes int) error {
if p.bytesPerSecond < 1 || bytes < 1 {
return nil
const fairPacerMaximumCatchup = 5 * time.Millisecond
func newFairPacer(kbps int64) *fairPacer {
pacer := &fairPacer{flows: make(map[string]fairPacerFlow)}
pacer.setKbps(kbps)
return pacer
}
func (p *fairPacer) setKbps(kbps int64) {
if p == nil {
return
}
now := time.Now()
if p.last.IsZero() || now.After(p.last) {
p.last = now
p.mu.Lock()
if kbps > 0 {
p.bytesPerSecond = kbps * 1000 / 8
} else {
p.bytesPerSecond = 0
}
delay := time.Duration(float64(bytes) / float64(p.bytesPerSecond) * float64(time.Second))
p.last = p.last.Add(delay)
if wait := time.Until(p.last); wait > 0 {
timer := time.NewTimer(wait)
p.mu.Unlock()
}
func (p *fairPacer) remove(flow string) {
if p == nil || flow == "" {
return
}
p.mu.Lock()
delete(p.flows, flow)
p.mu.Unlock()
}
func (p *fairPacer) reserveAt(now time.Time, flow string, bytes int) time.Time {
if p == nil || flow == "" || bytes < 1 {
return now
}
p.mu.Lock()
defer p.mu.Unlock()
if p.bytesPerSecond < 1 {
return now
}
for key, state := range p.flows {
if now.Sub(state.lastSeen) > time.Second {
delete(p.flows, key)
}
}
state := p.flows[flow]
state.lastSeen = now
p.flows[flow] = state
base := state.next
if base.IsZero() {
base = now
} else if lag := now.Sub(base); lag > fairPacerMaximumCatchup {
base = now.Add(-fairPacerMaximumCatchup)
}
numerator := int64(bytes) * int64(len(p.flows)) * int64(time.Second)
delay := time.Duration((numerator + p.bytesPerSecond - 1) / p.bytesPerSecond)
state.next = base.Add(delay)
p.flows[flow] = state
return state.next
}
func (p *fairPacer) wait(ctx context.Context, flow string, bytes int) error {
target := p.reserveAt(time.Now(), flow, bytes)
p.reservations.Add(1)
if delay := time.Until(target); delay > 0 {
timer := time.NewTimer(delay)
defer timer.Stop()
select {
case <-ctx.Done():
+1 -1
View File
@@ -1 +1 @@
RTSP/1.0 200 OK\r\nSession: fixture-session\r\nTransport: RTP/AVP/TCP;interleaved=0-1\r\n\r\n
RTSP/1.0 200 OK\r\nSession: fixture-session\r\nTransport: unicast;server_port=43000\r\n\r\n
+540 -80
View File
@@ -9,6 +9,7 @@ import (
"fmt"
"io"
"net"
"slices"
"sync"
"sync/atomic"
"time"
@@ -19,9 +20,14 @@ import (
)
const (
defaultHelloLimit = 16 * 1024
defaultControlLimit = 128 * 1024
applicationError = quic.ApplicationErrorCode(0x100)
defaultHelloLimit = 16 * 1024
defaultControlLimit = 128 * 1024
clientControlBacklog = 64
terminalAckTimeout = 2 * time.Second
applicationError = quic.ApplicationErrorCode(0x100)
controlFlowID = "control.ack.v1"
inputFlowID = "input.sequenced.v1"
clipboardFlowID = "clipboard.text.v1"
)
var (
@@ -53,25 +59,31 @@ type ProviderStateReporter interface {
ReportProviderState(context.Context, protocol.ProviderState) error
}
type ClipboardAuditReporter interface {
ReportClipboardAudit(context.Context, protocol.GatewayClipboardAudit) error
}
type ServerConfig struct {
ListenAddress string
TLSConfig *tls.Config
QUICConfig *quic.Config
GatewayID string
Capabilities protocol.CapabilityProfile
ProviderCapabilities protocol.CapabilityProfile
Admission Admission
ProviderStateReporter ProviderStateReporter
Provider Provider
ProviderProfile string
ProviderIdentity string
PacerKbps int64
ListenAddress string
TLSConfig *tls.Config
QUICConfig *quic.Config
GatewayID string
Capabilities protocol.CapabilityProfile
ProviderCapabilities protocol.CapabilityProfile
Admission Admission
ProviderStateReporter ProviderStateReporter
ClipboardAuditReporter ClipboardAuditReporter
Provider Provider
ProviderProfile string
ProviderIdentity string
PacerKbps int64
}
type Server struct {
listener *quic.Listener
config ServerConfig
metrics *Metrics
pacer *fairPacer
mu sync.Mutex
sessions map[*gatewaySession]struct{}
draining atomic.Bool
@@ -90,12 +102,15 @@ func NewServer(config ServerConfig) (*Server, error) {
if err := validateServerTLS(config.TLSConfig); err != nil {
return nil, err
}
if config.Capabilities == (protocol.CapabilityProfile{}) {
if capabilityProfileUnset(config.Capabilities) {
config.Capabilities = DefaultCapabilities()
}
if config.ProviderCapabilities == (protocol.CapabilityProfile{}) {
if capabilityProfileUnset(config.ProviderCapabilities) {
config.ProviderCapabilities = DefaultCapabilities()
}
if config.Capabilities.Validate() != nil || config.ProviderCapabilities.Validate() != nil {
return nil, ErrNoCapabilityOverlap
}
if config.ProviderProfile == "" {
config.ProviderProfile = ProviderProfileApollo
}
@@ -115,7 +130,7 @@ func NewServer(config ServerConfig) (*Server, error) {
if err != nil {
return nil, err
}
return &Server{listener: listener, config: config, metrics: &Metrics{}, sessions: make(map[*gatewaySession]struct{})}, nil
return &Server{listener: listener, config: config, metrics: &Metrics{}, pacer: newFairPacer(config.PacerKbps), sessions: make(map[*gatewaySession]struct{})}, nil
}
func validateServerTLS(config *tls.Config) error {
@@ -130,7 +145,9 @@ func (s *Server) Metrics() MetricsSnapshot { return s.metrics.Snapshot() }
func (s *Server) Draining() bool { return s.draining.Load() }
func (s *Server) BeginDrain() {
s.draining.Store(true)
if s.draining.CompareAndSwap(false, true) {
s.metrics.DrainTransitions.Add(1)
}
}
func (s *Server) Serve(ctx context.Context) error {
@@ -228,6 +245,24 @@ func (s *Server) handleConnection(parent context.Context, connection *quic.Conn)
_ = writeStableError(stream, "no_capability_overlap", err, false)
return
}
selected, err = selectApolloPolicyCapabilities(work.StreamPolicy, selected)
if err != nil {
_ = s.config.Admission.Release(context.Background(), authority)
s.metrics.AdmissionRejects.Add(1)
_ = writeStableError(stream, "no_capability_overlap", err, false)
return
}
clipboard, err := newClipboardGate(work.ClipboardPolicy, time.Now)
if err != nil {
_ = s.config.Admission.Release(context.Background(), authority)
_ = writeStableError(stream, "provider_work_unavailable", ErrAdmissionRejected, false)
return
}
if (work.ClipboardPolicy.ClientToProviderEnabled || work.ClipboardPolicy.ProviderToClientEnabled) && s.config.ClipboardAuditReporter == nil {
_ = s.config.Admission.Release(context.Background(), authority)
_ = writeStableError(stream, "clipboard_audit_unavailable", ErrAdmissionRejected, true)
return
}
if err := s.reportProviderState(ctx, protocol.ProviderState{Version: "1", SessionID: request.SessionID, State: ProviderStateStarting, CleanupPending: false, Channels: []string{"video", "audio", "input", "feedback"}}); err != nil {
_ = s.config.Admission.Release(context.Background(), authority)
_ = writeStableError(stream, "provider_state_unavailable", err, true)
@@ -256,9 +291,13 @@ func (s *Server) handleConnection(parent context.Context, connection *quic.Conn)
_ = s.config.Admission.Release(context.Background(), authority)
return
}
session := newGatewaySession(s, connection, stream, request, authority, providerSession)
session := newGatewaySession(s, connection, stream, request, authority, providerSession, clipboard)
s.addSession(session)
s.metrics.ActiveSessions.Add(1)
s.metrics.AdmittedSessions.Add(1)
if authority.ReconnectSequence > 0 {
s.metrics.Reconnects.Add(1)
}
defer func() {
s.removeSession(session)
s.metrics.ActiveSessions.Add(-1)
@@ -267,12 +306,13 @@ func (s *Server) handleConnection(parent context.Context, connection *quic.Conn)
}
func (s *Server) reportProviderState(ctx context.Context, state protocol.ProviderState) error {
if s.config.ProviderStateReporter == nil {
return nil
}
if err := state.Validate(); err != nil {
return err
}
s.metrics.observeProviderState(state.State)
if s.config.ProviderStateReporter == nil {
return nil
}
return s.config.ProviderStateReporter.ReportProviderState(ctx, state)
}
@@ -296,12 +336,39 @@ func (s *Server) validateProviderWork(work protocol.ProviderSessionWork, authori
}
if work.SessionID != authority.SessionID || work.GatewayID != authority.GatewayID ||
work.ReconnectSequence != authority.ReconnectSequence || work.ExpiresAt != authority.ExpiresAt ||
work.ProviderProfile != authority.ProviderProfile {
work.ProviderProfile != authority.ProviderProfile || !apolloPolicyMatchesCapabilities(work.StreamPolicy, authority.Capabilities) {
return ErrAdmissionRejected
}
return nil
}
func apolloPolicyMatchesCapabilities(policy protocol.ProviderStreamPolicy, capabilities protocol.CapabilityProfile) bool {
if validateApolloStreamPolicy(policy) != nil || capabilities.Audio != "encoded" {
return false
}
required := apolloPolicyProfile(policy)
return required != "" && slices.Contains(capabilities.ClientDecode, required)
}
func selectApolloPolicyCapabilities(policy protocol.ProviderStreamPolicy, capabilities protocol.CapabilityProfile) (protocol.CapabilityProfile, error) {
if !apolloPolicyMatchesCapabilities(policy, capabilities) {
return protocol.CapabilityProfile{}, ErrNoCapabilityOverlap
}
capabilities.ClientDecode = []string{apolloPolicyProfile(policy)}
return capabilities, nil
}
func apolloPolicyProfile(policy protocol.ProviderStreamPolicy) string {
switch policy.Codec {
case "H264":
return "h264-opus"
case "HEVC":
return "hevc-opus"
default:
return ""
}
}
func (s *Server) addSession(session *gatewaySession) {
s.mu.Lock()
s.sessions[session] = struct{}{}
@@ -315,25 +382,34 @@ func (s *Server) removeSession(session *gatewaySession) {
}
type gatewaySession struct {
server *Server
connection *quic.Conn
control *quic.Stream
request protocol.TunnelAdmissionRequest
authority protocol.SessionAuthority
provider ProviderSession
pacer *Pacer
ctx context.Context
cancel context.CancelFunc
cleanupOnce sync.Once
inputMu sync.Mutex
pressed map[string]struct{}
sequence atomic.Uint32
result chan error
server *Server
connection *quic.Conn
control *quic.Stream
request protocol.TunnelAdmissionRequest
authority protocol.SessionAuthority
provider ProviderSession
clipboard *clipboardGate
ctx context.Context
cancel context.CancelFunc
cleanupOnce sync.Once
inputMu sync.Mutex
controlWriteMu sync.Mutex
outputMu sync.Mutex
terminalMu sync.Mutex
pressed map[string]struct{}
sequence atomic.Uint32
mediaDrops uint64
mediaQuiesced bool
terminalSent atomic.Bool
terminalAwait bool
terminalAck chan struct{}
endReason error
result chan error
}
func newGatewaySession(server *Server, connection *quic.Conn, control *quic.Stream, request protocol.TunnelAdmissionRequest, authority protocol.SessionAuthority, provider ProviderSession) *gatewaySession {
func newGatewaySession(server *Server, connection *quic.Conn, control *quic.Stream, request protocol.TunnelAdmissionRequest, authority protocol.SessionAuthority, provider ProviderSession, clipboard *clipboardGate) *gatewaySession {
ctx, cancel := context.WithCancel(context.Background())
return &gatewaySession{server: server, connection: connection, control: control, request: request, authority: authority, provider: provider, pacer: NewPacer(server.config.PacerKbps), ctx: ctx, cancel: cancel, pressed: make(map[string]struct{}), result: make(chan error, 3)}
return &gatewaySession{server: server, connection: connection, control: control, request: request, authority: authority, provider: provider, clipboard: clipboard, ctx: ctx, cancel: cancel, pressed: make(map[string]struct{}), terminalAck: make(chan struct{}, 1), result: make(chan error, 3)}
}
func (s *gatewaySession) run() {
@@ -350,13 +426,150 @@ func (s *gatewaySession) run() {
go s.controlLoop()
go s.datagramLoop()
go s.mediaLoop()
go s.providerEventLoop()
go s.providerTelemetryLoop()
if s.clipboard != nil && s.clipboard.policy.ProviderToClientEnabled {
go s.clipboardLoop()
}
select {
case <-timer.C:
s.server.metrics.InputRejected.Add(1)
case <-s.ctx.Done():
case <-s.result:
case s.endReason = <-s.result:
}
s.cancel()
if s.terminalSent.Load() {
s.cleanup()
}
}
func (s *gatewaySession) providerEventLoop() {
events := s.provider.Events()
for events != nil {
select {
case <-s.ctx.Done():
return
case event, ok := <-events:
if !ok {
return
}
terminal := event.Kind == ProviderEventTerminated || event.Kind == ProviderEventDisconnected
if terminal {
s.outputMu.Lock()
s.mediaQuiesced = true
s.terminalMu.Lock()
}
payload, err := EncodeProviderEvent(event)
if err == nil {
err = s.sendControl(s.sequence.Add(1), payload)
}
if terminal {
s.terminalAwait = err == nil
s.terminalMu.Unlock()
s.outputMu.Unlock()
}
if err != nil {
s.result <- err
return
}
if terminal {
s.terminalSent.Store(true)
timer := time.NewTimer(terminalAckTimeout)
select {
case <-s.terminalAck:
case <-timer.C:
s.terminalMu.Lock()
s.terminalAwait = false
s.terminalMu.Unlock()
s.result <- context.DeadlineExceeded
return
case <-s.ctx.Done():
timer.Stop()
s.terminalMu.Lock()
s.terminalAwait = false
s.terminalMu.Unlock()
return
}
timer.Stop()
}
switch event.Kind {
case ProviderEventTerminated:
s.result <- ErrProviderTerminated
return
case ProviderEventDisconnected:
s.result <- ErrProviderDisconnected
return
}
}
}
}
func (s *gatewaySession) clipboardLoop() {
ticker := time.NewTicker(500 * time.Millisecond)
defer ticker.Stop()
for {
select {
case <-s.ctx.Done():
return
case <-ticker.C:
text, err := s.provider.ReadClipboard(s.ctx)
if err != nil {
if auditErr := s.reportClipboardAudit("provider_to_client", "rejected", clipboardAuditTextBytes(text), clipboardAuditReason(err)); auditErr != nil {
s.result <- auditErr
return
}
s.result <- err
return
}
value, suppress, err := s.clipboard.fromProvider(text)
if err != nil {
if auditErr := s.reportClipboardAudit("provider_to_client", "rejected", clipboardAuditTextBytes(text), clipboardAuditReason(err)); auditErr != nil {
s.result <- auditErr
return
}
s.result <- err
return
}
if suppress {
if err := s.reportClipboardAudit("provider_to_client", "suppressed", clipboardAuditTextBytes(text), "loop"); err != nil {
s.result <- err
return
}
continue
}
if err := s.sendClipboard(value); err != nil {
s.result <- err
return
}
if err := s.reportClipboardAudit(value.Direction, "forwarded", clipboardAuditTextBytes(value.Text), "forwarded"); err != nil {
s.result <- err
return
}
}
}
}
func (s *gatewaySession) providerTelemetryLoop() {
s.observeProviderTelemetry()
ticker := time.NewTicker(500 * time.Millisecond)
defer ticker.Stop()
for {
select {
case <-s.ctx.Done():
return
case <-ticker.C:
s.observeProviderTelemetry()
}
}
}
func (s *gatewaySession) observeProviderTelemetry() {
telemetry := s.provider.Telemetry()
if telemetry.MediaDrops >= s.mediaDrops {
s.server.metrics.MediaDrops.Add(telemetry.MediaDrops - s.mediaDrops)
s.mediaDrops = telemetry.MediaDrops
}
s.server.metrics.observeProviderTelemetry(telemetry)
}
func (s *gatewaySession) controlLoop() {
@@ -377,13 +590,33 @@ func (s *gatewaySession) controlLoop() {
return
}
switch frame.FlowID {
case "control":
if err := s.handleControl(payload); err != nil {
case controlFlowID:
sequence, sequenceErr := channelSequence(frame.Sequence)
if sequenceErr != nil {
s.result <- sequenceErr
return
}
if err := s.handleControl(payload, sequence); err != nil {
s.result <- err
return
}
case "input":
if err := s.handleInput(payload); err != nil {
case inputFlowID:
sequence, sequenceErr := channelSequence(frame.Sequence)
if sequenceErr != nil {
s.result <- sequenceErr
return
}
if err := s.handleInput(payload, sequence); err != nil {
s.result <- err
return
}
case clipboardFlowID:
value, decodeErr := protocol.DecodeGatewayClipboardText(payload)
if decodeErr != nil {
s.result <- ErrProviderMalformed
return
}
if err := s.handleClipboard(value); err != nil {
s.result <- err
return
}
@@ -408,19 +641,13 @@ func (s *gatewaySession) datagramLoop() {
}
switch frame.Channel {
case ChannelInput:
if err := s.handleInput(frame.Payload); err != nil {
if err := s.handleInput(frame.Payload, frame.Sequence); err != nil {
s.result <- err
return
}
case ChannelText:
if len(frame.Payload) > 4096 {
s.result <- ErrFramePayloadLimit
return
}
if err := s.provider.Feedback(s.ctx, Feedback{Sequence: frame.Sequence, Payload: append([]byte(nil), frame.Payload...)}); err != nil {
s.result <- err
return
}
s.result <- ErrFrameChannel
return
default:
s.result <- ErrFrameChannel
return
@@ -434,21 +661,21 @@ func (s *gatewaySession) mediaLoop() {
select {
case <-s.ctx.Done():
return
case payload, ok := <-video:
case media, ok := <-video:
if !ok {
video = nil
continue
}
if err := s.sendMedia(ChannelVideo, payload); err != nil {
if err := s.forwardMedia(ChannelVideo, media); err != nil {
s.result <- err
return
}
case payload, ok := <-audio:
case media, ok := <-audio:
if !ok {
audio = nil
continue
}
if err := s.sendMedia(ChannelAudio, payload); err != nil {
if err := s.forwardMedia(ChannelAudio, media); err != nil {
s.result <- err
return
}
@@ -457,47 +684,168 @@ func (s *gatewaySession) mediaLoop() {
s.result <- ErrProviderDisconnected
}
func (s *gatewaySession) sendMedia(channel byte, payload []byte) error {
frames, err := FragmentPayload(channel, s.sequence.Add(1), uint64(time.Now().UnixMilli()), payload)
func (s *gatewaySession) forwardMedia(channel byte, media ProviderMedia) error {
s.outputMu.Lock()
defer s.outputMu.Unlock()
state := s.provider.State().State
if s.mediaQuiesced || state == ProviderStateTerminated || state == ProviderStateDisconnected {
s.mediaQuiesced = true
return nil
}
return s.sendMedia(channel, media)
}
func (s *gatewaySession) sendMedia(channel byte, media ProviderMedia) error {
dequeuedAt := time.Now()
if media.EnqueuedAt.IsZero() || media.EnqueuedAt.After(dequeuedAt) {
media.EnqueuedAt = dequeuedAt
}
if media.ReceivedAt.IsZero() || media.ReceivedAt.After(media.EnqueuedAt) {
media.ReceivedAt = media.EnqueuedAt
}
processingStarted := time.Now()
frames, err := FragmentPayload(channel, s.sequence.Add(1), uint64(time.Now().UnixMilli()), media.Payload)
if err != nil {
return err
}
var pacingDelay time.Duration
for _, frame := range frames {
encoded, err := EncodeFrame(frame)
if err != nil {
return err
}
if err := s.pacer.Wait(s.ctx, len(encoded)); err != nil {
pacingStarted := time.Now()
if err := s.server.pacer.wait(s.ctx, s.authority.SessionID, len(encoded)); err != nil {
return err
}
pacingDelay += time.Since(pacingStarted)
if err := s.connection.SendDatagram(encoded); err != nil {
return err
}
s.server.metrics.MediaPackets.Add(1)
s.server.metrics.MediaBytes.Add(uint64(len(encoded)))
}
processingDelay := media.EnqueuedAt.Sub(media.ReceivedAt) + time.Since(processingStarted) - pacingDelay
s.server.metrics.QueueDelayNanos.Add(uint64(dequeuedAt.Sub(media.EnqueuedAt)))
s.server.metrics.ProcessingDelayNanos.Add(uint64(max(processingDelay, 0)))
s.server.metrics.PacingDelayNanos.Add(uint64(pacingDelay))
s.server.metrics.ProcessingSamples.Add(1)
return nil
}
func (s *gatewaySession) handleControl(payload []byte) error {
if len(payload) < 4 {
return ErrProviderMalformed
func (s *gatewaySession) sendControl(sequence uint32, payload []byte) error {
if len(payload) > 1024 {
return ErrFramePayloadLimit
}
switch string(payload[:4]) {
case "TERM":
return errors.New("client requested termination")
case "RECN":
return s.provider.Reconnect(s.ctx)
case "FBRK":
return s.provider.Feedback(s.ctx, Feedback{Payload: append([]byte(nil), payload[4:]...)})
frame := protocol.ChannelFrame{Version: "1", FlowID: controlFlowID, Sequence: int64(sequence), Flags: 0, FragmentIndex: 0, FragmentCount: 1, TimestampMs: time.Now().UnixMilli(), Payload: base64.StdEncoding.EncodeToString(payload)}
encoded, err := protocol.EncodeChannelFrame(frame)
if err != nil {
return err
}
s.controlWriteMu.Lock()
defer s.controlWriteMu.Unlock()
return writeWire(s.control, encoded, defaultControlLimit)
}
func (s *gatewaySession) sendClipboard(value protocol.GatewayClipboardText) error {
payload, err := protocol.EncodeGatewayClipboardText(value)
if err != nil {
return err
}
frame := protocol.ChannelFrame{Version: "1", FlowID: clipboardFlowID, Sequence: int64(s.sequence.Add(1)), Flags: 0, FragmentIndex: 0, FragmentCount: 1, TimestampMs: time.Now().UnixMilli(), Payload: base64.StdEncoding.EncodeToString(payload)}
encoded, err := protocol.EncodeChannelFrame(frame)
if err != nil {
return err
}
s.controlWriteMu.Lock()
defer s.controlWriteMu.Unlock()
return writeWire(s.control, encoded, defaultControlLimit)
}
func (s *gatewaySession) handleControl(payload []byte, sequence uint32) error {
feedback, err := DecodeClientFeedback(payload)
if err != nil {
return err
}
if feedback.Kind == FeedbackTerminalReceipt {
s.terminalMu.Lock()
defer s.terminalMu.Unlock()
if !s.terminalAwait {
return ErrProviderMalformed
}
s.terminalAwait = false
select {
case s.terminalAck <- struct{}{}:
return nil
default:
return ErrProviderMalformed
}
}
feedback.Sequence = sequence
return s.provider.Feedback(s.ctx, feedback)
}
func (s *gatewaySession) handleClipboard(value protocol.GatewayClipboardText) error {
if s.clipboard == nil {
return ErrClipboardDenied
}
suppress, err := s.clipboard.fromClient(value)
if err != nil {
if auditErr := s.reportClipboardAudit(value.Direction, "rejected", clipboardAuditTextBytes(value.Text), clipboardAuditReason(err)); auditErr != nil {
return auditErr
}
return err
}
if suppress {
return s.reportClipboardAudit(value.Direction, "suppressed", clipboardAuditTextBytes(value.Text), "loop")
}
if err := s.provider.WriteClipboard(s.ctx, value.Text); err != nil {
s.clipboard.retractClient(value)
if auditErr := s.reportClipboardAudit(value.Direction, "rejected", clipboardAuditTextBytes(value.Text), "provider"); auditErr != nil {
return auditErr
}
return err
}
return s.reportClipboardAudit(value.Direction, "forwarded", clipboardAuditTextBytes(value.Text), "forwarded")
}
func (s *gatewaySession) reportClipboardAudit(direction, outcome string, textBytes int, reason string) error {
if s.server.config.ClipboardAuditReporter == nil {
return ErrClipboardDenied
}
ctx, cancel := context.WithTimeout(s.ctx, 5*time.Second)
defer cancel()
return s.server.config.ClipboardAuditReporter.ReportClipboardAudit(ctx, protocol.GatewayClipboardAudit{
Version: "1", SessionID: s.authority.SessionID, Direction: direction, Outcome: outcome, TextBytes: int64(textBytes), Reason: reason,
})
}
func clipboardAuditTextBytes(text string) int {
if len(text) > 65536 {
return 65536
}
return len(text)
}
func clipboardAuditReason(err error) string {
switch {
case errors.Is(err, ErrClipboardRate):
return "rate"
case errors.Is(err, ErrProviderMalformed):
return "malformed"
case errors.Is(err, ErrClipboardDenied):
return "policy"
default:
return ErrProviderMalformed
return "provider"
}
}
func (s *gatewaySession) handleInput(payload []byte) error {
func (s *gatewaySession) handleInput(payload []byte, sequence uint32) error {
event, err := DecodeInputEvent(payload)
if err != nil {
return err
}
event.Sequence = sequence
if err := s.provider.Input(s.ctx, event); err != nil {
s.server.metrics.InputRejected.Add(1)
return err
@@ -513,9 +861,17 @@ func (s *gatewaySession) handleInput(payload []byte) error {
return nil
}
func channelSequence(sequence int64) (uint32, error) {
if sequence < 0 || sequence > int64(^uint32(0)) {
return 0, ErrProviderMalformed
}
return uint32(sequence), nil
}
func (s *gatewaySession) cleanup() {
s.cleanupOnce.Do(func() {
s.cancel()
s.server.pacer.remove(s.authority.SessionID)
cleanupCtx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
releaseInputsErr := s.provider.ReleaseAll(cleanupCtx)
@@ -538,6 +894,10 @@ func (s *gatewaySession) cleanup() {
_ = s.connection.CloseWithError(applicationError, "session closed")
return
}
if errors.Is(s.endReason, ErrProviderDisconnected) {
state.State = ProviderStateDisconnected
state.CleanupPending = false
}
if err := s.server.config.Admission.Release(cleanupCtx, s.authority); err != nil {
s.server.metrics.ProviderErrors.Add(1)
}
@@ -613,9 +973,12 @@ func readWire(reader io.Reader, max int) ([]byte, error) {
}
type Client struct {
connection *quic.Conn
control *quic.Stream
Authority protocol.SessionAuthority
connection *quic.Conn
control *quic.Stream
controlReadMu sync.Mutex
controlWriteMu sync.Mutex
pendingControl map[string][][]byte
Authority protocol.SessionAuthority
}
func Dial(ctx context.Context, address string, tlsConfig *tls.Config, request protocol.TunnelAdmissionRequest) (*Client, error) {
@@ -658,7 +1021,7 @@ func Dial(ctx context.Context, address string, tlsConfig *tls.Config, request pr
}
return nil, authorityErr
}
return &Client{connection: connection, control: stream, Authority: authority}, nil
return &Client{connection: connection, control: stream, pendingControl: make(map[string][][]byte), Authority: authority}, nil
}
func (c *Client) SendInput(event InputEvent) error {
@@ -666,20 +1029,51 @@ func (c *Client) SendInput(event InputEvent) error {
if err != nil {
return err
}
frame := protocol.ChannelFrame{Version: "1", FlowID: "input", Sequence: int64(event.Sequence), Flags: 0, FragmentIndex: 0, FragmentCount: 1, TimestampMs: time.Now().UnixMilli(), Payload: base64.StdEncoding.EncodeToString(payload)}
frame := protocol.ChannelFrame{Version: "1", FlowID: inputFlowID, Sequence: int64(event.Sequence), Flags: 0, FragmentIndex: 0, FragmentCount: 1, TimestampMs: time.Now().UnixMilli(), Payload: base64.StdEncoding.EncodeToString(payload)}
encoded, err := protocol.EncodeChannelFrame(frame)
if err != nil {
return err
}
return writeWire(c.control, encoded, defaultControlLimit)
return c.writeControl(encoded)
}
func (c *Client) SendControl(payload []byte) error {
frame := protocol.ChannelFrame{Version: "1", FlowID: "control", Sequence: 0, Flags: 0, FragmentIndex: 0, FragmentCount: 1, TimestampMs: time.Now().UnixMilli(), Payload: base64.StdEncoding.EncodeToString(payload)}
return c.sendControl(0, payload)
}
func (c *Client) SendFeedback(feedback Feedback) error {
payload, err := EncodeClientFeedback(feedback)
if err != nil {
return err
}
return c.sendControl(feedback.Sequence, payload)
}
func (c *Client) SendClipboard(value protocol.GatewayClipboardText) error {
payload, err := protocol.EncodeGatewayClipboardText(value)
if err != nil {
return err
}
frame := protocol.ChannelFrame{Version: "1", FlowID: clipboardFlowID, Sequence: 0, Flags: 0, FragmentIndex: 0, FragmentCount: 1, TimestampMs: time.Now().UnixMilli(), Payload: base64.StdEncoding.EncodeToString(payload)}
encoded, err := protocol.EncodeChannelFrame(frame)
if err != nil {
return err
}
return c.writeControl(encoded)
}
func (c *Client) sendControl(sequence uint32, payload []byte) error {
frame := protocol.ChannelFrame{Version: "1", FlowID: controlFlowID, Sequence: int64(sequence), Flags: 0, FragmentIndex: 0, FragmentCount: 1, TimestampMs: time.Now().UnixMilli(), Payload: base64.StdEncoding.EncodeToString(payload)}
encoded, err := protocol.EncodeChannelFrame(frame)
if err != nil {
return err
}
return c.writeControl(encoded)
}
func (c *Client) writeControl(encoded []byte) error {
c.controlWriteMu.Lock()
defer c.controlWriteMu.Unlock()
return writeWire(c.control, encoded, defaultControlLimit)
}
@@ -691,6 +1085,72 @@ func (c *Client) ReceiveFrame(ctx context.Context) (Frame, error) {
return DecodeFrame(data)
}
func (c *Client) ReceiveProviderEvent(ctx context.Context) (ProviderEvent, error) {
payload, err := c.receiveControlPayload(ctx, controlFlowID)
if err != nil {
return ProviderEvent{}, err
}
if len(payload) > 1024 {
return ProviderEvent{}, ErrProviderMalformed
}
event, err := DecodeProviderEvent(payload)
if err == nil && (event.Kind == ProviderEventTerminated || event.Kind == ProviderEventDisconnected) {
err = c.SendFeedback(Feedback{Kind: FeedbackTerminalReceipt})
}
return event, err
}
func (c *Client) ReceiveClipboard(ctx context.Context) (protocol.GatewayClipboardText, error) {
payload, err := c.receiveControlPayload(ctx, clipboardFlowID)
if err != nil {
return protocol.GatewayClipboardText{}, err
}
return protocol.DecodeGatewayClipboardText(payload)
}
func (c *Client) receiveControlPayload(ctx context.Context, flowID string) ([]byte, error) {
if c == nil || c.control == nil || (flowID != controlFlowID && flowID != clipboardFlowID) {
return nil, ErrProviderMalformed
}
c.controlReadMu.Lock()
defer c.controlReadMu.Unlock()
if err := ctx.Err(); err != nil {
return nil, err
}
if queued := c.pendingControl[flowID]; len(queued) > 0 {
payload := queued[0]
c.pendingControl[flowID] = queued[1:]
return payload, nil
}
if deadline, ok := ctx.Deadline(); ok {
if err := c.control.SetReadDeadline(deadline); err != nil {
return nil, err
}
defer c.control.SetReadDeadline(time.Time{})
}
for {
data, err := readWire(c.control, defaultControlLimit)
if err != nil {
return nil, err
}
frame, err := protocol.DecodeChannelFrame(data)
if err != nil || (frame.FlowID != controlFlowID && frame.FlowID != clipboardFlowID) {
return nil, ErrProviderMalformed
}
payload, err := base64.StdEncoding.DecodeString(frame.Payload)
if err != nil || len(payload) > maxFrameSize {
return nil, ErrProviderMalformed
}
if frame.FlowID == flowID {
return payload, nil
}
if len(c.pendingControl[frame.FlowID]) >= clientControlBacklog {
return nil, ErrFramePayloadLimit
}
c.pendingControl[frame.FlowID] = append(c.pendingControl[frame.FlowID], payload)
}
}
func (c *Client) Close() error {
return c.connection.CloseWithError(applicationError, "client closed")
}
+1 -1
View File
@@ -3,7 +3,7 @@ module git.sechmachine.io.vn/sechmachine/VerseVDI-Data-Plane
go 1.26.5
require (
git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol v1.0.0-phase3c-gateway-rc.5
git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol v1.0.0-phase3c-gateway-rc.8
github.com/quic-go/quic-go v0.61.0
)
+6
View File
@@ -4,6 +4,12 @@ git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol v1.0.0-phase3c-gateway-rc.4
git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol v1.0.0-phase3c-gateway-rc.4/go.mod h1:7PhFIDhjtr20btWoEb2GqB+7dBpzJt43olrnHVutWoc=
git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol v1.0.0-phase3c-gateway-rc.5 h1:F+Ig0OVpHcKr/G+uYy7Vm8BVxfei/Rk1yqYHtX9BcL0=
git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol v1.0.0-phase3c-gateway-rc.5/go.mod h1:7PhFIDhjtr20btWoEb2GqB+7dBpzJt43olrnHVutWoc=
git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol v1.0.0-phase3c-gateway-rc.6 h1:xCL6VYjn+alrcxIU85wmYOvLMyvIFzUWbTNxMP46ij4=
git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol v1.0.0-phase3c-gateway-rc.6/go.mod h1:7PhFIDhjtr20btWoEb2GqB+7dBpzJt43olrnHVutWoc=
git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol v1.0.0-phase3c-gateway-rc.7 h1:vQWELUD8bTjEI9rsJYinH2PegOlGwOkyndcD+jgLMBQ=
git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol v1.0.0-phase3c-gateway-rc.7/go.mod h1:7PhFIDhjtr20btWoEb2GqB+7dBpzJt43olrnHVutWoc=
git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol v1.0.0-phase3c-gateway-rc.8 h1:DqD2I3bjiVt+mr741o7hw4wDUp2vYZx32CNDSkqADwY=
git.sechmachine.io.vn/sechmachine/VerseVDI-Protocol v1.0.0-phase3c-gateway-rc.8/go.mod h1:7PhFIDhjtr20btWoEb2GqB+7dBpzJt43olrnHVutWoc=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-07-29
@@ -0,0 +1,54 @@
## Context
P3C-029 through P3C-033 require raw, reproducible processing, payload-rate,
fairness, cap, and impairment evidence. Existing focused tests cover the
framer, bounded queues, native Apollo fake, and fair pacer, but do not emit the
normative ten-minute or six-profile artifacts.
## Goals / Non-Goals
**Goals:**
- Run the three 20/50/80 Mbps encoded-payload profiles for ten wall-clock
minutes each after a recorded warm-up.
- Measure the existing gateway framing path with a monotonic clock and retain
compressed raw latency samples plus full summary statistics.
- Run the exact six Section 7.2 configurations once using a deterministic,
bounded virtual packet discipline and retain configured and observed values.
- Exercise one real mTLS/QUIC fake-provider traversal for every media profile
and reuse the real fair-pacer implementation for fairness and cap evidence.
**Non-Goals:**
- Live Apollo/macOS/firewall qualification, real encoder fidelity, codec
processing, host network mutation, or multi-host scale.
- A production impairment framework, new gateway API, dependency, cgo, or
sidecar.
## Decisions
- Implement the harness as an opt-in `go test` in package `gateway`. This
keeps qualification access to the actual unexported fair pacer without
adding a production API. Normal suites skip the long run unless an explicit
absolute evidence directory is supplied.
- Use wall-clock duration and target-rate pacing for performance profiles.
Measure only receive-to-framed-payload processing; pacing wait and raw-file
writes stay outside the measured interval and are reported separately.
- Stream every raw sample into gzip-compressed CSV while retaining one bounded
duration slice per profile for exact percentiles.
- Use a fixed-seed virtual FIFO for impairment. It records no host claim and
identifies its queue discipline and deterministic topology explicitly.
- Treat any payload mutation, p95 above 5 ms, catalog mismatch, fairness error
above 10%, cap excess above 5%, or step convergence beyond ten seconds as a
hard command failure.
## Risks / Trade-offs
- [Local virtual impairment cannot prove deployed route behavior] → label every
artifact deterministic and retain live Apollo/macOS/firewall as
deferred-owner-e2e.
- [Raw samples can be large] → stream gzip output and bound in-memory samples
to the exact profile packet budget.
- [Host load can invalidate latency] → record OS, architecture, Go version,
timing overhead, actual duration, packet count, and observed bitrate; fail
rather than substitute configured capacity for measured egress.
@@ -0,0 +1,35 @@
## Why
The Phase 3C gateway candidate has deterministic transport and scheduler tests
but no executable artifact generator for the normative ten-minute media
measurements and six bounded impairment profiles. Without that evidence,
P3C-029 through P3C-033 cannot be frozen truthfully.
## What Changes
- Add one stdlib-only qualification command for the three fixed encoded-media
profiles and the exact six Section 7.2 impairment profiles.
- Emit bounded machine-readable configuration, raw observations, summaries,
environment, topology, direction, queue discipline, and tool version.
- Fail the command when payload integrity, the 5 ms processing p95, impairment
bounds, fairness, capacity-step convergence, or aggregate cap gates fail.
- Keep live Apollo, macOS, physical firewall, real encoder fidelity, and real
multi-host scale explicitly deferred-owner-e2e.
## Capabilities
### New Capabilities
- `gateway-qualification`: Deterministic P3C-029 through P3C-033 media,
processing, fairness, cap, and impairment evidence generation.
### Modified Capabilities
None.
## Impact
The Data Plane gains a qualification-only Go command, focused tests, and
documented evidence output. It adds no dependency, production transport
abstraction, provider route, codec operation, cgo, sidecar, or Connection
Server code.
@@ -0,0 +1,63 @@
## ADDED Requirements
### Requirement: Fixed media processing qualification
The qualification harness SHALL run 1080p60 H.264 at 20 Mbps, 1440p120 HEVC
at 50 Mbps, and 4K60 HEVC at 80 Mbps for ten wall-clock minutes each after a
recorded warm-up. It SHALL preserve encoded payload bytes, record every
monotonic processing sample, report count, min, median, p90, p95, p99, max,
mean, standard deviation, timing overhead, and observed bitrate, and fail when
any p95 exceeds 5 ms.
#### Scenario: Healthy fixed profile
- **WHEN** a frozen candidate runs one fixed profile for the normative duration
- **THEN** the harness emits compressed raw samples and a summary tied to the
exact source commit, Protocol version, environment, and payload hash.
#### Scenario: Processing gate failure
- **WHEN** payload integrity fails or measured p95 exceeds 5 ms
- **THEN** the qualification command exits unsuccessfully without recording a
passing candidate.
### Requirement: Bounded impairment qualification
The harness SHALL run exactly the baseline, latency, jitter, loss, reorder,
and constrained Section 7.2 profiles once. Baseline SHALL cover all three
media profiles and the other profiles SHALL cover 1080p60. Each artifact SHALL
record tool version, exact command/configuration, direction, queue discipline,
topology, fixed seed, and observed RTT, jitter, loss, reorder, throughput,
drops, and capacity-step statistics.
#### Scenario: Complete six-profile run
- **WHEN** the frozen candidate runs impairment qualification
- **THEN** one result exists for each named profile, with no Cartesian
expansion and with observed rather than configured statistics.
#### Scenario: Unsupported or unbounded configuration
- **WHEN** a profile name, packet count, queue bound, loss, reorder, or
bandwidth step falls outside the fixed catalog
- **THEN** the harness rejects it before allocating or running the simulation.
### Requirement: Fairness and cap qualification
The harness SHALL exercise the production fair pacer with eight equal-tier
synthetic sessions for the required 60-second virtual interval, report every
share error and Jain's fairness index, and fail above 10% share error. It SHALL
apply 25% and 50% capacity steps, fail convergence beyond ten virtual seconds,
and fail aggregate egress above 105% of the cap over any rolling five-second
window.
#### Scenario: Equal-tier and capacity-step evidence
- **WHEN** the frozen candidate runs scheduler qualification
- **THEN** the artifact contains per-flow bytes, share errors, Jain's index,
step convergence, and rolling cap observations derived from the production
pacer.
### Requirement: Honest qualification boundary
Qualification artifacts SHALL contain no provider endpoint, credential,
clipboard text, input payload, secret, raw media content, or claim of live
Apollo/macOS/firewall interoperability. The harness SHALL add no codec
operation, production dependency, cgo, sidecar, or direct provider route.
#### Scenario: Deterministic evidence publication
- **WHEN** qualification completes
- **THEN** the manifest labels fake-provider, virtual impairment, and local
processing evidence separately and leaves live interoperability
deferred-owner-e2e.
@@ -0,0 +1,24 @@
## 1. Contract and focused regressions
- [x] 1.1 Add fixed catalog tests for the three media profiles, ten-minute
duration, exact six impairment profiles, and bounded output configuration.
- [x] 1.2 Add summary, payload-integrity, fairness, cap, and failure-threshold
tests before implementing the harness.
## 2. Qualification harness
- [x] 2.1 Implement opt-in real-duration processing measurement with compressed
raw samples, full statistics, timing overhead, and environment metadata.
- [x] 2.2 Implement deterministic bounded impairment observations and reuse the
production fair pacer for fairness and capacity-step evidence.
- [x] 2.3 Add one mTLS/QUIC fake-provider traversal per fixed encoded profile
and prove the artifact boundary contains no provider route or secret.
## 3. Freeze and evidence
- [x] 3.1 Run focused red/green checks, strict OpenSpec validation, `make
verify`, race/fuzz/resource checks, and freeze the harness commit.
- [x] 3.2 Run the opt-in ten-minute and six-profile command exactly once
against the frozen candidate and archive raw artifacts and hashes.
- [x] 3.3 Sync the canonical specification, archive the completed change, and
revalidate strictly without claiming live Apollo/macOS/firewall evidence.
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-07-29
@@ -0,0 +1,50 @@
## Context
The implementation already contains a source-shaped Apollo fake, native recovery, bounded gateway queues, fair pacing, Verse framing/QUIC, independent client support, lifecycle reporters, and low-cardinality telemetry. Audit defects arise where those existing pieces are bypassed or not connected.
## Goals / Non-Goals
**Goals:**
- Reuse the existing production path for policy, lifecycle, recovery, telemetry, and qualification.
- Delete duplicate qualification simulation.
- Preserve all trust, cleanup, and resource bounds.
**Non-Goals:**
- Add codecs, provider transports, dependencies, or a generic lifecycle/telemetry framework.
- Claim live Apollo/macOS/firewall interoperability.
- Run the normative qualification before immutable consumer resolution.
## Decisions
- Format ANNOUNCE from `ProviderStreamPolicy` using the pinned Moonlight common-c bitrate and codec attributes. H.264 and HEVC with audio enabled are supported; AV1 and audio disabled fail before management/network readiness.
- Advertise ordered registered `hevc-opus` and `h264-opus` profiles and delegate policy-compatible selection to generated Protocol intersection behavior. There is no Data Plane capability grammar.
- Before `/applist` or `/launch`, validate the selected policy against source-backed Apollo `/serverinfo` codec flags and HEVC luma bounds plus reviewed source limits for dimensions, frame rate, bitrate, and audio where Apollo exposes no dynamic field. Reject rather than cap or downgrade.
- Quiesce provider media sockets and the bounded forwarding path before emitting an existing terminal or disconnected event. Reuse current cleanup/release/reporting machinery and its cleanup-pending result; final control delivery has no fixed drain delay.
- On a full audio FEC map, evict the oldest block according to existing block ordering and increment existing drop telemetry.
- Carry provider receipt and queue-enqueue timestamps through the existing bounded media value. Queue residence, active processing, and scheduler pacing are sampled separately, once per complete provider media unit.
- Sample existing process counters at heartbeat time; calculate rate from byte and monotonic-time deltas while leaving configured capacity in registration.
- Build qualification on source-shaped pinned-mTLS Apollo management, encrypted RTSP, ENet, and provider UDP plus the public QUIC client path. Per-traversal stage deltas replace the standalone codec/parser and arithmetic impairment simulator; short smoke gates freeze the wiring, while normative durations remain deferred.
- Preserve the production fair-pacer schedule across short host-timer overshoots so
measured allocation can catch up within the already bounded provider queue
instead of accumulating timer granularity as lost capacity.
## Risks / Trade-offs
- [Apollo cannot represent disabled audio truthfully] → Reject it rather than silently streaming stereo.
- [Provider event races with queued or new media] → Quiesce ingestion and serialize forwarding with terminal event delivery before cleanup.
- [Counter reset or zero elapsed time] → Emit zero measured rate and establish a new baseline.
- [Corrected qualification is more expensive] → Run only short smoke tests until the immutable candidate is frozen.
- [Apollo exposes incomplete dynamic capability detail] → Use only source-backed fields and explicit reviewed bounds; never infer support by silent capping.
- [Pacer catch-up can emit a short burst after timer overshoot] → Clamp schedule
debt to five milliseconds in addition to the existing 16-packet provider
queue.
## Migration Plan
Land focused red/green repairs locally, verify through the temporary Protocol workspace, preserve old artifacts as superseded, and stop at the publication boundary. After a separately authorized immutable Protocol release is pinned, freeze inputs and run the corrected normative qualification once.
## Open Questions
None.
@@ -0,0 +1,28 @@
## Why
Fresh audit evidence shows the gateway ignores the immutable launch policy, leaves tunnels alive after provider termination/disconnect, can permanently stall audio after sustained loss, reports configured capacity as measured egress, and qualifies a standalone simulator instead of the production path.
## What Changes
- Negotiate registered decode profiles through the shared Protocol intersection, apply the effective policy to Apollo ANNOUNCE, and reject client or provider/source mismatch before launch (P3C-009, P3C-016, P3C-038).
- Quiesce media immediately on provider termination/disconnect, deliver the final typed event reliably, and preserve bounded tunnel and durable lifecycle transitions including cleanup-pending (P3C-018021, P3C-027).
- Evict bounded stale audio FEC blocks so newer recoverable media continues (P3C-001, P3C-026).
- Derive heartbeat egress and semantically separated queue, processing, and pacing observations from the production path (P3C-022, P3C-028).
- Replace standalone processing/impairment simulation with a driver around the source-shaped provider, production queues/pacer/framing, QUIC, and an independent client (P3C-029033, VER-008, VER-010).
## Capabilities
### New Capabilities
- `apollo-stream-policy`: Native Apollo launch consumes the authenticated effective stream policy without downgrade.
- `provider-session-lifecycle`: Provider terminal events close forwarding and report the correct durable lifecycle outcome.
- `audio-fec-resilience`: Bounded Apollo audio recovery continues after permanently incomplete blocks.
- `gateway-heartbeat-telemetry`: Authenticated heartbeat telemetry reports observed traffic and provider-path measurements.
### Modified Capabilities
- `gateway-qualification`: Normative evidence must traverse the production gateway path and retain raw resource, impairment, fairness, cap, and convergence observations.
## Impact
The native Apollo adapter, transport lifecycle, audio FEC state, heartbeat sampling, qualification driver, focused fixtures, and canonical qualification spec change. No dependency, cgo, sidecar, codec operation, direct provider route, or live interoperability claim is added.
@@ -0,0 +1,19 @@
## ADDED Requirements
### Requirement: Apollo launch consumes the effective policy
The native Apollo backend SHALL derive ANNOUNCE resolution, frame rate, supported codec, selected bitrate, and audio profile from authenticated `ProviderSessionWork`, and MUST NOT substitute local defaults.
#### Scenario: Supported HEVC policy reaches Apollo
- **WHEN** provider work selects HEVC at 2560×1440, 120 FPS, 40000 Kbps, with audio enabled
- **THEN** the encrypted ANNOUNCE carries those settings and the source-backed HEVC and bitrate attributes
### Requirement: Provider policy cannot downgrade
The gateway MUST use the generated Protocol intersection to select only a registered profile compatible with the immutable policy. It MUST reject invalid, unsupported, no-overlap, downgrade, audio-disabled, AV1, or provider/source-mismatched Apollo policy before `/applist`, `/launch`, or provider readiness because the current native path cannot truthfully honor those combinations.
#### Scenario: Unsupported policy fails closed
- **WHEN** authenticated provider work selects audio disabled, AV1, a codec outside the registered peer intersection, or a resolution, frame rate, bitrate, audio, or codec combination outside source-backed Apollo support
- **THEN** setup fails before application discovery or launch without falling back to H.264, stereo, a cap, or another local default
#### Scenario: Independent peers negotiate one registered profile
- **WHEN** a production gateway and independent client advertise overlapping registered H.264 or HEVC profiles
- **THEN** admission selects the first policy-compatible common profile using shared Protocol behavior
@@ -0,0 +1,8 @@
## ADDED Requirements
### Requirement: Bounded audio FEC state advances after loss
The Apollo audio recovery window SHALL remain bounded and SHALL evict the oldest incomplete block when accepting a newer block would otherwise be rejected.
#### Scenario: Newer complete block follows sustained loss
- **WHEN** more than the bounded number of permanently incomplete audio blocks arrive before a complete newer block
- **THEN** the oldest stale state is dropped, drop telemetry advances, and the newer encoded payload is relayed unchanged
@@ -0,0 +1,22 @@
## ADDED Requirements
### Requirement: Heartbeat egress is observed
Authenticated gateway heartbeat telemetry SHALL calculate egress from monotonic transmitted-byte deltas over monotonic elapsed time and MUST NOT report configured capacity as measured traffic.
#### Scenario: Controlled byte delta is sampled
- **WHEN** transmitted bytes increase by a known amount during a known interval
- **THEN** heartbeat egress equals the measured rate while configured capacity remains a separate registration value
### Requirement: Required telemetry remains bounded and low cardinality
The established authenticated path SHALL expose observed bytes, packets, drops, RTT, loss, jitter, queue delay, processing delay, pacing, reconnect, and provider state without session, route, credential, or payload labels.
#### Scenario: Telemetry snapshot is published
- **WHEN** the gateway emits a heartbeat after forwarding traffic
- **THEN** it carries the bounded process-level observations and no high-cardinality or secret-bearing value
### Requirement: Production delays have disjoint sample semantics
Queue delay SHALL measure provider-queue residence, processing delay SHALL measure provider recovery plus framing and QUIC handoff work excluding queue and pacing, and pacing delay SHALL measure scheduler waiting only. The gateway SHALL advance processing samples once per complete provider media unit even when it emits multiple Verse frames.
#### Scenario: Known production waits
- **WHEN** one provider media unit has controlled enqueue, processing, and pacing intervals and fragments across multiple frames
- **THEN** each cumulative total reports only its intended interval and exactly one processing sample is retained through authenticated Server persistence
@@ -0,0 +1,37 @@
## MODIFIED Requirements
### Requirement: Fixed media processing qualification
The qualification harness SHALL drive pinned-mTLS Apollo management, encrypted RTSP, ENet, and provider UDP through native source validation, `readUDPMedia`, recovery/FEC, bounded production queues, the production fair pacer, Verse framing/QUIC, and a public or independent client decoder for 1080p60 H.264 at 20 Mbps, 1440p120 HEVC at 50 Mbps, and 4K60 HEVC at 80 Mbps. After a recorded warm-up, the frozen candidate SHALL run each profile for ten wall-clock minutes, preserve encoded payload bytes, retain every monotonic processing sample plus bounded CPU, memory, goroutine, allocation, and provider-queue observations, and report count, min, median, p90, p95, p99, max, mean, standard deviation, timing overhead, and observed bitrate. Processing begins at complete provider-unit receipt and ends at QUIC handoff, excluding client transit. Any bypass, payload mutation, wall-duration violation, bitrate outside both lower and upper bounds, or p95 above 5 ms SHALL fail.
#### Scenario: Healthy fixed profile
- **WHEN** a frozen candidate runs one fixed profile for the normative duration
- **THEN** the harness emits compressed raw path and resource samples plus a summary tied to the exact command, topology, source commit, immutable Protocol version, environment, and payload hash
#### Scenario: Processing gate failure
- **WHEN** any production path stage lacks a per-traversal observation, payload integrity fails, duration or bitrate bounds fail, or measured p95 exceeds 5 ms
- **THEN** the qualification command exits unsuccessfully without recording a passing candidate
### Requirement: Bounded impairment qualification
The harness SHALL run exactly the baseline, latency, jitter, loss, reorder, and constrained Section 7.2 profiles once by applying impairment at the source-shaped provider UDP boundary while traffic traverses the production gateway path. Baseline SHALL cover all three media profiles and the other profiles SHALL cover 1080p60. Each artifact SHALL retain raw impairment and queue observations and record tool version, exact command/configuration, environment, candidate commit, immutable Protocol version, direction, queue discipline, topology, fixed seed, and observed RTT, jitter, loss, reorder, throughput, drops, and capacity-step statistics.
#### Scenario: Complete six-profile run
- **WHEN** the frozen candidate runs impairment qualification
- **THEN** one result exists for each named profile, with no Cartesian expansion and with raw observed rather than configured statistics from the real traversal
#### Scenario: Unsupported or unbounded configuration
- **WHEN** a profile name, packet count, queue bound, loss, reorder, or bandwidth step falls outside the fixed catalog
- **THEN** the harness rejects it before allocating or running traffic
### Requirement: Fairness and cap qualification
The harness SHALL exercise the production fair pacer with eight equal-tier synthetic sessions for the required 60-second virtual interval, retain every per-flow and aggregate observation, report every share error and Jain's fairness index, and fail above 10% share error. It SHALL apply 25% and 50% capacity steps, measure convergence of observed allocation rather than first delivery, fail convergence beyond ten virtual seconds, and fail aggregate egress above 105% of the cap over any rolling five-second window.
#### Scenario: Equal-tier and capacity-step evidence
- **WHEN** the frozen candidate runs scheduler qualification
- **THEN** the artifact contains raw per-flow bytes, aggregate-cap series, share errors, Jain's index, measured allocation convergence, and rolling cap observations derived from the production pacer
### Requirement: Honest qualification boundary
Qualification artifacts SHALL contain no provider endpoint, credential, clipboard text, input payload, secret, or raw media content and SHALL make no claim of live Apollo/macOS/firewall interoperability. The harness SHALL add no codec operation, production dependency, cgo, sidecar, direct provider route, or duplicate processing/impairment simulator. Deterministic smoke evidence SHALL remain distinct from the single normative run on the frozen immutable consumer candidate.
#### Scenario: Deterministic evidence publication
- **WHEN** qualification completes
- **THEN** the manifest labels fake-provider, path impairment, and local processing evidence separately and leaves live interoperability deferred-owner-e2e
@@ -0,0 +1,19 @@
## ADDED Requirements
### Requirement: Provider terminal events end forwarding
Encrypted provider termination and unexpected provider disconnect SHALL quiesce provider ingestion and queued/new media forwarding before the existing reliable typed terminal event is delivered, close the Verse tunnel within a bounded interval, release the session reservation, and report the appropriate durable provider/session state. A fixed drain delay MUST NOT stand in for reliable control delivery.
#### Scenario: Host termination closes the tunnel
- **WHEN** the native provider emits an authenticated termination event
- **THEN** queued and newly injected media cannot cross the Verse transport after observation, and the client tunnel, reservation, and durable lifecycle transition complete
#### Scenario: Unexpected provider disconnect is reconnectable
- **WHEN** required provider transport disconnects without acknowledged termination
- **THEN** forwarding stops and the Server receives the existing reconnectable lifecycle state rather than a termination claim
### Requirement: Cleanup failure remains durable
Gateway cleanup MUST preserve `cleanup_pending` when provider input release, transport cleanup, authorized cancellation, or durable reporting fails.
#### Scenario: Terminal cleanup fails
- **WHEN** a provider terminal event is handled but required cleanup cannot complete
- **THEN** the session is not reported released or reusable and durable state remains cleanup pending
@@ -0,0 +1,24 @@
## 1. Native Policy and Media
- [x] 1.1 Drive supported immutable policy values into encrypted Apollo ANNOUNCE
- [x] 1.2 Negotiate registered profiles through shared Protocol intersection behavior
- [x] 1.3 Reject unsupported client or provider/source policy before `/applist`, `/launch`, or readiness
- [x] 1.4 Evict oldest incomplete audio FEC state and pass sustained-loss relay regression
## 2. Lifecycle and Telemetry
- [x] 2.1 Quiesce queued and new media before terminal event delivery and report durable state
- [x] 2.2 Preserve cleanup-pending on terminal cleanup failure
- [x] 2.3 Report measured heartbeat egress and separately sampled queue, processing, and pacing telemetry
## 3. Qualification Path
- [x] 3.1 Add a red-to-green native UDP-to-public-client production-path smoke gate with per-traversal stage evidence
- [x] 3.2 Remove direct native internals, private QUIC/parser, and arithmetic impairment shortcuts
- [x] 3.3 Retain raw processing, impairment, fairness, cap, convergence, queue, and resource observations
- [x] 3.4 Pass short fixed-profile, impairment, fairness, race, parser fuzz, and resource smoke checks
## 4. Immutable Freeze
- [x] 4.1 Pin and verify a separately published never-reused Protocol version from an empty cache
- [x] 4.2 Freeze all normative inputs and run the corrected Section 7 qualification exactly once
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-07-30
@@ -0,0 +1,31 @@
## Context
The native production path and fair pacer already exist. The defect was evidence collection: provider sends waited synchronously for client delivery, RTT was arithmetic, CPU was available capacity, and the repository had no canonical Linux build target.
## Goals / Non-Goals
**Goals:**
- Observe concurrent source-shaped traversal without a duplicate transport.
- Distinguish actual one-way delivery, acknowledged RTT, queue/processing/pacing, and consumed process CPU.
- Produce reproducible inspectable Linux artifacts for the deployment architectures.
**Non-Goals:**
- A new transport, scheduler, dependency, codec operation, scanner, signer, or container framework.
- Live provider/client/firewall evidence or Connection Server image remediation.
## Decisions
- Keep the existing provider fixture and production path; overlap its UDP sender with the public QUIC receiver.
- Apply fixed-seed impairment before provider UDP injection and derive delivery statistics from decoded payload sequence and timestamps.
- Use native ENet acknowledgement timing for RTT instead of doubling one-way completion.
- Use OS process user plus system CPU for the isolated qualification command; retain memory, goroutine, and allocation series separately.
- Build both Linux architectures with `CGO_ENABLED=0`, `GOWORK=off`, `-trimpath`, no VCS stamping, and an empty build ID, then inspect ELF and embedded Go settings.
- Delete the dead exported pacer rather than consolidate it with the sole production `fairPacer`.
## Risks / Trade-offs
- [Short RTT smoke runs contain ENet smoothing history] → Classify the metric as acknowledged transport RTT and enforce profile tolerances on the full frozen run.
- [Process CPU includes the bounded fixture/client harness] → Run only the named qualification test in an isolated process and label the scope exactly; never call it host-wide or binary-only CPU.
- [No qualifying vulnerability scanner is installed] → Record unscanned status and deterministic dependency/artifact evidence without zero-finding claims.
@@ -0,0 +1,25 @@
## Why
The prior Phase 3C-G artifacts measured serialized simulator timing, runtime CPU capacity, and a Darwin test binary rather than the actual deployable gateway candidate. VER-009, VER-010, VER-015, and OPS-009 require observed production traversal, bounded resource evidence, and exact artifact provenance before engineering exit.
## What Changes
- Drive impairment concurrently through the source-shaped provider UDP, native recovery, bounded queue, production pacer, QUIC, and public decoder.
- Measure RTT from actual Apollo ENet acknowledgements, one-way latency and jitter from delivery observations, and CPU from isolated process user/system consumption.
- Remove the unused legacy pacer so qualification and production share one scheduler.
- Build and inspect reproducible pure-Go Linux amd64 and arm64 gateway artifacts.
- Report dependency, scanner, architecture, and security evidence only when actually generated.
## Capabilities
### New Capabilities
- `gateway-deployment-artifact`: Reproducible, inspectable Linux gateway build and evidence requirements.
### Modified Capabilities
- `gateway-qualification`: Replace serialized/synthetic timing and CPU-capacity evidence with actual bounded traversal and process-consumption observations.
## Impact
This affects only the GPLv3 Data Plane qualification harness, resource evidence, production scheduler inventory, and gateway packaging target. It adds no dependency, cgo, sidecar, codec operation, direct provider route, Server dependency, or proprietary source. Live Apollo, macOS-client, physical-firewall, promotion scanning/signing, and Connection Server Phase 3C-C images remain outside this deterministic gate.
@@ -0,0 +1,15 @@
## ADDED Requirements
### Requirement: Reproducible pure-Go Linux gateway artifacts
The candidate SHALL build the gateway with the normal immutable Protocol module boundary for Linux amd64 and arm64 using `CGO_ENABLED=0`, deterministic path/VCS/build-ID settings, and no sidecar. Two independent builds of each architecture MUST be byte-identical.
#### Scenario: Both Linux architectures are built
- **WHEN** the canonical gateway Linux target runs twice from the same frozen source and dependency inputs
- **THEN** both amd64 and arm64 outputs are byte-identical pure-Go ELF executables with matching embedded GOOS, GOARCH, and cgo settings
### Requirement: Artifact evidence is inspected and truthful
Candidate evidence SHALL record exact source and Protocol revisions, artifact hashes, architecture, embedded dependency inventory, container configuration when built, and the actual scanner/signing status. It MUST NOT claim an SBOM, vulnerability result, signature, image architecture, or deployment that was not produced and inspected.
#### Scenario: Supplemental scanner is unavailable
- **WHEN** no qualifying vulnerability scanner is available in the frozen environment
- **THEN** the artifact remains explicitly unscanned, deterministic compiler/dependency/boundary evidence is retained, and no zero-finding security claim is emitted
@@ -0,0 +1,23 @@
## MODIFIED Requirements
### Requirement: Fixed media processing qualification
The qualification harness SHALL drive pinned-mTLS Apollo management, encrypted RTSP, ENet, and provider UDP through native source validation, `readUDPMedia`, recovery/FEC, bounded production queues, the production fair pacer, Verse framing/QUIC, and a public or independent client decoder for 1080p60 H.264 at 20 Mbps, 1440p120 HEVC at 50 Mbps, and 4K60 HEVC at 80 Mbps. After a recorded warm-up, the frozen candidate SHALL run each profile for ten wall-clock minutes, preserve encoded payload bytes, retain every monotonic processing sample plus bounded process CPU, memory, goroutine, allocation, and provider-queue observations, and report count, min, median, p90, p95, p99, max, mean, standard deviation, timing overhead, and observed bitrate. Processing begins at complete provider-unit receipt and ends at QUIC handoff, excluding client transit and pacing. CPU SHALL be actual OS user plus system consumption of the isolated gateway qualification process and MUST NOT be GOMAXPROCS-times-wall capacity or unrelated parent test work. Any bypass, payload mutation, wall-duration violation, bitrate outside both lower and upper bounds, or p95 above 5 ms SHALL fail.
#### Scenario: Healthy fixed profile
- **WHEN** a frozen candidate runs one fixed profile for the normative duration in the isolated qualification command
- **THEN** the harness emits compressed raw path and resource samples plus a summary tied to the exact command, CPU scope, topology, source commit, immutable Protocol version, environment, and payload hash
#### Scenario: Processing gate failure
- **WHEN** any production path stage lacks a per-traversal observation, payload integrity fails, duration or bitrate bounds fail, measured p95 exceeds 5 ms, or idle capacity is reported as consumed CPU
- **THEN** the qualification command exits unsuccessfully without recording a passing candidate
### Requirement: Bounded impairment qualification
The harness SHALL run exactly the baseline, latency, jitter, loss, reorder, and constrained Section 7.2 profiles once by applying fixed-seed impairment at the source-shaped provider network boundary while traffic concurrently traverses the production gateway path. Baseline SHALL cover all three media profiles and the other profiles SHALL cover 1080p60. The harness MUST NOT serialize a complete provider-to-client traversal per source unit. Each artifact SHALL retain raw impairment and queue observations and record tool version, exact command/configuration, environment, candidate commit, immutable Protocol version, direction, queue discipline, topology, fixed seed, observed one-way latency, acknowledged Apollo ENet RTT, jitter, loss, reorder, throughput, drops, and capacity-step statistics.
#### Scenario: Complete six-profile run
- **WHEN** the frozen candidate runs impairment qualification
- **THEN** one result exists for each named profile, configured jitter remains observable within reviewed fixed-seed tolerances, RTT comes from real request/response acknowledgement timing, and raw statistics come from actual traversal
#### Scenario: Unsupported or unbounded configuration
- **WHEN** a profile name, packet count, queue bound, loss, reorder, or bandwidth step falls outside the fixed catalog
- **THEN** the harness rejects it before allocating or running traffic
@@ -0,0 +1,18 @@
## 1. Qualification observations
- [x] 1.1 Reproduce synthetic doubled one-way RTT and serialized traversal
- [x] 1.2 Overlap source-shaped provider sends with public QUIC receive and retain actual delivery observations
- [x] 1.3 Measure RTT from Apollo ENet acknowledgements and verify fixed-seed jitter
- [x] 1.4 Replace CPU capacity with isolated OS process consumption and prove idle/work/parent isolation
## 2. Production and packaging
- [x] 2.1 Prove the legacy pacer has no production caller and delete it
- [x] 2.2 Build and inspect byte-reproducible pure-Go Linux amd64 and arm64 gateway artifacts
- [ ] 2.3 Generate final dependency/artifact evidence and record scanner/signing status truthfully
## 3. Frozen verification
- [ ] 3.1 Pass focused race/resource/impairment tests and complete Data Plane verification
- [ ] 3.2 Freeze immutable Protocol consumer inputs and run corrected Section 7 qualification once
- [ ] 3.3 Retain raw artifacts and explicit live Apollo/macOS/firewall deferral
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-07-30
@@ -0,0 +1,29 @@
## Context
The native Apollo session and production QUIC gateway already quiesce media before terminal delivery. A public independent-client test proved that immediate `CloseWithError` can overtake the queued reliable stream frame, while waiting for client connection closure leaves tunnel ownership unbounded.
## Goals / Non-Goals
**Goals:**
- Deliver one terminal event before gateway-owned closure.
- Bound closure when a client remains open or omits the receipt.
- Preserve cleanup, input release, reservation, and durable state behavior.
**Non-Goals:**
- A generic acknowledgement or lifecycle framework.
- Any Apollo protocol, media, Server authority, or dependency change.
## Decisions
- Reuse Protocol `control.ack.v1` type `0x03` as an empty terminal receipt.
- Hold the receipt-state lock across the terminal write, arm one receipt slot only after a successful write, and consume it in the gateway rather than provider feedback.
- Wait at most two seconds for receipt, then close and clean up regardless.
- Serialize the bounded native event queue and evict one older feedback item only when necessary to retain a terminal event.
## Risks / Trade-offs
- [Client omits receipt] → Close at the two-second bound and retain durable cleanup behavior.
- [Feedback queue is saturated] → Sacrifice one older nonterminal feedback event rather than lose terminal ownership.
- [Receipt is malformed, duplicate, or early] → Fail the session closed without provider mutation.
@@ -0,0 +1,24 @@
## Why
P3C-018, P3C-019, P3C-021, and P3C-027 require terminal feedback, bounded cleanup, durable state, and explicit input release. A public independent-client regression proved that immediate QUIC closure loses the terminal event, while the old behavior left the tunnel open until the client closed it.
## What Changes
- Consume the Protocol-owned terminal receipt on `control.ack.v1` inside the gateway rather than forwarding it to Apollo.
- Quiesce media before terminal delivery and close the gateway-owned tunnel after receipt or a bounded receipt deadline.
- Guarantee a terminal event survives saturation of the bounded native feedback queue.
- Preserve provider cleanup, reservation release, reconnectable disconnect reporting, and `cleanup_pending`.
## Capabilities
### New Capabilities
None.
### Modified Capabilities
- `provider-session-lifecycle`: Make terminal delivery and gateway-owned bounded closure executable rather than dependent on client connection closure.
## Impact
The pure-Go GPLv3 gateway control and native Apollo session paths change. The Protocol repository remains the wire-contract owner; the Server remains the durable authority. No cgo, sidecar, direct provider route, decode/transcode path, dependency, or proprietary source is introduced. Failure to receive a valid receipt before the bound is a hard session close, not a fallback.
@@ -0,0 +1,23 @@
## MODIFIED Requirements
### Requirement: Provider terminal events end forwarding
Encrypted provider termination and unexpected provider disconnect SHALL quiesce provider ingestion and queued/new media forwarding before the existing reliable typed terminal event is delivered. The client SHALL return the Protocol-owned terminal receipt after decoding that event. The gateway SHALL close the Verse tunnel after that receipt or a bounded receipt deadline even when the client keeps the connection open, release the session reservation, and report the appropriate durable provider/session state. The receipt MUST be consumed by the gateway and MUST NOT be forwarded to the provider. A fixed drain delay MUST NOT stand in for reliable control delivery.
#### Scenario: Host termination closes the tunnel
- **WHEN** the native provider emits an authenticated termination event and an independent client returns its terminal receipt
- **THEN** queued and newly injected media cannot cross the Verse transport after observation, and the gateway closes the tunnel, releases the reservation, and completes the durable lifecycle transition
#### Scenario: Unexpected provider disconnect is reconnectable
- **WHEN** required provider transport disconnects without acknowledged provider termination
- **THEN** forwarding stops, the final typed disconnect reaches the client, and the Server receives the existing reconnectable lifecycle state rather than a termination claim
#### Scenario: Client omits terminal receipt
- **WHEN** the terminal event is written but the client remains open without returning a valid receipt
- **THEN** the gateway closes the tunnel at the bounded receipt deadline and continues cleanup without restoring media forwarding
### Requirement: Cleanup failure remains durable
Gateway cleanup MUST preserve `cleanup_pending` when provider input release, transport cleanup, authorized cancellation, or durable reporting fails.
#### Scenario: Terminal cleanup fails
- **WHEN** a provider terminal event is handled but required cleanup cannot complete
- **THEN** the session is not reported reusable and durable state remains cleanup pending
@@ -0,0 +1,16 @@
## 1. Regressions
- [x] 1.1 Reproduce terminal event loss with an independent QUIC client and immediate gateway closure
- [x] 1.2 Reproduce terminal loss behind a saturated native feedback queue
## 2. Lifecycle repair
- [x] 2.1 Consume the scoped terminal receipt without provider forwarding
- [x] 2.2 Close acknowledged and non-acknowledged terminal tunnels within bounds
- [x] 2.3 Preserve media quiescence, reservation release, durable state, and cleanup-pending
## 3. Verification
- [x] 3.1 Pin the final immutable Protocol version and pass focused lifecycle/race/resource checks
- [ ] 3.2 Pass complete Data Plane verification and the frozen normative Section 7 qualification
- [ ] 3.3 Record that deterministic fixtures do not prove live Apollo, macOS-client, or physical-firewall interoperability
@@ -0,0 +1,22 @@
# apollo-stream-policy Specification
## Purpose
Define fail-closed translation of authenticated immutable stream policy into source-backed Apollo launch and ANNOUNCE behavior.
## Requirements
### Requirement: Apollo launch consumes the effective policy
The native Apollo backend SHALL derive ANNOUNCE resolution, frame rate, supported codec, selected bitrate, and audio profile from authenticated `ProviderSessionWork`, and MUST NOT substitute local defaults.
#### Scenario: Supported HEVC policy reaches Apollo
- **WHEN** provider work selects HEVC at 2560×1440, 120 FPS, 40000 Kbps, with audio enabled
- **THEN** the encrypted ANNOUNCE carries those settings and the source-backed HEVC and bitrate attributes
### Requirement: Provider policy cannot downgrade
The gateway MUST use the generated Protocol intersection to select only a registered profile compatible with the immutable policy. It MUST reject invalid, unsupported, no-overlap, downgrade, audio-disabled, AV1, or provider/source-mismatched Apollo policy before `/applist`, `/launch`, or provider readiness because the current native path cannot truthfully honor those combinations.
#### Scenario: Unsupported policy fails closed
- **WHEN** authenticated provider work selects audio disabled, AV1, a codec outside the registered peer intersection, or a resolution, frame rate, bitrate, audio, or codec combination outside source-backed Apollo support
- **THEN** setup fails before application discovery or launch without falling back to H.264, stereo, a cap, or another local default
#### Scenario: Independent peers negotiate one registered profile
- **WHEN** a production gateway and independent client advertise overlapping registered H.264 or HEVC profiles
- **THEN** admission selects the first policy-compatible common profile using shared Protocol behavior
@@ -0,0 +1,11 @@
# audio-fec-resilience Specification
## Purpose
Define bounded Apollo audio FEC state advancement and recovery after sustained incomplete blocks.
## Requirements
### Requirement: Bounded audio FEC state advances after loss
The Apollo audio recovery window SHALL remain bounded and SHALL evict the oldest incomplete block when accepting a newer block would otherwise be rejected.
#### Scenario: Newer complete block follows sustained loss
- **WHEN** more than the bounded number of permanently incomplete audio blocks arrive before a complete newer block
- **THEN** the oldest stale state is dropped, drop telemetry advances, and the newer encoded payload is relayed unchanged
@@ -0,0 +1,19 @@
# gateway-deployment-artifact Specification
## Purpose
Define the reproducible Linux gateway artifacts and truthful inspection evidence required for a deterministic Phase 3C engineering candidate.
## Requirements
### Requirement: Reproducible pure-Go Linux gateway artifacts
The candidate SHALL build the gateway with the normal immutable Protocol module boundary for Linux amd64 and arm64 using `CGO_ENABLED=0`, deterministic path/VCS/build-ID settings, and no sidecar. Two independent builds of each architecture MUST be byte-identical.
#### Scenario: Both Linux architectures are built
- **WHEN** the canonical gateway Linux target runs twice from the same frozen source and dependency inputs
- **THEN** both amd64 and arm64 outputs are byte-identical pure-Go ELF executables with matching embedded GOOS, GOARCH, and cgo settings
### Requirement: Artifact evidence is inspected and truthful
Candidate evidence SHALL record exact source and Protocol revisions, artifact hashes, architecture, embedded dependency inventory, container configuration when built, and the actual scanner/signing status. It MUST NOT claim an SBOM, vulnerability result, signature, image architecture, or deployment that was not produced and inspected.
#### Scenario: Supplemental scanner is unavailable
- **WHEN** no qualifying vulnerability scanner is available in the frozen environment
- **THEN** the artifact remains explicitly unscanned, deterministic compiler/dependency/boundary evidence is retained, and no zero-finding security claim is emitted
@@ -0,0 +1,25 @@
# gateway-heartbeat-telemetry Specification
## Purpose
Define truthful measured gateway egress, disjoint delay semantics, and bounded authenticated heartbeat telemetry.
## Requirements
### Requirement: Heartbeat egress is observed
Authenticated gateway heartbeat telemetry SHALL calculate egress from monotonic transmitted-byte deltas over monotonic elapsed time and MUST NOT report configured capacity as measured traffic.
#### Scenario: Controlled byte delta is sampled
- **WHEN** transmitted bytes increase by a known amount during a known interval
- **THEN** heartbeat egress equals the measured rate while configured capacity remains a separate registration value
### Requirement: Required telemetry remains bounded and low cardinality
The established authenticated path SHALL expose observed bytes, packets, drops, RTT, loss, jitter, queue delay, processing delay, pacing, reconnect, and provider state without session, route, credential, or payload labels.
#### Scenario: Telemetry snapshot is published
- **WHEN** the gateway emits a heartbeat after forwarding traffic
- **THEN** it carries the bounded process-level observations and no high-cardinality or secret-bearing value
### Requirement: Production delays have disjoint sample semantics
Queue delay SHALL measure provider-queue residence, processing delay SHALL measure provider recovery plus framing and QUIC handoff work excluding queue and pacing, and pacing delay SHALL measure scheduler waiting only. The gateway SHALL advance processing samples once per complete provider media unit even when it emits multiple Verse frames.
#### Scenario: Known production waits
- **WHEN** one provider media unit has controlled enqueue, processing, and pacing intervals and fragments across multiple frames
- **THEN** each cumulative total reports only its intended interval and exactly one processing sample is retained through authenticated Server persistence
@@ -0,0 +1,41 @@
# gateway-qualification Specification
## Purpose
Define the deterministic processing, impairment, pacing, and evidence boundaries
for qualifying a frozen Phase 3C gateway candidate.
## Requirements
### Requirement: Fixed media processing qualification
The qualification harness SHALL drive pinned-mTLS Apollo management, encrypted RTSP, ENet, and provider UDP through native source validation, `readUDPMedia`, recovery/FEC, bounded production queues, the production fair pacer, Verse framing/QUIC, and a public or independent client decoder for 1080p60 H.264 at 20 Mbps, 1440p120 HEVC at 50 Mbps, and 4K60 HEVC at 80 Mbps. After a recorded warm-up, the frozen candidate SHALL run each profile for ten wall-clock minutes, preserve encoded payload bytes, retain every monotonic processing sample plus bounded process CPU, memory, goroutine, allocation, and provider-queue observations, and report count, min, median, p90, p95, p99, max, mean, standard deviation, timing overhead, and observed bitrate. Processing begins at complete provider-unit receipt and ends at QUIC handoff, excluding client transit and pacing. CPU SHALL be actual OS user plus system consumption of the isolated gateway qualification process and MUST NOT be GOMAXPROCS-times-wall capacity or unrelated parent test work. Any bypass, payload mutation, wall-duration violation, bitrate outside both lower and upper bounds, or p95 above 5 ms SHALL fail.
#### Scenario: Healthy fixed profile
- **WHEN** a frozen candidate runs one fixed profile for the normative duration in the isolated qualification command
- **THEN** the harness emits compressed raw path and resource samples plus a summary tied to the exact command, CPU scope, topology, source commit, immutable Protocol version, environment, and payload hash
#### Scenario: Processing gate failure
- **WHEN** any production path stage lacks a per-traversal observation, payload integrity fails, duration or bitrate bounds fail, measured p95 exceeds 5 ms, or idle capacity is reported as consumed CPU
- **THEN** the qualification command exits unsuccessfully without recording a passing candidate
### Requirement: Bounded impairment qualification
The harness SHALL run exactly the baseline, latency, jitter, loss, reorder, and constrained Section 7.2 profiles once by applying fixed-seed impairment at the source-shaped provider network boundary while traffic concurrently traverses the production gateway path. Baseline SHALL cover all three media profiles and the other profiles SHALL cover 1080p60. The harness MUST NOT serialize a complete provider-to-client traversal per source unit. Each artifact SHALL retain raw impairment and queue observations and record tool version, exact command/configuration, environment, candidate commit, immutable Protocol version, direction, queue discipline, topology, fixed seed, observed one-way latency, acknowledged Apollo ENet RTT, jitter, loss, reorder, throughput, drops, and capacity-step statistics.
#### Scenario: Complete six-profile run
- **WHEN** the frozen candidate runs impairment qualification
- **THEN** one result exists for each named profile, configured jitter remains observable within reviewed fixed-seed tolerances, RTT comes from real request/response acknowledgement timing, and raw statistics come from actual traversal
#### Scenario: Unsupported or unbounded configuration
- **WHEN** a profile name, packet count, queue bound, loss, reorder, or bandwidth step falls outside the fixed catalog
- **THEN** the harness rejects it before allocating or running traffic
### Requirement: Fairness and cap qualification
The harness SHALL exercise the production fair pacer with eight equal-tier synthetic sessions for the required 60-second virtual interval, retain every per-flow and aggregate observation, report every share error and Jain's fairness index, and fail above 10% share error. It SHALL apply 25% and 50% capacity steps, measure convergence of observed allocation rather than first delivery, fail convergence beyond ten virtual seconds, and fail aggregate egress above 105% of the cap over any rolling five-second window.
#### Scenario: Equal-tier and capacity-step evidence
- **WHEN** the frozen candidate runs scheduler qualification
- **THEN** the artifact contains raw per-flow bytes, aggregate-cap series, share errors, Jain's index, measured allocation convergence, and rolling cap observations derived from the production pacer
### Requirement: Honest qualification boundary
Qualification artifacts SHALL contain no provider endpoint, credential, clipboard text, input payload, secret, or raw media content and SHALL make no claim of live Apollo/macOS/firewall interoperability. The harness SHALL add no codec operation, production dependency, cgo, sidecar, direct provider route, or duplicate processing/impairment simulator. Deterministic smoke evidence SHALL remain distinct from the single normative run on the frozen immutable consumer candidate.
#### Scenario: Deterministic evidence publication
- **WHEN** qualification completes
- **THEN** the manifest labels fake-provider, path impairment, and local processing evidence separately and leaves live interoperability deferred-owner-e2e
@@ -0,0 +1,26 @@
# provider-session-lifecycle Specification
## Purpose
Define terminal media quiescence, bounded Verse tunnel ownership, provider cleanup, reservation release, and durable lifecycle outcomes.
## Requirements
### Requirement: Provider terminal events end forwarding
Encrypted provider termination and unexpected provider disconnect SHALL quiesce provider ingestion and queued/new media forwarding before the existing reliable typed terminal event is delivered. The client SHALL return the Protocol-owned terminal receipt after decoding that event. The gateway SHALL close the Verse tunnel after that receipt or a bounded receipt deadline even when the client keeps the connection open, release the session reservation, and report the appropriate durable provider/session state. The receipt MUST be consumed by the gateway and MUST NOT be forwarded to the provider. A fixed drain delay MUST NOT stand in for reliable control delivery.
#### Scenario: Host termination closes the tunnel
- **WHEN** the native provider emits an authenticated termination event and an independent client returns its terminal receipt
- **THEN** queued and newly injected media cannot cross the Verse transport after observation, and the gateway closes the tunnel, releases the reservation, and completes the durable lifecycle transition
#### Scenario: Unexpected provider disconnect is reconnectable
- **WHEN** required provider transport disconnects without acknowledged provider termination
- **THEN** forwarding stops, the final typed disconnect reaches the client, and the Server receives the existing reconnectable lifecycle state rather than a termination claim
#### Scenario: Client omits terminal receipt
- **WHEN** the terminal event is written but the client remains open without returning a valid receipt
- **THEN** the gateway closes the tunnel at the bounded receipt deadline and continues cleanup without restoring media forwarding
### Requirement: Cleanup failure remains durable
Gateway cleanup MUST preserve `cleanup_pending` when provider input release, transport cleanup, authorized cancellation, or durable reporting fails.
#### Scenario: Terminal cleanup fails
- **WHEN** a provider terminal event is handled but required cleanup cannot complete
- **THEN** the session is not reported reusable and durable state remains cleanup pending