Benchmarks & Provenance Protocol
The defining differentiator of this lab is that every number carries its own provenance. When I publish a latency, throughput, or cost figure, it is accompanied by the exact hardware tier, dataset source, and an unambiguous declaration of whether it was measured on physical silicon or projected analytically.
The Three Provenance Rules
Numbers from high-TFLOPS consumer cards (RTX 4090) and cloud virtualization instances (GCP L4) are never aggregated or blended into composite averages. They are presented in dedicated columns with their respective driver and PCIe setups.
If a metric was generated through trace replay or discrete-event simulation (e.g. Alibaba cluster logs), it must be labeled as "modeled" or "simulated" directly in the sentence that reports it, never relegated to fine-print footnotes.
An engineering trade-off where an optimization kernel runs slower or where an architectural assumption breaks down is published with the same prominence as positive gains.
Verified Hardware Inventory
All benchmarks cited across Anavind Labs systems were executed on the following documented testbed environments.
| Hardware Platform | VRAM & Interconnect | Peak Bandwidth / Compute | Primary Role in Lab | Key Projects Evaluated |
|---|---|---|---|---|
| NVIDIA GeForce RTX 4090 Bare-Metal Dedicated Host |
24GB GDDR6X PCIe 4.0 x16 |
1,008 GB/s Bandwidth ~82.6 TFLOPS FP32 |
48 paired runs (58,528 requests) across 4 serving modes; continuous burst stress tests | Aether Control, Clairvoyant |
| NVIDIA GeForce RTX 4070 (1x & 2x) Physical Dual-GPU Host |
12GB GDDR6X PCIe 4.0 x16 |
504 GB/s Bandwidth ~29.1 TFLOPS FP32 |
Physical multi-GPU silicon smoke tests, same-host cuda:0 → cuda:1 handoff, and Step 3 prompt length sweeps | HeteroDisagg (Steps 1, 2, 3) |
| Cloud Heterogeneous Pair (RTX 3090 → RTX 4090) Independent Cloud Nodes |
24GB GDDR6X (3090) 24GB GDDR6X (4090) |
936 GB/s (Ampere) 1,008 GB/s (Ada) |
Step 4B cross-machine WAN handoff across mismatched GPU microarchitectures (Ampere SM 8.6 → Ada SM 8.9) | HeteroDisagg (Step 4B) |
| NVIDIA GeForce RTX 3090 Dedicated Training Host |
24GB GDDR6X PCIe 4.0 x16 |
936 GB/s Bandwidth ~35.6 TFLOPS FP32 |
Tokenizer-aware training input validation, loss mask reconciliation, and SFT length-grouped batching benchmarks | TrainSight (PyPI package) |
| NVIDIA L4 Tensor Core GPU Google Cloud Platform (GCP GKE) |
24GB GDDR6 PCIe 4.0 x16 |
300 GB/s Bandwidth ~30.3 TFLOPS FP32 |
Cloud production emulation, Kubernetes custom scheduler plugin profiling | Kube-Sentinel (V2) |
| Apple Silicon M1 Edge Workstation |
16GB Unified Memory | 68.25 GB/s Bandwidth | Constrained edge LLM runtime validation (llama.cpp) | Clairvoyant (edge-llm-pipeline) |
Key Architectural Disclosures
Synthetic Instruction Skew in Length Predictors
When training the Clairvoyant output-length classifier, instruction-tuning datasets (e.g. Stanford Alpaca) created model collapse. Models trained on synthetic prompts achieved near-zero ranking accuracy on real conversational traces. Synthetic instruction pairs are constrained by GPT distillation brevity, whereas real human conversations exhibit multi-modal bimodal length distributions with high variance.
Stateless EMA vs. Deep LSTM in Job Routing
In a predictor ablation for GPU job routing, evaluating 14 predictor variants revealed that an Exponential Moving Average (EMA) with decay factor α=0.2 routed 89.2% of jobs to stable nodes, beating the LSTM's 75.4%. The LSTM suffered from catastrophic forgetting when burst patterns shifted, while the stateless EMA adapted in one step with zero inference latency.
TrainSight: Preflight Contract Verification & SFT Batching
Supervising prompt tokens or empty answer delimiters (<|im_end|>) quietly corrupts autoregressive model training while passing standard tensor checks. TrainSight audits sequence lengths, answer-only loss masks, and identical DPO pairs before execution. Benchmarked on an RTX 3090 (Qwen2.5-1.5B, 1,000 GSM8K examples), length-grouped batching achieved 25.76% ± 0.45% fewer padded tokens and a +29.28% ± 0.65% increase in useful-token throughput with zero GPU memory overhead.
Step 3 Compression Crossover: Loopback vs. Network Amortization
In HeteroDisagg, transforming FP16 KV-cache blocks into FP8 introduces +11ms to +32ms of isolated compute delay. On loopback sockets, uncompressed FP16 is strictly faster across all prompt lengths ($S \in [64, 1024]$), and AdaptiveCompressionPolicy correctly selects uncompressed mode with 0ms regret. However, on calibrated 500 Mbps network links at $S=1024$, FP8 wire compression achieves an end-to-end handoff latency speedup of +25.73ms (1.104x speedup), establishing the exact physical bandwidth-delay threshold where byte reduction overcomes codec cost.
Overload Goodput Sacrifice in Admission Control
Under sustained 48 RPS overload, direct unconstrained vLLM breaches the 2.50s E2E SLO (P99 = 2.57s) while delivering 44.98 RPS goodput. Introducing a static concurrency cap (Mode 3 in Aether Control) defends user-perceived latency, strictly bounding P99 E2E to 1.50s via clean HTTP 429 shedding. However, this latency protection incurs an explicit systems cost: compliant goodput drops to 30.27 RPS (a 32.7% goodput reduction, shedding 743.3 requests). Bounded queue admission cuts near-capacity shedding by 55%.