Measured, not estimated
Every number on this page is a real device benchmark, run in-cluster from the control plane. No projections, no marketing math. Where something isn't built yet, we say so.
Benchmarked 2026-09-24 · FP16 8192³ matmul · serving on Qwen2.5-7B (Q4), GPU-resident
Three GPU architectures under one control plane. The eight Blackwell cards land within 1.5% of each other — the fabric, drivers, and cooling all behave identically.
| Tier | Cards | Arch | FP16 | VRAM |
|---|---|---|---|---|
| NVIDIA — workhorse | 8× RTX 5060 Ti | Blackwell (sm_120) | ~50 TFLOP/s ea | 16 GB ea |
| NVIDIA — Ada | 1× RTX 4060 Ti | Ada (sm_89) | 45.8 TFLOP/s | 16 GB |
| AMD — modern | 1× RX 9060 XT | RDNA4 (gfx1200) | 40.4 TFLOP/s | 16 GB |
| AMD — HBM2 | 2× Radeon Pro VII | gfx906 | 15.5 + 16.6 TFLOP/s | 16 GB ea |
| Verified total · 12 GPUs | ~520 TFLOP/s | 201 GB | ||
Real token generation on a 7B (Q4) model, GPU-resident.
Serving pool = 7 Blackwell + 1 Ada + 1 RDNA4 for fast 7B. The RX 9060 XT and both HBM2 Radeon Pro VII cards are now cluster-joinable as GPU peers via Vulkan (RADV) — folding the full fleet into one large-model pool.
Every worker clears the ~30B (4-bit) bar on one card; the NVIDIA tier does it eight times over, in parallel. 70B-class has been benchmarked across a cross-GPU llama.cpp RPC pool (5 GPUs, measured ~7 tok/s) over the 2.5 GbE fabric — modest speed, but real. Larger models are in active bring-up.
The nodes are separate machines joined by a 2.5 GbE fabric. That's ideal for running many models side by side and serving many users in parallel — the fleet's real shape is 12 independent GPU endpoints, a throughput farm. Splitting one very large model across nodes is bottlenecked by the network, not the silicon; we shipped exactly that engineering path — a llama.cpp RPC pool (CUDA + Vulkan) that now serves 70B-class live at modest, honest tok/s. Two real shapes: a throughput farm of independent endpoints, and one pooled large-model engine.