TinyBox packs a punch with six of AMD's fastest gaming GPUs repurposed for AI — new box uses Radeon 7900 XTX and retails for $15K, now in production
Tiny Corp., the startup that maintains tinygrad, a neural network framework, aims to democratize PetaFLOPS-class performance for artificial intelligence. To that effect, it has unveiled its TinyBox system that packs six AMD's Radeon RX 7900 XTX graphics cards into a 12U rack case and will sell for $15,000. That figure includes all the necessary software and hardware.
The TinyBox system uses six AMD Radeon RX 7900 XTX boards (one of the best graphics cards) connected using 'full fabric' PCIe 4.0 x16 links to ensure maximum bandwidth. Apparently, these consumer-grade GPUs fully support the peer-to-peer interconnections necessary for large language models (unlike Nvidia's GeForce RTX 4090), so the company chose to use these GPUs over Nvidia's more popular options.
From a performance point of view, the TinyBox can offer up to 738 FP16 TFLOPS (0.738 FP16 TFLOPS) of performance along with 96 GB of GDDR6 memory that delivers 21 TB/s of aggregated memory bandwidth. To put the numbers into context, one TinyBox offers 37% of Nvidia H100 compute performance (FP16) but slightly more memory (96 GB instead of 80 GB) and considerably higher peak memory bandwidth (21 TB/s instead of 3.35 TB/s).
