Creating custom kernels for the AMD MI300
This blog post details the collaboration between Hugging Face and AMD to create optimized kernels for the AMD MI300X GPU. Three custom kernels—fused residual connection/RMS norm/FP8 conversion, fused SwiGLU activation/FP8 conversion, and skinny GEMM—were developed to improve VLLM serving of Llama 3.1 405B in FP8. Performance gains were measured with input size 1 and output size 128, using median decoding latency over 30 iterations.
Related Tech News
AI CuratedHyperX Omen 15 review: Strong gaming performance and colorful OLED display, with obvious cost-cutting measures
The HyperX Omen 15 provides solid gaming performance with an Intel Core Ultra 9, RTX 5070, 32GB RAM, and a 15.3-inch 1800p OLED display for $2,099.99. Benchmarks show competitive frame rates, though the plastic chassis lacks rigidity and there are no Thunderbolt ports. The OLED panel is bright and colorful, but the 120 Hz refresh rate is lower than rivals. Overall, a good value with compromises.
AMD Introduces Threadripper Halo Station at IFA 2026
At IFA 2026, AMD unveiled the Threadripper Halo Station, a workstation combining a 96-core Zen 5 CPU with up to 4 Instinct MI350P cards (each 144GB HBM3E, 600W TBP). The system can hold a trillion-parameter model entirely on-device. Both CPU and GPUs are liquid-cooled. Full specs, pricing, and availability have not been announced.
AMD unveils Threadripper Halo Station, an AI workstation packing 96 cores and dual liquid-cooled MI350P accelerators — 'the most powerful workstation in the world' can run trillion-parameter models, says AMD
AMD unveiled the Threadripper Halo Station, an AI workstation it calls 'the most powerful in the world,' at IFA 2026. The system combines a 96-core Zen 5 Threadripper Pro 9995WX CPU with dual liquid-cooled Instinct MI350P accelerators (144GB HBM3E each), 2TB DDR5, and support for up to four GPUs. AMD claims it can run trillion-parameter models. Estimated component cost exceeds $100,000, with fully configured systems potentially over $150,000. No price or release date was announced, and AMD has not yet named OEM partners.
MSI, GIGABYTE And Inno3D Jack Up RTX 50 And RX 9000 Prices Again, With RTX 5070 Buyers Facing A $90 Hit
Wccftech reports that MSI, Gigabyte, and Inno3D have joined ASUS and Galax in raising prices on NVIDIA GeForce RTX 50 and AMD Radeon RX 9000 series GPUs. This is the second price increase since August, affecting buyers of the RTX 5070 with a $90 hike. The article notes the trend but lacks detailed analysis or official confirmation from the vendors.