Zyphra Says Its AMD MI355X Attention Kernel Is 1.76x Faster

Zyphra says its new FP8 attention kernels for AMD’s MI355X GPU deliver “1.16–1.76×” the throughput of the default kernels in AITER, AMD’s own kernel library, on activations captured from three open models (Zyphra). The company published the kernels, called ZYFA, on 5 November.

The 1.76x figure is the best of three results. Across the three models Zyphra tested at 16,000 tokens, the FP8 speedup ranges from 1.16x to 1.76x, and the BF16 speedup from 1.01x to 1.02x, by Zyphra’s own measurements. These are a vendor’s benchmarks, not independent tests.

Business Pill · Concentration

An evergreen Business Engineer explainer on concentration. It teaches the general idea only and does not discuss Zyphra or AMD.

What Zyphra Reported

Zyphra says ZYFA “consists of causal forward attention kernels in BF16 and FP8 for the MI355X”. The kernels cover the prefill step, the pass over a prompt before a model writes its first token. Zyphra says they support head dimension 128 and the head shape of the multi-head latent attention used in DeepSeek-V2.

On activations from Mistral-7B, Qwen3-8B and DeepSeek-V2-Lite at 16K tokens, Zyphra reports FP8 speedups of 1.16x, 1.17x and 1.76x against AITER’s default kernels. On synthetic Gaussian inputs at 32K tokens, it reports “1.26–1.37×” at head dimension 128 and “1.99×” at the DeepSeek attention shape.

Zyphra says the real-activation gains are smaller because “the clock of the MI355X depends on the data”. On a Mistral-7B layer, it reports its FP8 kernel ran at 1,628 MHz against 2,108 MHz for AITER’s, because its per-block scaling keeps values that switch more bits.

Accuracy and Code

Zyphra reports that at head dimension 128 its FP8 error is “within a few percent of that of” AMD’s Composable Kernel and Triton kernels and “62–78%” lower than AITER’s. It says a rotation of the queries and keys halves most of the remaining FP8 error on Mistral-7B.

The code is on GitHub under the Apache 2.0 license (github.com/Zyphra/zyfa), and Zyphra published a technical report (arXiv). Zyphra says all experiments ran on “one node with eight MI355X GPUs”, on ROCm 7.2.3, against AITER 0.1.19, and that “all results are point-in-time”.

Our Numbers

Across Zyphra’s three real-model tests, the FP8 speedup over AITER ranges from 1.16x to 1.76x; on synthetic inputs it ranges from 1.26x to 1.99x. On the Mistral-7B layer, Zyphra’s FP8 kernel ran at 2,537 trillion operations per second against 1,223 for its BF16 kernel, about 2.07x, by our arithmetic from Zyphra’s appendix. Its FP8 kernel’s clock was about 23% below AITER’s on that layer, by our arithmetic.

Connecting the Coverage

AMD made 12.4% of the commits to the vLLM inference engine in a year (our piece), and a Meta paper found AMD ahead of Nvidia on 4 of 10 training models (our piece). A Stanford study put Nvidia’s B200 at 4.3x the MI350X’s value per joule (our piece). AMD expects its data center AI sales to top $6.4B in the fourth quarter (our piece), and TSMC is exploring 3D memory chips at Nvidia’s request (our piece).

The Structural Read

As we read the post, the gain depends on the workload: Zyphra reports 1.76x on DeepSeek-V2-Lite but 1.16x and 1.17x on Mistral-7B and Qwen3-8B.

As we read the figures, the clock matters as much as the code: Zyphra reports its FP8 kernel ran at 1,628 MHz against 2,108 MHz for AITER on a Mistral-7B layer.

As we read the scope, the results cover prefill attention only; Zyphra says all results are point-in-time.

Three Implications

FOR AMD Zyphra reports its FP8 kernels achieve a higher throughput than AITER, AMD’s own kernel library, at every sequence length from 1K to 32K tokens on Gaussian inputs.

FOR DEVELOPERS Zyphra says the source code is available on GitHub and the paper is on arXiv.

FOR ACCURACY Zyphra reports its FP8 error is 62-78% lower than AITER’s at head dimension 128.

The Business Engineer Lens

This story maps onto the Business Engineer framework The CFO’s Guide to the Token Economy.

The analysis says: “It is simultaneously the unit of cognition the model produces, the unit of compute the data center serves, the unit of price the lab charges, and the unit of value the enterprise extracts.”

As we read it, a faster attention kernel changes how much compute each token takes on a given chip; the analysis does not discuss Zyphra or AMD.

What Is Not Established

All figures are Zyphra’s own, from one node, and Zyphra says its kernels and the baselines change quickly. We found no independent test or AMD comment. The kernels cover causal prefill only, at two head shapes, so the results do not show the speed of a full model end to end or of the decoding step.

The Bottom Line

Zyphra says its open-source FP8 attention kernels run 1.16x to 1.76x faster than AMD’s default AITER kernels on three real models on the MI355X, with lower error than AITER at head dimension 128. The figures are Zyphra’s own and come from one node.