Fused Hadamard MXFP4 Block-Scaled Tensor Kernels Achieve 99.6 Percent FP16 Accuracy Recovery at 2.1x Inference Throughput

Quantization researchers and GPU kernel engineers across Silicon Valley, Santa Clara, San Jose, and Stanford published open-source low-bit inference kernels on Sunday afternoon through the OpenCompute Microscaling (MX) Working Group and vLLM Quantization SIG, demonstrating that Fused Hadamard Block-Scaled MXFP4 (E2M1) Tensor Kernels recover 99.6 percent of baseline BF16 accuracy across 70B and 405B reasoning models while delivering a 2.12x end-to-end throughput speedup and cutting HBM3e weight memory footprint by 73 percent. While fifth-generation tensor cores execute hardware-accelerated 4-bit microscaling (MXFP4, where every 32 contiguous FP4 E2M1 elements share an 8-bit E8M0 power-of-two block scale) at twice the FLOPs of FP8, naive post-training quantization collapses on transformer hidden states due to massive channel-wise activation outliers exceeding 100x the median magnitude.

To eliminate activation spikes before dynamic E2M1 rounding without runtime matrix-multiplication overhead, the open-source kernel library applies an orthogonal Fast Walsh-Hadamard Transform (FWHT) across 128-element activation sub-vectors and offline-rotates the corresponding linear projection weight matrices by the inverse Hadamard matrix H^T. Because H · H^T = I, the mathematical dot product is preserved exactly while outlier energy is redistributed uniformly into a near-Gaussian distribution whose crest factor fits cleanly inside the fifteen representable values of E2M1.

By fusing the butterfly-register O(d log d) Hadamard rotation directly into the preceding RMSNorm and SwiGLU epilogue kernels in shared SRAM, the transformation adds less than 1.8 percent latency overhead. Hardware block-scale factors (E8M0) are computed simultaneously in warp registers using fast reciprocal-max reductions before issuing fifth-generation tensor-core MMA instructions.

Evaluated across an 8-GPU server node at 2900 Semiconductor Drive in Santa Clara supported by a $390,000 open-source quantization grant, Hadamard-MXFP4 sustained 14,800 tokens per second on Llama-3.1-405B with less than 0.04 WikiText-2 perplexity degradation. CUTLASS and Triton kernels are available in the Fast-Hadamard-Transform GitHub Repository.