flâneur

What is the TensorFloat-32 Precision Format? | NVIDIA Blog

blogs.nvidia.com · 392 words · saved by 1 readers

As with all computing, you’ve got to get your math right to do AI well. Because deep learning is a young field, there’s still a lively debate about which types of math are needed, for both training and inferencing. In November, we explained the differences among popular formats such as single-, double-, half-, multi- and mixed-precision math used in AI and high performance computing. Today, the NVIDIA Ampere architecture introduces a new approach for improving training performance on the single-precision models widely used for AI. TensorFloat-32 is the new math mode in NVIDIA A100 GPUs for handling the matrix math also called tensor operations used at the heart of AI and certain HPC applications. TF32 running on Tensor Cores in A100 GPUs can provide up to 10x speedups compared to single-precision floating-point math (FP32) on Volta GPUs. Combining TF32 with structured sparsity on the A100 enables performance gains over Volta of up to 20x. It helps to step back for a second to see how T

Latest How Nations Are Deploying AI for Strategic Priorities Nations have long invested in domestic infrastructure to advance their economies, protect and use their data, and take advantage of technology opportunities in areas such as transportation, communications, commerce, entertainment... July 6, 2026 NVIDIA Blackwell: Born for Extreme-Scale AI Inference NVIDIA Blackwell is the core of an entire system architecture designed specifically to power AI factories that produce intelligence through inference using the largest and most complex AI models.... September 18, 2025 The More You…

related reading