Draft:HiFloat8

• Comment: The three don't have in-depth coverage of the topic, and the last source isn't usable per WP:ARXIV. JustARandomSquid (talk) 15:21, 8 May 2026 (UTC)

---

HiFloat8 (HiF8) is an 8-bit tapered-precision floating-point data format proposed by Huawei researchers for low-precision deep learning training and inference. Under an 8-bit encoding budget, HiF8 uses a prefix-coded dot field to vary the effective allocation of exponent and mantissa bits, and to indicate a denormal encoding mode.

HiFloat8 has been discussed in independent AI commentary as part of the broader industry trend toward mixed-precision and low-precision training. It has also been listed in academic and technology-media coverage of 8-bit or low-precision formats for AI accelerators, including coverage of Huawei's Ascend 950-series roadmap.

Background

Training and inference of large neural networks commonly use lower-precision numerical formats to reduce memory and computational requirements and improve throughput. HiFloat8 was proposed in this context as a custom 8-bit floating-point format for artificial-intelligence workloads. The original white paper situated HiF8 among earlier developments such as FP16, posit arithmetic and FP8-like formats, and described it as a format intended for both forward and backward passes in deep-learning training.

Independent commentary in Import AI described HiFloat8 as part of the general move from 16-bit and 32-bit computation toward lower-precision and mixed-precision training. The same commentary interpreted Huawei's development of a proprietary 8-bit data format as an example of full-stack AI hardware-software development.

A paper by Abdelfattah, Dongarra, Fasi, Mikaitis and Tisseur placed Huawei's Ascend HiFloat8 in a broader industry landscape of 8-bit floating-point formats, alongside formats associated with Graphcore, NVIDIA, Arm, Intel, the Open Compute Project and Tesla's Dojo.

Coverage and analysis

Import AI discussed HiFloat8 in a section on Huawei's approach to 8-bit training for large language models. The article described HiFloat8 as a specification for 8-bit data representation and connected it to the industry trend toward mixed-precision training.

Technology-media coverage of Huawei's Ascend roadmap has listed HiF8 among the low-precision formats supported by the Ascend 950 series. Tom's Hardware listed HiF8 among the supported formats for the Ascend 950PR, Ascend 950DT, Ascend 960 and Ascend 970, and described the Ascend 950 series as adding support for FP8, MXFP8, HiF8 and MXFP4 formats to balance performance and precision. TechRadar listed HiF8 among the supported formats for the Ascend 950DT and reported peak performance of one PFLOPS in FP8, MXFP8 and HiF8 modes. RCR Wireless similarly reported that the Ascend 950 series supports FP8, MXFP8, MXFP4 and Huawei's proprietary HiF8.

Definition

Fig. 1. Schematic of the proposed HiFloat8 encoding, including the sign field, dot field, exponent field, mantissa field and denormal mode.

HiF8 is defined as an 8-bit floating-point format consisting of a sign field, a dot field, an exponent field and a mantissa field. In the original white paper, the dot field is a variable-length prefix code that indicates the stored width of the exponent field and distinguishes normal and denormal encodings.

The format differs from fixed-field 8-bit floating-point formats in that the number of stored exponent bits and mantissa bits can vary across encoded ranges. The remaining bit budget after the sign and dot fields is allocated between exponent and mantissa fields. In normal encodings, the exponent is represented in sign-magnitude form, with the most significant magnitude bit treated as an implicit bit that is not stored.

Normal encoding mode

In the normal encoding mode described by the white paper, the dot field encodes five values, denoted Dot = 0 through Dot = 4, corresponding to stored exponent widths from 0 to 4 bits. Given the sign bit, the decoded dot value and the selected exponent width, the mantissa width is determined by the remaining 8-bit budget.

When Dot = 0, no exponent bits are stored and the exponent is defined as E = 0. When Dot = 1 through Dot = 4, HiF8 uses sign-magnitude exponent encoding and treats the leading magnitude bit as implicit. For example, Dot = 1 represents exponents E ∈ {−1, +1}, since E = 0 is represented by Dot = 0. Dot = 2 represents E ∈ {−3, −2, +2, +3}, excluding the exponent values already represented by Dot = 1. By extension, the normal encoding cases jointly cover exponents from −15 to +15 without duplicated exponent ranges.

The white paper summarizes the normal encoding as providing seven exponent values with a 3-bit mantissa, eight exponent values with a 2-bit mantissa and sixteen exponent values with a 1-bit mantissa.

Denormal encoding mode

In the denormal encoding mode, HiF8 extends the representable low-magnitude range. When the dot field indicates denormal mode, no exponent field is stored; the remaining mantissa bits are used to encode additional low-magnitude values. The white paper states that the denormal mode extends the 31 normal-mode exponent values over [−15, 15] by seven additional powers of two, extending the low-magnitude range to −22.

Together, the normal and denormal encodings are described as covering 38 binades, compared with 40 binades for FP16.

Special values

Fig. 2. Special-value encodings in HiF8.

In addition to regular numeric encodings, HiF8 defines encodings for zero, NaN, positive infinity and negative infinity. The white paper states that HiF8 does not distinguish between +0 and −0, instead using a single bit pattern to represent zero.

The Global Computing Consortium's HiFloat8 technical specification also states that the format defines unified encoding rules for zero, infinity and NaN values, as well as conversion rules between HiFloat8 and higher-precision floating-point formats such as FP32, FP16 and BF16.

Properties

Tapered precision

Fig. 3. Illustration of tapered precision in HiF8 compared with fixed-field FP8 formats.

The HiFloat8 white paper describes the format as a tapered-precision design. In this design, the effective precision varies with the magnitude of the represented value: some ranges allocate more bits to the mantissa, while other ranges allocate more of the 8-bit budget to exponent representation.

The white paper contrasts this design with fixed-field 8-bit floating-point formats such as E4M3 and E5M2, whose exponent and mantissa widths remain fixed across their representable ranges. In the authors' description, HiF8 is intended to balance numerical precision and dynamic range within a single 8-bit format for deep-learning workloads.

Non-redundant encoding

HiF8 represents normal-mode exponents using sign-magnitude encoding and an implicit leading magnitude bit. According to the white paper, this construction avoids duplicated exponent ranges across different stored exponent widths, resulting in what the authors call non-redundant encoding.

The Global Computing Consortium's HiFloat8 technical specification similarly describes HiFloat8 as using tapered accuracy and non-redundant encoding under a fixed 8-bit width.

Wide dynamic range

With denormal encoding, HiF8 extends the representable exponent range from [−15, 15] to [−22, 15]. The original white paper states that this gives HiF8 38 binades, close to the 40 binades covered by FP16, while retaining an 8-bit storage width.

Claims about HiF8's performance relative to other floating-point formats are primarily based on Huawei-authored or Huawei-affiliated sources. Huawei stated in its Ascend 950 roadmap presentation that HiF8 was intended to provide precision close to FP16 and efficiency comparable with FP8 on Ascend 950-series chips.

Standardization and ecosystem

On 23 June 2026, the Global Computing Consortium stated that it had released and implemented the group standard Technical Specification of HiFloat8 Data Format. GCC described the standard as applying to AI model training, inference, software development, test verification and performance evaluation based on 8-bit floating-point computation.

GCC stated that the standard specifies HiFloat8 binary-format requirements, including data-format definition, encoding rules and format-conversion requirements. The same announcement said that the standard was jointly completed by organizations from industry, academia and research institutions, including Huawei Technologies, Shanghai Artificial Intelligence Innovation Center, iFLYTEK, South China University of Technology, SenseTime, Tsinghua University, Peking University, Beihang University, Zhejiang Lab, China Electronics Standardization Institute and China Telecom Research Institute.

Evaluations and applications

A paper by researchers from Huawei Technologies in the ACL 2026 Industry Track evaluated the HiFloat family of formats, including HiF8 and HiF4, for low-bit inference on Ascend NPUs. The paper compared HiFloat formats with integer and floating-point quantization formats across weight-activation quantization and KV-cache quantization tasks.

Subsequent preprints have studied or used HiF8 in several low-precision AI-computing settings. One work studied HiF8 W8A8 quantization-aware training for OpenPangu-Embedded-1B through delayed tensor scaling, identifying amax saturation and catastrophic forgetting as separate failure modes. Another proposed a low-precision softmax workflow using HiF8 and block-aware precision rescaling for Transformer attention. A separate work applied W8A8 HiFloat8 quantization to a large-scale text-to-video diffusion transformer on Ascend 910B NPUs.

Hardware support

Huawei stated that Ascend 950-series chips would support low-precision data formats including FP8, MXFP8, MXFP4 and HiF8. In the same roadmap presentation, Huawei stated that the chips would deliver one PFLOPS in FP8, MXFP8 and HiF8 modes, and two PFLOPS in MXFP4 mode.

Tom's Hardware listed the Ascend 950PR and Ascend 950DT as supporting FP32, HF32, FP16, BF16, FP8, MXFP8, HiF8 and MXFP4. It also listed the later Ascend 960 and Ascend 970 as supporting HiF8 and HiF4 in addition to those formats. TechRadar listed the Ascend 950DT as supporting FP8, MXFP8, MXFP4 and HiF8, with one PFLOPS peak performance in FP8, MXFP8 and HiF8 modes. RCR Wireless reported that the Ascend 950 series supports FP8, MXFP8, MXFP4 and Huawei's proprietary HiF8.

See also

• FP8
• Half-precision floating-point format
• Bfloat16 floating-point format
• Block floating point
• Huawei Ascend