Draft:HiFloat8

• Comment: The three don't have in-depth coverage of the topic, and the last source isn't usable per WP:ARXIV. JustARandomSquid (talk) 15:21, 8 May 2026 (UTC)

---

The HiFloat8 (HiF8) is an 8‑bit tapered‑precision floating‑point format proposed for the training and inference of deep learning models. Under an 8‑bit encoding budget, HiF8 introduces an additional prefix‑coded dot field that selects the exponent field width (and a denormal mode), which allows the available mantissa bits to vary with the encoded exponent range and better balance numerical precision and dynamic range.

Definition

HiF8 is an 8-bit floating-point format designed for deep learning workloads. As illustrated in Fig. 1, in addition to the conventional sign, exponent, and mantissa fields (with the exponent using an implicit leading magnitude bit, shown in red), HiF8 introduces a variable-length dot field encoded with prefix codes. The dot field directly indicates the storage width of the exponent field and carries a denormal flag, thereby enabling dynamic allocation of exponent and mantissa bit widths within the 8-bit encoding budget.

Normal encoding mode

Fig. 1 HiFloat8 Format Definition

In the normal mode, the dot field encodes five values (0–4), corresponding to stored exponent widths from 0 to 4 bits. Given the sign bit, the dot field, and the selected exponent width, the mantissa width is determined by the 8-bit budget. Let Dot denote the decoded value of the dot field. When Dot = 0, no exponent bits are stored and the exponent is defined as E = 0. When Dot = 1–4, HiF8 encodes the exponent in sign-magnitude form and treats the most significant magnitude bit as an implicit 1 (shown in red in Fig. 1) and therefore does not store it. Specifically, with Dot = 1, the only possible exponents are E ∈ {−1, +1}, since E = 0 is represented by Dot = 0; when Dot = 2, the only possible exponents are E ∈ {−3, −2, +2, +3} (excluding {−1, +1} already covered by Dot = 1). By extension, the five cases Dot = 0–4 jointly represent all exponents in [-15, 15] without duplication.

Denormal encoding mode

In the denormal mode, HiF8 further extends its dynamic range. As shown in Fig. 1, when the dot field indicates the denormal mode, no exponent field is stored; instead, the remaining mantissa bits are used to encode additional low-magnitude exponent values. Specifically, the denormal mode extends the 31 exponents [-15, 15] available in normal mode by seven additional exponents [-22, -16], with one additional pattern reserved for special values, yielding 38 distinct exponents in total over [-22, 15].

Special values

Fig. 2 Special Value Support in HiF8

In addition to regular numeric encodings, HiF8 defines four special values: Zero, NaN, positive and negative infinity. HiF8 does not distinguish between +0 and −0, using a single bit pattern to represent zero. Together, these special values cover common exceptional and boundary cases encountered in floating-point computation.

Properties

Tapered precision

Fig. 3 Tapered Precision

As illustrated in Fig. 3, fixed-field 8-bit floating-point formats such as FP8-E4M3 and FP8-E5M2 use constant exponent and mantissa widths, which can be difficult to simultaneously meet the dynamic-range and numerical precision requirements of deep learning training and inference within an 8-bit budget. In contrast, HiF8 exhibits a tapered-precision profile in which the effective precision varies with magnitude: it allocates more mantissa bits for values with magnitude near 1 while allocating more bits to the exponent toward the tails.

Non-redundant encoding

HiF8 represents the exponent using sign-magnitude encoding and fixes the most significant magnitude bit to a known implicit value that is not stored. This construction ensures that the exponent ranges associated with different stored bit-widths do not overlap, resulting in a non-redundant encoding.

Wide dynamic range

With the denormal mode, HiF8 extends the representable exponent range from [-15, 15] to [-22, 15], yielding 38 distinct exponent values in total—close to FP16’s 40 exponents.

Evaluation and adoption

In October 2024, while discussing the trend towards mixed-precision training, Import AI discussed HiFloat8 and considered Huawei's reported performance improvements to be of practical value. It also interpreted Huawei's development of this format as evidence of vertical integration involving numerical representation, hardware, and software stacks. In October 2025, TechRadar reported that Ascend 950 would support HiF8, and later in December Tom's Hardware also reported it. In June 2026, Global Computing Consortium released Technical Specification of HiFloat8 Data Format,

A 2026 master's thesis from the University of British Columbia compared HiFloat8 with other tapered-precision formats. The thesis discussed how HiFloat8 uses different numbers of mantissa bits at different magnitudes and how it covers 38 binades. Meanwhile, the author pointed out that, despite HiFloat8 having been evaluated on multiple deep-learning models, the effect of using different levels of precision at different magnitudes had not been studied in depth as a separate factor. The author believed that the effect of tapered precision required further evaluation. A low-precision training survey published in 2026 at IEEE Transactions on Pattern Analysis and Machine Intelligence listed HiFloat8 outside Nvidia ecosystem and considered it as an ascend-oriented low-precision format. SAFE-Pruner combined HiF8 quantization and token pruning and applied it to OpenVLA-OFT; it's reported that this approach reduced FLOPs and GPU memory allocation while maintaining a similar success rate.