No project description provided
Project description
ml_dtypes
ml_dtypes
is a stand-alone implementation of several NumPy dtype extensions used in machine learning libraries, including:
bfloat16
: an alternative to the standardfloat16
formatfloat8_*
: several experimental 8-bit floating point representations including:float8_e3m4
float8_e4m3
float8_e4m3b11fnuz
float8_e4m3fn
float8_e4m3fnuz
float8_e5m2
float8_e5m2fnuz
- Microscaling (MX) sub-byte floating point representations including:
float4_e2m1fn
float6_e2m3fn
float6_e3m2fn
int2
,int4
,uint2
anduint4
: low precision integer types.
See below for specifications of these number formats.
Installation
The ml_dtypes
package is tested with Python versions 3.9-3.12, and can be installed
with the following command:
pip install ml_dtypes
To test your installation, you can run the following:
pip install absl-py pytest
pytest --pyargs ml_dtypes
To build from source, clone the repository and run:
git submodule init
git submodule update
pip install .
Example Usage
>>> from ml_dtypes import bfloat16
>>> import numpy as np
>>> np.zeros(4, dtype=bfloat16)
array([0, 0, 0, 0], dtype=bfloat16)
Importing ml_dtypes
also registers the data types with numpy, so that they may
be referred to by their string name:
>>> np.dtype('bfloat16')
dtype(bfloat16)
>>> np.dtype('float8_e5m2')
dtype(float8_e5m2)
Specifications of implemented floating point formats
bfloat16
A bfloat16
number is a single-precision float truncated at 16 bits.
Exponent: 8, Mantissa: 7, exponent bias: 127. IEEE 754, with NaN and inf.
float4_e2m1fn
Exponent: 2, Mantissa: 1, bias: 1.
Extended range: no inf, no NaN.
Microscaling format, 4 bits (encoding: 0bSEEM
) using byte storage (higher 4
bits are unused). NaN representation is undefined.
Possible absolute values: [0
, 0.5
, 1
, 1.5
, 2
, 3
, 4
, 6
]
float6_e2m3fn
Exponent: 2, Mantissa: 3, bias: 1.
Extended range: no inf, no NaN.
Microscaling format, 6 bits (encoding: 0bSEEMMM
) using byte storage (higher 2
bits are unused). NaN representation is undefined.
Possible values range: [-7.5
; 7.5
]
float6_e3m2fn
Exponent: 3, Mantissa: 2, bias: 3.
Extended range: no inf, no NaN.
Microscaling format, 4 bits (encoding: 0bSEEEMM
) using byte storage (higher 2
bits are unused). NaN representation is undefined.
Possible values range: [-28
; 28
]
float8_e3m4
Exponent: 3, Mantissa: 4, bias: 3. IEEE 754, with NaN and inf.
float8_e4m3
Exponent: 4, Mantissa: 3, bias: 7. IEEE 754, with NaN and inf.
float8_e4m3b11fnuz
Exponent: 4, Mantissa: 3, bias: 11.
Extended range: no inf, NaN represented by 0b1000'0000.
float8_e4m3fn
Exponent: 4, Mantissa: 3, bias: 7.
Extended range: no inf, NaN represented by 0bS111'1111.
The fn
suffix is for consistency with the corresponding LLVM/MLIR type, signaling this type is not consistent with IEEE-754. The f
indicates it is finite values only. The n
indicates it includes NaNs, but only at the outer range.
float8_e4m3fnuz
8-bit floating point with 3 bit mantissa.
An 8-bit floating point type with 1 sign bit, 4 bits exponent and 3 bits mantissa. The suffix fnuz
is consistent with LLVM/MLIR naming and is derived from the differences to IEEE floating point conventions. F
is for "finite" (no infinities), N
for with special NaN encoding, UZ
for unsigned zero.
This type has the following characteristics:
- bit encoding: S1E4M3 -
0bSEEEEMMM
- exponent bias: 8
- infinities: Not supported
- NaNs: Supported with sign bit set to 1, exponent bits and mantissa bits set to all 0s -
0b10000000
- denormals when exponent is 0
float8_e5m2
Exponent: 5, Mantissa: 2, bias: 15. IEEE 754, with NaN and inf.
float8_e5m2fnuz
8-bit floating point with 2 bit mantissa.
An 8-bit floating point type with 1 sign bit, 5 bits exponent and 2 bits mantissa. The suffix fnuz
is consistent with LLVM/MLIR naming and is derived from the differences to IEEE floating point conventions. F
is for "finite" (no infinities), N
for with special NaN encoding, UZ
for unsigned zero.
This type has the following characteristics:
- bit encoding: S1E5M2 -
0bSEEEEEMM
- exponent bias: 16
- infinities: Not supported
- NaNs: Supported with sign bit set to 1, exponent bits and mantissa bits set to all 0s -
0b10000000
- denormals when exponent is 0
float8_e8m0fnu
OpenCompute MX scale format E8M0, which has the following properties:
- Unsigned format
- 8 exponent bits
- Exponent range from -127 to 127
- No zero and infinity
- Single NaN value (0xFF).
int2
, int4
, uint2
and uint4
2 and 4-bit integer types, where each element is represented unpacked (i.e., padded up to a byte in memory).
NumPy does not support types smaller than a single byte: for example, the
distance between adjacent elements in an array (.strides
) is expressed as
an integer number of bytes. Relaxing this restriction would be a considerable
engineering project. These types therefore use an unpacked representation, where
each element of the array is padded up to a byte in memory. The lower two or four
bits of each byte contain the representation of the number, whereas the remaining
upper bits are ignored.
Quirks of low-precision Arithmetic
If you're exploring the use of low-precision dtypes in your code, you should be
careful to anticipate when the precision loss might lead to surprising results.
One example is the behavior of aggregations like sum
; consider this bfloat16
summation in NumPy (run with version 1.24.2):
>>> from ml_dtypes import bfloat16
>>> import numpy as np
>>> rng = np.random.default_rng(seed=0)
>>> vals = rng.uniform(size=10000).astype(bfloat16)
>>> vals.sum()
256
The true sum should be close to 5000, but numpy returns exactly 256: this is
because bfloat16
does not have the precision to increment 256
by values less than
1
:
>>> bfloat16(256) + bfloat16(1)
256
After 256, the next representable value in bfloat16 is 258:
>>> np.nextafter(bfloat16(256), bfloat16(np.inf))
258
For better results you can specify that the accumulation should happen in a
higher-precision type like float32
:
>>> vals.sum(dtype='float32').astype(bfloat16)
4992
In contrast to NumPy, projects like JAX which support low-precision arithmetic more natively will often do these kinds of higher-precision accumulations automatically:
>>> import jax.numpy as jnp
>>> jnp.array(vals).sum()
Array(4992, dtype=bfloat16)
License
This is not an officially supported Google product.
The ml_dtypes
source code is licensed under the Apache 2.0 license
(see LICENSE). Pre-compiled wheels are built with the
EIGEN project, which is released under the
MPL 2.0 license (see LICENSE.eigen).
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for ml_dtypes-0.5.0-cp313-cp313-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cb5cc7b25acabd384f75bbd78892d0c724943f3e2e1986254665a1aa10982e07 |
|
MD5 | e5569fd81ab1c49088e803176a37034a |
|
BLAKE2b-256 | 148730323ad2e52f56262019a4493fe5f5e71067c5561ce7e2f9c75de520f5e8 |
Hashes for ml_dtypes-0.5.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 54415257f00eb44fbcc807454efac3356f75644f1cbfc2d4e5522a72ae1dacab |
|
MD5 | 8f9e7e2086082e739e0c612c7e453dc2 |
|
BLAKE2b-256 | 4cb4d766586e24e7a073333c8eb8bd9275f3c6fe0569b509ae7b1699d4f00c74 |
Hashes for ml_dtypes-0.5.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e04fde367b2fe901b1d47234426fe8819909bd1dd862a5adb630f27789c20599 |
|
MD5 | 2c5d3a7e29eb10ef5961b8f2fb386f95 |
|
BLAKE2b-256 | edc6358d85e274e22d53def0c85f3cbe0933475fa3cf6922e9dca66eb25cb22f |
Hashes for ml_dtypes-0.5.0-cp313-cp313-macosx_10_13_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d3b3db9990c3840986a0e70524e122cfa32b91139c3653df76121ba7776e015f |
|
MD5 | 0effb40dfc11661839dfd5e08bde144e |
|
BLAKE2b-256 | b34a18f670a2703e771a6775fbc354208e597ff062a88efb0cecc220a282210b |
Hashes for ml_dtypes-0.5.0-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | afa08343069874a30812871d639f9c02b4158ace065601406a493a8511180c02 |
|
MD5 | 82775ba7db29539474a3bb3deddd3cb3 |
|
BLAKE2b-256 | 003a40c40b78a7eb456837817bfa2c5bc442db59aefdf21c5ecb94700037813d |
Hashes for ml_dtypes-0.5.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a38df8df61194aeaae1ab7579075779b4ad32cd1cffd012c28be227fa7f2a70a |
|
MD5 | 8f365d00020fc5709a2e9333344ee72d |
|
BLAKE2b-256 | 6fd31321715a95e856d4ef4fba24e4351cf5e4c89d459ad132a8cba5fe257d72 |
Hashes for ml_dtypes-0.5.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a988bac6572630e1e9c2edd9b1277b4eefd1c86209e52b0d061b775ac33902ff |
|
MD5 | 0cb3c4089d4906a8d504b14927768eaf |
|
BLAKE2b-256 | 3175bf571247bb3dbea73aa33ccae57ce322b9688003cfee2f68d303ab7b987b |
Hashes for ml_dtypes-0.5.0-cp312-cp312-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d4b1a70a3e5219790d6b55b9507606fc4e02911d1497d16c18dd721eb7efe7d0 |
|
MD5 | 9d42d39539020fdec89f46479d2a0027 |
|
BLAKE2b-256 | 1cb7a067839f6e435785f34b09d96938dccb3a5d9502037de243cb84a2eb3f23 |
Hashes for ml_dtypes-0.5.0-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dc74fd9995513d33eac63d64e436240f5494ec74d522a9f0920194942fc3d2d7 |
|
MD5 | 677406d7ee253d78c1e1e830ab1e2ce6 |
|
BLAKE2b-256 | bf31058b9bcf9a81abd51623985add78711a915e4b0f6045baa5f9a0b41eb039 |
Hashes for ml_dtypes-0.5.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2e7534392682c3098bc7341648c650864207169c654aed83143d7a19c67ae06f |
|
MD5 | 9339d09dba861a6b1e4dc3beb2f5febd |
|
BLAKE2b-256 | 8965ffdbf3489b0ba2213674ea347fad3a11747be64d2d23d888f9e5abe80a18 |
Hashes for ml_dtypes-0.5.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 76942f6aeb5c40766d5ea62386daa4148e6a54322aaf5b53eae9e7553240222f |
|
MD5 | c079d58d78eb8622c8d52ab173fb1a85 |
|
BLAKE2b-256 | 199314896596644dad2e041ac5ca7237e6233c484f7defa186ff88b18ee6110b |
Hashes for ml_dtypes-0.5.0-cp311-cp311-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 60275f2b51b56834e840c4809fca840565f9bf8e9a73f6d8c94f5b5935701215 |
|
MD5 | 8c9ee5027b50cfc72c87a5bfb9ef794f |
|
BLAKE2b-256 | fe298968fd7ee026c0d04c553fb1ce1cd67f9da668cd567d62c0cdc995ce989e |
Hashes for ml_dtypes-0.5.0-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 968fede07d1f9b926a63df97d25ac656cac1a57ebd33701734eaf704bc55d8d8 |
|
MD5 | 1ceef88901ad5634e124c0dac5746cea |
|
BLAKE2b-256 | e60ca89f5c0fe9e48ed6e7e27d53e045711ee3d5b850bece5ee22fb0fb24b281 |
Hashes for ml_dtypes-0.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c7a9152f5876fef565516aa5dd1dccd6fc298a5891b2467973905103eb5c7856 |
|
MD5 | daed8ab11730fd7f2a197ef6650e5ef1 |
|
BLAKE2b-256 | 9a5bd47361f882ff2ae27d764f314d18706c69859da60a6c78e6c9e81714c792 |
Hashes for ml_dtypes-0.5.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ab046f2ff789b1f11b2491909682c5d089934835f9a760fafc180e47dcb676b8 |
|
MD5 | 12844268a0b7308917a5c173d50dd1c5 |
|
BLAKE2b-256 | 0bb195e7995f031bb3890884ddb22e331f24c49b0a4a8f6c448ff5984c86012e |
Hashes for ml_dtypes-0.5.0-cp310-cp310-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8c32138975797e681eb175996d64356bcfa124bdbb6a70460b9768c2b35a6fa4 |
|
MD5 | e5dd5eb72e9657d1ea9377b0fe339de3 |
|
BLAKE2b-256 | 83500a2048895a764b138638b5e7a62436545eb206948a5e6f77d9d5a4b02479 |
Hashes for ml_dtypes-0.5.0-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7ee9c320bb0f9ffdf9f6fa6a696ef2e005d1f66438d6f1c1457338e00a02e8cf |
|
MD5 | 0be572c461cf05d4e399913711c816b5 |
|
BLAKE2b-256 | a850f883464cbedaa78ebb332391b258513eaecd4f67a7972d0def669ea93e63 |
Hashes for ml_dtypes-0.5.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a03fc861b86cc586728e3d093ba37f0cc05e65330c3ebd7688e7bae8290f8859 |
|
MD5 | 536f7535782c3037d53dbf4db2c2fd38 |
|
BLAKE2b-256 | a86f49effaafbc24c7665bcea42cacb22e7198bbab5b473d908c5900c6bb6a59 |
Hashes for ml_dtypes-0.5.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 099e09edd54e676903b4538f3815b5ab96f5b119690514602d96bfdb67172cbe |
|
MD5 | 84d607293140c464c981e4b77dc303be |
|
BLAKE2b-256 | e6c4a21c68253584c678c98490894bf809e943829a380018c0fdd2e34288b07b |
Hashes for ml_dtypes-0.5.0-cp39-cp39-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5f2b59233a0dbb6a560b3137ed6125433289ccba2f8d9c3695a52423a369ed15 |
|
MD5 | 43df0029964519796361499ab4ba62c3 |
|
BLAKE2b-256 | ecc76e4018b7de2189b8264f8787b413aa7c9a914332ea0e8c7e1057936594cd |