{"course":{"slug":"python-for-ai","title":"Python, from Zero, for AI","url":"https://addaly.com/learn/python-for-ai","level":"beginner","track":"build"},"lesson":{"slug":"dtypes-and-precision","title":"dtypes: what float16 costs, why int8 wraps round, and how to compare floats","url":"https://addaly.com/learn/python-for-ai/dtypes-and-precision","position":72,"of":89,"minutes":9,"answer":"The dtype fixes bytes per element and therefore memory, range and precision; float16 holds three significant figures and overflows at 65,504, integers wrap silently at their limit.","keyIdea":"The dtype fixes bytes per element and therefore memory, range and precision; float16 holds three significant figures and overflows at 65,504, integers wrap silently at their limit, and two floats should be compared with isclose rather than ==."},"body":{"format":"markdown","text":"## Bytes per element\n\nEvery array has a `dtype`, and the dtype decides three things at once: how much memory each element takes, the largest value it can hold, and how many significant figures it keeps.\n\n```python\nnp.zeros(1_000_000, dtype=np.float64).nbytes   # 8,000,000\nnp.zeros(1_000_000, dtype=np.float32).nbytes   # 4,000,000\nnp.zeros(1_000_000, dtype=np.float16).nbytes   # 2,000,000\nnp.zeros(1_000_000, dtype=np.int8).nbytes      # 1,000,000\n```\n\nA million float64 values is 8 MB. A model with 7 billion parameters stored as float16 is 14 GB; as float32, 28 GB; as int8 after quantisation, 7 GB. That arithmetic (parameters times bytes per parameter) is the single most useful calculation in deciding whether a model fits on a machine, and it is nothing more than `nbytes`.\n\n> Figure: One hundred thousand embeddings of 1,536 dimensions. Bars (megabytes): float64 1229; float32 614; float16 307; int8, quantised 154. Same vectors, four dtypes, exact arithmetic: 153.6 million numbers times the bytes each takes. float32 is the working standard for embeddings and costs nothing you can measure in a ranking; float16 halves it again and keeps about three significant figures; int8 needs a scale factor and a reason.\n\n## Floats: range and precision\n\n- **float64**: about 15 to 16 significant decimal figures, largest value around 1.8 × 10³⁰⁸. Python's own `float` is this. The default for most NumPy operations.\n- **float32**: about 7 significant figures, largest around 3.4 × 10³⁸. The standard for model weights in training and for most GPU arithmetic. Half the memory of float64 and rarely a problem for anything short of accumulating a long sum.\n- **float16**: about 3 significant figures, largest **65,504**. Half the memory again. Used for storing weights and for inference on hardware that supports it.\n- **bfloat16**: same 16 bits, but spent differently: the range of float32 with only about 2 to 3 significant figures. Used in training because the range matters more than the precision there. NumPy does not ship it natively; `ml_dtypes` adds it.\n\nThe practical consequence of float16's ceiling:\n\n```python\nx = np.array([12.0], dtype=np.float16)\nnp.exp(x)         # [inf]  — e^12 ≈ 162,755, above 65,504\n```\n\nA softmax over logits that reach 12 produces `inf`, and `inf / inf` is `nan`, and one `nan` poisons every later sum. This is why libraries compute softmax by subtracting the maximum logit first (every exponent is then at most `e⁰ = 1`) and why you see `-inf` and `nan` in a log the day someone stores intermediate values in half precision to save memory. Store in float16 if you must; compute in float32.\n\nPrecision has the same shape of consequence in the other direction:\n\n```python\nnp.float16(1000) + np.float16(0.2)      # 1000.0 — the 0.2 is lost\nnp.float32(1e8) + np.float32(1)         # 100000000.0 — the 1 is lost\n```\n\nThree significant figures means a value near 1,000 cannot represent a change of 0.5. Summing ten thousand small values into a float16 accumulator loses most of them. NumPy guards against the worst case by accumulating reductions in a wider type, but any arithmetic *you* write elementwise in float16 has no such guard.\n\n## Integers wrap\n\n```python\nnp.array([127], dtype=np.int8) + 1     # [-128]\nnp.array([255], dtype=np.uint8) + 1    # [0]\n```\n\nAn integer dtype has a fixed range (`int8` is −128 to 127, `uint8` is 0 to 255, `int64` is ±9.2 × 10¹⁸), and arithmetic that leaves it **wraps round** without a warning in most configurations. Image pixels are `uint8`; add brightness to a pixel at 250 and it becomes dark. Token IDs fit in `int32`; counts of tokens across a large corpus may not fit in `int32` and will go negative. Python's own `int` never does this, which is why the surprise lands on people arriving from plain Python.\n\n`np.iinfo(np.int8)` and `np.finfo(np.float16)` print the limits for any type. When a number in an array looks impossible, check them.\n\n## Conversion\n\n```python\na = np.array([1.7, 2.2, -0.5])\na.astype(np.int32)          # [1, 2, 0]: truncates toward zero, does not round\nnp.round(a).astype(int)     # [2, 2, -0]: round first if that is what you meant\n```\n\n`astype` makes a copy in the new type. Float to int truncates; it does not round. Int to a smaller int wraps. Anything to `bool` gives `False` for zero and `True` otherwise.\n\nMixed-type arithmetic promotes to the wider type: `int32 + float32` is `float32`, `float32 + float64` is `float64`. A single float64 scalar in a float32 expression can quietly upcast the whole array and double its memory. Check `.dtype` on the result when memory matters.\n\n## Comparing floats\n\n```python\n0.1 + 0.2 == 0.3              # False\nnp.isclose(0.1 + 0.2, 0.3)    # True\n```\n\nThis is the same fact as module 1's decimal surprise, in array form. Never compare computed floats with `==`. `np.isclose(a, b)` compares with a tolerance (relative `1e-5` and absolute `1e-8` by default) and `np.allclose` does it for whole arrays. In tests, `pytest.approx` or `numpy.testing.assert_allclose` are the tools. Two embeddings computed on different machines may differ in the last digit; a test that asserts equality is a test that fails on Tuesday.\n\n## Choosing\n\nUse float64 for anything small enough not to care and for statistics where accumulated error matters. Use float32 for model inputs, embeddings and anything that will meet a GPU. Use float16 for storage and for inference where the hardware and the library both promise to handle the overflow cases. Use the smallest integer type that certainly holds the range, and check the range rather than assuming it.\n\n## Try this now\n\nCompute `np.exp` of `np.arange(0, 20, dtype=np.float16)` and find the first `inf`. Then softmax those same logits in float32 with and without subtracting the max. Finally, make a `uint8` array of `[250, 251, 252]`, add 10, and look."},"check":{"question":"A script stores model logits as `np.float16` to halve memory, then computes `np.exp(logits).sum()` for a softmax. For some inputs the sum comes back as `inf`. What is the mechanism?","options":[{"text":"`np.exp` is not implemented for half-precision and silently promotes to an integer type, which overflows.","correct":false,"tempting":"`exp` is implemented for float16 and stays in float16; the overflow is a property of the type's range, not a missing implementation."},{"text":"float16 cannot represent zero exactly, so the sum of many small terms accumulates a rounding error that grows without bound.","correct":false,"tempting":"Zero is exactly representable in every float type; the failure is at the large end, not the small."},{"text":"The `.sum()` reduction on float16 uses a 16-bit accumulator that overflows after a few thousand elements regardless of their values, which is why only some inputs trigger it.","correct":false,"tempting":"NumPy accumulates float16 sums in a wider type internally; the `inf` is produced by `exp` before the sum runs."},{"text":"float16 tops out at 65,504, so `exp` of any logit above about 11 becomes `inf`, and one `inf` makes the sum `inf`.","correct":true}]},"prev":{"slug":"shapes-and-broadcasting","title":"Shapes, axes and broadcasting: the wrong-shape bug that runs without complaint","url":"https://addaly.com/learn/python-for-ai/shapes-and-broadcasting"},"next":{"slug":"masks-views-and-copies","title":"Masks, where, and the slice that is not a copy","url":"https://addaly.com/learn/python-for-ai/masks-views-and-copies"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Source and assessment review continues. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'dtypes: what float16 costs, why int8 wraps round, and how to compare floats', in Python, from Zero, for AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/python-for-ai/dtypes-and-precision (Accessed: 21 November 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"dtypes: what float16 costs, why int8 wraps round, and how to compare floats","courseTitle":"Python, from Zero, for AI","module":{"index":8,"slug":"numbers-in-bulk","title":"Numbers in bulk: NumPy, pandas and the vector under every model"},"answer":null,"keyIdea":"The dtype fixes bytes per element and therefore memory, range and precision; float16 holds three significant figures and overflows at 65,504, integers wrap silently at their limit, and two floats should be compared with isclose rather than ==.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"caption":"Same vectors, four dtypes, exact arithmetic: 153.6 million numbers times the bytes each takes. float32 is the working standard for embeddings and costs nothing you can measure in a ranking; float16 halves it again and keeps about three significant figures; int8 needs a scale factor and a reason.","series":[{"label":"float64","value":1229},{"highlight":true,"label":"float32","value":614},{"label":"float16","value":307},{"label":"int8, quantised","value":154}],"title":"One hundred thousand embeddings of 1,536 dimensions","type":"bars","unit":"megabytes"},"alt":"Figure: One hundred thousand embeddings of 1,536 dimensions. Bars (megabytes): float64 1229; float32 614; float16 307; int8, quantised 154. Same vectors, four dtypes, exact arithmetic: 153.6 million numbers times the bytes each takes. float32 is the working standard for embeddings and costs nothing you can measure in a ranking; float16 halves it again and keeps about three significant figures; int8 needs a scale factor and a reason."}},"blocks":[{"t":"h2","id":"bytes-per-element","runs":[{"text":"Bytes per element"}]},{"t":"p","runs":[{"text":"Every array has a "},{"text":"dtype","code":true},{"text":", and the dtype decides three things at once: how much memory each element takes, the largest value it can hold, and how many significant figures it keeps."}]},{"t":"code","lang":"python","label":"Python","text":"np.zeros(1_000_000, dtype=np.float64).nbytes   # 8,000,000\nnp.zeros(1_000_000, dtype=np.float32).nbytes   # 4,000,000\nnp.zeros(1_000_000, dtype=np.float16).nbytes   # 2,000,000\nnp.zeros(1_000_000, dtype=np.int8).nbytes      # 1,000,000"},{"t":"p","runs":[{"text":"A million float64 values is 8 MB. A model with 7 billion parameters stored as float16 is 14 GB; as float32, 28 GB; as int8 after quantisation, 7 GB. That arithmetic (parameters times bytes per parameter) is the single most useful calculation in deciding whether a model fits on a machine, and it is nothing more than "},{"text":"nbytes","code":true},{"text":"."}]},{"t":"h2","id":"floats-range-and-precision","runs":[{"text":"Floats: range and precision"}]},{"t":"ul","items":[[{"text":"float64","b":true},{"text":": about 15 to 16 significant decimal figures, largest value around 1.8 × 10³⁰⁸. Python’s own "},{"text":"float","code":true},{"text":" is this. The default for most NumPy operations."}],[{"text":"float32","b":true},{"text":": about 7 significant figures, largest around 3.4 × 10³⁸. The standard for model weights in training and for most GPU arithmetic. Half the memory of float64 and rarely a problem for anything short of accumulating a long sum."}],[{"text":"float16","b":true},{"text":": about 3 significant figures, largest "},{"text":"65,504","b":true},{"text":". Half the memory again. Used for storing weights and for inference on hardware that supports it."}],[{"text":"bfloat16","b":true},{"text":": same 16 bits, but spent differently: the range of float32 with only about 2 to 3 significant figures. Used in training because the range matters more than the precision there. NumPy does not ship it natively; "},{"text":"ml_dtypes","code":true},{"text":" adds it."}]]},{"t":"p","runs":[{"text":"The practical consequence of float16’s ceiling:"}]},{"t":"code","lang":"python","label":"Python","text":"x = np.array([12.0], dtype=np.float16)\nnp.exp(x)         # [inf]  — e^12 ≈ 162,755, above 65,504"},{"t":"p","runs":[{"text":"A softmax over logits that reach 12 produces "},{"text":"inf","code":true},{"text":", and "},{"text":"inf / inf","code":true},{"text":" is "},{"text":"nan","code":true},{"text":", and one "},{"text":"nan","code":true},{"text":" poisons every later sum. This is why libraries compute softmax by subtracting the maximum logit first (every exponent is then at most "},{"text":"e⁰ = 1","code":true},{"text":") and why you see "},{"text":"-inf","code":true},{"text":" and "},{"text":"nan","code":true},{"text":" in a log the day someone stores intermediate values in half precision to save memory. Store in float16 if you must; compute in float32."}]},{"t":"p","runs":[{"text":"Precision has the same shape of consequence in the other direction:"}]},{"t":"code","lang":"python","label":"Python","text":"np.float16(1000) + np.float16(0.2)      # 1000.0 — the 0.2 is lost\nnp.float32(1e8) + np.float32(1)         # 100000000.0 — the 1 is lost"},{"t":"p","runs":[{"text":"Three significant figures means a value near 1,000 cannot represent a change of 0.5. Summing ten thousand small values into a float16 accumulator loses most of them. NumPy guards against the worst case by accumulating reductions in a wider type, but any arithmetic "},{"text":"you","i":true},{"text":" write elementwise in float16 has no such guard."}]},{"t":"h2","id":"integers-wrap","runs":[{"text":"Integers wrap"}]},{"t":"code","lang":"python","label":"Python","text":"np.array([127], dtype=np.int8) + 1     # [-128]\nnp.array([255], dtype=np.uint8) + 1    # [0]"},{"t":"p","runs":[{"text":"An integer dtype has a fixed range ("},{"text":"int8","code":true},{"text":" is −128 to 127, "},{"text":"uint8","code":true},{"text":" is 0 to 255, "},{"text":"int64","code":true},{"text":" is ±9.2 × 10¹⁸), and arithmetic that leaves it "},{"text":"wraps round","b":true},{"text":" without a warning in most configurations. Image pixels are "},{"text":"uint8","code":true},{"text":"; add brightness to a pixel at 250 and it becomes dark. Token IDs fit in "},{"text":"int32","code":true},{"text":"; counts of tokens across a large corpus may not fit in "},{"text":"int32","code":true},{"text":" and will go negative. Python’s own "},{"text":"int","code":true},{"text":" never does this, which is why the surprise lands on people arriving from plain Python."}]},{"t":"p","runs":[{"text":"np.iinfo(np.int8)","code":true},{"text":" and "},{"text":"np.finfo(np.float16)","code":true},{"text":" print the limits for any type. When a number in an array looks impossible, check them."}]},{"t":"h2","id":"conversion","runs":[{"text":"Conversion"}]},{"t":"code","lang":"python","label":"Python","text":"a = np.array([1.7, 2.2, -0.5])\na.astype(np.int32)          # [1, 2, 0]: truncates toward zero, does not round\nnp.round(a).astype(int)     # [2, 2, -0]: round first if that is what you meant"},{"t":"p","runs":[{"text":"astype","code":true},{"text":" makes a copy in the new type. Float to int truncates; it does not round. Int to a smaller int wraps. Anything to "},{"text":"bool","code":true},{"text":" gives "},{"text":"False","code":true},{"text":" for zero and "},{"text":"True","code":true},{"text":" otherwise."}]},{"t":"p","runs":[{"text":"Mixed-type arithmetic promotes to the wider type: "},{"text":"int32 + float32","code":true},{"text":" is "},{"text":"float32","code":true},{"text":", "},{"text":"float32 + float64","code":true},{"text":" is "},{"text":"float64","code":true},{"text":". A single float64 scalar in a float32 expression can quietly upcast the whole array and double its memory. Check "},{"text":".dtype","code":true},{"text":" on the result when memory matters."}]},{"t":"h2","id":"comparing-floats","runs":[{"text":"Comparing floats"}]},{"t":"code","lang":"python","label":"Python","text":"0.1 + 0.2 == 0.3              # False\nnp.isclose(0.1 + 0.2, 0.3)    # True"},{"t":"p","runs":[{"text":"This is the same fact as module 1’s decimal surprise, in array form. Never compare computed floats with "},{"text":"==","code":true},{"text":". "},{"text":"np.isclose(a, b)","code":true},{"text":" compares with a tolerance (relative "},{"text":"1e-5","code":true},{"text":" and absolute "},{"text":"1e-8","code":true},{"text":" by default) and "},{"text":"np.allclose","code":true},{"text":" does it for whole arrays. In tests, "},{"text":"pytest.approx","code":true},{"text":" or "},{"text":"numpy.testing.assert_allclose","code":true},{"text":" are the tools. Two embeddings computed on different machines may differ in the last digit; a test that asserts equality is a test that fails on Tuesday."}]},{"t":"h2","id":"choosing","runs":[{"text":"Choosing"}]},{"t":"p","runs":[{"text":"Use float64 for anything small enough not to care and for statistics where accumulated error matters. Use float32 for model inputs, embeddings and anything that will meet a GPU. Use float16 for storage and for inference where the hardware and the library both promise to handle the overflow cases. Use the smallest integer type that certainly holds the range, and check the range rather than assuming it."}]},{"t":"h2","id":"try-this-now","runs":[{"text":"Try this now"}],"role":"try"},{"t":"p","runs":[{"text":"Compute "},{"text":"np.exp","code":true},{"text":" of "},{"text":"np.arange(0, 20, dtype=np.float16)","code":true},{"text":" and find the first "},{"text":"inf","code":true},{"text":". Then softmax those same logits in float32 with and without subtracting the max. Finally, make a "},{"text":"uint8","code":true},{"text":" array of "},{"text":"[250, 251, 252]","code":true},{"text":", add 10, and look."}]}],"check":{"question":[{"text":"A script stores model logits as "},{"text":"np.float16","code":true},{"text":" to halve memory, then computes "},{"text":"np.exp(logits).sum()","code":true},{"text":" for a softmax. For some inputs the sum comes back as "},{"text":"inf","code":true},{"text":". What is the mechanism?"}],"options":[{"text":[{"text":"np.exp","code":true},{"text":" is not implemented for half-precision and silently promotes to an integer type, which overflows."}],"correct":false,"tempting":[{"text":"exp","code":true},{"text":" is implemented for float16 and stays in float16; the overflow is a property of the type’s range, not a missing implementation."}]},{"text":[{"text":"float16 cannot represent zero exactly, so the sum of many small terms accumulates a rounding error that grows without bound."}],"correct":false,"tempting":[{"text":"Zero is exactly representable in every float type; the failure is at the large end, not the small."}]},{"text":[{"text":"The "},{"text":".sum()","code":true},{"text":" reduction on float16 uses a 16-bit accumulator that overflows after a few thousand elements regardless of their values, which is why only some inputs trigger it."}],"correct":false,"tempting":[{"text":"NumPy accumulates float16 sums in a wider type internally; the "},{"text":"inf","code":true},{"text":" is produced by "},{"text":"exp","code":true},{"text":" before the sum runs."}]},{"text":[{"text":"float16 tops out at 65,504, so "},{"text":"exp","code":true},{"text":" of any logit above about 11 becomes "},{"text":"inf","code":true},{"text":", and one "},{"text":"inf","code":true},{"text":" makes the sum "},{"text":"inf","code":true},{"text":"."}],"correct":true,"tempting":null}],"order":[2,1,3,0],"why":null},"prev":{"slug":"shapes-and-broadcasting","title":"Shapes, axes and broadcasting: the wrong-shape bug that runs without complaint","minutes":9},"next":{"slug":"masks-views-and-copies","title":"Masks, where, and the slice that is not a copy","minutes":8}}}