Skip to content

Floating Point Number

December 2, 2023
March 18, 2023

Floating point number in computer is only an *approximation* of the actual value

IEEE 754-2008 revision - Wikiwand
Floating-point arithmetic - Wikiwand
Unit in the last place - Wikiwand the spacing between two consecutive floating-point numbers

Since the number of segments between the power of twos are limited by the number of bits of the mantissa, this space will increase as the numbers increases.

What Every Computer Scientist Should Know About Floating-Point Arithmetic
What Every Computer Scientist Should Know About Floating-Point Arithmetic
Tutorial: Floating-Point Binary
Binary representation of the floating-point numbers | Trekhleb

Fast Inverse Square Root

Fast inverse square root - Wikiwand
Fast Inverse Square Root — A Quake III Algorithm - YouTube explaining IEEE 754
REVEALED: Quake III's SECRET Algorithm! - YouTube
The Truth about the Fast Inverse Square Root on the N64 - YouTube when it is applicable (normalization) and performance improvement

Visualizers

Float Exposed
IEEE-754 Floating Point representation explained
IEEE-754 Floating Point Converter

Floating Point Numbers (Part1: Fp vs Fixed) - Computerphile - YouTube
Floating Point Numbers - Computerphile - YouTube

Single-precision and Double-precision

Single-precision floating-point format - Wikiwand

# protocol "Fraction:23,Exponent:8,S:1"
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                   Fraction                  |    Exponent   |S|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Sign (S)
Exponent: −126 to +127 (binary - 127)

https://wikimedia.org/api/rest_v1/media/math/render/svg/908c155d6002beadf2df5a7c05e954ec2373ca16
{\displaystyle {\text{value}}=(-1)^{\text{sign}}\times 2^{(E-127)}\times \left(1+\sum _{i=1}^{23}b_{23-i}2^{-i}\right).}

Double-precision floating-point format - Wikiwand

# protocol "Fraction:52,Exponent:11,S:1"
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                            Fraction                           |
+                                       +-+-+-+-+-+-+-+-+-+-+-+-+
|                                       |       Exponent      |S|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Sign (S)
Exponent: -1022 to +1023 (binary - 1023)

https://wikimedia.org/api/rest_v1/media/math/render/svg/5f677b27f52fcd521355049a560d53b5c01800e1
{\displaystyle (-1)^{\text{sign}}\left(1+\sum _{i=1}^{52}b_{52-i}2^{-i}\right)\times 2^{e-1023}}

Inherent inaccuracy

In most languages,

0.1 + 0.2 - 0.3 == 5.551115123125783e-17
0.1 + 0.2 == 0.30000000000000004

double(2^53 + 1) == 2^53 == 9007199254740992
biggy = pow(2, 53)
float(biggy+1) == float(biggy) + 1 == float(biggy) == biggy
biggy = BigInt(Math.pow(2, 53));
((Number(biggy + BigInt(1)) == Number(biggy) + 1) == Number(biggy)) == biggy;

Deciphering Glyph :: The Numbers, They Lie
In Python, why does 8.5 - 8.4 give 0.099999999999999964? - Quora
Why is 0.1+0.2 not equal to 0.3 in most programming languages? - Quora

.110011001100110011001101 (0.1, scaled, 24 bits rounded, exponent -3)
.100110011001100110011010 (sum result, scaled, 24 bits, exponent -1)

Python's int type will promote to bignum automatically
Python's float is double precision
JavaScript's number type will promote to double automatically
JavaScript has bigint primitive type

Comparing floats

Wait, so comparisons in floating point only just KINDA work? What DOES work? - YouTube

The Right Way to Compare Floats in Python | by David Amos
Comparing Floating Point Numbers, 2012 Edition | Random ASCII – tech blog of Bruce Dawson

def isclose(a, b, rel_tol=1e-09, abs_tol=0.0):
    return abs(a-b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)

How to Round Numbers in Python – Real Python

See math.isclose() in Python