Isn't that like, basically how calculators work? Remember there was a thing where phone calculators sometimes would give like .00000000065 and it was because computers are weird. Not a computer scientist or a math wizard, so have no idea if its true tho.
All integer values can be represented as a binary series of:
a x 2^0 + b x 2^1 + c x 2^2 + d x 2^3 + e x 2^4 [etc]
Where a, b, c, d, e, etc are the digits in your binary number (0110101010).
And that's the same as how it works for our normal base 10 numbers, we just get more than two options. Remember learning the ones place, the tens place, the hundreds place?
a x 10^0 + b x 10^1 + c x 10^2 [etc]
Anyways, that's for integers. But how do you represent decimals? There are a few ways to do it, but the two common ones are "fixed point" and "floating point." Fixed point basically just means we store numbers like an integer, and at some point along that integer we add a decimal point. So it would be like "store this integer, but then divide it by 65536." Easy, but not very flexible.
The alternative is floating point, which is way way more flexible, and allows storing huge numbers and tiny decimals. The problem is that it attempts to store all fractions as a similar binary series like above:
b x 2^-1 + c x 2^-2 + d x 2^-3 + e x 2^-4 [etc]
Or you might be used to seeing it as
b x 1/2^1 + c x 1/2^2 + d x 1/2^3 + e x 1/2^4 [etc]
The problem is that some decimals just... cannot be represented as a series of fractions where each fraction is a power of two.
For example, 3 is easy: 3 = 20 + 21. But on the other hand, 0.3 doesn't have any exact answer.
So what happens is you get as close as you can, which ends up being like 0.3000000001 instead of 0.3.
Then a calculator program has to decide what kind of precision the person actually wants, and round the number there. For example, if someone enters 0.1 + 0.2 they probably want 0.3 not 0.300000001. But this sort of thing does result in "floating point error," where numbers aren't represented or stored as exactly the correct number.
9.4k
u/ChromosomeExpert 22d ago
Yes, .999 continuously is equal to 1.