Login
Community
DARK THEME

Minor innacuracies with math functions

There seem to be some minor errors with the math functions. For example, 1.1*100 = 110.00000000000001. This is extremely minor, but is starting to cause me some issues (and gets annoying when viewing variables in console).

I noticed these too. Usually they are close enough to the original number to get by, but sometimes they can be problems

It's just enough to cause my collision system to fall apart (objects going through walls and stopping in midair)

Google 'floating point precision', encountered this in a few programming languages. Normally I just ignore it.

There is also an article here giving examples how one could deal with them:
https://stackoverflow.com/questions/11695618/dealing-with-float-precision-in-javascript

As @TinkerSmith says, these inaccuracies are inherent to how floating point values are implemented on CPUs (with a limited number of binary digits, usually 64 for floating point values - of which 53 are used for the "significand"). You will actually find the same inaccuracies in many programming languages.

To explain why it is so, let's use the decimal system (base-10) and imagine you have a limited number of digits to store numbers. You can easily store 1/2 as 0.5 but how do you store 1/3? You will have to use an approximation fitting the number of digits you can use, e.g. 0.3333333333, which isn't exactly equal to 1/3.

The CPU has the exact same problem, not all numbers can be stored with perfect precision using a limited number of digits. The difference is that the CPU uses binary digits (base-2). Some numbers that have a finite representation in the decimal system (like 0.1 and 0.2) cannot be represented with a finite sequence of digits in binary (just like 1/3 cannot be represented with a finite sequence of digits in base-10). This is why this simple operation:

0.1+0.2

will give this approximate result:

0.30000000000000004

(you can test this in microStudio or in a JavaScript console)

Should you need to prevent these inaccuracies, one workaround could be:

Instead of 1.1*100, do 11*100/10
Instead of 0.1+0.2, do (1+2)/10

Post a reply

Progress

Status

Preview
Cancel
Post
Validate your e-mail address to participate in the community