[FIX] tools: epsilon magnitude

The following rounding is incorrect:
```
>>> float_round(6.6 * 0.175, precision_digits=2)
1.15
```

Indeed, 6.6 * 0.175 = 1.155 ≈ 1.16.

In this specific case, the `epsilon` computed is not sufficient. A
precision of 53 gives:
normalized_value = 115.49999999999997
epsilon = 1.2823075934420547e-14
=> new normalized_value = 115.49999999999999

Bad luck, this is just not enough to tip the value in the right
direction.

However, a precision of 52 is sufficient:
normalized_value = 115.49999999999997
epsilon = 2.5646151868841094e-14
=> new normalized_value = 115.5

The value of 53 was chosen from the `binary64` number format precision.
In case of Python, the corresponding machine epsilon is 2^-52 [1].
Therefore, using 52 instead of 53 does make sense.

It is worth noting that the value of the machine epsilon
2^-52 = 2.2204460492503131e-16, which is still 2 orders of magnitude
below our dynamic estimation.

[1] https://en.wikipedia.org/wiki/Machine_epsilon
[2] `numpy.finfo(float).eps = 2.2204460492503131e-16`

opw-2047368

closes odoo/odoo#35521

Signed-off-by: Nicolas Martinelli (nim) <nim@odoo.com>
This commit is contained in:
Nicolas Martinelli
2019-08-07 08:25:53 +00:00
parent 71af1a710a
commit fde3b69656
3 changed files with 4 additions and 2 deletions
+1 -1
View File
@@ -309,7 +309,7 @@ var utils = {
}
var normalized_value = value / precision;
var epsilon_magnitude = Math.log(Math.abs(normalized_value))/Math.log(2);
var epsilon = Math.pow(2, epsilon_magnitude - 53);
var epsilon = Math.pow(2, epsilon_magnitude - 52);
normalized_value += normalized_value >= 0 ? epsilon : -epsilon;
/**
+2
View File
@@ -27,6 +27,8 @@ class TestFloatPrecision(TransactionCase):
try_round(0.0049,'0.00') # 0.0049 is closer to 0 than to 0.01, so should round down
try_round(0.005,'0.01') # the rule is to round half away from zero
try_round(-0.005,'-0.01') # the rule is to round half away from zero
try_round(6.6 * 0.175, '1.16') # 6.6 * 0.175 is rounded to 1.15 with epsilon = 53
try_round(-6.6 * 0.175, '-1.16')
def try_zero(amount, expected):
self.assertEqual(currency.is_zero(amount), expected,
+1 -1
View File
@@ -73,7 +73,7 @@ def float_round(value, precision_digits=None, precision_rounding=None, rounding_
normalized_value = value / rounding_factor # normalize
sign = math.copysign(1.0, normalized_value)
epsilon_magnitude = math.log(abs(normalized_value), 2)
epsilon = 2**(epsilon_magnitude-53)
epsilon = 2**(epsilon_magnitude-52)
# TIE-BREAKING: UP/DOWN (for ceiling[resp. flooring] operations)
# When rounding the value up[resp. down], we instead subtract[resp. add] the epsilon value