About JS numerical floating point rounding error can cause problems in the

Symptoms:

  Results plus 0.1 to 0.2: 0.30000000000000004, as shown:

 

problem causes:

  We know that the computer can understand the binary rather than decimal, so we put 0.1 and 0.2 into a binary look:  

0.1 == "0.1.toString (2) == " 0.0001100110011 ( infinite loop ..) 
 0.2 == "0.2.toString (2) ==" 0.001100110011 (infinite loop ..)

  Fractional part of double precision floating point support up to 52, so the two get so 0.0100110011001100110011001100110011001100110011001100 a string of binary digits due to the limitations of floating-point decimal places and truncated after the addition, at this time, we then convert it to a decimal, it becomes .30000000000000004.

  One thing to be clear: This is the value of using the IEEE754 floating point calculations common problem, not only ECMAScript have this problem. But in C ++ / C # / Java these languages ​​have a good package of ways to avoid problems of accuracy, while JavaScript is a loosely typed language, there is no strict data types to float from design thinking, so accuracy the error problem is particularly prominent.

 

solution:

 A program: if accuracy requirements, can be treated with toFixed method (method toFixed numerical rounding operation, this method has great limitations)

was num1 = 0.1; 
was num2 = 0.2; 
alert (parseFloat ((num1 num2 +) .toFixed (2)) === 0.30);

 Scheme II: General Solution: the need to calculate the number multiplied by 10 to the power n, converted into an integer of the computer can accurately identify, and then divided by the n-th power of 10

Copy the code
formatNum = function(f, digit) { 
    var m = Math.pow(10, digit); 
    return parseInt(f * m, 10) / m; 
} 
var num1 = 0.1; 
var num2 = 0.2;
alert(Math.formatFloat(num1 + num2, 1) === 0.3)
Copy the code

Original link: https://www.cnblogs.com/lvmylife/p/9001256.html

Guess you like

Origin www.cnblogs.com/love-yu/p/10978416.html