js calculation accuracy problem

When doing calculations with js, the following problems often occur:

0.1 + 0.2 = 0.30000000000000004
0.3 - 0.2 = 0.09999999999999998
20.123/100 = 0.201230000000000000000002
2.425*100 = 2.424999999999997

IEEE 754 64-bit floating point type
IEEE 754

IEEE 754 specifies four ways to represent floating-point values: single precision (32 bits), double precision (64 bits), extended single precision (above 43 bits, rarely used) and extended double precision (79 bits) The above is usually implemented in 80 bits).

The full name of this standard is IEEE Binary Floating Point Arithmetic Standard (ANSI/IEEE Std 754-1985), also known as IEC 60559:1989, the binary floating point arithmetic of microprocessor systems (the original number is IEC 559:1989).
Single-precision floating
-point number The single -precision floating -point number format is a data type that occupies 4 bits (32 bits) in computer memory. Using the "floating point" (floating decimal point) method, it can represent a large range of values.

In the definition of IEEE 754-2008, the 32-bit base 2 format is officially called the binary32 format. This format is defined as single in IEEE 754-1985, that is, single precision. It should be noted that in some earlier computer systems, other 4-byte floating-point number formats also exist.

definition

The first digit indicates positive and negative, the middle 8 digits indicate the exponent, and the last 23 digits store the effective digits (the effective digit is 24 digits).

The middle eight bits can represent a total of 28=256 numbers, and the exponent can be two's complement; or 0 to 255, 0 to 126 for -127 to -1, 127 for zero, 128-255 for 1-128.

The leftmost 1 of the significant digits will not be stored because it must exist (the first significant digit of the binary must be 1). In other words, the effective digits are 24 bits, and 23 bits are actually stored.

js calculation accuracy problem
Double-precision floating
-point number Double -precision floating -point number (double) is a data type used by computers. Compared with single-precision floating-point numbers, double-precision floating-point numbers (double) use 64 bits (8 bytes) to store a floating-point number. It can represent 15 or 16 significant digits in the decimal system, and the absolute value range of the number it can represent is approximately [2.23e-308,1.79e308]

definition

Similar to single precision, the first digit represents positive and negative, the last 11 digits are exponent digits, and the last 52 digits represent precision (the effective digits are 53 digits).

js calculation accuracy problem

0.1/0.2 binary bits

(0.1).toString(2) === "0.000 110011001100110011001100110011001100110011001100110 1“

(0.2).toString(2) === "0.00 1100110011001100110011001100110011001100110011001101"

(0.30000000000000004).toString(2) === "0.0100110011001100110011001100110011001100110011001101"

(0.3).toString(2) === "0.010011001100110011001100110011001100110011001100110011"

So the final 0.1 in binary representation is 0.0001 1001 1001 1001..., but we look at the last six digits of (0.1).toString() above 001101, the normal cycle should be 001100, so after truncation, the value of 0.1 binary representation becomes larger ! ! ! . 0.2 converted to binary means that it also becomes larger after truncation.

By comparing the binary representation of 0.1, 0.2 and their sum, it can be found that the length of the string has changed, but the accuracy has not changed, that is, the length of the string from 1 to the end is 52.

0.1 + 0.2 should have a length of 57, but because it cannot represent such a number, counting from the number starting from 1 again, the last three numbers will be truncated (the final accuracy is 52 or 53)

After the addition of binary digits, 0.1+0.2 is
0.0100 1100 .......... 1100 111 total 57, and then 54 from the first 1, but the maximum is 53 digits, which will be truncated Becomes 52 or 53 and then becomes bigger

It should be noted that different decimal places save different digits at the end.

The specific solution is recommended to use the library to solve, and then to study the solution of the library.
A more general solution is to convert decimals into integers and then back to decimals to solve!
math.js bigNumber.js decimal.js number-precision can be used to solve

Guess you like

Origin blog.51cto.com/13934921/2569895