Daily practice of 2023 Wang Zha interview questions - why there is a loss of accuracy

Why does precision loss occur in basic data types and how to avoid precision loss?

loat 32 bit

Reasons for accuracy loss:

The input value is decimal, and during the calculation process, the decimal decimal value must be converted into a binary decimal with limited digits, which will cause a loss of accuracy.

Conclusion: Binary decimals cannot express decimal decimals very accurately.

To avoid loss of accuracy:

a. You can use BigDecimal. Note: When performing calculations, convert to String type and use the methods provided by BigDecimal to implement calculations.

b. Convert the decimal places into integer digits for storage, such as using the Long type, such as money 3.99. The calculation of the unit value of money can be expressed by quantiles, that is, 399 cents.

Additional knowledge points: The underlying principle of BigDecimal is to expand the value into a Long type through intCompact (a Long type expanded by N times, such as: 199.99->19999) and scale (number of decimal places), and then perform further calculations.
 

Guess you like

Origin blog.csdn.net/weixin_42450130/article/details/132640718