Mysql 5.6 "implicit conversion" caused index failure and inaccurate data

background

  • During an SQl query, I tried to remove the single quote query for the vachar type field in the where condition. At this time, I found that this sentence that should be fast was actually very slow. This varchar field has a composite index. The total number of items is 58,989, and even the data found without single quotes is not the data we want.
  • Using mysql 5.6 version, innoDB engine

The actual situation is as follows

  • Let's take a look at the results of the execution
    Insert picture description here

  • In the above description, we have to pay attention to that, the string of your where condition must be a full number without single quotes. Otherwise it will report an error
    Insert picture description here

  • It is also possible that the data found is not the data we want. As shown below

Insert picture description here

analysis

  1. Judging from the execution result, the corresponding index is gone with single quotation marks. Without using single quotation marks, there is no index, and a full table scan is performed.
  2. Why is this so? Why doesn't mysql optimizer directly perform type conversion?
  • The introduction of single quotes in SQL statements means that this type is the string data type CHAR, VARCHAR, BINARY, VARBINARY, BLOB, TEXT, ENUM, and SET. .
  • Without single quotes, it means that this is a type other than a string, such as int, bigDecimal, etc.
  • If single quotes are not added to a string with subtitles and special symbols, the consequence is that the type conversion fails and SQl cannot be executed. As described above:
1054 - Unknown column '000w1993521' in 'where clause', Time: 0.008000s
  1. Let's first look at the execution process of a SQL
    Insert picture description here

(Net map)

  • We first come to the conclusion: if you perform a function operation on the index field (in this case, the cast function does an implicit conversion), the order of the index value may be destroyed, so the optimizer decides to abandon the tree search function. (Https://dev.mysql.com/doc/refman/5.7/en/cast-functions.html)
    [External link image transfer failed, the source site may have an anti-leech link mechanism, it is recommended to save the image and upload it directly (img -l5AwT0xu-1607244327891)(http://note.youdao.com/yws/res/23689/CE6F785994E6476D816B23787CE65217)]
    means: Please note that if you use BINARY, CAST() or CONVERT() to convert index columns, MySQL The index may not be used effectively.
  • The detected data is inaccurate, but also because of implicit conversion, which results in different numeric types after conversion, resulting in inequality becoming equal.

Implicit conversion

1. Production conditions

  1. When the operator is used with operands of different types, type conversion occurs to make the operands compatible. Implicit conversion
  2. Conditions for implicit conversion:
    • When at least one of the two parameters is NULL, the result of the comparison is also NULL. The exception is when using <=> to compare two NULLs, it will return 1. In both cases, no type conversion is required.
    • Both parameters are strings, and will be compared according to strings without type conversion
    • Both parameters are integers and are compared according to integers without type conversion
    • When the hexadecimal value is compared with a non-number, it will be treated as a binary string
    • One parameter is TIMESTAMP or DATETIME, and the other parameter is a constant, the constant will be converted to timestamp
    • One parameter is of decimal type. If the other parameter is decimal or integer, the integer will be converted to decimal and then compared. If the other parameter is a floating point number, the decimal will be converted to floating point for comparison.
    • In all other cases, the two parameters will be converted to floating point numbers and then compared

2. Analyze the actual situation

  1. Then we will be clear. The example I mentioned above is a comparison of integers and strings, which belongs to other situations. Then let's first analyze the reasons for index failure
    1. Due to other cases of implicit conversion, the comparison value must be converted to a floating point number for comparison
    2. We first convert the query condition value to a floating point number, and then convert the record value of the table, so at this time, the index sorting that has been created before can no longer take effect. Because the implicit conversion (function) has changed the original value, it is said that the optimizer does not directly use the index here, and directly uses the full table scan.
  2. Query the unmatched value (or the partially matched value), as in the above query result. This really has to look at the source code, which is the implicit conversion rule of MYsql. I will not analyze it in detail here (because the relevant documents were not found).
    For historical reasons, it is necessary to be compatible with the old design. You can use MySQL's type conversion functions cast and convert to perform the conversion explicitly.

to sum up

  1. The use of implicit conversion and function will cause index failure and inaccurate data selected
  2. Conditions and rules for implicit conversion
  3. The specific reason for the index failure caused by the implicit conversion is that the comparison value needs to be type converted, which causes the failure.
  4. Avoid implicit type conversion. The types of implicit conversion mainly include inconsistent field types, in parameters contain multiple types, character set types, or inconsistent collation rules, etc.

reference

  • https://dev.mysql.com/doc/refman/5.7/en/type-conversion.html
  • https://xiaomi-info.github.io/2019/12/24/mysql-implicit-conversion/
  • https://zhuanlan.zhihu.com/p/95170837

Guess you like

Origin blog.csdn.net/weixin_40413961/article/details/110743406