1. Background
There is a requirement that the SQL statement be used as the key of the distributed cache.
But doing so will cause the key to be too long. Keys that are too long have some disadvantages.
But the problem of too long key can also be solved.
This article will describe in detail the shortcomings and solutions of the key being too long, and I hope it will be helpful to everyone.
2. Disadvantages of too long key
Generally speaking, SQL is too long as a Key, which has many disadvantages:
- If the key is too long, it will take up more memory space and reduce the cache efficiency and hit rate.
- If the key is too long, it will increase the overhead of network transmission and affect the response speed and throughput of the cache.
- If the key is too long, data fragmentation will be uneven, which will increase the load difference between cache servers and the cost of data migration.
- If the key is too long, it will increase the difficulty of cache management and maintenance, such as deleting, updating, monitoring and other operations.
Therefore, it is recommended to use reasonable key design specifications and avoid using too long or unnecessary prefixes, suffixes, separators, etc.
3. Solutions
3.1 Hash (to resolve conflicts)
You can use hash algorithms (such as MD5, SHA-1, etc.) to convert SQL statements into fixed-length strings as cache keys. This method can shorten the length of the key, reduce the cache space occupied, and also improve the search efficiency.
However, there may be conflicts in the hash algorithm, and attention should be paid to the uniqueness of the cache key.
How to solve the uniqueness problem caused by the conflict?
- Adding prefixes, such as different tenants, store IDs or Codes as Key prefixes, can greatly reduce the probability of conflicts.
- You can customize the structure of the value
(1) Store SQL in the Value object for secondary confirmation.
(2) If the original value is a single object, it can be defined as a collection.
When reading the value through the hash value as the key, compare whether the SQL stored in the current Sql and Value are consistent.
If it is consistent, it means that there is no hash conflict; if it is inconsistent, it means that there is a hash conflict, you can put the new value in the collection, and traverse the collection to get the corresponding value when reading.
The pseudo code is as follows:
public class Value{
String sql;
Object value;
}
public class CacheValueWrapper{
// 不冲突时存这里
Value value;
// 冲突时存这里
List<Value> values;
Value getValue(String sql){
// 先匹配 value
// 再匹配 values
}
}
3.2 Statement mapping (transformed into a unique value)
You can use the characteristics of the unique key of the table.
Create a new mapping table, including SQL string, business code (business ID).
| sql | id |
|---|---|
| select * from user where id=21 | 10086 |
| select name,age from student where id=22 | 10087 |
Business Code or business ID can use the feature of database self-increment or distributed ID generator.
SQL string set as unique key.
Query through SQL first, if used directly, if there is no such SQL value in the table, insert it.
Then the corresponding business code or business ID can be used as an important part of the cache key.
Cache Key: some_biz_prefix_10086
Four. Summary
This article provides some ideas to solve the problem that the key is too long, hoping to inspire students who encounter similar problems.
Creation is not easy. If this article is helpful to you, please like, bookmark and pay attention. Your support and encouragement are the biggest motivation for my creation.
Welcome to join my Knowledge Planet, Knowledge Planet ID: 15165241 to communicate and learn together.
https://t.zsxq.com/Z3bAiea is marked from CSDN when applying.
Welcome to our slack workspace where you can ask questions of ai and me.
https://join.slack.com/t/ai-yx51081/shared_invite/zt-1t8cp1lk3-ZMAFutZcN3PCW~8WQDGjPg