For some reason this afternoon, a bloom filter suddenly appeared in my mind. I've been a crawler for so long, and I haven't found an application scenario yet, so I just want to implement a play by myself. The principle is very simple. First define an n-long array, each of which is 0, perform k hashes when adding records, then use the hashed int % n as the index, and set the corresponding index bit to 1. Do the same operation to determine whether each bit is 1. As long as one bit is not 1, the record definitely does not exist. But if all 1s are all 1, it does not necessarily exist.
There are too many descriptions of the bloom filter principle on the Internet, directly link to https://blog.csdn.net/hguisu/article/details/7866173
According to the principle, a simple bloom filter that supports three working modes of redis, memory, and file is implemented.