The basic idea of Bit-map is to use a bit to mark the Value corresponding to an element, and the Key is the element. As the unit of Bit is used to store data, the storage space can be greatly saved. (PS: focus on saving storage space )
Post a few basic concepts:
8 basic types occupy bytes in Java:
Integer type:
byte 1 byte
short 2 bytes
int 4 bytes
long 8 bytes
Floating point type:
float 4 bytes
double 8 bytes
Character type:
char 2 bytes
Boolean:
boolean 1 byte
One byte is equal to 8 bits: 1byte = 8bit
Bit operation:
In Java, the bottom layer of int data is stored in the form of complement. The int variable uses 32bit to store data, the highest bit is the sign bit, 0 means positive number, 1 means negative number, which can be Integer.toBinaryString()converted to bit string,
Output:
Shift left << For example: 5 << 2 = 20
On the premise that the number does not overflow, for both positive and negative numbers, shifting one bit to the left is equivalent to multiplying by the 1st power of 2, and shifting n bits to the left is equivalent to multiplying by the nth power of 2.
For example, convert 5 to binary representation: 0000 0000 0000 0000 0000 0000 0000 0101
Then after shifting to the left by 2 digits, add 0 to the low digit: 0000 0000 0000 0000 0000 0000 0001 0100 converted to decimal 20
In fact, as long as it is multiplied or divided by an integer, you can use the shift method to get the result, such as:
a=a*5
Analysis a9 can be split into a(4+1) that is a4+a1, so it can be changed to: a=(a<<2)+a
[Note] Since the +/- operator has a higher priority than the shift operator, you must remember to add parentheses when writing formulas, not a = a*12 is equivalent to a = a<<3 +a <<2 ; To be written as a = (a<<3)+(a <<2 ).
Shift right >> For example: 5 >> 2 = 1
On the premise that the number does not overflow, for positive and negative numbers, shifting one bit to the right is equivalent to dividing by 2, and shifting n bits to the right is equivalent to dividing by the nth power of 2.
Or first convert 5 to binary representation: 0000 0000 0000 0000 0000 0000 0000 0101
Then shift to the right by 2 digits, and add 0 to the high digits: 0000 0000 0000 0000 0000 0000 0000 0001 After being converted into decimal system, it is 1 (the extra part is cut directly )
Unsigned right shift >>>
5>>>3
We know that the int type occupies 32 bits in Java, which can represent a positive number or a negative number. The highest bit of a positive number converted to binary is 0, and the highest bit of a negative number is 1. For the addition operation of 2's complement code, it is the same as usual calculation, and the sign bit is also involved in the operation, but only 32 bits are retained in the end.
-5 is converted to binary: 1111 1111 1111 1111 1111 1111 1111 1011
-5 is shifted right by 3 digits: 1111 1111 1111 1111 1111 1111 1111 1111 // (Padded with 1, the result is -1, the high digit of positive number is filled with 0)
-5 Unsigned right shift by 3 bits: 0001 1111 1111 1111 1111 1111 1111 1111 // (Padded with 0, the result is 536870911)
Bit and &
The nth of the first operand is at the nth position of the second operand. If both are 1, then the nth of the result is also 1, otherwise it is 0
Bit or |
The nth bit of the first operand is located at the nth bit of the second operand. As long as one of them is 1, it is 1, otherwise it is 0
^ XOR
The nth of the first operand is located at the nth position of the second operand, as long as it is different ( different values, it is a 0 and a 1 ), it is 1, otherwise it is 0
Equivalent to finding the remainder, for example: 47^32 is equivalent to 48%32=15
Here comes the point:
Now there is such a demand: find out whether a certain number m exists among 2 billion random integers, and assume a 32-bit operating system, 4G memory
If each number is stored in int, that is 2 billion ints, so the occupied space is about (2000000000*4/1024/1024/1024)≈ 7.45 G
If bitwise storage is different, 2 billion is 2 billion bits, which takes up about (2000000000/8/1024/1024/1024)≈ 0.23 G
No need to say more
So, the question is, how to represent a number?
As I just said, each digit represents a number, 0 means it does not exist, and 1 means it exists, which is in line with binary
In this way, we can easily express the numbers {1,2,4,6}:
Steal a picture. Can't draw by myself
The smallest unit of computer memory allocation is byte, which is 8 bits. What if you want to represent {12,13,15}?
Of course it’s expressed on another 8-bit
How to determine which subscript of the int number is in the tmp array, this can actually be obtained by directly dividing by 32 to get the integer part, for example: the integer 8 divided by 32 is rounded to 0, then 8 is on tmp[0]. In addition, how do we know which of the 32 bits of 8 is in tmp[0]? In this case, directly mod 32 is ok, and like the integer 8, 32 is equal to the 8th mod in tmp[0] 8, then the integer 8 is in the eighth bit of tmp[0] (counting from the right).
Paste the bitmap source code
private long length;
private static int[] bitsMap;
private static final int[] BIT_VALUE = {0x00000001, 0x00000002, 0x00000004, 0x00000008, 0x00000010, 0x00000020,
0x00000040, 0x00000080, 0x00000100, 0x00000200, 0x00000400, 0x00000800, 0x00001000, 0x00002000, 0x00004000,
0x00008000, 0x00010000, 0x00020000, 0x00040000, 0x00080000, 0x00100000, 0x00200000, 0x00400000, 0x00800000,
0x01000000, 0x02000000, 0x04000000, 0x08000000, 0x10000000, 0x20000000, 0x40000000, 0x80000000};
public BitMap2(long length) { this.length = length; /** * Calculate according to the length, the required array size * When length%32=0, the size is equal to * = length/32 * When length%32>0, the size is equal to * = length/32+l */ bitsMap = new int[(int) (length >> 5) + ((length & 31)> 0? 1: 0)]; }
/**
* @param n The value to be set is n
*/
public void setN(long n) { if (n <0 || n> length) { throw new IllegalArgumentException("length value "+n+" is illegal! "); } // Find the subscript of the bitMap where n is, equivalent to "n/5" int index = (int) n>>5; // Find the offset of the value (remainder), Equivalent to "n%31" int offset = (int) n & 31; /** * Equivalent to * int bits = bitsMap[index]; * bitsMap[index]=bits| BIT_VALUE[offset]; * For example, When n=3, set the fourth position of byte to 1 (counting from 0, the number represented by bitsMap[0] is: 0~31, each bit from left to right represents a number)
* bitsMap[0]=00000000 00000000 00000000 00000000 | 00000000 00000000 00000000 00001000=00000000 00000000 00000000 00000000 00001000
* That is: bitsMap[0] = 0 | 0x00000008 = 3
*
* For example, when n=4, set the fifth position of byte as 1
* bitsMap[0]=00000000 00000000 00000000 00001000 | 00000000 00000000 00000000 00010000=00000000 00000000 00000000 00000000
00011000 * That is: bitsMap[0]=3 | 0x00000010 = 12
*/
bitsMap[index] |= BIT_VALUE[offset];
}
/**
* 获取值N是否存在
* @return 1：存在，0：不存在
*/
public int isExist(long n) {
if (n < 0 || n > length) {
throw new IllegalArgumentException("length value illegal!");
}
int index = (int) n>>5;
int offset = (int) n & 31;
int bits = (int) bitsMap[index];
// System.out.println("n="+n+",index="+index+",offset="+offset+",bits="+Integer.toBinaryString(bitsMap[index]));
return ((bits & BIT_VALUE[offset])) >>> offset;
}
Add to
Here is a question, how do we put a number in it? For example, if you want to put the number 5 in, how do you do it?
First, 5/32=0, 5%32=5, which means that it should be in the 5th position of tmp[0], then we move 1 to the left by 5 places, and then bitwise OR
To binary is
This is equivalent to 86 | 32 = 118
86 | (1<<5) = 118
b[0] = b[0] | (1<<5)
In other words, if you want to insert a number, move 1 to the left with the one that represents the number, and then perform bitwise OR operation with the original number
To simplify, it is 86 + (5/8) | (1<<(5%8))
Therefore, the formula can be summarized as: p + (i/8)|(1<<(i%8)) where p represents the current value and i represents the number to be inserted
Clear
The above is an addition, what should I do if I want to clear it?
Or the above example, suppose we want to remove 6, what should we do?
From the picture, only the position of the number is 0
Shift 1 to the left by 6 digits to reach the digit represented by the number 6, then reverse it by bit, and finally and the original number by bitwise AND, so that the position is 0.
b[0] = b[0] & (~(1<<6))
b[0] = b[0] & (~(1<<(i%8)))、
Above code:
public class BitMap { // private byte[] bits; //how much data can be stored private int capacity; public BitMap(int capacity){ this.capacity = capacity; //1bit can store 8 data, then capacity data How many bits are needed, capacity/8+1, right shifting by 3 bits is equivalent to dividing by 8 bits = new byte[(capacity >>3 )+1]; } public void add(int num){ // num/8 Get the index of byte[] int arrayIndex = num >> 3; // num%8 gets the position of byte[index] int position = num & 0x07; // After shifting 1 to the left position, that position is naturally 1, then Do | with the previous data, so that position is replaced with 1. bits[arrayIndex] |= 1 << position; } public boolean contain(int num){
// num/8 gets the index of byte[]
int arrayIndex = num >> 3;
// num%8 gets the position of byte[index]
int position = num & 0x07;
//After shifting 1 to the left position, that position Naturally it is 1, and then do & with the previous data to determine whether it is 0.
Return (bits[arrayIndex] & (1 << position)) !=0;
}
public void clear(int num){ // num/8 Get the index of byte[] int arrayIndex = num >> 3; // num%8 gets the position of byte[index] int position = num & 0x07; // After shifting 1 to the left position, that position is naturally 1, then Take the opposite and do & with the current value to clear the current position. bits[arrayIndex] &= ~(1 << position);
}
public static void main(String[] args) { BitMap bitmap = new BitMap(100); bitmap.add(7); System.out.println("Insert 7 success"); boolean isexsit = bitmap.contain(7) ; System.out.println("Does 7 exist:"+isexsit); bitmap.clear(7); isexsit = bitmap.contain(7); System.out.println("Does 7 exist:"+isexsit); } }
Find
We also said earlier that each digit represents a number, 1 means there is (or exists), and 0 means nothing (or does not exist). By setting the value to 1 or 0 to achieve the addition and removal of the guy, then judging whether a number exists or not is judging whether the bit of the number is 0 or 1.
Suppose, we want to know whether 3 is present, then we only need to judge b[0] & (1<<3) If this value is 0, it does not exist, if it is 1, it means there is
Application scenarios:
1: Look at a small scene> Find out the non-repeating integers among 300 million integers, and limit the memory to be insufficient to hold 300 million integers.
For this kind of scenario, I can use 2-BitMap to solve it, that is, allocate 2 bits for each integer, and use different combinations of 0 and 1 to identify special meanings. For example, 00 means that the integer has not appeared, 01 means once, and 11 means it appears. After many times, you can find the repeated integers. The memory space required is twice the normal BitMap, which is: 300 million*2/8/1024/1024=71.5MB.
The specific process is as follows:
Scanning 300 million integers, group BitMap, first check the corresponding position in BitMap, if 00 becomes 01, if 01 becomes 11, if it is 11, it remains unchanged. After scanning 300 million integers, that is Said that the entire BitMap has been assembled. Finally, check BitMap and output the integer corresponding to the bit 11.
For example: the number 0 occupies the position of index [0] and [1]
2: It is known that a certain file contains some telephone numbers, each number is 8 digits, and the number of different numbers is counted.
8 bits up to 99 999 999, about 99m bits, about 10 m bytes of memory is enough. (It can be understood as a number from 0-99 999 999, each number corresponds to a Bit bit, so only 99M Bit==1.2MBytes are needed. In this way, a small 1.2M memory is used to represent all 8 Digit phone)