1. The concept of CRC
1. What is CRC?
In the previous article, through the metaphor of a small example, I briefly introduced what CRC is, and I will explain it in detail again here.
CRC (Cyclic Redundancy Checksum) is an error correction technology that stands for Cyclic Redundancy Checksum.
One of the most commonly used error checking codes in the field of data communication, the length of the information field and the check field can be specified arbitrarily, but the CRC standards defined by both communication parties are required to be consistent. It is mainly used to detect or verify errors that may occur after data transmission or storage. The way it is used can be illustrated as shown in the following diagram:
In the process of data transmission, no matter how perfect the design of the transmission system is, there will always be errors, which may cause one or more frames transmitted on the link to be destroyed (bit errors occur, 0 becomes 1, or 1 becomes 0), so that the receiver receives wrong data.
In order to improve the accuracy of the data received by the receiver as much as possible, it is necessary to perform error detection on the data before the receiver receives the data, and the receiver will actually accept the data only when the detection result is correct. There are many detection methods, common ones are parity check , Internet check and cyclic redundancy check .
2. Overview of usage
Cyclic redundancy check is a calculation method used to verify the accuracy of digital transmission on communication links (the agreed relationship between data bits and check bits is established through certain mathematical operations).
The sending computer uses a certain formula to calculate a value of the information contained in the transmitted data, and attaches this value to the transmitted data, and the receiving computer performs the same calculation on the same data, and should get the same result.
If the two CRC results are inconsistent, it means that an error occurred in the transmission, and the receiver computer can ask the sender computer to resend the data.
3. Wide range of applications
Among many error detection methods, CRC is the most famous one. The full name of CRC is Cyclic Redundancy Check, which is characterized by strong error detection capability, low overhead, and easy implementation with encoders and detection circuits. From the perspective of its error detection ability, the probability of errors that it cannot find is only below 0.0047%.
In terms of performance and overhead, it is far superior to parity check and arithmetic sum check.
Therefore, in the field of data storage and data communication, CRC is ubiquitous: the FCS (frame error detection sequence) of the famous communication protocol X.25 adopts CRC-CCITT, and the compression tool software such as WinRAR, NERO, ARJ, and LHA adopts It is CRC32, the read and write of the disk drive adopts CRC16, and the common image storage formats GIF, TIFF, etc. also use CRC as an error detection method.
2. Definition of CRC name
Here you need to know several components or calculation concepts: polynomial formula, polynomial abbreviation, data width, initial value, result XOR value, input value inversion, output value inversion, parameter model.
1. Polynomial formula
For the CRC standard divisor, it is generally represented by a polynomial (or binomial) formula. The binomial of the divisor 11011 (poly value 0x1b) in the figure below is G(X)=X4+X3+X+1, and the index of X is It means that the data on this bit is 1, (the lowest bit is 0).
Pay special attention to the number of digits here. The number of digits in the divisor is the highest power of the binomial + 1 (4+1=5), which is very important.
2. Polynomial abbreviation
Through the basic understanding of CRC, we know that the beginning and end of the polynomial must be 1, and the position of this 1 must be 0 in the next calculation, so the previous 1 is omitted, and a thing called abbreviation appears, as above The abbreviation of the divisor 11011 in the example is 1011. Many people who have read the CRC high-level language source code will know that the poly value of G(X)=X16+X15+X2+1 (16#18005) under the CRC_16 standard is actually 8005, the abbreviation is used here. This usage will be explained later.
3. Data width
The data width refers to the length of the CRC check code (binary digits). Knowing the concept of CRC operations and polynomials, you can understand this concept. The length of the CRC is always 1 less than the number of digits of the divisor. The length of the shorthand formula is consistent.
The above three data are the basic data that we can often use
4. Initial value and result XOR value
In some standards, the initial value is stipulated. Before the data is subjected to the above binomial operation, the data to be calculated needs to be XORed with the lowest byte of the initial value, and then calculated with the polynomial.
In the case that the result XOR value is not zero, it is necessary to perform an XOR calculation on the calculated CRC result value and the result XOR value, and the final value obtained is the CRC check code we need.
It can be seen here that the number of digits required for the initial value and the result value is consistent with the data width.
5. Input value inversion and output value inversion
Reversing the input values means inverting the binomial before the calculation, and then performing the calculation with the resulting new values and data. For example, for G(X)=X16+X15+X2+1 (16#18005), the forward value is 1 1000 0000 0000 0101, and the reverse value is 1010 0000 0000 0001 1
Output value inversion is to invert the final CRC result.
Usually, the result value after the input value is reversed will also be reversed, so these two options are generally in the same direction. We can only see the free choice of forward and reverse in the online CRC calculator.
Three, common CRC algorithm
Although CRC can define binomial, data length, etc. arbitrarily, without a unified standard, it will make the whole calculation very troublesome. But in fact, different manufacturers often use different standard algorithms. Here are some internationally commonly used model tables:
| name | polynomial | Notation | Application examples |
|---|---|---|---|
| CRC-8 | X8+X2+X+1 | 0X107 | |
| CRC-12 | X12+X11+X3+X2+X+1 | 0X180F | telecom systems |
| CRC-16 | X16+X15+X2+1 | 0X18005 | Bisync, Modbus, USB, ANSI X3.28, SIA DC-07, many others; also known as CRC-16 and CRC-16-ANSI |
| CRC-CCITT | X16+X12+X5+1 | 0X11021 | ISO HDLC, ITU X.25, V.34/V.41/V.42, PPP-FCS |
| CRC-32 | X32+X26+X23+X22+X16+X12+X11+X10+X8+X7+X5+X4+X2+X+1 | 0x104C11DB7 | ZIP, RAR, IEEE 802 LAN/FDDI, IEEE 1394, PPP-FCS |
| CRC-32C | X32+X28+X27+X26+X25+X23+X22+X20+X19+X18+X14+X13+X11+X10+X9+X8+X6+1 | 0x11EDC6F41 | iSCSI, SCTP, G.hn payload, SSE4.2, Btrfs, ext4, Ceph |
4. Pre-knowledge of CRC check algorithm
Before learning the CRC check algorithm, first review the main algorithms involved in CRC.
1. XOR
Exclusive or, that is, the difference is 1, the same is 0, and the operation symbol is ^.
0^0 = 0
0^1 = 1
1^1 = 0
1^0 = 1XOR operation has the following rules, which need to be understood.
0^x = x 即0 异或任何数等于任何数
1^x = ~x 即1异或任何数等于任何数取反
x^x = 0 即任何数与自己异或，结果为0
a ^ b = b ^ a 交换律
a ^ (b ^ c) = (a ^ b) ^c 结合律2. Addition modulo 2
Compared with ordinary arithmetic addition, the main difference between modulo 2 addition is that modulo 2 addition does not perform carry processing. The specific results are as follows. 0+0 = 0 0+1 = 1 1+1 = 0 1+0 = 1 We found that the calculation result of modulo 2 addition is exactly the same as that of XOR operation. Further deduction, we will find that the five laws of XOR operation are also suitable for modulo 2 addition. Here, I will not list them one by one.
3. Subtraction modulo 2
Compared with ordinary arithmetic subtraction, the main difference between modulo 2 subtraction is that modulo 2 subtraction does not do borrow processing. The specific results are as follows. 0-0 = 0 0-1 = 1 1-1 = 0 1-0 = 1 We found that the calculation result of modulo 2 subtraction is exactly the same as that of modulo 2 addition and XOR. Further deduction, we will find that the five rules of XOR operation are also suitable for modulo 2 subtraction. Here, I will not list them one by one.
4. Modulo 2 division
Compared with ordinary arithmetic division, the main difference between modulo 2 division and modulo 2 division is that it does not borrow from the upper bit, nor does it compare the size of the same digit value of the divisor and the dividend, as long as the division is performed with the same digit.
Five, CRC principle
CRC principle: After the K-bit information code (target sending data), the R-bit check code is spliced to make the entire code length N bits, so this code is also called (N, K) code.
In layman's terms, it is to append a number (that is, a check code) to the information that needs to be sent to generate a new sending data and send it to the receiving end. This data requirement is such that the new data generated is divisible by a specific number. The integer division here needs to introduce the concept of modulo 2 division.
Then, the specific method of CRC check is
(1) Select a standard divisor (K-bit binary data string)
(2) Add K-1 bit 0 after the data to be sent (m bits), and then divide this new number (M+K-1 bits) by the above standard divisor by modulo 2 division, the obtained The remainder is the CRC check code of the data (Note: The remainder must be less than the divisor and only one bit less, if it is not enough, fill in 0)
(3) Attach this check code to the original m-bit data to form a new M+K-1-bit data and send it to the receiving end.
(4) The receiving end divides the received data by the standard divisor, and if the remainder is 0, the data is considered correct.
Note: There are two key points in the CRC check:
One is to predetermine a binary bit string (or polynomial) that both the sender and the receiver use as a divisor;
The second is to perform binary division on the original frame and the division selected above to calculate the FCS.
The former can be selected randomly or according to internationally accepted standards, but the highest bit and the lowest bit must both be "1"
6. Calculation of Cyclic Redundancy
Example:
Since the encoding process of CRC-32, CRC-16, CCITT and CRC-4 is basically the same, only the number of digits and the generator polynomial are different, the following is an example to illustrate the CRC check code generation process.
For the data 1110 0101 (16#E5), to specify the divisor 11011 to find its CRC check code, the process is as follows:
Using the checksum calculated above and the message data, the codeword to be transmitted can be created.
Sometimes, we need to fill the checksum to the specified position, which involves byte order issues, it is recommended to use memcpy() to copy.
7. Code implementation
The implementation algorithm refers to the relevant codes of the network, organizes and verifies, and can be used directly.
crc.c
#include "crc.h"
#include <stdio.h>
typedef enum {
REF_4BIT = 4,
REF_5BIT = 5,
REF_6BIT = 6,
REF_7BIT = 7,
REF_8BIT = 8,
REF_16BIT = 16,
REF_32BIT = 32
}REFLECTED_MODE;
uint32_t ReflectedData(uint32_t data, REFLECTED_MODE mode)
{
data = ((data & 0xffff0000) >> 16) | ((data & 0x0000ffff) << 16);
data = ((data & 0xff00ff00) >> 8) | ((data & 0x00ff00ff) << 8);
data = ((data & 0xf0f0f0f0) >> 4) | ((data & 0x0f0f0f0f) << 4);
data = ((data & 0xcccccccc) >> 2) | ((data & 0x33333333) << 2);
data = ((data & 0xaaaaaaaa) >> 1) | ((data & 0x55555555) << 1);
switch (mode)
{
case REF_32BIT:
return data;
case REF_16BIT:
return (data >> 16) & 0xffff;
case REF_8BIT:
return (data >> 24) & 0xff;
case REF_7BIT:
return (data >> 25) & 0x7f;
case REF_6BIT:
return (data >> 26) & 0x7f;
case REF_5BIT:
return (data >> 27) & 0x1f;
case REF_4BIT:
return (data >> 28) & 0x0f;
}
return 0;
}
uint8_t CheckCrc4(uint8_t poly, uint8_t init, bool refIn, bool refOut, uint8_t xorOut,
const uint8_t *buffer, uint32_t length)
{
uint8_t i;
uint8_t crc;
if (refIn == true)
{
crc = init;
poly = ReflectedData(poly, REF_4BIT);
while (length--)
{
crc ^= *buffer++;
for (i = 0; i < 8; i++)
{
if (crc & 0x01)
{
crc >>= 1;
crc ^= poly;
}
else
{
crc >>= 1;
}
}
}
return crc ^ xorOut;
}
else
{
crc = init << 4;
poly <<= 4;
while (length--)
{
crc ^= *buffer++;
for (i = 0; i < 8; i++)
{
if (crc & 0x80)
{
crc <<= 1;
crc ^= poly;
}
else
{
crc <<= 1;
}
}
}
return (crc >> 4) ^ xorOut;
}
}
uint8_t CheckCrc5(uint8_t poly, uint8_t init, bool refIn, bool refOut, uint8_t xorOut,
const uint8_t *buffer, uint32_t length)
{
uint8_t i;
uint8_t crc;
if (refIn == true)
{
crc = init;
poly = ReflectedData(poly, REF_5BIT);
while (length--)
{
crc ^= *buffer++;
for (i = 0; i < 8; i++)
{
if (crc & 0x01)
{
crc >>= 1;
crc ^= poly;
}
else
{
crc >>= 1;
}
}
}
return crc ^ xorOut;
}
else
{
crc = init << 3;
poly <<= 3;
while (length--)
{
crc ^= *buffer++;
for (i = 0; i < 8; i++)
{
if (crc & 0x80)
{
crc <<= 1;
crc ^= poly;
}
else
{
crc <<= 1;
}
}
}
return (crc >> 3) ^ xorOut;
}
}
uint8_t CheckCrc6(uint8_t poly, uint8_t init, bool refIn, bool refOut, uint8_t xorOut,
const uint8_t *buffer, uint32_t length)
{
uint8_t i;
uint8_t crc;
if (refIn == true)
{
crc = init;
poly = ReflectedData(poly, REF_6BIT);
while (length--)
{
crc ^= *buffer++;
for (i = 0; i < 8; i++)
{
if (crc & 0x01)
{
crc >>= 1;
crc ^= poly;
}
else
{
crc >>= 1;
}
}
}
return crc ^ xorOut;
}
else
{
crc = init << 2;
poly <<= 2;
while (length--)
{
crc ^= *buffer++;
for (i = 0; i < 8; i++)
{
if (crc & 0x80)
{
crc <<= 1;
crc ^= poly;
}
else
{
crc <<= 1;
}
}
}
return (crc >> 2) ^ xorOut;
}
}
uint8_t CheckCrc7(uint8_t poly, uint8_t init, bool refIn, bool refOut, uint8_t xorOut,
const uint8_t *buffer, uint32_t length)
{
uint8_t i;
uint8_t crc;
if (refIn == true)
{
crc = init;
poly = ReflectedData(poly, REF_7BIT);
while (length--)
{
crc ^= *buffer++;
for (i = 0; i < 8; i++)
{
if (crc & 0x01)
{
crc >>= 1;
crc ^= poly;
}
else
{
crc >>= 1;
}
}
}
return crc ^ xorOut;
}
else
{
crc = init << 1;
poly <<= 1;
while (length--)
{
crc ^= *buffer++;
for (i = 0; i < 8; i++)
{
if (crc & 0x80)
{
crc <<= 1;
crc ^= poly;
}
else
{
crc <<= 1;
}
}
}
return (crc >> 1) ^ xorOut;
}
}
uint8_t CheckCrc8(uint8_t poly, uint8_t init, bool refIn, bool refOut, uint8_t xorOut,
const uint8_t *buffer, uint32_t length)
{
uint32_t i = 0;
uint8_t crc = init;
while (length--)
{
if (refIn == true)
{
crc ^= ReflectedData(*buffer++, REF_8BIT);
}
else
{
crc ^= *buffer++;
}
for (i = 0; i < 8; i++)
{
if (crc & 0x80)
{
crc <<= 1;
crc ^= poly;
}
else
{
crc <<= 1;
}
}
}
if (refOut == true)
{
crc = ReflectedData(crc, REF_8BIT);
}
return crc ^ xorOut;
}
uint16_t CheckCrc16(uint16_t poly, uint16_t init, bool refIn, bool refOut, uint16_t xorOut,
const uint8_t *buffer, uint32_t length)
{
uint32_t i = 0;
uint16_t crc = init;
while (length--)
{
if (refIn == true)
{
crc ^= ReflectedData(*buffer++, REF_8BIT) << 8;
}
else
{
crc ^= (*buffer++) << 8;
}
for (i = 0; i < 8; i++)
{
if (crc & 0x8000)
{
crc <<= 1;
crc ^= poly;
}
else
{
crc <<= 1;
}
}
}
if (refOut == true)
{
crc = ReflectedData(crc, REF_16BIT);
}
return crc ^ xorOut;
}
uint32_t CheckCrc32(uint32_t poly, uint32_t init, bool refIn, bool refOut, uint32_t xorOut,
const uint8_t *buffer, uint32_t length)
{
uint32_t i = 0;
uint32_t crc = init;
while (length--)
{
if (refIn == true)
{
crc ^= ReflectedData(*buffer++, REF_8BIT) << 24;
}
else
{
crc ^= (*buffer++) << 24;
}
for (i = 0; i < 8; i++)
{
if (crc & 0x80000000)
{
crc <<= 1;
crc ^= poly;
}
else
{
crc <<= 1;
}
}
}
if (refOut == true)
{
crc = ReflectedData(crc, REF_32BIT);
}
return crc ^ xorOut;
}
uint32_t CrcCheck(CRC_Type crcType, const uint8_t *buffer, uint32_t length)
{
switch (crcType.width)
{
case 4:
return CheckCrc4(crcType.poly, crcType.init, crcType.refIn, crcType.refOut,
crcType.xorOut, buffer, length);
case 5:
return CheckCrc5(crcType.poly, crcType.init, crcType.refIn, crcType.refOut,
crcType.xorOut, buffer, length);
case 6:
return CheckCrc6(crcType.poly, crcType.init, crcType.refIn, crcType.refOut,
crcType.xorOut, buffer, length);
case 7:
return CheckCrc7(crcType.poly, crcType.init, crcType.refIn, crcType.refOut,
crcType.xorOut, buffer, length);
case 8:
return CheckCrc8(crcType.poly, crcType.init, crcType.refIn, crcType.refOut,
crcType.xorOut, buffer, length);
case 16:
return CheckCrc16(crcType.poly, crcType.init, crcType.refIn, crcType.refOut,
crcType.xorOut, buffer, length);
case 32:
return CheckCrc32(crcType.poly, crcType.init, crcType.refIn, crcType.refOut,
crcType.xorOut, buffer, length);
}
return 0;
}crc.h
#ifndef __CRC_H__
#define __CRC_H__
#include <stdint.h>
#include <stdbool.h>
typedef struct {
uint8_t width;
uint32_t poly;
uint32_t init;
bool refIn;
bool refOut;
uint32_t xorOut;
}CRC_Type;
uint32_t CrcCheck(CRC_Type crcType, const uint8_t *buffer, uint32_t length);
#endifmain.c
#include <stdio.h>
#include <stdint.h>
#include <stdbool.h>
#include "crc.h"
#define LENGTH 8
const uint8_t data[3][LENGTH] = {
{ 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08 },
{ 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80 },
{ 0xfe, 0xfd, 0xfb, 0xf7, 0xef, 0xdf, 0xbf, 0x7f }};
typedef struct {
CRC_Type crcType;
uint32_t result[3];
}CRC_Test;
CRC_Test crc4_ITU = { { 4, 0x03, 0x00, true, true, 0x00 }, { 0x0f, 0x0a, 0x0e } };
CRC_Test crc5_EPC = { { 5, 0x09, 0x09, false, false, 0x00 }, { 0x00, 0x0c, 0x17 } };
CRC_Test crc5_ITU = { { 5, 0x15, 0x00, true, true, 0x00 }, { 0x16, 0x0a, 0x17 } };
CRC_Test crc5_USB = { { 5, 0x05, 0x1f, true, true, 0x1f }, { 0x10, 0x09, 0x17 } };
CRC_Test crc6_ITU = { { 6, 0x03, 0x00, true, true, 0x00 }, { 0x1d, 0x30, 0x00 } };
CRC_Test crc7_MMC = { { 7, 0x09, 0x00, false, false, 0x00 }, { 0x57, 0x30, 0x5b } };
CRC_Test crc8 = { { 8, 0x07, 0x00, false, false, 0x00 }, { 0x3e, 0xe1, 0x36 } };
CRC_Test crc8_ITU = { { 8, 0x07, 0x00, false, false, 0x55 }, { 0x6b, 0xb4, 0x63 } };
CRC_Test crc8_ROHC = { { 8, 0x07, 0xff, true, true, 0x00 }, { 0x6b, 0x78, 0x93 } };
CRC_Test crc8_MAXIM = { { 8, 0x31, 0x00, true, true, 0x00 }, { 0x83, 0x60, 0xa9 } };
CRC_Test crc16_IBM = { { 16, 0x8005, 0x0000, true, true, 0x0000 }, { 0xc4f0, 0x2337, 0xa776 } };
CRC_Test crc16_MAXIM = { { 16, 0x8005, 0x0000, true, true, 0xffff }, { 0x3b0f, 0xdcc8, 0x5889 } };
CRC_Test crc16_USB = { { 16, 0x8005, 0xffff, true, true, 0xffff }, { 0x304f, 0xd788, 0x53c9 } };
CRC_Test crc16_MODBUS = { { 16, 0x8005, 0xffff, true, true, 0x0000 }, { 0xcfb0, 0x2877, 0xac36 } };
CRC_Test crc16_CCITT = { { 16, 0x1021, 0x0000, true, true, 0x0000 }, { 0xeea7, 0xfe7c, 0x7919 } };
CRC_Test crc16_CCITT_FALSE = { { 16, 0x1021, 0xffff, false, false, 0x0000 }, { 0x4792, 0x13a7, 0xb546 } };
CRC_Test crc16_X25 = { { 16, 0x1021, 0xffff, true, true, 0xffff }, { 0x6dd5, 0x7d0f, 0xfa6a } };
CRC_Test crc16_XMODEM = { { 16, 0x1021, 0x0000, false, false, 0x0000 }, { 0x76ac, 0x2299, 0x8478 } };
CRC_Test crc16_DNP = { { 16, 0x3D65, 0x0000, true, true, 0xffff }, { 0x7bda, 0x0535, 0x08c4 } };
CRC_Test crc32 = { { 32, 0x04c11db7, 0xffffffff, true, true, 0xffffffff }, { 0x3fca88c5, 0xe0631a53, 0xa4051a26 } };
CRC_Test crc32_MPEG2 = { { 32, 0x4c11db7, 0xffffffff, false, false, 0x00000000 }, { 0x14dbbdd8, 0x6509b4b6, 0xcb09d294 } };
void CrcTest(CRC_Test crcTest)
{
uint32_t i;
for (i = 0; i < 3; i++)
{
printf("%08x\t%08x\r\n", CrcCheck(crcTest.crcType, data[i], LENGTH), crcTest.result[i]);
}
printf("\r\n");
}
int main(void)
{
CrcTest(crc4_ITU);
CrcTest(crc5_EPC);
CrcTest(crc5_ITU);
CrcTest(crc5_USB);
CrcTest(crc6_ITU);
CrcTest(crc7_MMC);
CrcTest(crc8);
CrcTest(crc8_ITU);
CrcTest(crc8_ROHC);
CrcTest(crc8_MAXIM);
CrcTest(crc16_IBM);
CrcTest(crc16_MAXIM);
CrcTest(crc16_USB);
CrcTest(crc16_MODBUS);
CrcTest(crc16_CCITT);
CrcTest(crc16_CCITT_FALSE);
CrcTest(crc16_X25);
CrcTest(crc16_XMODEM);
CrcTest(crc16_DNP);
CrcTest(crc32);
CrcTest(crc32_MPEG2);
return 0;
}Tips：
Different CRC algorithms have different calculation results for 00H or FFH data streams, and some algorithms have check results that are also 00H or FFH (that is, when the storage space is in the initialization state: all 0s or all 1s, CRC checks The experience is correct), and it needs to be avoided in the application.