0 Preface
Derived from 0.1websocket
To learn websocket, you first need to pay attention to the origin of websocket and some usage scenarios. First of all, because http has flaws in communication, the client can only initiate a request and the server responds, which makes the server unable to actively push messages, and the client needs to poll to bring about efficiency problems. Websocket was born in 2008 and became an international standard in 2011. At present, all browsers support websocket, which is used to establish peer-to-peer communication between the server and the client. The initiator of the communication can be the client or the server, so that the server can actively push relevant information. .
0.2 websocket features
- Built on top of the TCP protocol
- Good compatibility with http
- Supports data formats such as data text and binary
- Protocol identifier: ws; encryption is wss
1 websocket protocol format
1.1 Protocol header definition
The study of the standard protocol cannot be avoided is the study of the RFC document [RFC6455] . First, the protocol format of the websocket is interpreted, including the protocol header and the protocol data. The details are shown in the following figure:
**1 FIN: **End identifier, when this bit is 1, the session ends;
**2 RSV1-3: **Reserved bits must be 0; unless both parties specify special meanings;
**3 opcode: **data type definition;
* %x0：连续帧
* %x1：文本帧
* %x2：二进制帧
* %x3-7：为非控制帧保留
* %x8：连接关闭
* %x9：ping
* %xA：pong
* %xB-F：为制帧保留
There are three types of WebSocket control frames: Close (close frame), Ping, and Pong. The opcode of the control frame defines 0x08 (shutdown frame), 0x09 (Ping frame), 0x0A (Pong frame). The Close frame is easy to understand. If the client receives it, it closes the connection, and the client can also send a close frame to the server. Ping and Pong are heartbeats in websocket to ensure that the client is online. Generally speaking, only the server sends Ping to the client, and the client sends Pong in response.
**4 MASK: **Mask bit, when this bit is 1, it means that the current protocol is encrypted transmission, and the masking-key in the protocol header needs to be set.
5 Payload len: If it is 0-125, it is the payload length. If 126, the following 2 bytes interpreted as a 16-bit unsigned integer are the payload length. If 127, the following 8 bytes interpreted as a 64-bit unsigned integer (most significant bit must be 0) are the payload length.
pyload_len = 7bit (0-125)
= 7+16bit(126)
= 7+64bit(127)
**6 Masking-key:** This field does not exist when the mask is 0, and this field is 4 bytes when the mask is 1. All frames sent from the client to the server must be covered with a mask, that is, for all For frames sent from the client to the server, this field must exist, and the value of this field is generated by the client using a random number generator with a sufficiently large entropy value.
**7 payload data: **data segment, which stores the actual transmission data.
1.2 Protocol header data structure
Define the data structure according to the protocol definition, divide the protocol header into two parts and one part is the public part, and parse the first two public fields. The second part is the extended length part, which is divided into two cases. One is the case where the payload_len length is 126; the other is the case where the payload_len length is 127.
typedef struct WS_COM_HDR_T {
unsigned char opcode:4,
resv3:1,
resv2:1,
resv1:1,
fin:1;
unsigned char payload_len:7,
mask:1;
}WS_COM_HDR;
typedef struct WS_HDR_EXT126_T {
unsigned short extended_payload_len;
char mask_key[4];
}WS_HDR_EXT126 ;
typedef struct WS_HDR_EXT127_T {
long long extended_payload_len;
char mask_key[4];
}WS_HDR_EXT127 ;
2 websoket interaction process
The websocket state includes handshake (Handshake) state, data transfer (Data Transfer) state and close (close) state.
2.1 websocket handshake
Receive the request from the client The basic format of the first request connection packet is as follows, the main purpose is to be compatible with the server or service relay based on the http protocol. Sec-WebSocket-Key, used to prove that the server received a websocket service request. After splicing with the global unique GUID, after SHA1 hashing and base64 encryption, the server returns to the client through the Sec-WebSocket-Accept field to verify the websocket connection.
GET /chat HTTP/1.1
Host: server.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Origin: http://example.com
Sec-WebSocket-Protocol: chat, superchat
Sec-WebSocket-Version: 13
The basic format of the server's response connection is as follows:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
Sec-WebSocket-Protocol: chat
The pseudocodes for generating the Sec-WebSocket-Key field and the Sec-WebSocket-Accept field are as follows:
const char GUID = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
char str_key[16] = random();
Sec_WebSocket_Key = base64(str_key);
Sec_WebSocket_Accept = Sec_WebSocket_Key + GUID;
Sec_WebSocket_Accept = SHA1(Sec_WebSocket_Accept);
Sec_WebSocket_Accept = base64(Sec_WebSocket_Accept);
2.2 Data transmission process
After the handshake is established, the websocket can perform data transmission. The data transmission method is divided into two forms: one is plaintext transmission, and the other is ciphertext transmission. Generally, ciphertext transmission is used. The main setting is to set the mask to 1. . Parsing the websocket data packet first requires parsing the public header, obtaining the data size and classifying the data according to the size. The data classification includes three cases: 0-125, 126 and 127. After that, the data decryption part is decrypted. The basic receiving process and the analysis work have been completed. The next step is data processing and data return to the client.
The websocket decryption process is relatively simple as a whole, mainly including two parts. The first part is to take out the Masking-key and perform a 4-remainder operation in a loop; the second step is to perform an XOR operation on the remainder data and the received data to obtain the decoded real data
poyload[i] = payload[i] ^ masking_key[i % 4];
2.3 websocket disconnected
The close handshake is intended to complement the TCP close handshake (FIN/ACK), which is not always end-to-end reliable, especially in the presence of intercepting proxies and other intermediaries. WebSocket avoids certain situations where data may be lost unnecessarily by sending a close frame and waiting for a close frame in response. The opcode is set to 0x8, which can carry the status code and shutdown reason.
3 Actual websocket server
This article mainly implements the websocket echo server, and sends the received data back to the client. See the specific source code: my_websocket_server.c
3.1 Connection handling
Connection processing is divided into several steps:
- 1 Get the Sec-WebSocket-Keykey value in the received data
- 2 Combine the key value with the GUID field
- 3 Perform SHA1 hash encryption operation;
- 4 Encrypted by base64;
- 5 Finally, reply to the client according to the fixed format
int websocket_handshake(event_item *event)
{
char line_buf[MAX_LINE_BUF] = {
0};
int index = 0;
char sec_data[128] = {
0};
char sec_accept[32] = {
0};
do {
memset(line_buf, 0, sizeof(line_buf));
index = readline(event->buffer, index, line_buf);
printf("line:%s\n", line_buf);
if(strstr(line_buf, "Sec-WebSocket-Key"))
{
strcat(line_buf, GUID);
//Sec-WebSocket-Key:
//linebuf:
//Sec-WebSocket-Key: c1WhjNBGlWE8hIh5IFaoyQ==258EAFA5-E914-47DA-95CA-C5AB0DC85B11
int skip_len = strlen("Sec-WebSocket-Key: ");
SHA1(line_buf+skip_len, strlen(line_buf+skip_len), sec_data);
base64_encode(sec_data, strlen(sec_data), sec_accept);
memset(event->buffer, 0, MAX_BUFFER_LENGTH);
event->length = sprintf(event->buffer, "HTTP/1.1 101 Switching Protocols\r\n"
"Upgrade: websocket\r\n"
"Connection: Upgrade\r\n"
"Sec-WebSocket-Accept: %s\r\n\r\n", sec_accept);
printf("websocket_handshake response : %s\n", event->buffer);
break;
}
}while((line_buf[index] != '\r' || line_buf[index+1] != '\n') && (index != -1));
return 0;
}
3.2 Data Analysis
The key points of data analysis are as follows:
- 1 Websocket header analysis, get the mask value;
- 2 Use the mask masking_key to analyze;
- 3 It should be noted that different length data headers are distinguished.
int websocket_transfer(event_item *event)
{
WS_COM_HDR *hdr = (WS_COM_HDR*)event->buffer;
unsigned char *payload = NULL;
unsigned char *masking_key = NULL;
int head_len = 0;
int payload_len = 0;
printf("length: %d mask:%d\n", hdr->payload_len, hdr->mask);
if (hdr->payload_len < 126) {
//
payload_len = hdr->payload_len;
if (hdr->mask) {
// mask set 1
head_len = sizeof(WS_COM_HDR) + MASKING_KEY_LEN;
payload = event->buffer + head_len;
masking_key = event->buffer + head_len - MASKING_KEY_LEN;
umask(payload, hdr->payload_len, masking_key);
}
else
{
head_len = sizeof(WS_COM_HDR);
payload = event->buffer + head_len;
}
} else if (hdr->payload_len == 126) {
WS_HDR_EXT126 *hdr126 = (WS_HDR_EXT126*)(event->buffer + sizeof(WS_COM_HDR));
payload_len = ntoh16(hdr126->extended_payload_len);
if (hdr->mask) {
// mask set 1
head_len = sizeof(WS_COM_HDR) + sizeof(WS_HDR_EXT126) + MASKING_KEY_LEN;
payload = event->buffer + head_len;
masking_key = event->buffer + head_len - MASKING_KEY_LEN;
umask(payload, ntoh16(hdr126->extended_payload_len), masking_key);
}
else
{
head_len = sizeof(WS_COM_HDR) + sizeof(WS_HDR_EXT126);
payload = event->buffer + head_len;
}
} else {
WS_HDR_EXT127 *hdr127 = (WS_HDR_EXT127*)(event->buffer + sizeof(WS_COM_HDR));
payload_len = ntoh64(hdr127->extended_payload_len);
if (hdr->mask) {
// mask set 1
head_len = sizeof(WS_COM_HDR) + sizeof(WS_HDR_EXT127) + MASKING_KEY_LEN;
payload = event->buffer + head_len;
masking_key = event->buffer + head_len - MASKING_KEY_LEN;
umask(payload, ntoh64(hdr127->extended_payload_len), masking_key);
}
else
{
head_len = sizeof(WS_COM_HDR) + sizeof(WS_HDR_EXT127);
payload = event->buffer + head_len;
}
}
printf("payload : %s\n", payload);
int mk = ntoh32(*(int *)masking_key);
printf("masking_key : %08x\n", mk);
//build send buffer
int buffer_len = payload_len + sizeof(WS_COM_HDR) + sizeof(WS_HDR_EXT127) + 4;
char *buffer = (char *)malloc(buffer_len);
int send_len = 0;
if(!buffer)
printf("malloc err(%d).\n", errno);
memset(buffer, 0, buffer_len);
if(hdr->opcode == WS_OPCODE_TEXT_FRAME)
{
send_len = encode_packet(buffer, payload, payload_len);
}
else if(hdr->opcode == WS_OPCODE_CLOSE_FRAME)
{
send_len = wetsocket_close(buffer);
}
memset(event->buffer, 0, event->length);
memcpy(event->buffer, buffer, send_len);
event->length = send_len;
printf("send_len:%d\n", send_len);
free(buffer);
return 0;
}
3.3 Reply data encapsulation
Key points of reply data encapsulation:
- 1 Need to encapsulate the data websocket header;
- 2 Note that the server reply mask needs to be set to 0; masking_key is empty.
int encode_packet(char *buffer, char *stream, int length) {
printf("length:%d stream:%s\n", length, stream);
WS_COM_HDR head = {
0};
head.fin = 1;
head.opcode = WS_OPCODE_TEXT_FRAME;
int size = 0;
if (length < 126) {
head.payload_len = length;
size = 2;
memcpy(buffer, &head, sizeof(WS_COM_HDR));
} else if (length < 0xffff) {
WS_HDR_EXT126 hdr = {
0};
hdr.extended_payload_len = length;
size = sizeof(WS_COM_HDR) + sizeof(WS_HDR_EXT126);
memcpy(buffer, &head, sizeof(WS_COM_HDR));
memcpy(buffer+sizeof(WS_COM_HDR), &hdr, sizeof(WS_HDR_EXT126));
} else {
WS_HDR_EXT127 hdr = {
0};
hdr.extended_payload_len = length;
size = sizeof(WS_COM_HDR)+sizeof(WS_HDR_EXT127);
memcpy(buffer, &head, sizeof(WS_COM_HDR));
memcpy(buffer+sizeof(WS_COM_HDR), &hdr, sizeof(WS_HDR_EXT127));
}
memcpy(buffer+size, stream, length);
return length + size;
}
3.4 Closing the connection
Close connection focus:
- 1 Set opcode to 8;
- 2 With the status code, you can understand the cause of the error more clearly.
int wetsocket_close(char *buffer) {
WS_COM_HDR head = {
0};
head.fin = 1;
head.opcode = WS_OPCODE_CLOSE_FRAME;
int size = sizeof(WS_COM_HDR);
head.payload_len = WS_STATUS_CODES_LEN;
memcpy(buffer, &head, sizeof(WS_COM_HDR));
short nStatusCode = hton16(WS_STATUS_CODES_NORMAL);
memcpy(buffer+size, &nStatusCode, WS_STATUS_CODES_LEN);
size += WS_STATUS_CODES_LEN;
return size;
}