About the Author:
The flower name is "Kaku", system development engineer of Baishan Cloud Technology
API development and management Lao Xianrou has rich experience in product development and operation and maintenance. He has worked in well-known Internet companies such as Sohu and Sina. He has participated in Sina Cloud SAE platform CC firewall project, providing security protection for hundreds of thousands of users and ensuring SAE platform. Stable performance; joined Baishan in 2016 and became the system development engineer of the largest dimple in Jiuxianqiao area.
For real-time web applications (such as: real-time communication, stock fund applications, sports updates, multi-player games, etc.), in order to obtain real-time server-side data in traditional web, the client-side usually sends HTTP requests periodically, and the server-side responds and returns data. Since the client sends requests to the server regularly, when there is no data update on the server, the client still sends requests, which results in a waste of bandwidth and CPU usage on the server.
In order to solve the above problems, more and more enterprises are thinking about how to solve the problem of long connection, and WebSocket is one of the more commonly used methods. WebSocket establishes a TCP connection through the first HTTP request, and subsequent data exchanges do not need to send HTTP requests, creating a real long connection. At the same time, WebSocket is also a dual-channel connection, which can send and receive information on the same TCP connection.
Baishan Cloud Aggregation Platform also integrates WebSocket, which can provide users with the conversion function from WebSocket protocol to HTTP protocol, so that the user's Client can connect to the Cloud Aggregation Platform in the way of long connection WebSocket protocol, and Cloud Aggregation only needs an HTTP connection to connect to the cloud aggregation platform. The back-end of the enterprise greatly reduces the pressure on the back-end, and also avoids the problem of adapting the WebSocket protocol on the user's server side. We encountered an interesting problem during the development and testing process, which may have been encountered by many developers: using different WebSocket clients to communicate with the WebSocket Server, the WebSocket Server returns inconsistent data.
The problem scenario
1. Different client access
(1) Python interacts with WebSocket Server ws://2abe356fc.bsclink.com/ through the WebSocket client, and the output is normal;
(python client output content)
(2) After the Chrome browser loads the ws.html page, the js in the page calls the WebSocket Client that comes with the browser to interact with the WebSocket Server ws://2abe356fc.bsclink.com/, and outputs ERROR;
(Chrome browser output content)
(3) After the ws.html page is loaded by the Safari browser, the js in the page calls the WebSocket Client that comes with the browser to interact with the WebSocket Server ws://2abe356fc.bsclink.com/, and the output is normal;
(Safari browser output content)
2. Browser request flow chart
The following is the process of the browser requesting the server through the WebSocket protocol:
(browser request flow chart)
2. Problem Analysis
Only the communication between Chrome and Websocket Server is abnormal. It is judged that the ERROR is probably caused by a problem with the Chrome browser. Based on this, we can analyze the specific cause of the problem.
1. View the error-related information through the browser console:
As shown at the bottom of the above figure, the WebSocket protocol fails to decode a text frame when it is converted to uft-8 encoding.
Since the WebSocket Server uses the text frame method to return data to the Client, we began to investigate the cause of the decoding failure caused by the WebSocket Server returning data.
2. Print the WebSocket Server log and view the returned content
Through the log, it is observed that the content sent by longloop to the WebSocket Server is consistent with the content output by the WebSocket Server to the Client, both of which are garbled. Based on this, we can determine that there is no abnormality in the WebSocket Server, so we need to determine whether there is an abnormality in the longloop.
3. Capture packets through longloop to view the content returned by backend
You can use TCPDUMP to capture packets to determine whether there is a problem with longloop.
(backend data returned to longloop)
(longloop returns data to WebSocket Server)
通过对比以上两组数据，可以得出如下结论：
经过longloop后，真实返回给Client的数据并未发生变化。
（1）backend的返回数据被gzip压缩；
（2）压缩的响应数据被发送至WebSocket Server；
（3）最终由WebSocket Server发送到WebSocket客户端。
4. backend返回的数据为什么被压缩了？
首先，backend端必须开启gzip压缩，并支持对此返回的数据类型的gzip压缩，才能返回压缩后的响应数据；
其次，客户端要明确声明能接收gzip压缩的响应数据，backend端才能够返回gzip压缩过的数据。
经确认，backend server上的配置开启了gzip压缩功能，并对content-type为text/html的数据支持gzip压缩。
可以判断问题有可能出现在client环节：
Client没有要求返回压缩数据，但是backend端返回了压缩数据；
通过不同浏览器访问，返回不同数据，可以判定不是backend端的问题。
Client主动要求backend端返回被压缩的数据；
只有Chrome浏览器返回了gzip压缩数据，可以推断可能是因为Chrome请求backend端时，在request header中包含了可以接收gzip压缩数据的header，导致backend端返回了gzip压缩数据。
5. 抓包对比Chrome和Safari请求头信息
Chrome相关信息：
（1）Chrome浏览器请求ws.html静态文件的请求头中带有Accept-Encoding：
（2）Chrome浏览器将ws.html加载到本地后，ws.html文件中的js WebSocket 客户端向WebSocket Server发送请求的请求头中带有Accept-Encoding：
（3）Chrome浏览器的请求发送到longloop之后，longloop到backend的请求头中带有Accept-Encoding：
Safari相关信息
（1）Safari浏览器请求ws.html静态文件的请求头中带有Accept-Encoding：
（2）Safari浏览器将ws.html加载到本地后，ws.html文件中的js WebSocket 客户端向WebSocket Server发送请求的请求头中未带有Accept-Encoding：
（3）Safari浏览器的请求发送到longloop之后，longloop到backend的请求头中未带有Accept-Encoding：
通过对比Chrome和Safari相关请求数据，我们可以判断出WebSocket Server返回数据不一致的原因如下：
Chrome，Safari浏览器发送请求时，为了提高网络传输效率、减少网络带宽占用，默认自带gzip压缩支持，两种浏览器加载ws.html时均无异常。但当js调用Chrome浏览器WebSocket 客户端向WebSocket Server端发送请求时，在请求头Accept-Encoding中添加了对gzip的支持，backend收到HTTP请求后，认为客户端能够对gzip压缩的响应数据进行解压缩，从而backend返回了gzip压缩过的响应数据，而WebSocket客户端接收到gzip压缩的数据后，不支持gzip数据解压缩，最终导致了decode出错。
而js调用Safari浏览器WebSocket客户端向WebSocket Server端发送请求时，请求头未带有Accept-Encoding，backend收到http请求后，不会返回被gzip压缩的响应数据，从而WebSocket客户端正常解析访问正常。
三、解决办法
为解决上述问题，我们需要在longloop这一层进行判断：如果user agent为Chrome浏览器，则需要去掉request header中的Accept-Encoding这个header，明确告知服务器端不接受gzip压缩过的数据，这样服务器端就不会返回gzip压缩过的数据，Chrome浏览器即可正常访问。