This article is published by NetEase Cloud .
In the current Internet products, there are more and more scenarios such as short video, online KTV, and online multimedia interaction. Such products are very dependent on value creators, such as beauty anchors, small video screen producers, music producers, etc. It is particularly important for these Internet products to create a high-quality user experience for value providers.
So, how can this be guaranteed? Let's talk about the user experience of value providers producing and disseminating value (uploading data).
As shown in the figure above, this should be a problem faced by many APPs today. Upload failures and slow uploads lead to unsatisfactory user experience. From a macro level, we can roughly summarize the reasons as follows:
Mobile
There are various types of mobile networks, such as wifi, 2g, 3g, and 4g. Compared with the PC, one of the more significant features of the mobile Internet is that the environment is very unstable and the packet loss is serious, which directly leads to the connection between the client and the server. The rate is low, resulting in slow file upload and download speed and low success rate.
Long journey
Another point is the problem that both the mobile network and the PC network must face - the high latency of the wide area network. From the delay monitoring of NetEase cloud object storage distributed among the computer rooms in various regions, we can see that the delay from the nodes in North China, Northwest, Southwest and other regions to the Hangzhou computer room is basically about 30ms to 50ms. When the network is busy at night, there may be delays. It will often reach the level of 100 ms, and the packet loss rate will also increase accordingly.
You are in china
Another typical problem in the domestic network environment is that telecommunications, China Unicom are separated from north to south, and there are many network problems of small operators. Not to mention foreign and cross-border visits. Whether it is domestic visits abroad or foreign visits at home, you can basically only be in a hurry, and the delay packet loss is terrifyingly high.
Drilling down to the technical level, it's clear to technologists that it's because of the Internet's roots—the great TCP protocol is a little stretched in such a mobile, wide-area network environment. As shown in the figure below, we are facing a very poor quality underlying TCP data transmission channel. Packet loss (High loss rate) and high latency (High RTT) make this data transmission channel narrow and congested.
NetEase Cloud provides solutions
As a first-class storage service provider (this is our goal), NetEase Cloud provides high-quality object storage and storage-based upstream and downstream data transmission acceleration and other value-added services for NetEase Group and many partners, providing a one-stop solution for enterprises It solves the management problem of unstructured data in the mobile Internet era.
Difficulty and slow uploading at home, abroad and across borders are the common problems we reflect when connecting with Internet products from all walks of life. To this end, since the beginning of 2014, NetEase Cloud has started to create a unified solution to help products overcome this problem. At present, we also have a very complete solution. Let's take a look at some of the results we have achieved:
The acceleration effect is obvious
The solution provided by NetEase Cloud has helped the product to achieve a good improvement in transmission speed and upload success rate, which has been affirmed by users. (The picture above is the objective data obtained from the keynote test at all levels across the country)
Access convenience
Enterprises only need to use SDK (Android, iOS, Web PC) to solve all kinds of problems such as failure to upload, slow upload, and secure upload in a short time, so that product users have a first-class upload experience.
If a company wants to build a customer upload system by itself, it will involve all aspects of investment:
○ Reliable upload protocol: support file fragmentation, breakpoint upload, streaming upload, secure upload (HTTPS)
○ Upload Server-side system: supports high concurrency and high throughput (it is often difficult to design a server-side with a large amount of data interaction)
○ All-platform SDK: including mobile terminals Android, iOS; Web side; PC side
○ A lot of resource investment, etc.: including A large number of human resources (development, operation and maintenance), edge nodes in various regions, private line resources at home and abroad, etc.
are invested, and the capital cost is at least millions. With NetEase Cloud, enterprises can access the upload system at zero cost, and build products on the same technical starting line as NetEase's partners.
Technology revealed
Next, I will share with you how we technically create the above uploading solution.
We have invested a lot in resources, architecture, system optimization, etc. Among them, the main optimization work includes:
○ Edge deployment
○ TCP protocol tuning
○ Application layer protocol optimization
○ Mobile upload optimization
○ Routing optimization system
1 Edge layout and TCP optimization
The main problem between the client and the base station is High Loss Rate, that is, the problem of high packet loss rate; the main problem between the base station and the data center is the problem of High RTT and high delay.
Our solution is to divide into two. In order to solve the problem of high latency in the second half of the network, we deploy the edge node server to the place closest to the user, and combine the high-speed dedicated line to quickly upload the user data to the data center.
At present, the NetEase cloud object storage direct transfer acceleration network has covered several major regions in China, including Central China, North China, South China, East China, Southwest China, and Northwest China; foreign countries mainly include the United States, Japan, Southeast Asia, Europe and other regions, and other regions are also covered. In progress.
The following picture shows the domestic coverage area:
In foreign countries, we use the nodes in the AWS computer room for coverage, and access the domestic computer room through a foreign high-speed dedicated line.
The network between the edge node and the data center (NOS central computer room) is in the hands of NetEase Cloud itself, so the optimization is to overcome the high latency problem of the WAN first. We have established a long connection pool between the edge nodes and the central computer room, and made certain parameter tuning for TCP connections, such as tcpslowstartafteridle, tcp_wmem and so on. This avoids the slow start process of uploading data each time, and ensures that a piece of data can be sent to the data center after only one RTT (theoretical optimal effect).
The following are the statistics from the edge node to the central computer room before and after the optimization of the Beijing node in the online keynote test (the time from the edge node to the central computer room, including writing NOS). It can be seen that after the optimization, compared with Hangzhou BGP edge node to Hangzhou center In the computer room (same computer room), the difference between Beijing AWS is about 30ms in RTT.
2 Application layer protocol optimization
The traditional standard object storage service (AWS S3 is basically the de facto standard) is originally designed for the server side, including the system design and the interfaces it provides, which are not well suited to the needs of mobile networks. The most important point is that the storage interface of traditional (and standard) object storage cannot support resuming the upload from a breakpoint, and its block upload protocol is mainly aimed at the scenario where users upload large files (the minimum block size is 5M).
As far as the current mobile Internet application scenarios are concerned, in order to provide users with a better experience, resources including voice, pictures and videos are generally subject to substantial data compression on the basis of ensuring that the user experience is not affected. The size is often not more than 1M, and the minimum block of 5M is completely useless.
Therefore, it is necessary to design a set of general protocols for direct uploading to support mobile uploading. We mainly consider the following two basic design goals: Resume upload from a
breakpoint: support the resumed upload of small files in a short period of time, and support large files Long-term breakpoint resume.
Streaming upload: Supports streaming uploading of large and small files, that is, performing streaming uploading part by part without knowing the final file size, such as supporting uploading while recording.
The following is the core interface PostPart
POST /${bucketName}/${objectName}?offset=${Offset}&complete=${Complete}&context={context}&version=1.0 HTTP/1.1
Content-Length: ${length}
Content- Type: ${contentType}
x-nos-token: ${token}
<data of body>
○ offset is the offset of the uploaded data in the whole file
○ x-nos-token is the upload token
○ complete identifier is the data of the last file segment
○ context is the identifier returned by the server, which is unique in the scenario of resume uploading Identifies this file upload
3 Mobile upload optimization
In order to cope with the high packet loss rate of the mobile network, in addition to designing the above protocol for fragment uploading, we have also made the following optimizations:
HTTP PipeLine
In the mobile network environment, in order to improve the success rate of file upload, the client often slices the file. For example, a 1M file is uploaded with 16K as a shard, one shard by one shard.
The traditional HTTP 1.1 request mode (that is, the way most users currently use) is the following no pipelining mode. Each multipart upload waits for an RTT before uploading. In a WAN environment, for example, when an overseas user uploads to Hangzhou, the next segment upload must wait for the previous segment upload to complete, that is, several hundreds of ms before the next segment can be uploaded.
Obviously, the traditional HTTP non-piping protocol mode is not suitable in the WAN environment. Currently, NetEase Cloud's SDK supports Http pipeling mode, and uses Http pipling mode for uploading by default. The upload bandwidth of the client is fully utilized, and the upload speed is not very sensitive to the client block size.
In the experimental environment, NetEase Cloud used Raspberry Pi + Facebook Augmented Traffic Control (FaceBook open source network environment simulation tool, which is mainly used to test the performance of FaceBook social network in some weak network environments) to conduct a round of testing on the pipeline, as follows to test the effect. And its performance online is also very good, and it can nearly double the speed on the basis of server and network optimization.
domestic:
foreign:
Connection pool management
The time to complete a tcp 3-way handshake is basically hundreds of ms, so SDKs such as NOS Andriod SDK and iOS SDK maintain a connection pool with the uploading node to avoid the time consumption of connection establishment before each upload.
4 Routing Optimization System
In addition, the WAN system is also in the process of constant adjustment. In order to obtain the best uploading effect, we build a closed-loop system that uses the tone to dynamically track the best route in the WAN and find the optimal strategy.
Online keynote random routing data => statistics of each routing quality => production of the best system routing => update online routing
more exciting
In addition to the direct transmission acceleration service, NetEase cloud object storage service also provides other diverse service ecosystems in typical data resources, such as pictures, audio and video, and anti-spam, to solve the problem of unstructured data management in the Internet era in one-stop, helping Start your business efficiently.
○ s Rich image processing
○ Native support for video on demand
○ Video screenshot, transcoding service
○ Yidun one-click anti-spam
○ Event notification
○ Rich access control
Learn about NetEase Cloud:
NetEase Cloud Official Website: https://www.163yun.com/
New User Gift Package: https://www.163yun.com/gift
NetEase Cloud Community: https://sq.163yun.com/