Intranet penetration imitating Shenzhuo Internet's delayed bandwidth technology implementation

writing background:

Because the public security patrol system of a city previously developed by the company uses a web architecture, the team has been testing it before it goes online. Due to the epidemic situation, several colleagues work from home, which requires the use of intranet penetration technology to talk about the intranet The project is mapped to the public network. The company installed the Shenzhuo Internet intranet penetration client, but the company opened a version with 2M bandwidth. I am worried that the insufficient bandwidth will make the product manager feel that the page loading is slow because I did not make an interface. Optimization, but when I actually used it, I didn't find that the page access was slow, but it was very smooth, which aroused my curiosity.

Why is the initial speed of Shenzhuo Internet so fast?

The first thing I thought was that the bandwidth must be greater than 2M, so I downloaded a large file to test the download speed, and found that the download speed was the fastest at first, and then the speed gradually tended to be stable. The final download speed was indeed 2M. It's really no more, no less, just 2M standard bandwidth.
 

Latency Bandwidth Technology

After continuous testing, I finally found a technology called delayed bandwidth, which is to set a highest bandwidth at the beginning and gradually reduce it to the specified bandwidth. For example, if the user's bandwidth is limited to 1M, then the user uses 1M bandwidth speed It is definitely not fast. If the user experience is good, the speed is fast, and the bandwidth cannot be increased, then the delayed bandwidth technology is the best solution. When we watch videos at home, it is often the most bandwidth-intensive loading at the beginning. After the loading is completed In fact, the bandwidth requirements are not very large. If you use the delayed bandwidth technology, you should increase the bandwidth appropriately at the beginning of loading, and then control the flow to the final bandwidth after loading.

How to achieve latency bandwidth

With the attitude of breaking the casserole and asking the end, I decided to try to make this function myself

Choosing between C and java, I decided to try it with java first, because after all, the development efficiency of C language is significantly lower than that of java.

 

Checked the relevant information

Here's what I saw online:

1. Assuming that the upper limit of the download or upload speed is m (KB/s), then the time spent sending a fixed byte of data (assumed to be n bytes) is: n/m; 2. Assuming that you want to send n bytes
now data, then the time required in theory should be n/m, but in reality, it only takes t seconds to send n bytes of data, then the sending thread should sleep for n/mt seconds, so that it is basically realized speed control.
 

However, I personally think that such a method has relatively large errors and cannot achieve precise flow control, so I continued to consult relevant information.

Summarized as follows:

Limiting interface traffic in JAVA can be implemented through the RateLimiter class of Guava or the Semaphore class that comes with JDK. The two are somewhat similar, but there are differences, and should be used according to the actual situation.

The RateLimiter class is to control the access interface at a certain rate.

The Semaphore class is to control the number of concurrent access interfaces allowed at the same time.

The token bucket algorithm is mostly used in traffic limitation, and only when a thread gets a token can it be released, and the passing rate can be controlled by setting the number of tokens generated per second. A simple demo:

public class TestRateLimiter implements Runnable {

    public static final SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
// 允许每秒最多500个任务

    public static final RateLimiter limiter = RateLimiter.create(500); 
    public static void main(String[] arg) {
        for (int i = 0; i < 50; i++) {
            limiter.acquire(); 
            Thread t = new Thread(new TestRateLimiter());
            t.start();
        }
    }

    public void run() {
        System.out.println(sdf.format(new Date()) + " 任务结束");
    }
}

There is another way to use semaphores. By setting the total number of semaphores, the thread can only execute when it gets the semaphore, and release the semaphore when the execution is completed. In order to control the number of concurrency of the interface. A simple demo:

The second parameter true in the initialization means to obtain the semaphore in a fair way, that is, the principle of first-in-first-out.

public class TestSemaphore implements Runnable {

    public static final SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");

// 允许并发的任务量限制为20个

    public static final Semaphore semaphore = new Semaphore(20, true); 
    public static void main(String[] arg) {
        for (int i = 0; i < 60; i++) {
             Thread t = new Thread(new TestSemaphore());
             t.start();
        }
    }

    public void run() {
        try {
// 获取信号量,不足会阻塞
            semaphore.acquire(); 
            System.out.println(sdf.format(new Date()) + " 任务开始");
            Thread.sleep(5000);
            System.out.println(sdf.format(new Date()) + " 任务结束");
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
// 释放信号量
            semaphore.release(); 
        }
    }
}

The above are two ideas of using token bucket algorithm and semaphore to realize flow control, and the completed version is still being improved.

 

Just now I saw someone saying why flow control is needed. Wouldn’t it be better not to control it? Isn’t it delicious? In fact, flow control is something that must be done.

Why flow control settings are needed

1. The concept of flow control

The speed at which the receiving end can process data is limited. If the speed of the sending end is too fast, the buffer will be filled. If you continue to send data at this time, it will cause a series of chain reactions such as packet loss.

Therefore, TCP supports determining the sending speed of the sender according to the capabilities of the receiver. This mechanism is called flow control.

2. Flow Control

Its specific operation is that the receiving end host notifies the sending end host of the size of the data that it can receive, so the sending end will send data that does not exceed this limit. This size limit is called the window size. The value of the window size is determined by the receiving host.

3. Window size

In the header of the TCP message, there is a 16-bit window field, which is used to indicate the buffer size of the receiver. The receiving host puts the buffer size it can receive into this field to notify the sender. The larger the value of this field, the higher the throughput of the network. However, once the buffer at the receiving end faces data overflow, the value of the window size will also be set to a smaller value to notify the sending end, thereby controlling the amount of data sent. That is to say, the sending host will control the amount of sent data according to the instructions of the receiving host.

Flow control for TCP is provided by each end of the connection through a declared window size. The window size is the number of bytes starting at the value specified in the Acknowledgment Sequence Number field, which is the number of bytes the receiver expects to receive. The window size is a 16-bit field, so the maximum window size is 65535 bytes.

4. Window expansion factor M

TCP is based on the mechanism of advertising the window size, allowing the sender to send multiple packets in succession before stopping and waiting for an acknowledgment. Since the sender does not have to stop and wait for confirmation every time a packet is sent, the protocol can speed up the transmission of data. The TCP transmission rate is positively correlated with the window size, and in some cases, increasing the window size can increase the transmission rate.

However, the size of the TCP window is only 16 bits, and the maximum value is 65535 bytes, which is already standard for the current Gigabit interface. In the reality that the data center starts to deploy 10G interfaces on the server, the window of 65535 bytes is obviously not enough.

There are 7 common TCP options, among which kind=3 is the window expansion factor option. When the TCP connection is initialized, the communication parties use this option to negotiate the window expansion factor of the received pass. Assuming that the notification window size in the TCP header is N, and the window expansion factor (displacement number) is M, then the actual receiving notification window size of the TCP segment is: N * (2 *M). The value range of M is 0-14. In this case, the maximum notification window is about 1GB, which can meet the needs of most applications.
 

Host A sends 1000bit data to host B. At this time, host B tells host A that my window size is 3000, and you can send more.

When host B receives data segments starting from 3001, its buffer is full.

Host A had to be told to temporarily stop sending data, and then the window was notified of the update before it could continue.

If this notification is lost en route, further communication may not be possible.

So the sender will send a window detection data segment from time to time.

This data port contains only one byte to get the latest window size.

When host B finishes processing, it will tell host A that I have 2000 space left. You can continue sending.