Latency, Chinese translation delays. Throughput, Chinese translation throughput. They are the two most common measure of software systems.
Generally comprises a delay way delay (One-way Latency), and round trip delay (Round Trip Latency), and generally the actual round trip delay measurement. It is measured in units of ms, s, min, h the like.
Generally refers to a certain number of tasks and measured for quite some time in the system unit time or the number of transaction (TPS). Note that "quite some time", not a few seconds, but may be ten minutes, half an hour, a day, weeks or even months. It is measured in units TPS, per unit time, the number of bytes written to the disk, and the like.
Think about a problem:
Low-latency high-throughput necessarily mean it? If not, try counter example.
If there is a website system, each time the client requests the server site, network transmission time (including roundtrip) is 200ms, the server processing the request to 10ms. So if a synchronous request, the delay of 210ms. At this time, if raising the network transmission speed, such as up to 100ms, then the delay is 110ms. This situation does seem to reduce latency throughput can be improved to some extent, mainly because: The system is not the performance bottleneck server processing speed, but in network transmission speed.
Continue to assume that a synchronization request changed to asynchronous request, so now the delay is 100ms, the delay is reduced, but the throughput remains unchanged. So this is a counter-example.
In addition to the above anti exception, there is a more lively counter-example:
火车、飞机运煤：从山西到广州运煤，一列火车100小时（包括往返）可以运输10000t煤，而一架飞机20小时（包括往返）可以运输100t煤
Obviously aircraft coal train delay was significantly lower than coal, but coal 10000t if the test run, the train's coal throughput is much higher than the aircraft:
- The train carrying coal throughput of 100t / h
- Aircraft coal throughput for 5t / h
We can be above the coal scene analogy software systems, train, plane can be compared to coal Web server processes the request, such as Apache and Nginx. When the number of concurrent requests is not high, such as 10000 (I assume) less, perhaps Apache throughput may be better than Nginx, but began to decline dramatically in throughput greater than 10,000, Apache, Nginx and relative to the previous throughput comparison stable. Therefore, when comparing the throughput of Web servers, you must observe their respective performance in the case of gradually increasing the number of concurrent requests.
We can also calculate the delay concurrency and throughput (Concurrency), the following formula:
并发度 = 吞吐量 * 延迟
For example, a task processing takes 1ms, throughput 1000tps, then the degree of concurrency is equal to 1/1000 * 1000 = 1 can be drawn tasking threading model is single-threaded model.
Another example is a disk HDD delay of 8ms, but the throughput can be achieved per second write 40MB, then the amount of data per disk seek can be written as (40 * 10 ^ 6) * (8 * 10 ^ -3) B = 320,000B = 320KB.
Throughput and latency
The following analogy is about the throughput (throughput) and delay (Latency) a. If you engage in network performance optimization, these two concepts you need to know, they are seemingly simple but in reality is not. I believe a lot of people, including me, were once considered large throughput means low latency, high throughput delay would mean smaller. The following analogy may explain this view simply wrong. The analogy from here , I'll make a general translation (non-literal translation).
We can send packets to the network likened to street ATM withdrawals. Everyone started using ATM to withdraw money from the end of the whole process takes a minute, so here is delayed 60 seconds, then the throughput it? Of course 1/60 people / sec. Now the bank upgraded their ATM operating system, each person as long as 30 seconds to complete the remittance! Delay is 30 seconds, the throughput is 1/30 human / sec. Well understood, but in front of the problem still exists, right? Do not panic, see below.
Because this neighborhood came to touch more people, here and now decided to add a bank ATM machine, a total of two ATM machines. Now, one minute can make withdrawals to complete the four people in front of the ATM machine to withdraw money although you still have to use to line 30 seconds! In other words, the delay has not changed, but the throughput increases! Be seen, the throughput can not be improved by decreasing the delay.
Well, now the bank in order to improve service and make a new decision: to withdraw money each customer must fill out a questionnaire at the next after taking their money, but also time with 30 seconds. Well, now you start to withdraw money from an ATM, then the time to complete the questionnaire and leave 60 seconds! In other words, the delay is 60 seconds. The throughput has not changed at all! Within one minute you can still come in four people! It is seen, the delay increases, throughput is not changed.
From this analogy, we can see that the length of the delay is measured for each customer (per application) felt the time, and throughput measures the processing efficiency of the entire bank (the entire operating system), and are two completely different the concept of. With the author's original words are:
In short, the throughput is a function of how many stages are in parallel while latency is a function of how many are in series when there are multiple stages in the processing. The stage with the lowest throughput determines the overall throughput.
As banks in order to improve customer satisfaction not only to its own efficiency, but also at the same time minimize the customer service bank spent, not just the operating system to try to make the network throughput, but also to allow each application to send data delay as small as possible. These are two different objectives.