Targeted: single 5,000
Large multi shouting, 1,000 or 10,000? Well, tentatively scheduled for 5000 or more. Bandwidth is not enough? Gigabit Ethernet. Hard disk is too slow? SSD.
This article does not consider limiting the IO to discuss the structure and mode only.
Open source world Voip field loudest brand should be FreeSwitch, many users, it can achieve such a large single concurrent it? I think: no.
why not? It is because too many threads, one thread channel, the threads 5000, could not handle: "CPU busy thread context switching, and how dry down to business time" ( "Concurrent way the GO language").
Why not multithreaded?
All in all, I was once multi-threaded fans, designed in 2003 Blue Star voice platform, press 1 channel corresponds to 1 thread way. At that time dozens of lines to normal, even if hundreds of lines of large-scale applications. (Note: FreeSwitch 1.0 only appeared in 2008)
Let's say the benefits of multi-threading.
I designed a scripting language called Koodoo platform for voice, each channel operates independently of the script, the equivalent of running the application on each thread, for example, to run a IVR:
WaitRing (); // wait for a call blockage
Play ( "welcome.wav"); // play welcome words, put over until the next statement
k = "";
Getkeys (k, 1, 20); // Up key, 20 a second reception
if( k=="1" )
Play("menu1.wav");
else if( k=="2" )
Play("menu2.wav");
else
Sleep (5); // 5 seconds delay
Advantage is reflected multithreaded came out, a thread a channel that allows casual jam, the write process becomes very simple, because you do not care about a state machine, a programmer need not convert programming mode, writing a multi-channel single-channel program like writing the same .
In addition to convenient development and application brought by a thread so that a channel structure of the system becomes clear, Bug fewer, more stable.
It brings this advantage is not without cost: the cost of operating system threads too. To memory, for example, a thread of default under Windows stack is 1M, even if doing nothing, running 5,000 empty thread must occupy 5G memory.
Memory not the main problem, insufficient memory can be extended, the main problem in the CPU: thread synchronization channel and growth, and the logic of the CPU unit is limited, typically 4-8, strong server 32, running over 5,000 threads, assuming 1,000 threads running on each CPU,
Then each CPU needs to switch back and forth between on the one thousand threads, resulting in low efficiency of the CPU, and soon all the CPU load increases to 100%, almost paralyzed, can not do anything else.
Because Koodoo language is my own design, approach to the.
The solution: coroutine
We all know that the so-called "coroutine" GO language, this concept should be inherited from Erlang, to solve large concurrent IO of a key.
Coroutine called, the function is performed at the language level is divided into time slots, required to achieve the language level scheduler to allocate these time slots coroutine.
In an operating system to run a thread scheduler, you can execute thousands of coroutine, because coroutines language level is achieved, the context of the overhead is extremely small, and therefore will greatly enhance the efficiency of the CPU.
I have to transform under Koodoo language, but also to achieve a coroutine version, but to do this source code compatible, developers do not need to do anything, just change what configuration, can greatly enhance the performance.
Koodoo each language statement corresponding to a C ++ objects, each object has a pointer nextObj, pointing to the next statement (object), the scheduling granularity is apparent that our statements all objects. It is easy to make a scheduler to switch between the channels.
We also consider the congestion, but the operation is a lot of IO class is clogged, for example: a channel such as a sound done and then switch to the next channel ten seconds later, which is obviously not. Of course, Koodoo IO classes of functions are substantially non-blocking version of the lazy method requires developers to use non-blocking versions, contrary to the principles of the source code that is compatible. Moreover, some statements can not replace, such as Sleep (delay in seconds).
To have surgery on the statement object.
Statement object type for clogging, increase a member variable RunCount, used to record the operation state, the initial value is 0, the following pseudo-code:
Statement Object :: Run ()
{
if( runCount==0 ){
Start here to perform IO, for example, start playback
runCount = 1;
}
else{
if (IO has been completed) {
Close IO
runCount = 0;
}
}
}
At the same time, transform what function to get the next statement:
Obj * statement object :: GetNext ()
{
return(runCount ? this : nextObj);
}
If the IO is not completed on or implementation of this statement object, executing the normal jump to the next one.
The IO and IO because determines whether the execution completion time is very short, can be seen as a non-clogging, with this scheduling method, entirely feasible.
Flexible configuration of the scheduler
Each scheduler runs on an operating system thread. You can specify the number of scheduler (thread).
Scheduling by the scheduler actually the average CPU time allocation, which is the default configuration. Also bind specifically to certain specific channel to the dispenser, there are some channels perform special tasks, such as automatic call distribution queue (ACD), coroutine scheduler runs less, there may be a higher priority.
Contrast Measured: Stunning
test environment:
MacBook Pro 15.2017 版 MacOS Catalina 10.15.3
windows10 64-bit Home Edition under Paralles Desktop virtual machines assigned to two processors, 6G memory.
operation result:
Traditional model (1 1 thread channel), the channel configuration 2000, CPU: 100%, the system is very card. 5000 channel would not have tried.
Coroutine mode (2 scheduler), the channel configuration 2000, CPU: between 6-10%, the system smoothly.
Coroutine mode (2 scheduler), the channel configuration 5000, CPU: between 26-37%, the system smoothly.
