Previously, in the Java new features column , we briefly introduced the virtual thread officially released in Java 21 .
Yesterday, I happened to see a video explaining this content. It was very good, so DD has translated it for everyone. If you are interested, you can take a look. You can learn more about virtual threads.
What is a virtual thread? What will happen if 10 million are activated at one time?
This video uses the Chrome plug-in Youtube Chinese Dubbing for translation and dubbing processing. If you usually watch cutting-edge videos on YouTube, you can also install one, which can effectively improve your learning efficiency .
If your current environment is not suitable for watching videos, you can also learn from the text content below. The following content is summarized by me based on the video content, which will be more concise.
What is a virtual thread
Virtual threads are a new concept added in the field of Java concurrency, so what exactly are virtual threads used for?
According to the content in the JEP, virtual threads are lightweight threads that can significantly help us reduce the workload of writing, maintaining, and observing high-throughput applications. Its implementation goals include the following:
- Programs written in a thread-per-request style can scale with near-optimal hardware utilization.
What is one thread per request style?
For an HTTP server, this means that each HTTP request is handled by its own thread. For a relational database server, this means that each SQL transaction is also handled by its own thread. If you've ever used a Java EE server, this is how it works. So, what is the thread-per-request style is: one request = one transaction = one thread .
So, how much does this model cost? To understand this cost, you need to understand the cost of threads in Java. Costs in platform threads and CPU usage.
Java threads were created in earlier versions of Java as platform threads, also known as thin wrappers over operating system threads. There are two things you need to know about them.
- Platform threads need to store their call stacks in memory
- It is a system resource and it takes about a millisecond to start the platform thread
In fact, platform threads are a fairly expensive resource. How can such threads be used to optimize hardware utilization?
Assume that your application has 16 GB of available memory. Divided by the thread size of 20 MB, there is room for 800 threads on such a machine. Let's say these threads are doing some I/O, like accessing resources on the network. Assume that the resource is accessed within 100 milliseconds. Preparing the request and processing the response will be completed within 10 nanoseconds. Assume that all these in-memory calculations take 1000 nanoseconds. This means there is a factor of about 100000 between preparing the request and processing the response, and the time it takes to get the response, during which time your thread is just sitting there doing nothing. So if you have 800 of these threads, the CPU utilization is only a measly 0.8%.
If you double the memory to 32 GB, then the CPU utilization can reach 1.3%, but this is still low.
Think about it the other way around, if we want to achieve 90% CPU utilization. This would require 90,000 threads, which would take 90 seconds to start, and consume 1.8 TB of memory.
It's clear that platform threads are too expensive to scale with near-optimal hardware utilization. Therefore, we need another threading model to solve such problems.
- Enables existing code based on classic Java threads to use virtual threads with minimal change .
This goal means that everything done by classic threads can be easily converted to a virtual thread processing method. Several key points are covered here.
- A virtual thread can run any Java code or any native code.
- You don't need to learn any new concepts.
- But you need to unlearn certain thoughts, such as:
- Virtual threads are cheap, about 1,000 times cheaper than traditional platform threads.
- The cost of blocking a virtual thread is also very low, so trying to avoid blocking a virtual thread is useless.
- It's OK to write classic blocking code, which is good news because blocking code is easier to write than asynchronous code. At this point, you may be wondering, is pooling virtual threads a good idea? Well, the answer is no. Don't do that. You are just wasting your time.
Two more good news about virtual threads: thread-local variables work the same way; synchronization also works. There are a few things to say about synchronization. The virtual thread still runs on top of the platform thread, and there is a platform thread below. However, this virtual thread can be detached from its platform thread so that the platform thread can run another virtual thread. When can we leave? A virtual thread can be detached from its platform thread once blocked. It may be blocked on I/O operations or synchronization operations, or it may be put to sleep. If a virtual thread is executing some code inside a synchronized block, it cannot detach from its platform thread. Therefore, it blocks the platform thread while this synchronized code block is running. If this time is short, that's okay. There is no need to panic or take any steps to prevent this from happening. If this takes a long time, that is, if it's doing some long I/O operations, then things aren't so good. You can prevent this from happening by simply synchronizedreplacing the call to with a reentrant lock.
Dig deeper into coding
Regarding how to create a virtual thread, it was mentioned in the previous Java 21 new feature virtual thread . Just pass Thread.ofVirtual(), for example:
Thread.ofVirtual()
.name("didispace-virtual-thread")
.start(runnable);
Tips: If you want to create a platform thread, you can use:
Thread.ofPlatform()
Virtual threads work on top of platform threads. You might think there's no performance gain, just overhead. So what's going on? There is more to say about virtual threads. Let’s take a look at how this code works.
In this code, streaming mode is used to create 10 virtual, unstarted threads. The task these threads are running is just printing the current thread. Then, let them sleep for 10 milliseconds, and then print the thread's name again. Finally, start these unstarted threads and call the join method to ensure everything is visible on the console.
So run this code and you'll see that something really unexpected is going on here.
When thread 7 of this ForkJoinPool comes back from sleep, it does not continue to run on the original platform thread, but jumps to another platform thread. If you're doing this on your own machine, make sure you start enough virtual threads, as you probably won't observe this with just one or two threads.
How it works behind the scenes
In fact, when a virtual thread is blocked due to some operation, the corresponding stack is moved from the platform thread on which it runs to heap memory. So, now this platform thread is free to run another virtual thread. When the task receives a signal that it can continue running, its stack is moved from the heap back to the platform thread, but not necessarily the same. So, that's the cost of blocking a virtual thread, moving that virtual thread's stack to main memory and back. Blocking virtual threads is not free, but it is much more cost-effective than blocking platform threads.
Tips: There is a graphical explanation in this logic video. It is recommended to watch it in conjunction with the video animation to make it easier to understand.
Happily, all of the JDK's blocking operations have been refactored to take advantage of it. These include I/O operations, synchronization and Thread.sleep.
How many platform threads are needed to run the virtual thread
Regarding this issue, we can test it. Let me create virtual threads and collect all corresponding platform thread names.
The code basically starts five virtual threads and then uses some code to extract the pool name and platform thread name. Finally, it just prints different statistics, the time it took to run this code, the number of cores on the CPU, the number of thread pools, and the number of platform threads.
So let me run this code and I can see the following results:
For 5 virtual threads, it uses 3 platform threads and takes 2 milliseconds.
Let me use 10 virtual threads and run the code again.
With 10 threads, it still used 3 platform threads and took 4 milliseconds.
Let me use 100 virtual threads and run the code again.
Now it uses 7 platform threads.
Let's see what happens with 1,000 virtual threads.
It still uses 7 platform threads.
How about trying 100,000 virtual threads?
Now it uses 8 platform threads and takes 156 milliseconds.
By the way, even if these threads aren't doing much, just some string manipulation and adding elements to the concurrent set, you can see that it only takes 156 milliseconds to run all of these threads.
Now let me increase this to 1 million threads.
It took less than a second and still used 8 platform threads.
Start 10 million virtual threads
How about we try launching 10 million virtual threads? Have you ever tried doing this? Launch 10 million platform threads on your machine? Well, normally this is not possible, but using virtual threads we might be able to do it. We can get the following results:
This was just tested on an old laptop and it took less than 7 seconds, which is pretty awesome!
This is virtual threading in Java! Isn't it great? So, have you already started upgrading to Java 21 and started using this feature to improve the performance of your applications? Let’s chat together in the message area.
Coding is not easy. If you like this article, please like, read, and forward to support it. What if you encounter difficulties while studying? You can join our ultra-high-quality technical exchange group to participate in exchanges and discussions for better learning and progress! Also, don’t go away and follow me! Continuously update the Java new features column !