Chapter 6: Task Execution
Most concurrent applications around the implementation of tasks to manage (task) of. The so-called task is abstract, discrete units of work (unit of work).
- An application to work (Work) isolated tasks can be simplified management procedures;
- This separation is still divided between different transactions of the natural dividing line, you can easily program to recover when things go wrong;
- At the same time this separation can also work in parallel to provide a natural structure is conducive to improve concurrency program.
Perform tasks in a thread
Ideally, the task is to separate activities: its work does not depend on the status of other tasks, or the result of boundary effects (side effect).
Independent favor of concurrency, if we can get the appropriate processor resources, independent tasks can also be performed in parallel.
Under normal load, the server application should both good throughput and rapid responsiveness .
Further, when the application should overloading gently deteriorated , a load should not simply another job failed.
For these purposes, you have to choose a clear task boundaries , and with a clear task execution policy .
Most server applications are selected following a natural boundary task: individual customer requests.
Web server, mail server, file server, EJB container, and database servers that accept remote client connection requests sent over the network.
The independent task request as a boundary, allowing tasks both independence and appropriate size.
For example, the result after submitting a message to the mail server, and will not be affected by other messages are processed simultaneously.
Sequentially perform tasks
Internal task scheduling application, there are a variety of possible scheduling policies, which can play a potential concurrency in varying degrees.
The simplest strategy is to perform tasks sequentially in a single thread.
/**
* 顺序化的 Web Server
*/
class SingleThreadWebServer {
public static void main(String[] args) throws IOException {
ServerSocket socket = new ServerSocket(80);
while (true) {
Socket connection = socket.accept();
handleRequest(connection);
}
}
}
On this example, the main thread running constantly alternating between "accept connection" and "process-related request", and the main thread until the completion of the current request and call again acceptafter the new request must wait.
The process comprises performing a Web request for IO operations and operation.
The server must handle Socket I / O, in response to read requests and writeback, or a communication network congestion problems will result from blocking this.
Also document processing server I / O, database requests transmission, which will also cause clogging of the operation.
This efficiency in production environments will be very bad.
In some cases, sequential treatment has advantages in simplicity or security; most GUI framework uses a single thread, and sequentially processing tasks.
We will discuss again the order of the model in Chapter 9.
Explicitly create threads for the task
In order to provide better responsiveness, create a new thread for each service may be requested.
/**
* Web Server为每个请求启动一个新的线程
*/
class ThreadPerTaskWebServer {
public static void main(String[] args) throws IOException {
ServerSocket socket = new ServerSocket(80);
while (true) {
final Socket connection = socket.accept();
Runnable task = () -> handleRequest(connection);
new Thread(task).start();
}
}
}
ThreadPerTaskWebServerSingle-threaded version is similar in structure - the main thread is still constantly alternating operation "Accept external connections" and "forward requests."
The difference is that each connection creates a new thread to process the request, instead of inside the main processing loop of the main loop it is. This results in the following three main conclusions:
- Load mission has been out of the main thread, which makes the main loop to more quickly resume waiting for the next connection. This makes it can accept a new request before the completion of the foregoing request, thereby improving the responsiveness.
- Parallel processing tasks, which makes it possible to obtain a plurality of service requests simultaneously. If there are multiple processors, or for the I / O is not completed, any factor lock request and the resource availability required blocked task, the program will be improved throughput.
- Tasking code must be thread-safe, because there are multiple tasks concurrently calls it.
Under moderate load levels, a "per each task thread (thread-per-task)" is a good method of improving the order of execution.
As long as the arrival rate of request server request processing capability has not been exceeded, then this method can lead to faster response and greater throughput at the same time. .
Unlimited create disadvantages thread
Will become more prominent When used in a production environment, there is a method "task per each thread (thread-per-task)" some real shortcomings, especially in the need to create a large number of threads:
- Thread lifecycle overhead . Creating and closing threads is not "free". The actual cost varies depending on the platform, but create a thread does take time to bring the delay processing of the request, and the need for appropriate processing activities between the JVM and the operating system. If the request is frequent and lightweight, the same as most server programs, then for each request to create a new thread approach would consume a large amount of computing resources.
- Resource consumption . Active threads consume system resources, especially memory. If the number of threads that can run more than the available number of processors, the threads will be idle. A large number of idle threads take up more memory, to bring pressure on the garbage collector, and a large number of threads in competing for CPU resources, but also have other performance overhead. If you already have enough thread to keep the CPU busy all, then re-create more threads are harmful to one's interest.
- Stability . You should limit the number of threads that can be created. By limiting the number of different platform dependent, but also by the JVM startup parameters, and other factors affect the stack size Thread constructor requested, and to limit the underlying operating system threads. If you break these restrictions, the most likely outcome is a receipt
OutOfMemoryError. Attempt to recover from this error is very dangerous, simpler way is to avoid exceeding these limits when constructing your program.
Executor framework
Mission is to work on a logical unit, the task thread is executed asynchronously mechanism.
We have analyzed the strategies of two types of threads to perform tasks - all tasks are performed sequentially in a single thread, and the execution of each task in its own thread.
Each strategy has serious limitations: the order of execution will produce poor responsiveness and throughput, "Every task per-thread" resource management will bring trouble.
Thread pool (ThreadPool) provides benefits for thread management.
In the Java class libraries, the primary task execution is not abstract Thread, but Executor.
public interface Executor {
void execute(Runnable command);
}
It is a job submission and task execution decoupling between provides a standard method for use Runnableprovides a common way to describe the task.
ExecutorBased on producer - consumer model.
Perform tasks submitted by the producer (to be completed to produce a unit of work), mission thread is the consumer (consume these work units).
If you are implementing a program producer - consumer design, Executoroften the easiest way.
/**
* 使用线程池的 WebServer
*/
class TaskExecutionWebServer {
private static final int NTHREADS = 100;
private static final Executor exec = Executore.newFixedThreadPool(NTHREADS);
public static void main(String[] args) throws IOException {
ServerSocket socket = new ServerSocket(80);
while (true) {
final Socket connection = socket.accept();
Runnable task = () -> handleRequest(connection);
exec.execute(task);
}
}
}
Thread Pool
As the name referred to as a managed thread pool worker thread pool of homogeneous (homogeneous pool).
Thread pool is a work queue (work queue) tightly bound.
The so-called work queue, its role is to hold all the tasks waiting to be executed.
Life from the worker thread relaxed: it gets the next task from the work queue, execute it, and then come back to wait for another thread.
Thread pool Advantages:
- Reuse existing threads, rather than create a new thread, which can be offset by a thread created when processing multiple requests, the demise of expenses incurred.
- When a request arrives, the worker thread is usually already exist, the waiting time for creating threads and will not delay the implementation of tasks, thus improving responsiveness.
- By appropriately adjusting the size of the thread pool threads can be enough to keep the processor busy, but also can prevent too many threads competing resources, cause the application to fail or run out of memory.
Library provides a flexible thread pool implementation and some useful preset configuration. By calling Executorsto create a thread pool in a static factory method:
- newFixedThreadPool
- newCachedThreadPool
- newSingleThreadExecutor
- newScheduledThreadPool
Alibaba Java Developer's Handbook on Executor, is as follows:
Thread pool are not allowed
Executorsto create, but byThreadPoolExecutorthe way, this approach allows the students to write more explicit operating rules thread pool, to avoid the risk of resource depletion.
.
Description:Executorsshortcomings thread pool object is returned as follows:
1)FixedThreadPoolandSingleThreadPool: the request queue length is allowedInteger.MAX_VALUE, a large number of requests may accumulate, resulting in the OOM.
2)CachedThreadPool: Allows the number of threads createdInteger.MAX_VALUEcould create a large number of threads, resulting OOM.
Executor lifecycle
JVM will in all (non-background, nondaemon) after the termination of all the threads exit. So, if you can not shut down properly Executor, it will prevent the end of the JVM.
To solve this problem perform the service lifecycle, Executorserviceinterface extends Executorand adds a number of methods for life-cycle management.
/**
* ExecutorService 中的生命周期方法
*/
public interface ExecutorService extends Executor {
void shutdown();
List<Runnable> shutdownNow();
boolean isShutdown();
boolean isTerminated();
boolean awaitTermination(long t imeout, TimeUnit unit) throws InterruptedException;
// ...其他用于任务提交的便利方法
}
ExecutorserviceIt implies that life cycle has three states: running (running), close (shutting down) and termination (terminated).
ExecutorServiceThe initial state after initially created is running.
shutdownThe method starts a gradual shutdown process: Stop accepting new tasks while waiting for the completion of the task has been submitted - including task execution has not begun.
shutdownNowThe method starts a forced shutdown process: Try to cancel all running tasks and scheduling tasks in the queue has not yet started.
After closing submission to ExecutorServicethe task, it will be rejected execution processor (rejected execution handler) process (see Chapter VIII).
/**
* 支持关闭操作的 Web Server
*/
class LifecycleWebServer {
private final ExecutorService exec = ...;
public void start() throws IOException {
ServerSocket socket = new ServerSocket(80);
while (!exec.isShutdown()) {
try {
final Socket conn = socket.accept();
exec.execute((Runnable) () -> handleRequest(conn));
} catch (RejectedExecutionException e) {
if (!exec.isShutdown()) {
log("task submission rejected", e);
}
}
}
}
void handleRequest(Socket connection){
Request req = readRequest(connection);
if (isShutdownRequest(req)) {
stop();
} else {
dispatchRequest(req);
}
}
private void stop() {
exec.shutdown();
}
}
Looking parallelism can be enhanced
The results can carry tasks: Callable and Future
You can be a Runnableor a Callablesubmission to executor, and then get a Futureuse it to retrieve the results of task execution, or cancel the job.
You can explicitly for a given Runnableor Callableinstantiate a FutureTask. ( FutureTaskAchieved Runnable, they can either submit it to Executorbe performed, they can directly call the runmethod run.)
If the task has been completed, get returns immediately or throw a Exception; if the task is not finished, get blocks until it is completed.
If the task throws an exception, get an exception will be the package is ExecutionExceptionthen re-raise;
If a task is canceled, get throws CancellationException.
When thrown ExecutionException, the it can be used getCauseto regain the original encapsulated exception.
A large number of independent and kind of concurrent processing tasks, assignments will be assigned to a program of different tasks in order to really get performance improvements.
CompletionService: When Executor met BlockingQueue
CompletionServiceIntegration Executorand BlockingQueuefunctionality.
You can Callablesubmit it to the task to perform in the queue and then use similar takeand pollmethods, to obtain this result in the complete results are available as a packaged Future.
ExecutorCompletionserviceIt is to achieve CompletionServicea class interface, and a computing tasks entrusted to Executor.
ExecutorCompletionServiceCreate a constructor inBlockingQueue, use it to save the results accomplished. When the calculation is complete it will callFutureTaskthedonemethod.- When submitted to a task, the first task of a package
QueueingFuture, which isFutureTaska sub-class, and then overridedonethe method, the results placedBlockingQueuein.takeAndpollmethods entrusted toBlockingQueueit will be blocked while the results are not available.
Use CompletionService, two ways we can improve the performance of page renderer: reduce the overall run time and improve responsiveness.
We can be created for each image to download a separate task, and execute them in the thread pool, the download process sequence is converted to parallel: It can reduce the total time to download all images.
And from the Completionserviceget results, as long as any one image download is complete, on the show immediately. As a result, we can provide a more dynamic and have more responsive user interface to the user.
/**
* 使用 CompletionService渲染可用的页面元素
*/
public class Renderer {
private final ExecutorService executor;
Renderer(ExecutorService executor) { this.executor = executor; }
void renderPage(CharSequence source) {
final List<ImageInfo> info = scanForImageInfo(source);
CompletionService<ImageData> completionService = new ExecutorCompletionService<ImageData>(executor);
for (final ImageInfo imageInfo : info) {
completionService.submit(new Callable<ImageData>() {
public ImageData call() {
return imageInfo.downloadImage();
}
});
renderText(source);
try {
for(int t = 0, n = info.size(); t < n; t++){
Future<ImageData> f = comp1etionService.take();
ImageData imageData = f.get();
renderImage(imageData);
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} catch (ExecutionException e) {
throw launderThrowable(e.getCause());
}
}
}
}
Set time limits for the task
Sometimes, if an activity can not be completed within a determined time, then its result becomes ineffective, then the program can give up the activity.
Future.getTime-limited versions to meet this condition: it is ready to return after the results immediately, if not ready within the time limit, they will be thrown TimeoutException.
If a limit of getthrows TimeoutException, you can Futurecancel the task.
/**
* 在预定时间内获取广告信息
*/
Page renderPageWithAd() throws InterruptedException{
long endNanos = System.nanoTime() + TIME_BUDGET;
Future<Ad> f = exec.submit(new FetchAdTask());
// 等待广告是呈现页面
Page page = renderPageBody();
Ad ad;
try{
// 在预计时间内等待
long timeLeft = endNanos - System.nanoTime();
ad = f.get(timeLeft, NANOSECONDS);
}catch(ExecutionException e){
ad = DEFAULT_AD;
}catch(TimeoutException e){
ad = DEFAULT_AD;
f.cancel(true);
}
page.setAd(ad);
return page;
}
to sum up
To construct applications that can simplify development, facilitate synchronous around the mission.
ExecutorFramework helps you decoupling between the submission and implementation of the policy task of the task, and also supports many different types of execution strategies.
When you find yourself and create a thread to perform a task, consider Executorreplacing the previous method.
The application into different tasks, in order to produce the greatest benefits of this behavior, you must specify a clear mission boundaries.
In some applications, there are obvious good working border task, but there are some programs, you need to do further analysis to reveal more can enhance parallelism.
