Tabla de contenido
Combinación de cola y grupo de subprocesos
Objetivos
- Datos comerciales del búfer de cola
- Reconocimiento de hilo inactivo y ocupado
- Control de progreso del procesamiento de tareas
Análisis de objetivos
Control de progreso: Es necesario registrar el total de tareas en la cola y las tareas consumidas por subprocesos para lograr un control completo / total.
Control del grupo de subprocesos: el grupo de subprocesos utiliza contadores para realizar operaciones atómicas y mantener el número de contadores al completar y agregar tareas.
Diseño de grupo de hilos
El grupo de subprocesos es en realidad el proceso de agrupación de un subproceso. Generalmente, se inicializan varios subprocesos. Cuando se utilizan los subprocesos, se toman del grupo. La capacidad total en el grupo se puede utilizar para marcar los subprocesos como ocupados e inactivos. El diseño del grupo de subprocesos es muy similar al diseño de la base de datos JDBC. Por ejemplo, cuánto tiempo para obtener el tiempo de espera de conexión, etc., también se puede lograr en el grupo de subprocesos, el ejemplo del subproceso se usa para realizar tareas, el número de tamaño del grupo de subprocesos no debe ser demasiado grande, generalmente núcleo * 2 o core * 2 + 1, y luego O crea un entero decimal.
Cola: La cola puede ser una lista enlazada o una colección simple. Es necesario diseñar la estrategia de cola de cola correspondiente (entrada, sacar de cola, prioridad), y también es necesario diseñar los métodos correspondientes para facilitar las operaciones externas. Las colas de Java incluyen principalmente lo siguiente métodos de operación:
Grupo de subprocesos: también hay muchas implementaciones de grupos de subprocesos, como los múltiples tipos de grupos de subprocesos de Excutors más utilizados, como:
- Grupo de subprocesos de tipo de escalado automático
- Grupo de hilos de tipo de tamaño fijo
- Grupo de un solo hilo
- Programación del grupo de subprocesos
- Grupo de subprocesos no configurable ...
package java.util.concurrent;
import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;
import java.security.AccessControlContext;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.security.PrivilegedExceptionAction;
import java.security.PrivilegedActionException;
import java.security.AccessControlException;
import sun.security.util.SecurityConstants;
/**
* Factory and utility methods for {@link Executor}, {@link
* ExecutorService}, {@link ScheduledExecutorService}, {@link
* ThreadFactory}, and {@link Callable} classes defined in this
* package. This class supports the following kinds of methods:
*
* <ul>
* <li> Methods that create and return an {@link ExecutorService}
* set up with commonly useful configuration settings.
* <li> Methods that create and return a {@link ScheduledExecutorService}
* set up with commonly useful configuration settings.
* <li> Methods that create and return a "wrapped" ExecutorService, that
* disables reconfiguration by making implementation-specific methods
* inaccessible.
* <li> Methods that create and return a {@link ThreadFactory}
* that sets newly created threads to a known state.
* <li> Methods that create and return a {@link Callable}
* out of other closure-like forms, so they can be used
* in execution methods requiring {@code Callable}.
* </ul>
*
* @since 1.5
* @author Doug Lea
*/
public class Executors {
/**
* Creates a thread pool that reuses a fixed number of threads
* operating off a shared unbounded queue. At any point, at most
* {@code nThreads} threads will be active processing tasks.
* If additional tasks are submitted when all threads are active,
* they will wait in the queue until a thread is available.
* If any thread terminates due to a failure during execution
* prior to shutdown, a new one will take its place if needed to
* execute subsequent tasks. The threads in the pool will exist
* until it is explicitly {@link ExecutorService#shutdown shutdown}.
*
* @param nThreads the number of threads in the pool
* @return the newly created thread pool
* @throws IllegalArgumentException if {@code nThreads <= 0}
*/
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
}
/**
* Creates a thread pool that maintains enough threads to support
* the given parallelism level, and may use multiple queues to
* reduce contention. The parallelism level corresponds to the
* maximum number of threads actively engaged in, or available to
* engage in, task processing. The actual number of threads may
* grow and shrink dynamically. A work-stealing pool makes no
* guarantees about the order in which submitted tasks are
* executed.
*
* @param parallelism the targeted parallelism level
* @return the newly created thread pool
* @throws IllegalArgumentException if {@code parallelism <= 0}
* @since 1.8
*/
public static ExecutorService newWorkStealingPool(int parallelism) {
return new ForkJoinPool
(parallelism,
ForkJoinPool.defaultForkJoinWorkerThreadFactory,
null, true);
}
/**
* Creates a work-stealing thread pool using all
* {@link Runtime#availableProcessors available processors}
* as its target parallelism level.
* @return the newly created thread pool
* @see #newWorkStealingPool(int)
* @since 1.8
*/
public static ExecutorService newWorkStealingPool() {
return new ForkJoinPool
(Runtime.getRuntime().availableProcessors(),
ForkJoinPool.defaultForkJoinWorkerThreadFactory,
null, true);
}
/**
* Creates a thread pool that reuses a fixed number of threads
* operating off a shared unbounded queue, using the provided
* ThreadFactory to create new threads when needed. At any point,
* at most {@code nThreads} threads will be active processing
* tasks. If additional tasks are submitted when all threads are
* active, they will wait in the queue until a thread is
* available. If any thread terminates due to a failure during
* execution prior to shutdown, a new one will take its place if
* needed to execute subsequent tasks. The threads in the pool will
* exist until it is explicitly {@link ExecutorService#shutdown
* shutdown}.
*
* @param nThreads the number of threads in the pool
* @param threadFactory the factory to use when creating new threads
* @return the newly created thread pool
* @throws NullPointerException if threadFactory is null
* @throws IllegalArgumentException if {@code nThreads <= 0}
*/
public static ExecutorService newFixedThreadPool(int nThreads, ThreadFactory threadFactory) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>(),
threadFactory);
}
/**
* Creates an Executor that uses a single worker thread operating
* off an unbounded queue. (Note however that if this single
* thread terminates due to a failure during execution prior to
* shutdown, a new one will take its place if needed to execute
* subsequent tasks.) Tasks are guaranteed to execute
* sequentially, and no more than one task will be active at any
* given time. Unlike the otherwise equivalent
* {@code newFixedThreadPool(1)} the returned executor is
* guaranteed not to be reconfigurable to use additional threads.
*
* @return the newly created single-threaded Executor
*/
public static ExecutorService newSingleThreadExecutor() {
return new FinalizableDelegatedExecutorService
(new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>()));
}
/**
* Creates an Executor that uses a single worker thread operating
* off an unbounded queue, and uses the provided ThreadFactory to
* create a new thread when needed. Unlike the otherwise
* equivalent {@code newFixedThreadPool(1, threadFactory)} the
* returned executor is guaranteed not to be reconfigurable to use
* additional threads.
*
* @param threadFactory the factory to use when creating new
* threads
*
* @return the newly created single-threaded Executor
* @throws NullPointerException if threadFactory is null
*/
public static ExecutorService newSingleThreadExecutor(ThreadFactory threadFactory) {
return new FinalizableDelegatedExecutorService
(new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>(),
threadFactory));
}
/**
* Creates a thread pool that creates new threads as needed, but
* will reuse previously constructed threads when they are
* available. These pools will typically improve the performance
* of programs that execute many short-lived asynchronous tasks.
* Calls to {@code execute} will reuse previously constructed
* threads if available. If no existing thread is available, a new
* thread will be created and added to the pool. Threads that have
* not been used for sixty seconds are terminated and removed from
* the cache. Thus, a pool that remains idle for long enough will
* not consume any resources. Note that pools with similar
* properties but different details (for example, timeout parameters)
* may be created using {@link ThreadPoolExecutor} constructors.
*
* @return the newly created thread pool
*/
public static ExecutorService newCachedThreadPool() {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
}
/**
* Creates a thread pool that creates new threads as needed, but
* will reuse previously constructed threads when they are
* available, and uses the provided
* ThreadFactory to create new threads when needed.
* @param threadFactory the factory to use when creating new threads
* @return the newly created thread pool
* @throws NullPointerException if threadFactory is null
*/
public static ExecutorService newCachedThreadPool(ThreadFactory threadFactory) {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>(),
threadFactory);
}
/**
* Creates a single-threaded executor that can schedule commands
* to run after a given delay, or to execute periodically.
* (Note however that if this single
* thread terminates due to a failure during execution prior to
* shutdown, a new one will take its place if needed to execute
* subsequent tasks.) Tasks are guaranteed to execute
* sequentially, and no more than one task will be active at any
* given time. Unlike the otherwise equivalent
* {@code newScheduledThreadPool(1)} the returned executor is
* guaranteed not to be reconfigurable to use additional threads.
* @return the newly created scheduled executor
*/
public static ScheduledExecutorService newSingleThreadScheduledExecutor() {
return new DelegatedScheduledExecutorService
(new ScheduledThreadPoolExecutor(1));
}
/**
* Creates a single-threaded executor that can schedule commands
* to run after a given delay, or to execute periodically. (Note
* however that if this single thread terminates due to a failure
* during execution prior to shutdown, a new one will take its
* place if needed to execute subsequent tasks.) Tasks are
* guaranteed to execute sequentially, and no more than one task
* will be active at any given time. Unlike the otherwise
* equivalent {@code newScheduledThreadPool(1, threadFactory)}
* the returned executor is guaranteed not to be reconfigurable to
* use additional threads.
* @param threadFactory the factory to use when creating new
* threads
* @return a newly created scheduled executor
* @throws NullPointerException if threadFactory is null
*/
public static ScheduledExecutorService newSingleThreadScheduledExecutor(ThreadFactory threadFactory) {
return new DelegatedScheduledExecutorService
(new ScheduledThreadPoolExecutor(1, threadFactory));
}
/**
* Creates a thread pool that can schedule commands to run after a
* given delay, or to execute periodically.
* @param corePoolSize the number of threads to keep in the pool,
* even if they are idle
* @return a newly created scheduled thread pool
* @throws IllegalArgumentException if {@code corePoolSize < 0}
*/
public static ScheduledExecutorService newScheduledThreadPool(int corePoolSize) {
return new ScheduledThreadPoolExecutor(corePoolSize);
}
/**
* Creates a thread pool that can schedule commands to run after a
* given delay, or to execute periodically.
* @param corePoolSize the number of threads to keep in the pool,
* even if they are idle
* @param threadFactory the factory to use when the executor
* creates a new thread
* @return a newly created scheduled thread pool
* @throws IllegalArgumentException if {@code corePoolSize < 0}
* @throws NullPointerException if threadFactory is null
*/
public static ScheduledExecutorService newScheduledThreadPool(
int corePoolSize, ThreadFactory threadFactory) {
return new ScheduledThreadPoolExecutor(corePoolSize, threadFactory);
}
/**
* Returns an object that delegates all defined {@link
* ExecutorService} methods to the given executor, but not any
* other methods that might otherwise be accessible using
* casts. This provides a way to safely "freeze" configuration and
* disallow tuning of a given concrete implementation.
* @param executor the underlying implementation
* @return an {@code ExecutorService} instance
* @throws NullPointerException if executor null
*/
public static ExecutorService unconfigurableExecutorService(ExecutorService executor) {
if (executor == null)
throw new NullPointerException();
return new DelegatedExecutorService(executor);
}
/**
* Returns an object that delegates all defined {@link
* ScheduledExecutorService} methods to the given executor, but
* not any other methods that might otherwise be accessible using
* casts. This provides a way to safely "freeze" configuration and
* disallow tuning of a given concrete implementation.
* @param executor the underlying implementation
* @return a {@code ScheduledExecutorService} instance
* @throws NullPointerException if executor null
*/
public static ScheduledExecutorService unconfigurableScheduledExecutorService(ScheduledExecutorService executor) {
if (executor == null)
throw new NullPointerException();
return new DelegatedScheduledExecutorService(executor);
}
/**
* Returns a default thread factory used to create new threads.
* This factory creates all new threads used by an Executor in the
* same {@link ThreadGroup}. If there is a {@link
* java.lang.SecurityManager}, it uses the group of {@link
* System#getSecurityManager}, else the group of the thread
* invoking this {@code defaultThreadFactory} method. Each new
* thread is created as a non-daemon thread with priority set to
* the smaller of {@code Thread.NORM_PRIORITY} and the maximum
* priority permitted in the thread group. New threads have names
* accessible via {@link Thread#getName} of
* <em>pool-N-thread-M</em>, where <em>N</em> is the sequence
* number of this factory, and <em>M</em> is the sequence number
* of the thread created by this factory.
* @return a thread factory
*/
public static ThreadFactory defaultThreadFactory() {
return new DefaultThreadFactory();
}
/**
* Returns a thread factory used to create new threads that
* have the same permissions as the current thread.
* This factory creates threads with the same settings as {@link
* Executors#defaultThreadFactory}, additionally setting the
* AccessControlContext and contextClassLoader of new threads to
* be the same as the thread invoking this
* {@code privilegedThreadFactory} method. A new
* {@code privilegedThreadFactory} can be created within an
* {@link AccessController#doPrivileged AccessController.doPrivileged}
* action setting the current thread's access control context to
* create threads with the selected permission settings holding
* within that action.
*
* <p>Note that while tasks running within such threads will have
* the same access control and class loader settings as the
* current thread, they need not have the same {@link
* java.lang.ThreadLocal} or {@link
* java.lang.InheritableThreadLocal} values. If necessary,
* particular values of thread locals can be set or reset before
* any task runs in {@link ThreadPoolExecutor} subclasses using
* {@link ThreadPoolExecutor#beforeExecute(Thread, Runnable)}.
* Also, if it is necessary to initialize worker threads to have
* the same InheritableThreadLocal settings as some other
* designated thread, you can create a custom ThreadFactory in
* which that thread waits for and services requests to create
* others that will inherit its values.
*
* @return a thread factory
* @throws AccessControlException if the current access control
* context does not have permission to both get and set context
* class loader
*/
public static ThreadFactory privilegedThreadFactory() {
return new PrivilegedThreadFactory();
}
/**
* Returns a {@link Callable} object that, when
* called, runs the given task and returns the given result. This
* can be useful when applying methods requiring a
* {@code Callable} to an otherwise resultless action.
* @param task the task to run
* @param result the result to return
* @param <T> the type of the result
* @return a callable object
* @throws NullPointerException if task null
*/
public static <T> Callable<T> callable(Runnable task, T result) {
if (task == null)
throw new NullPointerException();
return new RunnableAdapter<T>(task, result);
}
/**
* Returns a {@link Callable} object that, when
* called, runs the given task and returns {@code null}.
* @param task the task to run
* @return a callable object
* @throws NullPointerException if task null
*/
public static Callable<Object> callable(Runnable task) {
if (task == null)
throw new NullPointerException();
return new RunnableAdapter<Object>(task, null);
}
/**
* Returns a {@link Callable} object that, when
* called, runs the given privileged action and returns its result.
* @param action the privileged action to run
* @return a callable object
* @throws NullPointerException if action null
*/
public static Callable<Object> callable(final PrivilegedAction<?> action) {
if (action == null)
throw new NullPointerException();
return new Callable<Object>() {
public Object call() { return action.run(); }};
}
/**
* Returns a {@link Callable} object that, when
* called, runs the given privileged exception action and returns
* its result.
* @param action the privileged exception action to run
* @return a callable object
* @throws NullPointerException if action null
*/
public static Callable<Object> callable(final PrivilegedExceptionAction<?> action) {
if (action == null)
throw new NullPointerException();
return new Callable<Object>() {
public Object call() throws Exception { return action.run(); }};
}
/**
* Returns a {@link Callable} object that will, when called,
* execute the given {@code callable} under the current access
* control context. This method should normally be invoked within
* an {@link AccessController#doPrivileged AccessController.doPrivileged}
* action to create callables that will, if possible, execute
* under the selected permission settings holding within that
* action; or if not possible, throw an associated {@link
* AccessControlException}.
* @param callable the underlying task
* @param <T> the type of the callable's result
* @return a callable object
* @throws NullPointerException if callable null
*/
public static <T> Callable<T> privilegedCallable(Callable<T> callable) {
if (callable == null)
throw new NullPointerException();
return new PrivilegedCallable<T>(callable);
}
/**
* Returns a {@link Callable} object that will, when called,
* execute the given {@code callable} under the current access
* control context, with the current context class loader as the
* context class loader. This method should normally be invoked
* within an
* {@link AccessController#doPrivileged AccessController.doPrivileged}
* action to create callables that will, if possible, execute
* under the selected permission settings holding within that
* action; or if not possible, throw an associated {@link
* AccessControlException}.
*
* @param callable the underlying task
* @param <T> the type of the callable's result
* @return a callable object
* @throws NullPointerException if callable null
* @throws AccessControlException if the current access control
* context does not have permission to both set and get context
* class loader
*/
public static <T> Callable<T> privilegedCallableUsingCurrentClassLoader(Callable<T> callable) {
if (callable == null)
throw new NullPointerException();
return new PrivilegedCallableUsingCurrentClassLoader<T>(callable);
}
// Non-public classes supporting the public methods
/**
* A callable that runs given task and returns given result
*/
static final class RunnableAdapter<T> implements Callable<T> {
final Runnable task;
final T result;
RunnableAdapter(Runnable task, T result) {
this.task = task;
this.result = result;
}
public T call() {
task.run();
return result;
}
}
/**
* A callable that runs under established access control settings
*/
static final class PrivilegedCallable<T> implements Callable<T> {
private final Callable<T> task;
private final AccessControlContext acc;
PrivilegedCallable(Callable<T> task) {
this.task = task;
this.acc = AccessController.getContext();
}
public T call() throws Exception {
try {
return AccessController.doPrivileged(
new PrivilegedExceptionAction<T>() {
public T run() throws Exception {
return task.call();
}
}, acc);
} catch (PrivilegedActionException e) {
throw e.getException();
}
}
}
/**
* A callable that runs under established access control settings and
* current ClassLoader
*/
static final class PrivilegedCallableUsingCurrentClassLoader<T> implements Callable<T> {
private final Callable<T> task;
private final AccessControlContext acc;
private final ClassLoader ccl;
PrivilegedCallableUsingCurrentClassLoader(Callable<T> task) {
SecurityManager sm = System.getSecurityManager();
if (sm != null) {
// Calls to getContextClassLoader from this class
// never trigger a security check, but we check
// whether our callers have this permission anyways.
sm.checkPermission(SecurityConstants.GET_CLASSLOADER_PERMISSION);
// Whether setContextClassLoader turns out to be necessary
// or not, we fail fast if permission is not available.
sm.checkPermission(new RuntimePermission("setContextClassLoader"));
}
this.task = task;
this.acc = AccessController.getContext();
this.ccl = Thread.currentThread().getContextClassLoader();
}
public T call() throws Exception {
try {
return AccessController.doPrivileged(
new PrivilegedExceptionAction<T>() {
public T run() throws Exception {
Thread t = Thread.currentThread();
ClassLoader cl = t.getContextClassLoader();
if (ccl == cl) {
return task.call();
} else {
t.setContextClassLoader(ccl);
try {
return task.call();
} finally {
t.setContextClassLoader(cl);
}
}
}
}, acc);
} catch (PrivilegedActionException e) {
throw e.getException();
}
}
}
/**
* The default thread factory
*/
static class DefaultThreadFactory implements ThreadFactory {
private static final AtomicInteger poolNumber = new AtomicInteger(1);
private final ThreadGroup group;
private final AtomicInteger threadNumber = new AtomicInteger(1);
private final String namePrefix;
DefaultThreadFactory() {
SecurityManager s = System.getSecurityManager();
group = (s != null) ? s.getThreadGroup() :
Thread.currentThread().getThreadGroup();
namePrefix = "pool-" +
poolNumber.getAndIncrement() +
"-thread-";
}
public Thread newThread(Runnable r) {
Thread t = new Thread(group, r,
namePrefix + threadNumber.getAndIncrement(),
0);
if (t.isDaemon())
t.setDaemon(false);
if (t.getPriority() != Thread.NORM_PRIORITY)
t.setPriority(Thread.NORM_PRIORITY);
return t;
}
}
/**
* Thread factory capturing access control context and class loader
*/
static class PrivilegedThreadFactory extends DefaultThreadFactory {
private final AccessControlContext acc;
private final ClassLoader ccl;
PrivilegedThreadFactory() {
super();
SecurityManager sm = System.getSecurityManager();
if (sm != null) {
// Calls to getContextClassLoader from this class
// never trigger a security check, but we check
// whether our callers have this permission anyways.
sm.checkPermission(SecurityConstants.GET_CLASSLOADER_PERMISSION);
// Fail fast
sm.checkPermission(new RuntimePermission("setContextClassLoader"));
}
this.acc = AccessController.getContext();
this.ccl = Thread.currentThread().getContextClassLoader();
}
public Thread newThread(final Runnable r) {
return super.newThread(new Runnable() {
public void run() {
AccessController.doPrivileged(new PrivilegedAction<Void>() {
public Void run() {
Thread.currentThread().setContextClassLoader(ccl);
r.run();
return null;
}
}, acc);
}
});
}
}
/**
* A wrapper class that exposes only the ExecutorService methods
* of an ExecutorService implementation.
*/
static class DelegatedExecutorService extends AbstractExecutorService {
private final ExecutorService e;
DelegatedExecutorService(ExecutorService executor) { e = executor; }
public void execute(Runnable command) { e.execute(command); }
public void shutdown() { e.shutdown(); }
public List<Runnable> shutdownNow() { return e.shutdownNow(); }
public boolean isShutdown() { return e.isShutdown(); }
public boolean isTerminated() { return e.isTerminated(); }
public boolean awaitTermination(long timeout, TimeUnit unit)
throws InterruptedException {
return e.awaitTermination(timeout, unit);
}
public Future<?> submit(Runnable task) {
return e.submit(task);
}
public <T> Future<T> submit(Callable<T> task) {
return e.submit(task);
}
public <T> Future<T> submit(Runnable task, T result) {
return e.submit(task, result);
}
public <T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks)
throws InterruptedException {
return e.invokeAll(tasks);
}
public <T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks,
long timeout, TimeUnit unit)
throws InterruptedException {
return e.invokeAll(tasks, timeout, unit);
}
public <T> T invokeAny(Collection<? extends Callable<T>> tasks)
throws InterruptedException, ExecutionException {
return e.invokeAny(tasks);
}
public <T> T invokeAny(Collection<? extends Callable<T>> tasks,
long timeout, TimeUnit unit)
throws InterruptedException, ExecutionException, TimeoutException {
return e.invokeAny(tasks, timeout, unit);
}
}
static class FinalizableDelegatedExecutorService
extends DelegatedExecutorService {
FinalizableDelegatedExecutorService(ExecutorService executor) {
super(executor);
}
protected void finalize() {
super.shutdown();
}
}
/**
* A wrapper class that exposes only the ScheduledExecutorService
* methods of a ScheduledExecutorService implementation.
*/
static class DelegatedScheduledExecutorService
extends DelegatedExecutorService
implements ScheduledExecutorService {
private final ScheduledExecutorService e;
DelegatedScheduledExecutorService(ScheduledExecutorService executor) {
super(executor);
e = executor;
}
public ScheduledFuture<?> schedule(Runnable command, long delay, TimeUnit unit) {
return e.schedule(command, delay, unit);
}
public <V> ScheduledFuture<V> schedule(Callable<V> callable, long delay, TimeUnit unit) {
return e.schedule(callable, delay, unit);
}
public ScheduledFuture<?> scheduleAtFixedRate(Runnable command, long initialDelay, long period, TimeUnit unit) {
return e.scheduleAtFixedRate(command, initialDelay, period, unit);
}
public ScheduledFuture<?> scheduleWithFixedDelay(Runnable command, long initialDelay, long delay, TimeUnit unit) {
return e.scheduleWithFixedDelay(command, initialDelay, delay, unit);
}
}
/** Cannot instantiate. */
private Executors() {}
}
Combinación de cola y grupo de subprocesos
Tomemos como ejemplo el almacenamiento de archivos OSS en la nube de Alibaba:
package com.forestar.aliyun.service.oss.queue;
import com.forestar.aliyun.service.oss.bean.FileInfo;
import com.forestar.aliyun.service.oss.tds.OssHttpService;
import lombok.extern.slf4j.Slf4j;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
/**
* @Copyright: 2019-2021
* @FileName: FileUploadQueue.java
* @Author: PJL
* @Date: 2020/12/3 10:35
* @Description: 文件上传队列
*/
@Slf4j
public class FileUploadQueue {
private ConcurrentLinkedQueue<FileInfo> queue = new ConcurrentLinkedQueue<>();
private AtomicLong total = new AtomicLong(0);
private AtomicLong completed = new AtomicLong(0);
private AtomicInteger exeCount = new AtomicInteger(0);
private ExecutorService executorService;
private OssHttpService ossHttpService;
private Boolean started = false;
private int poolSize;
/**
* 文件上传队列初始化
*
* @param ossHttpService
* @param executorPoolSize
* @param startConsumer
*/
public FileUploadQueue(OssHttpService ossHttpService, int executorPoolSize, boolean startConsumer) {
this.poolSize = executorPoolSize;
this.ossHttpService = ossHttpService;
this.executorService = Executors.newWorkStealingPool(this.poolSize);
if (startConsumer) {
this.start();
}
}
/**
* 文件入队列
*
* @param fileInfo
*/
public Long enqueue(FileInfo fileInfo) {
queue.add(fileInfo);
return total.incrementAndGet();
}
/**
* 空闲判断
*
* @return
*/
public Boolean isUnFull() {
return exeCount.get() < poolSize;
}
/**
* 空闲判断
*
* @return
*/
public Boolean isBusy() {
return exeCount.get() > 5 && isUnFull();
}
/**
* 空闲判断
*
* @return
*/
public Boolean isIdle() {
return exeCount.get() <= 5;
}
/**
* 开启消费线程
*/
public void start() {
if (!started) {
new Thread(() -> {
while (true) {
try {
int count = queue.size();
// 线程池消费
if (count > 0 && isUnFull()) {
consumer();
}
// 忙碌延长休眠
if (count > 0 && isBusy()) {
Thread.sleep(100);
}
// 空闲缩短休眠
else if (count > 0 && isIdle()) {
Thread.sleep(50);
}else{
Thread.sleep(1000);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}).start();
started = true;
}
}
/**
* 执行业务处理
*/
private void consumer() {
FileInfo fileInfo = queue.poll();
if (null != fileInfo) {
// 增加线程占用数量
exeCount.incrementAndGet();
// 提交执行任务
executorService.submit(() -> {
// 处理业务数据
ossHttpService.syncToAliyunOss(fileInfo);
// 执行个数增加
completed.incrementAndGet();
// 线程池占用减少
if (exeCount.get() > 0) {
exeCount.decrementAndGet();
}
// 打印处理进度
log.info("===队列消费进度==={}/{}", completed.get(), total.get());
// 处理完成通知
if (completed.get() == total.get()) {
log.info("=====================所有文件上传完成!=======================");
}
});
}
}
}
Al final obtenemos una salida de efecto similar a la siguiente:
2020-12-03 12:38:08.802 INFO 9792 --- [pool-3-thread-1] c.f.a.service.oss.tds.OssHttpService : >>>开始解析....D:/TDSpath/list.txt
2020-12-03 12:38:08.804 INFO 9792 --- [pool-3-thread-1] c.f.a.service.oss.tds.OssHttpService : httpUrl = http://remote_host/upload/eventAttach/eventpic/original/202007/20200711/05290729-bf44-41c6-8da7-44251e131d15.jpg , objectName = /upload/eventAttach/eventpic/original/202007/20200711/05290729-bf44-41c6-8da7-44251e131d15.jpg
2020-12-03 12:38:08.804 INFO 9792 --- [pool-3-thread-1] c.f.a.service.oss.tds.OssHttpService : ==文件队列长度== size =1
2020-12-03 12:38:08.805 INFO 9792 --- [pool-3-thread-1] c.f.a.service.oss.tds.OssHttpService : httpUrl = http://remote_host/upload/eventAttach/eventpic/thumb/202007/20200711/05290729-bf44-41c6-8da7-44251e131d15.jpg , objectName = /upload/eventAttach/eventpic/thumb/202007/20200711/05290729-bf44-41c6-8da7-44251e131d15.jpg
2020-12-03 12:38:08.805 INFO 9792 --- [pool-3-thread-1] c.f.a.service.oss.tds.OssHttpService : ==文件队列长度== size =2
2020-12-03 12:38:08.805 INFO 9792 --- [pool-3-thread-1] c.f.a.service.oss.tds.OssHttpService : httpUrl = http://remote_host/upload/eventAttach/eventpic/original/202007/20200711/15d1724a-a969-41f7-abe8-171904a746da.jpg , objectName = /upload/eventAttach/eventpic/original/202007/20200711/15d1724a-a969-41f7-abe8-171904a746da.jpg
2020-12-03 12:38:08.805 INFO 9792 --- [pool-3-thread-1] c.f.a.service.oss.tds.OssHttpService : ==文件队列长度== size =3
2020-12-03 12:38:08.806 INFO 9792 --- [pool-3-thread-1] c.f.a.service.oss.tds.OssHttpService : httpUrl = http://remote_host/upload/eventAttach/eventpic/thumb/202007/20200711/15d1724a-a969-41f7-abe8-171904a746da.jpg , objectName = /upload/eventAttach/eventpic/thumb/202007/20200711/15d1724a-a969-41f7-abe8-171904a746da.jpg
2020-12-03 12:38:08.807 INFO 9792 --- [pool-3-thread-1] c.f.a.service.oss.tds.OssHttpService : ==文件队列长度== size =4
2020-12-03 12:38:09.805 INFO 9792 --- [Pool-3-worker-9] c.f.a.service.oss.queue.FileUploadQueue : ===队列消费进度===1/4
2020-12-03 12:38:09.856 INFO 9792 --- [Pool-3-worker-9] c.f.a.service.oss.queue.FileUploadQueue : ===队列消费进度===2/4
2020-12-03 12:38:09.907 INFO 9792 --- [Pool-3-worker-9] c.f.a.service.oss.queue.FileUploadQueue : ===队列消费进度===3/4
2020-12-03 12:38:09.957 INFO 9792 --- [Pool-3-worker-9] c.f.a.service.oss.queue.FileUploadQueue : ===队列消费进度===4/4
2020-12-03 12:38:09.957 INFO 9792 --- [Pool-3-worker-9] c.f.a.service.oss.queue.FileUploadQueue : =====================所有文件上传完成!=======================