"Multi-thread implementation, synchronization, asynchrony, optimization" of the interview questions for Java architects in Beijing, Shanghai, Guangzhou and Shenzhen

Create unlimited threads

In a web server, under normal load conditions, assigning a thread to each task can improve performance under serial execution conditions. As long as the arrival rate of requests does not exceed the server's request processing capacity, this approach can bring both faster responsiveness and higher throughput. If the rate of arrival of requests is very high, and the processing of requests is lightweight, creating a new thread for each request will consume a lot of computing resources.

raised problems

1. Threads have very high lifetime overhead

2. Excessive CPU resource consumption

   If the number of runnable threads is more than the number of available processors, then some threads will be idle. A large number of idle threads will take up a lot of memory, put pressure on the garbage collector, and there will be other performance overhead when a large number of threads compete for CPU resources.

3. reduce stability

   The JVM has a limit on the number of threads that can be created. This limit will vary from platform to platform and is subject to multiple factors, including the JVM startup parameters, the size of the request stack in the Thread constructor, and the underlying operations. System restrictions on threads, etc. If these restrictions are violated, an OutOfMemoryError may be thrown.

Tuning strategy

A thread pool can be used, which refers to a resource pool that manages a set of homogeneous worker threads.

The essence of the thread pool is: there is a queue, and tasks will be submitted to this queue. A certain number of threads will take tasks from this queue and execute them. The results of the task can be sent back to the client, can be written to a database, can be stored in an internal data structure, and so on. But after the task execution is complete, this thread returns to the task queue, retrieves another task and executes it.

Using a thread pool can bring the following benefits:

1. By reusing existing threads instead of creating new threads, the huge overhead incurred in thread creation and destruction can be amortized when processing multiple requests.

2. When the request arrives, the worker thread already exists, so the execution of the task is not delayed by waiting for the thread to be created, which improves responsiveness.

3. By properly sizing the thread pool, you can create enough threads to keep the processor busy, while preventing too many threads competing for resources and causing the application to run out of memory or fail.

thread synchronization

raised problems

reduce scalability

In some problems, the more resources available, the faster the problem can be solved. If multithreading is used primarily to exploit the processing power of multiple processors, then a reasonable parallel decomposition of the problem must be done and the program can effectively use this potential parallelism.

However, most concurrent programs are composed of a series of parallel work and serial work. Therefore, Amdhl's law describes: in the case of increasing computing resources, the program can theoretically achieve the highest acceleration ratio, which depends on the proportion of parallel components (1-F) and serial components (F) in the program .

Speedup≤1F+1−FN

• As N approaches infinity, the maximum acceleration ratio approaches 1/F

• If the program has 50% of the computing resources that need to be executed serially, then the highest acceleration ratio can be 2 (regardless of how many threads are available).

• If 10% of the calculations in the program need to be performed serially, the highest acceleration ratio will be close to 10.

• If 10% of the program needs to be executed serially

• In a system with 10 processors, the highest acceleration ratio is 5.3 (53% utilization);

• In a system with 100 processors, the acceleration ratio can reach 9.2 (9% utilization);

Therefore, as the value of F increases (that is, more code is executed serially), the advantage of introducing multithreading decreases. So it also shows that it is very important to limit the code amount of the serial block.

context switch overhead

If the main thread is the only thread, then it's basically not scheduled out. If the number of runnable threads is greater than the number of CPUs, then the operating system will eventually schedule a running thread out so that other threads can use the CPU. This will cause a context switch, which will save the execution context of the currently running thread and set the execution context of the newly scheduled thread to the current context.

Then the following overhead will be incurred during context switching

1. Access to data structures shared by the operating system and the JVM is required during thread scheduling.

2. The application, the operating system, and the JVM all use the same set of CPUs, and the more CPU clock cycles are consumed in the code of the JVM and the operating system, the fewer CPU clock cycles are available to the application.

3. When a new thread is switched in, the data it needs may not be in the current processor's local cache, so the context switch will cause some cache misses, so the thread will be slower the first time it is scheduled to run.

That's why the scheduler assigns a minimum execution time to each runnable thread, even if there are many other threads executing - it spreads the overhead of context switching over more uninterrupted execution time, improving the overall throughput (at the cost of losing responsiveness).

When a thread is blocked waiting for a competing lock, the JVM usually suspends the thread and allows it to be swapped out. If the thread blocks frequently, it will not get the full scheduling time slice. The more blocking occurs in a program, the more context switches occur with CPU-intensive programs, increasing scheduling overhead and thus reducing throughput (non-blocking algorithms also help reduce context switches).

Memory synchronization overhead

1. Indirect effects of memory fences

   Some special instructions may be used in the visibility guarantees provided by synchronized and volatile, namely memory barriers, which can flush caches, invalidate caches, flush hardware write buffers, and stop execution pipelines.

   Memory fences may also have an indirect impact on performance, as they will suppress some compiler optimizations. And within the memory fence, most operations cannot be reordered.

2. Synchronization from contention may require OS intervention, adding overhead

   When there is competition on the lock, the thread that fails the competition will definitely block. When the JVM implements blocking behavior, it can use spin-waiting (Spin-Waiting, which refers to continuously trying to acquire a lock through a loop until it succeeds), or suspend the blocked thread through the operating system. The efficiency of these two methods depends on the overhead of context switching and the time to wait before successfully acquiring the lock. If the waiting time is short, the spin-waiting method is suitable, and if the waiting time is long, the thread suspending method is suitable.

   Synchronization in one thread can affect the performance of other threads, synchronization increases the traffic on the memory bus, the bandwidth of the bus is limited, and all processors will share this bus. If there are multiple threads competing for synchronization bandwidth, all threads using synchronization are affected.

3. The overhead of uncontended synchronization is negligible

   The synchronized mechanism is optimized for non-competitive synchronization, removing some locks that do not compete, thereby reducing unnecessary synchronization overhead. So, don't worry about the overhead of uncontended synchronization, the basic mechanism is already very fast, and the JVM can do additional optimizations to further reduce or eliminate the overhead.

• If an object can only be accessed by the current thread, then the JVM can optimize to get rid of this lock acquisition operation.

• Some sophisticated JVMs can use escape analysis to find local object references that are not posted to the heap (these references are thread local)

During the execution of getStoogeNames(), the lock on the Vector will be acquired and released at least 4 times, and it will be executed once each time add or toString is called. However, an intelligent runtime compiler will usually analyze these calls so that the stooges and their internal state don't escape, so the 4 lock acquisitions can be eliminated.

• public String getStoogeNames(){ List<String> stooges = new Vector<>();stooges.add("Moe");stooges.add("Larry");stooges.add("Curly"); return stooges.toString();}

Even without escape analysis, the compiler can perform lock-graining operations to combine adjacent synchronized blocks of code with the same lock. In getStoogeNames, if the JVM is coarsening the lock granularity, it is possible to combine 3 add and 1 toString calls into a single lock acquire/release operation, and use heuristics to evaluate the synchronization operation in the synchronization code block and the difference between the instructions relative cost between. This not only reduces the overhead of synchronization, but also enables the optimization to handle larger blocks of code, potentially enabling further optimizations.

Tuning strategy

avoid synchronization

1. Using thread local variables ThreadLocal

The ThreadLocal class enables a thread's value to be associated with the thread object that holds the value. ThreadLocal provides get和setmethods such as making an independent copy of each thread using the variable, so get always returns the latest value set by the current executing thread in the call to set.

When a thread calls the ThreadLocal.get method for the first time, it calls initialValue to get the initial value. These thread-specific values ​​are kept in the Thread object and are garbage collected when the thread terminates.

1. Use a CAS-based alternative

   In a sense, this is not to avoid synchronization, but to reduce the performance penalty of synchronization. In general, when comparing CAS and traditional synchronization, the following principles are used:

• CAS-based protection is slightly faster than traditional synchronization if the access is to a resource that does not have contention (no protection at all is faster);

• CAS-based protection is faster than traditional synchronization (often in blocks) if there is mild or moderate contention for the accessed resources;

• If the resource access is particularly competitive, then traditional synchronization is a better choice.

  This conclusion is understandable and holds true in other fields: traffic lights can achieve higher throughput when traffic is congested, and roundabouts can achieve higher throughput when traffic is low. This is because locks suspend threads when contention occurs, reducing CPU usage and synchronization traffic on the shared memory bus. Similar to the producer-consumer model, where the producer can be blocked, it reduces the workload on the consumer so that the processing speed of the consumer catches up with the processing speed of the producer.

Reduce lock contention

Serial operations reduce scalability. In concurrent programs, the main threat to scalability is exclusive resource locking. Both scalability and context switching issues arise when lock contention occurs, so reducing lock contention improves performance and scalability.

The likelihood of contention on a lock is primarily affected by two factors: how often the lock is requested and how long the lock is held each time.

• If the product of the two is small, then most operations to acquire a lock will not compete, so contention on that lock will have no impact on scalability.

• If the number of requests on the lock is very high, the thread that needs to acquire the lock will be blocked and wait.

Therefore, there are 3 ways to reduce lock contention:

1. Reduce lock holding time - mainly by narrowing the scope of the lock, fast in and out

• Remove a non-lock related operation from a synchronized block of code, especially those that are expensive and may block.

• Its performance is further improved by delegating thread safety to other thread safety classes. This eliminates the need to use explicit synchronization, reduces lock scope, and reduces the risk of future code maintenance inadvertently breaking thread safety.

• While shrinking synchronized blocks can improve scalability, synchronized blocks can't be too small—some operations that need to be performed atomically must be contained within the same block. Synchronization also requires a certain overhead, and when a synchronized code block is decomposed into multiple synchronized code blocks, it will have a negative impact on performance.

1. Reduce the frequency of lock requests

   Implemented through techniques such as lock decomposition and lock segmentation, multiple independent locks will be used to protect independent state variables, thereby changing the situation where these variables were previously protected by a single lock. That is, if a lock needs to protect multiple independent state variables, the lock can be decomposed into multiple locks, and each lock only protects one variable, thereby improving scalability and ultimately reducing the cost of each lock being The frequency of requests. However, the more locks are used, the higher the risk of deadlocks.

   Both lock decomposition and lock segmentation improve scalability because they both enable different threads to operate on different data (or different parts of the same data) without interfering with each other. If the program uses lock segmentation, it must show that the frequency of contention on the lock is higher than the frequency of contention on the data protected by the lock.

• If there is moderate rather than intense contention on locks, performance can be maximized by splitting a lock into two locks. Decomposing locks that are not highly contested will result in very limited improvements in performance and throughput, but will also improve the inflection point at which performance decreases with contention. When decomposing locks with moderate contention, it is actually transforming these locks into non-contention locks, thereby effectively improving performance and scalability.

• In some cases, the lock decomposition technique can be further extended to decompose locks on a set of independent objects, a situation known as lock segmentation.

  In the implementation of ConcurrentHashMap, an array of 16 locks is used, each lock protects 1/16 of all hash buckets, and the Nth hash is protected by (N mod 16N mod 16) locks. Assuming that the hash function is reasonably distributed, and the keys can be uniformly distributed, then the lock request can be reduced to about 1/16 of the original. It is this technology that enables ConcurrentHashMap to support up to 16 concurrent writers.

• One disadvantage of lock segmentation is that it is more difficult and expensive to acquire multiple locks for exclusive access than with a single lock for exclusive access. When ConcurrentHashMap needs to expand the mapping range and recalculate the hash value of the key value to be distributed to a larger set of buckets, it needs to acquire all the locks in the segment lock set.

Avoid hot spots

In a common optimization measure, which is to cache the result of a repeated calculation, some hot spots are introduced, and these hot spots often limit scalability. In the container class, in order to obtain the number of elements of the container, a shared counter is used to count the size. In a single-threaded or fully synchronized implementation, using a separate counter can speed up methods like size and isEmpty well, but leads to much harder scalability, so every map-modifying operation is To update this shared counter. Even using lock segmentation to implement hash chains, synchronizing access to counters recreates the scalability problems that existed when using exclusive locks.

To avoid this problem, size in ConcurrentHashMap will enumerate each segment and add up the number of elements in each segment instead of maintaining a global count. To avoid enumerating each count, ConcurrentHashMap maintains a separate count for each segment and maintains this value through a lock on each segment.

Abandon the use of exclusive locks and use a concurrency-friendly way to manage shared state

• ReadWriteLock: Implements a locking rule in the case of multiple read operations and a single write operation.

  If multiple read operations do not modify the shared resource, then the read operations can access the shared resource concurrently, but the lock must be acquired exclusively for the write operation.

  For read-majority data structures, ReadWriteLock can provide higher concurrency than exclusive locks. For read-only data structures, the invariants contained in them may not require locking operations at all.

• Atomic Variables: Provides a way to reduce the overhead of updating hotspot domains.

  A static counter, sequencer, or a reference to the head node in a linked list data structure. If the class contains only a small amount of shared state, and the shared state does not participate in the immutability condition with other variables, then replacing them with atomic variables can improve scalability.

Use bias lock

When a lock is contended, the JVM can choose how to allocate the lock.

• Locks can be granted fairly, and each thread acquires locks in a round-robin manner;

• There is also an alternative where the lock can be biased towards the thread that accesses it most frequently.

The rationale for biased locking is that if a thread has recently used a lock, the data needed by the thread to execute code protected by the same lock next time may still be held in the processor's cache. If this thread is given priority to acquire the lock, then the cache hit rate will increase (supporting old users and avoiding the overhead associated with new users). Then the performance will be improved because the overhead of new threads creating new caches on the current processor is avoided.

However, disabling biased locking -XX:-UseBiasedLocking improves performance if the programming model used is for different thread pools to contend for locks with equal chance.

Use spin locks

When dealing with synchronization lock contention, the JVM has two options.

• You can let the current thread enter a busy loop, execute some instructions, and then check the lock again;

• It is also possible to put the thread into a queue to suspend (making the CPU available to other threads) and notify him when the lock is available.

If multiple threads contend for a lock that is held for a short time, then a spinlock is a better solution. If the lock is held for a long time, it would be better to have the second thread wait for the notification.

If you want to influence the way the JVM handles spinlocks, the only reasonable way is to keep synchronized blocks as short as possible.

false sharing

raised problems

Among the possible effects of synchronization is false sharing, which arises from the way the CPU handles its caches. Here's an extreme example with a DataHolder class:

Each long value here is stored in an adjacent memory location. For example, l1 might be stored at 0xF20, l2 at 0xF28, and so on. When the program wants to operate l2, a large chunk of memory (including before and after l2) is loaded into the cache line of a CPU core currently in use.

In most cases, this makes sense: if a program accesses a particular instance of an object, it may also access adjacent instance variables. If these instance variables are loaded into the current core's cache, then memory accesses are particularly fast.

Then the disadvantage of this mode is that when the program updates a value in the local cache, the core where the current thread is located must notify all other cores that this memory has been modified. Other cores must invalidate their cache lines and reload from memory. Then as the number of threads increases, the operations on volatile become more and more frequent, and the performance will gradually decrease.

The Java memory model requires that data must be written to main memory only at the end of synchronization primitives (including CAS and volatile constructs). Strictly speaking, false sharing does not necessarily involve synchronized (volatile) variables. If the long variable is not volatile, then the compiler will put these values ​​in registers, so the performance impact is not that big. However, whenever any data in the CPU cache is written, other caches holding the same range of data must be invalidated.

Tuning strategy

Obviously this is an extreme example, but raises the question, how can false sharing be detected and corrected? False sharing cannot be solved yet, because of the expertise related to processor architecture, but you can start with code:

1. Avoid frequent writes to the variables involved

   A local variable can be used instead, and only the final result is written back to the volatile variable. As the number of writes decreases, contention for cache lines decreases.

2. Populate related variables to prevent them from being loaded into the same cache line.

Filling variables with arrays may not work, because the JVM may rearrange the layout of instance variables so that all arrays are next to each other, so all long variables are still next to each other.

If you populate the structure with values ​​of primitive types, it's likely to work, but it's hard to control the number of variables.

In addition, it is difficult to predict the size of the padding, because different CPU cache sizes are different, and padding will increase the instance, which has a large impact on garbage collection.

However, in the absence of algorithmic improvements, padding the data can sometimes have clear advantages.

If you want to learn Java engineering, high performance and distributed, high performance, simple language. Friends of performance tuning, Spring, MyBatis, Netty source code analysis can add my Java advanced group, 694549689, in the group there are Ali Daniel live broadcast technology, and Java large-scale Internet technology video to share with everyone for free. Those with 1-5 work experience, who do not know where to start in the face of the current popular technology, and who need to break through the technical bottleneck can join the group. After staying in the company for a long time, I lived very comfortably, but the interview hit a wall when I changed jobs. Those who need to study in a short period of time and change jobs to get high salaries can join the group. If you have no work experience, but have a solid foundation, you can join the group if you are proficient in the working mechanism of java, common design ideas, and common java development frameworks.

Thread Pool

raised problems

thread starvation deadlock

Whenever a task in a thread pool needs to wait indefinitely for some resource or condition that must be provided by other tasks in the pool, such as a task waiting for a return value or execution result of another task, then unless the thread pool is large enough, it will happen Thread starvation deadlock.

Therefore, whenever a dependent Executor task is submitted, it is clear that thread starvation deadlock may occur, so it is necessary to record the thread pool size or configuration limit in the code or in the configuration file that configures the Executor.

If the task blocks for too long, the responsiveness of the thread pool will become poor even if there is no deadlock. Tasks that take too long to execute will not only block the thread pool, but even increase the service time of tasks that take too long to execute.

Too large a thread pool can adversely affect performance

There is one very critical factor in implementing a thread pool: tuning the size of the thread pool is critical to getting the best performance. The thread pool can set the maximum and minimum number of threads. There will be threads with the minimum number of threads on standby at any time in the pool. If the number of tasks increases, you can add threads to the pool. The maximum number of threads can be used as the upper limit of the number of threads to prevent running too many threads. cause performance degradation.

Tuning strategy

Set the maximum number of threads

The ideal size of the thread pool depends on the type of tasks being submitted and the characteristics of the deployed system. At the same time, setting the size of the thread pool needs to avoid the extreme cases of "too large" and "too small".

• If the thread pool is too large, a large number of threads will compete on relatively few CPU and memory resources, which will not only lead to higher memory usage, but may also run out of resources.

• If the thread pool is too small, many idle processors will be unable to perform work, reducing throughput.

Therefore, to properly size the thread pool, the characteristics of the computing environment, resource budget, and tasks must be analyzed. How many CPUs are there in the deployed system? How much memory? Is the task computationally intensive, I/O intensive, or both? Do they need scarce resources like JDBC connections? If different classes of tasks need to be performed and their behavior varies widely, then multiple thread pools should be considered so that each thread can be tuned to its respective workload.

If the processor reaches the desired utilization rate, the optimal size of the thread pool is equal to:

Nthreads=Ncpu∗Ucpu∗(1+W/C)

• Ncpu: Indicates the number of processors, which can be obtained through Runtime.getRuntime().avaliableProcessors();

• Ucpu: CPU usage, 0⩽Ucpu⩽1;

• WC: the ratio of waiting time to calculation time;

In addition, CPU cycles are not the only resource that affects thread pool size, including memory, file handles, socket handles, and database connections. The upper limit on the size of the thread pool is explained by calculating the demand for that resource by each task, and dividing the total available amount of that resource by the demand for each task.

Set the minimum (core) number of threads

The number of threads can be set to some other value, such as 1. The starting point is to prevent the system from creating too many threads in order to save system resources.

Additionally, the system should be sized to handle the expected maximum throughput, which will require the system to start all threads at the maximum number of threads set.

Also, the negative impact of specifying a minimum number of threads is very small, even if many tasks run the first time, but this one-time cost has little negative impact.

Set extra thread survival time

When the number of threads is greater than the number of core threads, the maximum survival time for excess idle threads to wait for new tasks before terminating.

In general, once a new thread is created, it should be around for at least a few minutes to handle any spikes in load. If the task achievement rate has a better model, you can set the idle time based on this model. Also, idle time should be measured in minutes and should be at least between 10 and 30 minutes.

Select thread pool queue

• SynchronousQueue

SynchronousQueue is not a real queue and cannot hold tasks, it is a mechanism for handover between threads. If you want to put an element into the SynchronousQueue, there must be another thread waiting to accept the element. If no threads are waiting, all threads are busy, and the maximum number of threads in the pool has not been reached, the ThreadPoolExecutor will create a new thread. Otherwise the task will be rejected according to the saturation policy.

Using direct handover will be more efficient, and SynchronousQueue has real value only if the thread pool is unbounded or tasks can be rejected. SynchronousQueue is used in the newCachedThreadPool factory method.

• unbounded queue

If the ThreadPoolExecutor is using an unbounded queue, no tasks will be rejected. In this case, the ThreadPoolExecutor will only create threads with the minimum number of threads at most, and the maximum number of threads will be ignored.

If the maximum number of threads and the minimum number of threads are the same, this choice is closest to the traditional thread pool operation mechanism configured with a fixed number of threads. newFixedThreadPool and newSingleThreadExecutor use an unbounded LinkedBlockingQueue by default.

• bounded queue

A more secure resource management strategy is to use bounded queues, such as ArrayBlockingQueue, bounded LinkedBlockingQueue, PriorityBlockingQueue.

The maximum number of threads to run is the set number of core threads (minimum number of threads) before the bounded queue fills up. If the queue is full and a new task is added, and the maximum thread limit is not reached, a new thread is started for the current new task. If the maximum number of threads limit is reached, it will be processed according to the saturation policy.

In general, if the thread pool is small and the queue is large, it will help reduce memory usage, reduce CPU usage, and reduce context switching, but at the cost of limiting throughput.

Choose an appropriate saturation strategy

When the bounded queue is filled, the saturation policy will come into play, and the saturation policy of the ThreadPoolExecutor can be achieved by calling

setRejectedExecutionHandler

to modify. The saturation strategy is also used if a task is submitted to a closed Executor. The JDK provides several different implementations of the RejectedExecutionHandler saturation strategy:

AbortPolicy

• This strategy is the default saturation strategy;

• An unchecked RejectedExecutionException will be thrown, and the caller can catch this exception and write its own processing code as needed;

DiscardPolicy (discard)

• When the submitted task cannot be saved to the queue for execution, the Discard strategy will silently discard the task.

DiscardOldestPolicy (discard the oldest)

• The next task to be executed will be discarded, and a new task that will be resubmitted will be attempted.

• If the work queue is a priority queue, then discarding the oldest policy will discard the highest priority task, so it is best not to use the discarding oldest saturation policy together with the priority queue.

CallerRunsPolicy (caller runs)

• This strategy neither discards tasks nor throws exceptions, but when all threads in the thread pool are occupied and the work queue is filled, the next task will call

  execute

  is executed in the main thread, thus reducing the traffic of new tasks. Since it takes a certain amount of time to execute a task, the main thread cannot submit any tasks for at least a certain amount of time, so that the worker thread has time to process the task being executed.

• On the other hand, during this period, the main thread will not call accept, then incoming requests will be kept in the TCP layer's queue instead of the application's queue. If it continues to be overloaded, then the TCP layer will eventually find its request queue full and will start dropping requests as well.

• When the server is overloaded, this overload situation gradually spreads outward - from the thread pool to the work queue to the application to the TCP layer and finally to the client, causing the server to achieve a kind of flattening under high load Reduced performance.

There is no predetermined saturation strategy to block execute when the work queue is full. Therefore, this function can be achieved by limiting the arrival rate of tasks through the semaphore Semaphore.

Choose the right thread pool

• The newCachedThreadPool factory method is a good default choice that provides better queuing performance than a fixed-size thread pool;

• When it is necessary to limit the number of current tasks to meet the needs of the resource manager, a fixed-size thread pool can be selected. For example, in a server program that accepts network requests, if it is not limited, it will easily lead to overload problems.

• Only when tasks are independent of each other, it is reasonable to set boundaries for thread pools; if there are dependencies between tasks, then bounded thread pools or queues may lead to thread starvation deadlock problems, then unbounded thread pools should be used.

• Another way to configure tasks that submit tasks and wait for their results is to use a bounded thread pool with a SynchronousQueue as the work queue, and the caller to run a saturation policy.