OutOfMemoryError系列（1）: Java heap space

https://blog.csdn.net/renfufei/article/details/76350794

This is the first article in this series, a list of related articles:

Every Java program can only use a certain amount of memory, this limit is determined by the JVM startup parameters. But more complicated in that case, Java program memory is divided into two parts: the heap (Heap space) and permanent substituting (Permanent Generation, abbreviated PermGen):

The maximum memory size of the two regions, by the JVM startup parameters -Xmx and -XX:MaxPermSize specified. If not specified, it is determined according to the type of platform (OS version + JVM version) and the size of physical memory.

If when you create a new object, the heap memory space is insufficient to store the newly created object, it will lead to java.lang.OutOfMemoryError: Java heap space error.

Regardless of no free physical memory on the machine, as long as the heap memory usage to achieve maximum memory limit, it will throw java.lang.OutOfMemoryError: Java heap spacean error.

Cause Analysis

It generates java.lang.OutOfMemoryError: Java heap space the wrong reasons, very often, is similar to the XXL number of objects, to the Java heap space S No. stuffed. In fact, the reason is clear, it is easy to solve as long as the right to increase the size of the heap memory, the program will be able to run properly and there are some more complex cases, the main problem is caused by the code?:

Than-expected number of visits / data volume. Application of the system design, there typically is a "capacity" is defined, deployment so many machines for handling certain amount of data / traffic. If a sudden surge in traffic, more than the expected threshold, similar to the time coordinate system, map the shape of the tip, then at the peak time period where the program will most likely stuck, and trigger java.lang.OutOfMemoryError: Java heap space error .
Memory leaks (Memory leak). This case is also a frequent occurrence. Due to some errors in the code, leading to more and more occupied by the system memory. If a method / a piece of code memory leak, Each time, there will be (more garbage objects) take up more memory over the running time of the leak object swallowed up all the memory heap, then java.lang.OutOfMemoryError: Java heap space error broke out.

Specific examples

A very simple example

The following code is very simple, the program tries to allocate capacity to int array of 2M. If you specify startup parameters -Xmx12m, it will occur java.lang.OutOfMemoryError: Java heap space error. And as long as the parameters a little tinkering, into -Xmx13mthe error will not happen again.

public class OOM {
    static final int SIZE=2*1024*1024; public static void main(String[] a) { int[] i = new int[SIZE]; } }

Memory leak example

This example more realistic. In Java, when you create a new object, for example Integer num = new Integer(5); , you do not need to manually allocate memory. Automatic packaging because the JVM and the memory allocation process. During program execution, JVM may be used as well as what the object is still necessary to check the memory, and those objects no longer needed it is discarded, and the memory occupied recycling and reuse. This process is called garbage collection . JVM is responsible for garbage collection module called a garbage collector (GC) .

Java's automatic memory management depends GC , GC will again and again to scan the memory area, the object will not be used to delete. In simple terms, the Java memory leaks, that is, those objects logically no longer in use, but it is not garbage collection program to get rid of garbage objects continue to occupy leading to heap memory, the gradual accumulation, the final result. java.lang.OutOfMemoryError: the Java heap Space error.

BUG is easy to write a program to simulate memory leaks:

import java.util.*;

public class KeylessEntry {

    static class Key {
        Integer id;

        Key(Integer id) {
        this.id = id;
        }

        @Override
        public int hashCode() { return id.hashCode(); } } public static void main(String[] args) { Map m = new HashMap(); while (true){ for (int i = 0; i < 10000; i++){ if (!m.containsKey(new Key(i))){ m.put(new Key(i), "Number:" + i); } } System.out.println("m.size()=" + m.size()); } } }

At first glance, you may think there is a problem, because the cache up to 10,000 elements Well! But a closer review reveals that Key this class only overrides the hashCode() method, but does not override equals() method, so they have to add more HashMap Key.

Please refer to: conventions in Java hashCode and equals methods and principles rewrite

Over time, "cached" As more and more objects will leak objects filled all the heap memory,. GC can not they clean up, will throw java.lang.OutOfMemoryError: Java heap space error.

The solution is simple, the Key proper implementation class equals() method can be:

@Override
public boolean equals(Object o) {
    boolean response = false; if (o instanceof Key) { response = (((Key)o).id).equals(this.id); } return response; }

To be honest, in the search for the real causes of memory leaks, you may be a lot of dead brain cells.

SpringMVC in a scene

Translators have come across such a scenario:

For compatibility to easily migrate from Struts2 SpringMVC code, access request directly in the Controller.

So in ControllerBase class by ThreadLocal caching the request object held by the current thread:

public abstract class ControllerBase {

    private static ThreadLocal<HttpServletRequest> requestThreadLocal = new ThreadLocal<HttpServletRequest>(); public static HttpServletRequest getRequest(){ return requestThreadLocal.get(); } public static void setRequest(HttpServletRequest request){ if(null == request){ requestThreadLocal.remove(); return; } requestThreadLocal.set(request); } }

Then SpringMVC interceptor (Interceptor) implementation class, in the preHandle method, the object is to save the request in ThreadLocal:

/**
 * 登录拦截器
 */
public class LoginCheckInterceptor implements HandlerInterceptor { private List<String> excludeList = new ArrayList<String>(); public void setExcludeList(List<String> excludeList) { this.excludeList = excludeList; } private boolean validURI(HttpServletRequest request){ // 如果在排除列表中 String uri = request.getRequestURI(); Iterator<String> iterator = excludeList.iterator(); while (iterator.hasNext()) { String exURI = iterator.next(); if(null != exURI && uri.contains(exURI)){ return true; } } // 可以进行登录和权限之类的判断 LoginUser user = ControllerBase.getLoginUser(request); if(null != user){ return true; } // 未登录,不允许 return false; } private void initRequestThreadLocal(HttpServletRequest request){ ControllerBase.setRequest(request); request.setAttribute("basePath", ControllerBase.basePathLessSlash(request)); } private void removeRequestThreadLocal(){ ControllerBase.setRequest(null); } @Override public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception { initRequestThreadLocal(request); // 如果不允许操作,则返回false即可 if (false == validURI(request)) { // 此处抛出异常,允许进行异常统一处理 throw new NeedLoginException(); } return true; } @Override public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView) throws Exception { removeRequestThreadLocal(); } @Override public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) throws Exception { removeRequestThreadLocal(); } }

In postHandle and afterCompletion methods to clean up the request object ThreadLocal.

However, in actual use, developers will be a great business object (such as a memory footprint of around 200MB List) is set to Attributes request is passed to the JSP.

JSP code may abnormality occurs, then the SpringMVC postHandle and afterCompletion methods will not be executed.

Thread scheduling in Tomcat, you may not have been scheduled thread that throws an exception, then ThreadLocal has been hold live request. Over the running time, the available memory is full, has been in the implementation of Full GC, the system directly stuck.

Subsequent amendments: The Filter, cleaning ThreadLocal in the finally block.

@WebFilter(value="/*", asyncSupported=true)
public class ClearRequestCacheFilter implements Filter{ @Override public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { clearControllerBaseThreadLocal(); try { chain.doFilter(request, response); } finally { clearControllerBaseThreadLocal(); } } private void clearControllerBaseThreadLocal() { ControllerBase.setRequest(null); } @Override public void init(FilterConfig filterConfig) throws ServletException {} @Override public void destroy() {} }

The lesson is: You can use ThreadLocal, but there must be a controlled release measures, it is the general try-finally code format.

Description: SpringMVC of the Controller, in fact, can be @Autowired injected into the request, the actual injection is an HttpServletRequestWrapper object that is calling the current request is executed by ThreadLocal mechanism.

In a conventional manner: receiving a request directly to the controller process parameters.

solution

Running the program does not meet the maximum memory if provided, need only to increase the heap memory, the configuration parameters can refer hereinafter.

But in many cases, increase the heap memory space does not solve the problem. For example, a memory leak, increasing the heap memory will only postpone java.lang.OutOfMemoryError: Java heap space error triggering time.

Of course, increasing the heap memory, it may increase the GC pauses time, thus affecting the program throughput or latency .

From fundamentally solve the problem, you need to troubleshoot memory allocation code simple, the need to address these issues:

What kind of objects are taking up the most memory?
These objects are allocated in which part of the code.

To clarify this point, it may take several days. The following is a general procedure:

Obtain permission to perform heap dump (heap dump) on a production server. "Dump" (Dump) is a snapshot of heap memory, the memory can be used for later analysis. These snapshots may contain confidential information, such as passwords, credit card numbers and so on, so sometimes, due to security restrictions business, to obtain a production environment heap dump is not easy.
Heap dump at the appropriate time. In general, the analysis requires comparing the plurality of memory heap dump file, if the acquired timing was not right, it could be a snapshot of "waste" In addition, each time the dump heap, the JVM will be "frozen" so a production environment, you can not perform a lot of Dump operation, otherwise the system slow or stuck, you are in big trouble.
Use another machine to load Dump file. In general, if the problem JVM memory is 8GB, then the analysis Heap Dump machine memory requires more than 8GB. Open the dump analysis software (we recommend the Eclipse MAT , of course you can also use other tools).
Detection snapshot memory-largest GC roots. For details, please refer to: Solving OutOfMemoryError (Part 6) - A Waste Dump IS not . This may be a bit difficult for the novice, but this will deepen your understanding of heap memory structure and navigation mechanisms.
Next, the code may be assigned to identify a large number of objects. If you are very familiar with the whole system, may soon be positioned.

Make advertising, we recommend Plumbr, only the Java Monitoring Solution at The root the cause is with Automatic Detection . Plumbr able to capture all java.lang.OutOfMemoryError , and find other performance issues, such as the most memory-intensive data structures, and so on.

Plumbr responsible for collecting data in the background - including heap memory usage (only statistical distribution of the object, it does not involve actual data), as well as heap dump is not easy to find a variety of issues. If there is java.lang.OutOfMemoryError , but also in the case of non-stop, make the necessary data processing Here is Plumbr one. Java.lang.OutOfMemoryError reminder:

Powerful it, do not need additional tools and analysis, can be seen directly:

What kind of objects are taking up the most memory (here 271 com.example.map.impl.PartitionContainer example, consumes 173MB of memory, and heap memory is only 248MB)
These objects are created where (mostly in MetricManagerImpl class, the first line at 304)
Who is currently refer to these objects (GC root intact from the beginning of the chain of references)

That information, you can navigate to the root of the problem, such as streamlining the local data structure / model only takes the necessary memory can be.

Of course, according to the results memory analysis, and reporting Plumbr generated if the memory occupied by objects found very reasonable, you do not need to modify the source code, then it would increase the heap memory of it. In this case, to modify the JVM startup parameters (proportionally) increases following values:

-Xmx1024m

Java heap memory configuration up to here 1024MB. You can use g/G represent GB, m/M on behalf of MB, k/K represented KB.

The following are all equivalent, set the maximum Java heap space for the 1GB:

# 等价形式: 最大1GB内存
java -Xmx1073741824 com.mycompany.MyClass
java -Xmx1048576k com.mycompany.MyClass
java -Xmx1024m com.mycompany.MyClass
java -Xmx1g com.mycompany.MyClass

Original link: https://plumbr.eu/outofmemoryerror/java-heap-space

Translation Date: July 29, 2017

Translators: anchor: http://blog.csdn.net/renfufei

[Reprint] OutOfMemoryError series (1): Java heap space