Netty之Buffer(一)ByteBuf内存泄漏检测2

io.netty.util.ResourceLeakTracker ,内存泄露追踪器接口,上文知每个资源( 例如:ByteBuf 对象 ),会创建一个追踪它是否内存泄露的 ResourceLeakTracker 对象,接口方法定义:

public interface ResourceLeakTracker<T>  {

    /**
     * 记录
     *
     * Records the caller's current stack trace so that the {@link ResourceLeakDetector} can tell where the leaked
     * resource was accessed lastly. This method is a shortcut to {@link #record(Object) record(null)}.
     */
    void record();
    /**
     * 记录
     *
     * Records the caller's current stack trace and the specified additional arbitrary information
     * so that the {@link ResourceLeakDetector} can tell where the leaked resource was accessed lastly.
     */
    void record(Object hint);

    /**
     * 关闭
     *
     * Close the leak so that {@link ResourceLeakTracker} does not warn about leaked resources.
     * After this method is called a leak associated with this ResourceLeakTracker should not be reported.
     *
     * @return {@code true} if called first time, {@code false} if called already
     */
    boolean close(T trackedObject);

}

ReferenceCounted中touch(…) 方法,会调用 record(…) 方法。
close(T trackedObject) 方法,关闭 ResourceLeakTracker 。
如果资源( 例如:ByteBuf 对象 )被正确释放,则会调用 close(T trackedObject) 方法,关闭 ResourceLeakTracker ,从而结束追踪。这样,在 ResourceLeakDetector的reportLeak() 方法,就不会提示该资源泄露。

DefaultResourceLeak ,继承 java.lang.ref.WeakReference 类,实现 ResourceLeakTracker 接口,默认 ResourceLeakTracker 实现类。同时,它是 ResourceLeakDetector 内部静态类:

public class ResourceLeakDetector<T> {

    private static final class DefaultResourceLeak<T> extends WeakReference<Object> implements ResourceLeakTracker<T>, ResourceLeak {
    }

}

构造:

/**
 * {@link #head} 的更新器
 */
@SuppressWarnings("unchecked") // generics and updaters do not mix.
private static final AtomicReferenceFieldUpdater<DefaultResourceLeak<?>, Record> headUpdater =
        (AtomicReferenceFieldUpdater)
                AtomicReferenceFieldUpdater.newUpdater(DefaultResourceLeak.class, Record.class, "head");

/**
 * {@link #droppedRecords} 的更新器
 */
@SuppressWarnings("unchecked") // generics and updaters do not mix.
private static final AtomicIntegerFieldUpdater<DefaultResourceLeak<?>> droppedRecordsUpdater =
        (AtomicIntegerFieldUpdater)
                AtomicIntegerFieldUpdater.newUpdater(DefaultResourceLeak.class, "droppedRecords");

/**
 * Record 链的头节点
 *
 * 看完 {@link #record()} 方法后,实际上,head 是尾节点,即最后( 新 )的一条 Record 。
 */
@SuppressWarnings("unused")
private volatile Record head;
/**
 * 丢弃的 Record 计数
 */
@SuppressWarnings("unused")
private volatile int droppedRecords;

/**
 * DefaultResourceLeak 集合。来自 {@link ResourceLeakDetector#allLeaks}
 */
private final ConcurrentMap<DefaultResourceLeak<?>, LeakEntry> allLeaks;
/**
 * hash 值
 *
 * 保证 {@link #close(Object)} 传入的对象,就是 {@link #referent} 对象
 */
private final int trackedHash;

  DefaultResourceLeak(
         Object referent,
          ReferenceQueue<Object> refQueue,
          ConcurrentMap<DefaultResourceLeak<?>, LeakEntry> allLeaks) {
      // 父构造方法 将 referent( 资源,例如:ByteBuf 对象 )和 refQueue( 引用队列 )传入父 WeakReference 构造方法。
      super(referent, refQueue);
  
      assert referent != null;
  
      // Store the hash of the tracked object to later assert it in the close(...) method.
      // It's important that we not store a reference to the referent as this would disallow it from
      // be collected via the WeakReference.
     trackedHash = System.identityHashCode(referent);
      allLeaks.put(this, LeakEntry.INSTANCE);
      // allLeaks 属性,DefaultResourceLeak 集合。来自 ResourceLeakDetector.allLeaks 属性,
      //会将自己添加到 allLeaks 中。trackedHash 属性,hash 值。保证在 close(T trackedObject) 方法,传入的对象,就是 referent 属性,即就是 DefaultResourceLeak 指向的资源
      // Create a new Record so we always have the creation stacktrace included.
      headUpdater.set(this, new Record(Record.BOTTOM));
      this.allLeaks = allLeaks;
  }

head 属性,Record 链的头节点,实则head 是尾节点,即最后( 新 )的一条 Record 记录。默认创建尾节点 Record.BOTTOM。
droppedRecords 属性,丢弃的 Record 计数。

引用队列(Reference Queue)

一旦弱引用对象开始返回null,该弱引用指向的对象就被标记成了垃圾。而这个弱引用对象(非其指向的对象)就没有什么用了。通常这时候需要进行一些清理工作。比如WeakHashMap会在这时候移除没用的条目来避免保存无限制增长的没有意义的弱引用。

引用队列可以很容易地实现跟踪不需要的引用。当你在构造WeakReference时传入一个ReferenceQueue对象,当该引用指向的对象被标记为垃圾的时候,这个引用对象会自动地加入到引用队列里面。接下来,你就可以在固定的周期,处理传入的引用队列,比如做一些清理工作来处理这些没有用的引用对象。

上面将 referent( 资源,例如:ByteBuf 对象 )和 refQueue( 引用队列 )传入父 WeakReference 构造方法。referent 被标记为垃圾的时候,它对应的 WeakReference 对象会被添加到 refQueue 队列中。在此处,即将 DefaultResourceLeak 添加到 referent 队列中。

所以,假设 referent 为 ByteBuf 对象。如果它被正确的释放,即调用了 release() 方法,从而调用了 AbstractReferenceCountedByteBuf中closeLeak() 方法,最终调用到 ResourceLeakTracker的close(trackedByteBuf) 方法,那么该 ByteBuf 对象对应的 ResourceLeakTracker 对象,将从 ResourceLeakDetector.allLeaks 中移除。

record(…) 方法,创建 Record 对象,添加到 head 链中:

@Override
public void record() {
    record0(null);
}
@Override
public void record(Object hint) {
    record0(hint);
}

/**
 * This method works by exponentially backing off as more records are present in the stack. Each record has a
 * 1 / 2^n chance of dropping the top most record and replacing it with itself. This has a number of convenient
 * properties:
 *
 * <ol>
 * <li>  The current record is always recorded. This is due to the compare and swap dropping the top most
 *       record, rather than the to-be-pushed record.
 * <li>  The very last access will always be recorded. This comes as a property of 1.
 * <li>  It is possible to retain more records than the target, based upon the probability distribution.
 * <li>  It is easy to keep a precise record of the number of elements in the stack, since each element has to
 *     know how tall the stack is.
 * </ol>
 *
 * In this particular implementation, there are also some advantages. A thread local random is used to decide
 * if something should be recorded. This means that if there is a deterministic access pattern, it is now
 * possible to see what other accesses occur, rather than always dropping them. Second, after
 * {@link #TARGET_RECORDS} accesses, backoff occurs. This matches typical access patterns,
 * where there are either a high number of accesses (i.e. a cached buffer), or low (an ephemeral buffer), but
 * not many in between.
 *
 * The use of atomics avoids serializing a high number of accesses, when most of the records will be thrown
 * away. High contention only happens when there are very few existing records, which is only likely when the
 * object isn't shared! If this is a problem, the loop can be aborted and the record dropped, because another
 * thread won the race.
 */
  private void record0(Object hint) {
      // Check TARGET_RECORDS > 0 here to avoid similar check before remove from and add to lastRecords
      if (TARGET_RECORDS > 0) {
          Record oldHead;
          Record prevHead;
          Record newHead;
          boolean dropped;
          do {
              // 已经关闭,则返回
              if ((prevHead = oldHead = headUpdater.get(this)) == null) {
                  // already closed.
                  return;
              }
              // 当超过 TARGET_RECORDS 数量时,随机丢到头节点。
              final int numElements = oldHead.pos + 1;
              if (numElements >= TARGET_RECORDS) {
                  final int backOffFactor = Math.min(numElements - TARGET_RECORDS, 30);
                  if (dropped = PlatformDependent.threadLocalRandom().nextInt(1 << backOffFactor) != 0) {
                      prevHead = oldHead.next;
                  }
              } else {
                  dropped = false;
              }
              // 创建新的头节点
              newHead = hint != null ? new Record(prevHead, hint) : new Record(prevHead);
          } while (!headUpdater.compareAndSet(this, oldHead, newHead)); // cas 修改头节点
          // 若丢弃,增加 droppedRecordsUpdater 计数
          if (dropped) {
              droppedRecordsUpdater.incrementAndGet(this);
          }
      }
  }

当前 DefaultResourceLeak 对象所拥有的 Record 数量超过 TARGET_RECORDS 时,随机丢弃当前 head 节点的数据。也就是说,尽量保留老的 Record 节点。这是为什么呢?越是老( 开始 )的 Record 节点,越有利于排查问题。另外,随机丢弃的的概率,按照 1 - (1 / 2^n) 几率,越来越大。

dispose() 方法, 清理,并返回是否内存泄露:

// 清理,并返回是否内存泄露
boolean dispose() {
    // 清理 referent 的引用
    clear();
    // 移除出 allLeaks 。移除成功,意味着内存泄露。
    return allLeaks.remove(this, LeakEntry.INSTANCE);
}

close(T trackedObject) 方法,关闭 DefaultResourceLeak 对象:

 @Override
 public boolean close(T trackedObject) {
     // 校验一致
     // Ensure that the object that was tracked is the same as the one that was passed to close(...).
     assert trackedHash == System.identityHashCode(trackedObject);
 
     // 关闭
     // We need to actually do the null check of the trackedObject after we close the leak because otherwise
     // we may get false-positives reported by the ResourceLeakDetector. This can happen as the JIT / GC may
     // be able to figure out that we do not need the trackedObject anymore and so already enqueue it for
     // collection before we actually get a chance to close the enclosing ResourceLeak.
     return close() && trackedObject != null;
 }

调用 close() 方法,关闭 DefaultResourceLeak 对象,关闭时,会将 DefaultResourceLeak 对象,从 allLeaks 中移除。

@Override
public boolean close() {
    // 移除出 allLeaks
    // Use the ConcurrentMap remove method, which avoids allocating an iterator.
    if (allLeaks.remove(this, LeakEntry.INSTANCE)) {
        // 清理 referent 的引用
        // Call clear so the reference is not even enqueued.
        clear();
        // 置空 head
        headUpdater.set(this, null);
        return true; // 返回成功
    }
    return false; // 返回失败
}

当 DefaultResourceLeak 追踪到内存泄露,会在 ResourceLeakDetector的reportLeak() 方法中,调用 DefaultResourceLeak的toString() 方法,拼接提示信息:

@Override
public String toString() {
    // 获得 head 属性,并置空! 
    Record oldHead = headUpdater.getAndSet(this, null);
    // 若为空,说明已经关闭。
    if (oldHead == null) {
        // Already closed
        return EMPTY_STRING;
    }

    final int dropped = droppedRecordsUpdater.get(this);
    int duped = 0;

    int present = oldHead.pos + 1;
    // Guess about 2 kilobytes per stack trace
    StringBuilder buf = new StringBuilder(present * 2048).append(NEWLINE);
    buf.append("Recent access records: ").append(NEWLINE);

    // 拼接 Record 练
    int i = 1;
    Set<String> seen = new HashSet<String>(present);
    for (; oldHead != Record.BOTTOM; oldHead = oldHead.next) {
        String s = oldHead.toString();
        if (seen.add(s)) { // 是否重复
            if (oldHead.next == Record.BOTTOM) {
                buf.append("Created at:").append(NEWLINE).append(s);
            } else {
                buf.append('#').append(i++).append(':').append(NEWLINE).append(s);
            }
        } else {
            duped++;
        }
    }

    // 拼接 duped ( 重复 ) 次数
    if (duped > 0) {
        buf.append(": ")
                .append(dropped)
                .append(" leak records were discarded because they were duplicates")
                .append(NEWLINE);
    }

    // 拼接 dropped (丢弃) 次数
    if (dropped > 0) {
        buf.append(": ")
           .append(dropped)
           .append(" leak records were discarded because the leak record count is targeted to ")
           .append(TARGET_RECORDS)
           .append(". Use system property ")
           .append(PROP_TARGET_RECORDS)
           .append(" to increase the limit.")
           .append(NEWLINE);
    }

    buf.setLength(buf.length() - NEWLINE.length());
    return buf.toString();
}

LeakEntry ,用于 ResourceLeakDetector.allLeaks 属性的 value 值:

private static final class LeakEntry {

    /**
     * 单例
     */
    static final LeakEntry INSTANCE = new LeakEntry();

    /**
     * hash 值,避免重复计算
     */
    private static final int HASH = System.identityHashCode(INSTANCE);

    private LeakEntry() { // 禁止创建,仅使用 INSTANCE 单例
    }

    @Override
    public int hashCode() {
        return HASH;
    }

    @Override
    public boolean equals(Object obj) {
        return obj == this;
    }

}

Record 记录。每次调用 ResourceLeakTracker的touch(…) 方法后,会产生响应的 Record 对象:

private static final class Record extends Throwable {

    private static final long serialVersionUID = 6065153674892850720L;

    /**
     * 尾节点的单例
     */
    private static final Record BOTTOM = new Record();

    /**
     * hint 字符串
     */
    private final String hintString;
    /**
     * 下一个节点
     */
    private final Record next;
    /**
     * 位置
     */
    private final int pos;

    // =========== 构造方法 ===========

    Record(Record next, Object hint) {
        // This needs to be generated even if toString() is never called as it may change later on.
        hintString = hint instanceof ResourceLeakHint ? ((ResourceLeakHint) hint).toHintString() : hint.toString(); // 如果传入的 hint 类型为 ResourceLeakHint 类型,会调用对应的 toHintString() 方法,拼接更友好的字符串提示信息。
        this.next = next;
        this.pos = next.pos + 1;
    }

    Record(Record next) {
       hintString = null;
       this.next = next;
       this.pos = next.pos + 1;
    }

    // Used to terminate the stack
    private Record() {
        hintString = null;
        next = null;
        pos = -1;
    }

    @Override
    public String toString() {
        StringBuilder buf = new StringBuilder(2048);
        if (hintString != null) {
            buf.append("\tHint: ").append(hintString).append(NEWLINE);
        }

        // Append the stack trace.
        StackTraceElement[] array = getStackTrace();
        // Skip the first three elements.
        out: for (int i = 3; i < array.length; i++) {
            StackTraceElement element = array[i];
            // 跳过忽略的方法 (如果调用栈的方法在 ResourceLeakDetector.exclusions 属性中,进行忽略。)
            // Strip the noisy stack trace elements.
            String[] exclusions = excludedMethods.get();
            for (int k = 0; k < exclusions.length; k += 2) {
                if (exclusions[k].equals(element.getClassName())
                        && exclusions[k + 1].equals(element.getMethodName())) {
                    continue out;
                }
            }

            buf.append('\t');
            buf.append(element.toString());
            buf.append(NEWLINE);
        }
        return buf.toString();
    }

}

由于 next 属性,我们可以得知,Record 是链式结构。呼应前文record对象。

io.netty.util.ResourceLeakHint ,接口,提供人类可读( 易懂 )的提示信息,使用在 ResourceLeakDetector 中:

/**
 * A hint object that provides human-readable message for easier resource leak tracking.
 */
public interface ResourceLeakHint {

    /**
     * Returns a human-readable message that potentially enables easier resource leak tracking.
     */
    String toHintString();

}

它的实现类是 AbstractChannelHandlerContext 。
对应的实现方法如下:

/**
 * 名字
 */
private final String name;

@Override
public String toHintString() {
    return '\'' + name + "' will handle the message from this point.";
}
发布了46 篇原创文章 · 获赞 6 · 访问量 3847

猜你喜欢

转载自blog.csdn.net/weixin_43257196/article/details/104974725