presto UI 分析及相关源码分析

CLUSTER OVERVIEW

访问coordinator 的 http://IP:PORT，首先呈现的是 cluster overview

访问coordinator

九个参数窗口，唯独 “ RESERVED MEMORY ” 不太好理解，presto 相关文档又没有解释。

而且容易和presto的配置“resources.reserved-system-memory”相混，容易以为是同一个概念。

RESERVED MEMORY

最后通过presto 源码阅读确认，确认为部分 Operator 操作产生的内存消耗。
web UI 先发HTTP请求至coordinator，coordinator，server代码最后调用StageStateMachine.java相关代码。

//StageStateMachine.java
totalMemoryReservation += taskStats.getMemoryReservation().toBytes();

继续溯源

#TaskContext.java
succinctBytes(memoryReservation.get()),

继续溯源

public synchronized ListenableFuture<?> reserveMemory(long bytes)
    {
        checkArgument(bytes >= 0, "bytes is negative");

        ListenableFuture<?> future = queryContext.reserveMemory(bytes);
        memoryReservation.getAndAdd(bytes);
        return future;
    }

继续溯源

//DriverContext.java
public ListenableFuture<?> reserveMemory(long bytes)
    {
        ListenableFuture<?> future = pipelineContext.reserveMemory(bytes);
        long newMemoryReservation = memoryReservation.addAndGet(bytes);
        peakMemoryReservation.accumulateAndGet(newMemoryReservation, Math::max);
        return future;
    }

继续溯源，省掉一些环节

//xxOperator.java
public void addInput(Page page)
    {
        checkState(needsInput(), "Operator is already finishing");
        requireNonNull(page, "page is null");

        long memorySize = 0;
        for (Aggregator aggregate : aggregates) {
            aggregate.processPage(page);
            memorySize += aggregate.getEstimatedSize();
        }
        if (partial) {
            systemMemoryContext.setBytes(memorySize);
        }
        else {
            operatorContext.setMemoryReservation(memorySize);
        }
    }

// Driver.java
// if we got an output page, add it to the next operator
                    if (page != null && page.getPositionCount() != 0) {
                        next.getOperatorContext().startIntervalTimer();
                        next.addInput(page);
                        next.getOperatorContext().recordAddInput(page);
                        movedPage = true;
                    }