[源码解析] elastic search 8.0.0 的查询原理（一）

每个查询请求都是先将request封装为一个task，然后再异步执行，如下所示

public <Request extends ActionRequest, Response extends ActionResponse>
    Task registerAndExecute(String type, TransportAction<Request, Response> action, Request request,
                            BiConsumer<Task, Response> onResponse, BiConsumer<Task, Exception> onFailure) {
         //1 来一个请求，会生成一个task
        final Task task;
        try {
            task = register(type, action.actionName, request);
        } catch (TaskCancelledException e) {
            unregisterChildNode.close();
            throw e;
        }
         //2 task 生成完后，会根据action的类型找对应的实现类真正执行
        action.execute(task, request, new ActionListener<Response>() {
            @Override
            public void onResponse(Response response) {
                try {
                    Releasables.close(unregisterChildNode, () -> unregister(task));
                } finally {
                    onResponse.accept(task, response);
                }
            }

             
        });
        return task;
    }

进入execute之后，具体过程如下。

经过一个请求链的验证

RequestFilterChain<Request, Response> requestFilterChain = new RequestFilterChain<>(this, logger);
        requestFilterChain.proceed(task, actionName, request, listener);

@Override
        public void proceed(Task task, String actionName, Request request, ActionListener<Response> listener) {
            int i = index.getAndIncrement();
            try {
                //1、2个拦截器进行各自的验证
                if (i < this.action.filters.length) {
                    this.action.filters[i].apply(task, actionName, request, listener, this);
                //2、通过后开始执行
                } else if (i == this.action.filters.length) {
                    this.action.doExecute(task, request, listener);
                } else {
                    listener.onFailure(new IllegalStateException("proceed was called too many times"));
                }
            } catch(Exception e) {
                logger.trace("Error during transport action execution.", e);
                listener.onFailure(e);
            }
        }

上面是模板方法设计模式，先用拦截器过滤一遍，看是否符合要求的action

在上面代码的第一步中，涉及到的具体的拦截器实例是下面这2个

第一个

第二个

在执行doExecute的时候就会到了Action真正的实现类去执行。在文档查询里面是TransportSearchAction

@Override
    protected void doExecute(Task task, SearchRequest searchRequest, ActionListener<SearchResponse> listener) {
//1、会对其他集群的索引和当前集群的索引分组
final Map<String, OriginalIndices> remoteClusterIndices = remoteClusterService.groupIndices(searchRequest.indicesOptions(),
                searchRequest.indices());
}

重点方法executeSearch。由于该方法很长，接下来分段解析。

扫描二维码关注公众号，回复： 11953803 查看本文章

private void executeSearch(SearchTask task, SearchTimeProvider timeProvider, SearchRequest searchRequest,
                               OriginalIndices localIndices, List<SearchShardIterator> remoteShardIterators,
                               BiFunction<String, String, DiscoveryNode> remoteConnections, ClusterState clusterState,
                               Map<String, AliasFilter> remoteAliasMap, ActionListener<SearchResponse> listener,
                               SearchResponse.Clusters clusters) {
1 集群的状态是否可用，是有有异常

clusterState.blocks().globalBlockedRaiseException(ClusterBlockLevel.READ);
2 解析索引名称 正常的别名或者索引es不允许以_开始

final Index[] indices = resolveLocalIndices(localIndices, clusterState, timeProvider);
3、构建别名过滤器 如果是内部索引查询比如.ml-config，是获取不到任何的别名过滤器的
Map<String, AliasFilter> aliasFilter = buildPerIndexAliasFilter(searchRequest, clusterState, indices, remoteAliasMap);
4、路由的处理
Map<String, Set<String>> routingMap = indexNameExpressionResolver.resolveSearchRouting(clusterState, searchRequest.routing(),
            searchRequest.indices());
        routingMap = routingMap == null ? Collections.emptyMap() : Collections.unmodifiableMap(routingMap);
        String[] concreteIndices = new String[indices.length];
        for (int i = 0; i < indices.length; i++) {
            concreteIndices[i] = indices[i].getName();
        }
}

在第2步解析索引名称中 ，系统预制2个解析器

private final DateMathExpressionResolver dateMathExpressionResolver = new DateMathExpressionResolver();
    private final WildcardExpressionResolver wildcardExpressionResolver = new WildcardExpressionResolver();
    private final List<ExpressionResolver> expressionResolvers = List.of(dateMathExpressionResolver, wildcardExpressionResolver);

对于日期类型的索引而言，判断的依据是看下当前这个索引是不是以<开头 或者>结尾

对于通配符匹配的解析器而言，先判断是不是_all或者*，也就是查询所有，还会与es已有的用户创建的索引进行对比。

如何获取当前es中的索引信息

第1种方法

在Metadata中

 public SortedMap<String, IndexAbstraction> getIndicesLookup() {
        return indicesLookup;
    }

第2种方法：

这个字段中也包含，并通过该方法获取

private final ImmutableOpenMap<String, IndexMetadata> indices;

public IndexMetadata index(String index) {
        return indices.get(index);
    }

 

继续回到主线 

private void executeSearch(SearchTask task, SearchTimeProvider timeProvider, SearchRequest searchRequest,
                               OriginalIndices localIndices, List<SearchShardIterator> remoteShardIterators,
                               BiFunction<String, String, DiscoveryNode> remoteConnections, ClusterState clusterState,
                               Map<String, AliasFilter> remoteAliasMap, ActionListener<SearchResponse> listener,
                               SearchResponse.Clusters clusters) {
 //5 
Map<String, Long> nodeSearchCounts = searchTransportService.getPendingSearchRequests();
//6 根据路由获取对应的分片
GroupShardsIterator<ShardIterator> localShardsIterator = clusterService.operationRouting().searchShards(clusterState,
                concreteIndices, routingMap, searchRequest.preference(), searchService.getResponseCollectorService(), nodeSearchCounts);
//是将本地集群和其他集群的迭代器对象列表进行合并
        GroupShardsIterator<SearchShardIterator> shardIterators = mergeShardsIterators(localShardsIterator, localIndices,
            searchRequest.getLocalClusterAlias(), remoteShardIterators);



}

 在第6步中，会涉及IndexRoutingTable，它表示单个索引的路由信息，该对象包含索引的所有分片。ShardRouting表示分片

如图所示，可以看出IndexRoutingTable和ShardRouting的关系。最后有几个分片就会有几个分片迭代器对象

继续回到主线 

private void executeSearch(SearchTask task, SearchTimeProvider timeProvider, SearchRequest searchRequest,
                               OriginalIndices localIndices, List<SearchShardIterator> remoteShardIterators,
                               BiFunction<String, String, DiscoveryNode> remoteConnections, ClusterState clusterState,
                               Map<String, AliasFilter> remoteAliasMap, ActionListener<SearchResponse> listener,
                               SearchResponse.Clusters clusters) {

//7 处理请求中的boost
Map<String, Float> concreteIndexBoosts = resolveIndexBoosts(searchRequest, clusterState);
if (shardIterators.size() == 1) {
            // if we only have one group, then we always want Q_T_F, no need for DFS, and no need to do THEN since we hit one shard
            searchRequest.searchType(QUERY_THEN_FETCH);
        }
//8 如果用户不设置 此处默认给修改为true
        if (searchRequest.allowPartialSearchResults() == null) {
           // No user preference defined in search request - apply cluster service default
            searchRequest.allowPartialSearchResults(searchService.defaultAllowPartialSearchResults());
        }
//9 获取当前的节点 查看当前集群中的节点数量会调用ClusterState中的nodes方法
final DiscoveryNodes nodes = clusterState.nodes();
//10 获取异步执行线程执行器
final Executor asyncSearchExecutor = asyncSearchExecutor(indices, clusterState);

}

第10步的详细分析：

同样都是查询，类型也都是query_then_fetch

但是系统索引和我们定义的索引的查询使用的是不同的线程池

Executor asyncSearchExecutor(final Index[] indices, final ClusterState clusterState) {
        final boolean onlySystemIndices =
            Arrays.stream(indices).allMatch(index -> clusterState.metadata().index(index.getName()).isSystem());
        return onlySystemIndices ? threadPool.executor(ThreadPool.Names.SYSTEM_READ) : threadPool.executor(ThreadPool.Names.SEARCH);
    }

EsThreadPoolExecutor[name = /system_read, queue capacity = 2000, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@1f1d38e1[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]]

上面是系统内置索引的线程执行器

下面是用户自定义索引使用的线程池

EWMATrackingEsThreadPoolExecutor[name = /search, queue capacity = 1000, task execution EWMA = 0s, org.elasticsearch.common.util.concurrent.EWMATrackingEsThreadPoolExecutor@470b5f26[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]]

继续回到主线 

private void executeSearch(SearchTask task, SearchTimeProvider timeProvider, SearchRequest searchRequest,
                               OriginalIndices localIndices, List<SearchShardIterator> remoteShardIterators,
                               BiFunction<String, String, DiscoveryNode> remoteConnections, ClusterState clusterState,
                               Map<String, AliasFilter> remoteAliasMap, ActionListener<SearchResponse> listener,
                               SearchResponse.Clusters clusters) {

//11 开始查询

searchAsyncAction(task, searchRequest, asyncSearchExecutor, shardIterators, timeProvider, connectionLookup, clusterState,
            Collections.unmodifiableMap(aliasFilter), concreteIndexBoosts, routingMap, listener, preFilterSearchShards, clusters).start();
}

第11步分析：

查询类型共这2种，其他的已经不再支持

switch (searchRequest.searchType()) {
                case DFS_QUERY_THEN_FETCH:
                    searchAsyncAction = new SearchDfsQueryThenFetchAsyncAction(logger, searchTransportService, connectionLookup,
                        aliasFilter, concreteIndexBoosts, indexRoutings, searchPhaseController, executor, searchRequest, listener,
                        shardIterators, timeProvider, clusterState, task, clusters, exc -> cancelTask(task, exc));
                    break;
                case QUERY_THEN_FETCH:
                    searchAsyncAction = new SearchQueryThenFetchAsyncAction(logger, searchTransportService, connectionLookup,
                        aliasFilter, concreteIndexBoosts, indexRoutings, searchPhaseController, executor, searchRequest, listener,
                        shardIterators, timeProvider, clusterState, task, clusters, exc -> cancelTask(task, exc));
                    break;
                default:
                    throw new IllegalStateException("Unknown search type: [" + searchRequest.searchType() + "]");
            }

searchAsyncAction生成一个抽象的异步执行Action ,AbstractSearchAsyncAction。这个类是干什么的呢？

它是一个抽象的基础类，描述的是对分片的操作。会查询该请求所有涉及到的分片并收集返回的结果。如果其中一个分片返回失败，比如此分片不可用了，它会自动请求该分片的副本来完成请求。所以这个类处理的是请求和收集，那要请求的分片是哪些呢？是上文传入的GroupShardsIterator。要请求的分片都在GroupShardsIterator。SearchQueryThenFetchAsyncAction就是一个具体的查询并收集的实例。

具体开始的时候的代码

SearchQueryThenFetchAsyncAction(final Logger logger, final SearchTransportService searchTransportService,
                                    final BiFunction<String, String, Transport.Connection> nodeIdToConnection,
                                    final Map<String, AliasFilter> aliasFilter,
                                    final Map<String, Float> concreteIndexBoosts, final Map<String, Set<String>> indexRoutings,
                                    final SearchPhaseController searchPhaseController, final Executor executor,
                                    final SearchRequest request, final ActionListener<SearchResponse> listener,
                                    final GroupShardsIterator<SearchShardIterator> shardsIts,
                                    final TransportSearchAction.SearchTimeProvider timeProvider,
                                    ClusterState clusterState, SearchTask task, SearchResponse.Clusters clusters,
                                    Consumer<Exception> onPartialMergeFailure) {
//1 默认先开始query阶段
        super("query", logger, searchTransportService, nodeIdToConnection, aliasFilter, concreteIndexBoosts, indexRoutings,
                executor, request, listener, shardsIts, timeProvider, clusterState, task,
                searchPhaseController.newSearchPhaseResults(executor, task.getProgressListener(),
                    request, shardsIts.size(), onPartialMergeFailure), request.getMaxConcurrentShardRequests(), clusters);
//2 es默认只查10条
        this.topDocsSize = getTopDocsSize(request);
        this.trackTotalHitsUpTo = request.resolveTrackTotalHitsUpTo();
        this.searchPhaseController = searchPhaseController;
        this.progressListener = task.getProgressListener();
        final SearchSourceBuilder sourceBuilder = request.source();
        progressListener.notifyListShards(SearchProgressListener.buildSearchShards(this.shardsIts),
            SearchProgressListener.buildSearchShards(toSkipShardsIts), clusters, sourceBuilder == null || sourceBuilder.size() != 0);
    }

真正开始执行之前会调用SearchProgressListener中的方法进行通知

protected void onListShards(List<SearchShard> shards, List<SearchShard> skippedShards, Clusters clusters, boolean fetchPhase) {}

开始执行时返回 SearchQueryThenFetchAsyncAction后调用start方法

public final void start() {
        
        executePhase(this);
    }

private void executePhase(SearchPhase phase) {
         
            phase.run();
         
    }

 org.elasticsearch.action.search.AbstractSearchAsyncAction#run

@Override
    public final void run() {
         
        if (shardsIts.size() > 0) {
            //在每个分片上都会执行一次
            for (int index = 0; index < shardsIts.size(); index++) {
                final SearchShardIterator shardRoutings = shardsIts.get(index);
                assert shardRoutings.skip() == false;
                performPhaseOnShard(index, shardRoutings, shardRoutings.nextOrNull());
            }
        }
    }

private void performPhaseOnShard(final int shardIndex, final SearchShardIterator shardIt, final ShardRouting shard) {
       
          
            Runnable r = () -> {
                final Thread thread = Thread.currentThread();
                try {
                    executePhaseOnShard(shardIt, shard,
                        new SearchActionListener<Result>(shardIt.newSearchShardTarget(shard.currentNodeId()), shardIndex) {
                            @Override
                            public void innerOnResponse(Result result) {
                                try {
                                    onShardResult(result, shardIt);
                                } finally {
                                    executeNext(pendingExecutions, thread);
                                }
                            }
                        });
                } catch (final Exception e) {
                     
                }
            };
            
                r.run();
            
        }
    }

线程的名字，

protected void executePhaseOnShard(final SearchShardIterator shardIt, final ShardRouting shard,
                                       final SearchActionListener<SearchPhaseResult> listener) {
//1 组装一个请求对象
        ShardSearchRequest request = rewriteShardSearchRequest(super.buildShardSearchRequest(shardIt));
        
//2 将请求发送出去
getSearchTransport().sendExecuteQuery(getConnection(shardIt.getClusterAlias(), shard.currentNodeId()),
            request, getTask(), listener);
    }

向当前集群的当前节点申请一个连接

@Override
    public final Transport.Connection getConnection(String clusterAlias, String nodeId) {
        return nodeIdToConnection.apply(clusterAlias, nodeId);
    }

申请的时候回判断是不是本地，否则会从连接管理器获取

public Transport.Connection getConnection(DiscoveryNode node) {
        if (isLocalNode(node)) {
            return localNodeConnection;
        } else {
            return connectionManager.getConnection(node);
        }
    }

主分片是否为1，对查询来说有非常大的影响。

org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor#interceptSender

@Override
    public AsyncSender interceptSender(AsyncSender sender) {
        return new AsyncSender() {
            @Override
            public <T extends TransportResponse> void sendRequest(Transport.Connection connection, String action, TransportRequest request,
                                                                  TransportRequestOptions options, TransportResponseHandler<T> handler) {
                
              
                    sendWithUser(connection, action, request, options, handler, sender, requireAuth);
                 
            
        };
    }

 org.elasticsearch.transport.TransportService#sendLocalRequest

private void sendLocalRequest(long requestId, final String action, final TransportRequest request, TransportRequestOptions options) {
        final DirectResponseChannel channel = new DirectResponseChannel(localNode, action, requestId, this, threadPool);
           
            final RequestHandlerRegistry reg = getRequestHandler(action);
             
            final String executor = reg.getExecutor();
            if (ThreadPool.Names.SAME.equals(executor)) {
                //noinspection unchecked
                reg.processMessageReceived(request, channel);
            } 

    }

查询过程中，内部发送请求过程中，先是使用的same名称的线程池，后使用search名称的线程池。

当前节点收到search阶段的请求

org.elasticsearch.transport.RequestHandlerRegistry#processMessageReceived

public void processMessageReceived(Request request, TransportChannel channel) throws Exception {
        final Task task = taskManager.register(channel.getChannelType(), action, request);
        Releasable unregisterTask = () -> taskManager.unregister(task);
        try {
            
            final TaskTransportChannel taskTransportChannel = new TaskTransportChannel(channel, unregisterTask);
//这里的handler实例是 ProfileSecuredRequestHandler{action='indices:data/read/search[phase/query]', executorName='same', forceExecution=false}
            handler.messageReceived(request, taskTransportChannel, task);
            unregisterTask = null;
        } finally {
            Releasables.close(unregisterTask);
        }
    }

我们看到了如果一个索引设置多分片的弊端，这些各个发送的调用都是串行执行的，因为最上游的外层有一个for循环。

就是这里

@Override
    public final void run() {
         
        if (shardsIts.size() > 0) {
            //在每个分片上都会执行一次
            for (int index = 0; index < shardsIts.size(); index++) {
                final SearchShardIterator shardRoutings = shardsIts.get(index);
                assert shardRoutings.skip() == false;
                performPhaseOnShard(index, shardRoutings, shardRoutings.nextOrNull());
            }
        }
    }

未完待续……