1.背景
Skywalking7.0が完了し、その他の機能は正常です。
プロファイルタスクタスクを作成しようとすると、応答データが空になり、作成に失敗します。
スカイウォーキングログを確認し、es書き込みエラーが原因の障害であることがわかりました。vim skywalking-oap-server.log
2.エラー分析：
重要な情報は次のとおりです。
Exception while fetching data (/createTask) :
Elasticsearch exception [
type=illegal_argument_exception,
reason=no write index is defined for alias [skywalking_profile_task]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index
]
org.elasticsearch.ElasticsearchStatusException:
Elasticsearch exception [
type=illegal_argument_exception,
reason=no write index is defined for alias [skywalking_profile_task]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index
]
エラーレポートによると、2つの理由が考えられます。
- is_write_index = false、インデックスは書き込みが無効になり、書き込みは失敗します
- エイリアスは複数のインデックスを指していますが、どれも記述できません
esデータを確認し、それが2番目のタイプであることを確認します。
3.解決策
すべてのログをクリアし、
curl -XDELETE http://localhost:9200/skywalking*
スカイウォーキングを再開します
./bin/startup.sh
4.詳細ログ
2020-06-16 20:14:09,050 - graphql.execution.SimpleDataFetcherExceptionHandler - 23 [qtp888378046-47] WARN [] - Exception while fetching data (/createTask) : Elasticsearch exception [type=illegal_argument_exception, reason=no write index is defined for alias [skywalking_profile_task]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index]
org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=illegal_argument_exception, reason=no write index is defined for alias [skywalking_profile_task]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index]
at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177) ~[elasticsearch-7.0.0.jar:7.0.0]
at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1706) ~[elasticsearch-rest-high-level-client-7.0.0.jar:7.0.0]
at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1683) ~[elasticsearch-rest-high-level-client-7.0.0.jar:7.0.0]
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1446) ~[elasticsearch-rest-high-level-client-7.0.0.jar:7.0.0]
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1403) ~[elasticsearch-rest-high-level-client-7.0.0.jar:7.0.0]
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1373) ~[elasticsearch-rest-high-level-client-7.0.0.jar:7.0.0]
at org.elasticsearch.client.RestHighLevelClient.index(RestHighLevelClient.java:821) ~[elasticsearch-rest-high-level-client-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.server.storage.plugin.elasticsearch7.client.ElasticSearch7Client.forceInsert(ElasticSearch7Client.java:207) ~[storage-elasticsearch7-plugin-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.server.storage.plugin.elasticsearch.base.NoneStreamEsDAO.insert(NoneStreamEsDAO.java:45) ~[storage-elasticsearch-plugin-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.server.core.analysis.worker.NoneStreamPersistentWorker.in(NoneStreamPersistentWorker.java:49) ~[server-core-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.server.core.analysis.worker.NoneStreamingProcessor.in(NoneStreamingProcessor.java:56) ~[server-core-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.server.core.profile.ProfileTaskMutationService.createTask(ProfileTaskMutationService.java:100) ~[server-core-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.query.graphql.resolver.ProfileMutation.createProfileTask(ProfileMutation.java:51) ~[query-graphql-plugin-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.query.graphql.resolver.ProfileMutationMethodAccess.invoke(Unknown Source) ~[reflectasm-1.11.7.jar:7.0.0]
at com.coxautodev.graphql.tools.MethodFieldResolverDataFetcher.get(MethodFieldResolver.kt:147) ~[graphql-java-tools-5.2.3.jar:?]
at graphql.execution.ExecutionStrategy.fetchField(ExecutionStrategy.java:227) ~[graphql-java-8.0.jar:?]
at graphql.execution.ExecutionStrategy.resolveField(ExecutionStrategy.java:170) ~[graphql-java-8.0.jar:?]
at graphql.execution.AsyncSerialExecutionStrategy.lambda$execute$1(AsyncSerialExecutionStrategy.java:42) ~[graphql-java-8.0.jar:?]
at graphql.execution.Async.eachSequentiallyImpl(Async.java:72) [graphql-java-8.0.jar:?]
at graphql.execution.Async.eachSequentially(Async.java:62) [graphql-java-8.0.jar:?]
at graphql.execution.AsyncSerialExecutionStrategy.execute(AsyncSerialExecutionStrategy.java:37) [graphql-java-8.0.jar:?]
at graphql.execution.Execution.executeOperation(Execution.java:158) [graphql-java-8.0.jar:?]
at graphql.execution.Execution.execute(Execution.java:100) [graphql-java-8.0.jar:?]
at graphql.GraphQL.execute(GraphQL.java:558) [graphql-java-8.0.jar:?]
at graphql.GraphQL.parseValidateAndExecute(GraphQL.java:500) [graphql-java-8.0.jar:?]
at graphql.GraphQL.executeAsync(GraphQL.java:470) [graphql-java-8.0.jar:?]
at graphql.GraphQL.execute(GraphQL.java:401) [graphql-java-8.0.jar:?]
at org.apache.skywalking.oap.query.graphql.GraphQLQueryHandler.execute(GraphQLQueryHandler.java:93) [query-graphql-plugin-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.query.graphql.GraphQLQueryHandler.doPost(GraphQLQueryHandler.java:83) [query-graphql-plugin-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.server.library.server.jetty.JettyJsonHandler.doPost(JettyJsonHandler.java:59) [library-server-7.0.0.jar:7.0.0]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) [javax.servlet-api-3.1.0.jar:3.1.0]
at org.apache.skywalking.oap.server.library.server.jetty.JettyJsonHandler.service(JettyJsonHandler.java:107) [library-server-7.0.0.jar:7.0.0]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [javax.servlet-api-3.1.0.jar:3.1.0]
at org.apache.skywalking.oap.server.library.server.jetty.JettyJsonHandler.service(JettyJsonHandler.java:112) [library-server-7.0.0.jar:7.0.0]
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841) [jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:543) [jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1239) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:481) [jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1141) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.Server.handle(Server.java:564) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279) [jetty-io-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110) [jetty-io-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124) [jetty-io-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672) [jetty-util-9.4.2.v20170220.jar:9.4.2.v20170220]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Suppressed: org.elasticsearch.client.ResponseException: method [PUT], host [http://localhost:9200], URI [/skywalking_profile_task/_doc/1592309649047_2?refresh=true&timeout=1m], status line [HTTP/1.1 400 Bad Request]
{
"error":{
"root_cause":[{
"type":"illegal_argument_exception","reason":"no write index is defined for alias [skywalking_profile_task]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index"}],"type":"illegal_argument_exception","reason":"no write index is defined for alias [skywalking_profile_task]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index"},"status":400}
at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:260) ~[elasticsearch-rest-client-7.0.0.jar:7.0.0]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:238) ~[elasticsearch-rest-client-7.0.0.jar:7.0.0]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:212) ~[elasticsearch-rest-client-7.0.0.jar:7.0.0]
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1433) ~[elasticsearch-rest-high-level-client-7.0.0.jar:7.0.0]
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1403) ~[elasticsearch-rest-high-level-client-7.0.0.jar:7.0.0]
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1373) ~[elasticsearch-rest-high-level-client-7.0.0.jar:7.0.0]
at org.elasticsearch.client.RestHighLevelClient.index(RestHighLevelClient.java:821) ~[elasticsearch-rest-high-level-client-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.server.storage.plugin.elasticsearch7.client.ElasticSearch7Client.forceInsert(ElasticSearch7Client.java:207) ~[storage-elasticsearch7-plugin-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.server.storage.plugin.elasticsearch.base.NoneStreamEsDAO.insert(NoneStreamEsDAO.java:45) ~[storage-elasticsearch-plugin-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.server.core.analysis.worker.NoneStreamPersistentWorker.in(NoneStreamPersistentWorker.java:49) ~[server-core-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.server.core.analysis.worker.NoneStreamingProcessor.in(NoneStreamingProcessor.java:56) ~[server-core-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.server.core.profile.ProfileTaskMutationService.createTask(ProfileTaskMutationService.java:100) ~[server-core-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.query.graphql.resolver.ProfileMutation.createProfileTask(ProfileMutation.java:51) ~[query-graphql-plugin-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.query.graphql.resolver.ProfileMutationMethodAccess.invoke(Unknown Source) ~[reflectasm-1.11.7.jar:7.0.0]
at com.coxautodev.graphql.tools.MethodFieldResolverDataFetcher.get(MethodFieldResolver.kt:147) ~[graphql-java-tools-5.2.3.jar:?]
at graphql.execution.ExecutionStrategy.fetchField(ExecutionStrategy.java:227) ~[graphql-java-8.0.jar:?]
at graphql.execution.ExecutionStrategy.resolveField(ExecutionStrategy.java:170) ~[graphql-java-8.0.jar:?]
at graphql.execution.AsyncSerialExecutionStrategy.lambda$execute$1(AsyncSerialExecutionStrategy.java:42) ~[graphql-java-8.0.jar:?]
at graphql.execution.Async.eachSequentiallyImpl(Async.java:72) [graphql-java-8.0.jar:?]
at graphql.execution.Async.eachSequentially(Async.java:62) [graphql-java-8.0.jar:?]
at graphql.execution.AsyncSerialExecutionStrategy.execute(AsyncSerialExecutionStrategy.java:37) [graphql-java-8.0.jar:?]
at graphql.execution.Execution.executeOperation(Execution.java:158) [graphql-java-8.0.jar:?]
at graphql.execution.Execution.execute(Execution.java:100) [graphql-java-8.0.jar:?]
at graphql.GraphQL.execute(GraphQL.java:558) [graphql-java-8.0.jar:?]
at graphql.GraphQL.parseValidateAndExecute(GraphQL.java:500) [graphql-java-8.0.jar:?]
at graphql.GraphQL.executeAsync(GraphQL.java:470) [graphql-java-8.0.jar:?]
at graphql.GraphQL.execute(GraphQL.java:401) [graphql-java-8.0.jar:?]
at org.apache.skywalking.oap.query.graphql.GraphQLQueryHandler.execute(GraphQLQueryHandler.java:93) [query-graphql-plugin-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.query.graphql.GraphQLQueryHandler.doPost(GraphQLQueryHandler.java:83) [query-graphql-plugin-7.0.0.jar:7.0.0]
at org.apache.skywalking.oap.server.library.server.jetty.JettyJsonHandler.doPost(JettyJsonHandler.java:59) [library-server-7.0.0.jar:7.0.0]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) [javax.servlet-api-3.1.0.jar:3.1.0]
at org.apache.skywalking.oap.server.library.server.jetty.JettyJsonHandler.service(JettyJsonHandler.java:107) [library-server-7.0.0.jar:7.0.0]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [javax.servlet-api-3.1.0.jar:3.1.0]
at org.apache.skywalking.oap.server.library.server.jetty.JettyJsonHandler.service(JettyJsonHandler.java:112) [library-server-7.0.0.jar:7.0.0]
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841) [jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:543) [jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1239) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:481) [jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1141) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.Server.handle(Server.java:564) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279) [jetty-io-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110) [jetty-io-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124) [jetty-io-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672) [jetty-util-9.4.2.v20170220.jar:9.4.2.v20170220]
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590) [jetty-util-9.4.2.v20170220.jar:9.4.2.v20170220]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
参照リンク：https：//www.cnblogs.com/Neeo/articles/10897280.html