前端响应超时、API-server 服务内存不足...碰见这类 DolphinScheduler 资源中心相关问题怎么办?

作者 | 刘森 卡特加特 大数据工程师

最近,有些用户小伙伴反映在使用 Apache DolphinScheduler 资源中心时会遇到问题,社区小伙伴整理了一些常见问题,希望帮大家解决燃眉之急。

[WARN] 2023-04-25 03:02:58.463 +0000 org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver:[434] - Failure in @ExceptionHandler org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler#exceptionHandler(Exception, HandlerMethod)
java.lang.IllegalStateException: Could not resolve parameter [1] in public org.apache.dolphinscheduler.api.utils.Result org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler.exceptionHandler(java.lang.Exception,org.springframework.web.method.HandlerMethod): No suitable resolver
    at org.springframework.web.method.support.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:176)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:146)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
    at org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver.doResolveHandlerMethodException(ExceptionHandlerExceptionResolver.java:428)
    at org.springframework.web.servlet.handler.AbstractHandlerMethodExceptionResolver.doResolveException(AbstractHandlerMethodExceptionResolver.java:75)
    at org.springframework.web.servlet.handler.AbstractHandlerExceptionResolver.resolveException(AbstractHandlerExceptionResolver.java:142)
    at org.springframework.web.servlet.handler.HandlerExceptionResolverComposite.resolveException(HandlerExceptionResolverComposite.java:80)
    at org.springframework.web.servlet.DispatcherServlet.processHandlerException(DispatcherServlet.java:1330)
    at org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1141)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1087)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:517)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:584)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)

问题原因:前端响应超时

解决办法:

需要修改以下4个文件。

/api-server/ui/assets/service.33119ae9.js
/api-server/ui/assets/service.33119ae9.js.gz

/ui/assets/service.33119ae9.js
/ui/assets/service.33119ae9.js.gz

并找到以下配置,然后将 15e3 修改为 15e5 或更高版本。

baseURL:"/dolphinscheduler",timeout:15e3

1.这里 service.33119ae9.并不是固定的名称,只要找到 service..js*的文件修改就行

2.如果使用docker部署的dolphinscheduler。是没有/ui/assets/service.33119ae9.js和/ui/assets/service.33119ae9.js.gz文件,忽略就行

3..gz文件使用vim就可以打开

4.重启api服务

5.清除浏览器缓存

[WARN] 2023-04-25 03:05:50.855 +0000 org.eclipse.jetty.server.HttpChannel:[776] - /dolphinscheduler/resources
org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.springframework.web.multipart.MultipartException: Failed to parse multipart servlet request; nested exception is java.lang.OutOfMemoryError: Java heap space
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:517)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:584)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
    at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
    at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)

问题原因:

api-server服务内存不足

解决办法:

更改bin/dolphinscheduler-daemon.sh的api-server项内存大小参数

Docker部署解决办法:

1.进入Docker实例

docker exec -it bbd60fe717af /bin/bash

2.修改/opt/dolphinscheduler/bin/start.sh

vim /opt/dolphinscheduler/bin/start.sh

3.内存参数从‘-Xms1g -Xmx1g -Xmn512m’ 修改为‘-Xms2g -Xmx2g -Xmn1g’。具体数值根据需要资源修改

#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

BIN_DIR=$(dirname $0)
DOLPHINSCHEDULER_HOME=${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)}

source "$DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh"

JAVA_OPTS=${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Xms2g -Xmx2g -Xmn1g -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"}

if [[ "
$DOCKER" == "true" ]]; then
  JAVA_OPTS="
${JAVA_OPTS} -XX:-UseContainerSupport"
fi

$JAVA_HOME/bin/java $JAVA_OPTS \
  -cp "
$DOLPHINSCHEDULER_HOME/conf":"$DOLPHINSCHEDULER_HOME/libs/*" \
  org.apache.dolphinscheduler.api.ApiApplicationServer

3.12 BUG

  • 修复任务组可能被多个任务抢走的问题 #13115

如果设置任务组,多个任务会抢占任务组不释放,导致master一直重试释放,无法继续执行任务

解决办法:

1.升级到3.13

2.取消任务组

[INFO] 2023-04-25 08:51:06.044 +0000 org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteRunnable:[286] - [WorkflowInstance-357][TaskInstance-4553] - Begin to handle state event, TaskStateEvent(processInstanceId=357, taskInstanceId=4553, taskCode=0, status=null, type=WAIT_TASK_GROUP, key=357-4553, channel=null, context=null)
[INFO] 2023-04-25 08:51:06.044 +0000 org.apache.dolphinscheduler.server.master.event.TaskWaitTaskGroupStateHandler:[34] - [WorkflowInstance-357][TaskInstance-4553] - Handle task instance wait task group event, taskInstanceId: 4553
[INFO] 2023-04-25 08:51:06.044 +0000 org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteRunnable:[286] - [WorkflowInstance-356][TaskInstance-4554] - Begin to handle state event, TaskStateEvent(processInstanceId=356, taskInstanceId=4554, taskCode=0, status=null, type=WAIT_TASK_GROUP, key=356-4554, channel=null, context=null)
[INFO] 2023-04-25 08:51:06.044 +0000 org.apache.dolphinscheduler.server.master.event.TaskWaitTaskGroupStateHandler:[34] - [WorkflowInstance-356][TaskInstance-4554] - Handle task instance wait task group event, taskInstanceId: 4554
[INFO] 2023-04-25 08:51:06.044 +0000 org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteRunnable:[286] - [WorkflowInstance-360][TaskInstance-4613] - Begin to handle state event, TaskStateEvent(processInstanceId=360, taskInstanceId=4613, taskCode=0, status=null, type=WAIT_TASK_GROUP, key=360-4613, channel=null, context=null)
[INFO] 2023-04-25 08:51:06.044 +0000 org.apache.dolphinscheduler.server.master.event.TaskWaitTaskGroupStateHandler:[34] - [WorkflowInstance-360][TaskInstance-4613] - Handle task instance wait task group event, taskInstanceId: 4613
[INFO] 2023-04-25 08:51:06.044 +0000 org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteRunnable:[286] - [WorkflowInstance-355][TaskInstance-4549] - Begin to handle state event, TaskStateEvent(processInstanceId=355, taskInstanceId=4549, taskCode=0, status=null, type=WAIT_TASK_GROUP, key=355-4549, channel=null, context=null)
[INFO] 2023-04-25 08:51:06.044 +0000 org.apache.dolphinscheduler.server.master.event.TaskWaitTaskGroupStateHandler:[34] - [WorkflowInstance-355][TaskInstance-4549] - Handle task instance wait task group event, taskInstanceId: 4549

本文由 白鲸开源科技 提供发布支持!

猜你喜欢

转载自blog.csdn.net/DolphinScheduler/article/details/130581151