1) First, look at the situation gitlab services:
[root@mail gitaly]# gitlab-ctl status
run: alertmanager: (pid 15356) 672s; run: log: (pid 4533) 28089s
run: gitaly: (pid 15373) 672s; run: log: (pid 4548) 28089s
run: gitlab-exporter: (pid 15392) 672s; run: log: (pid 4554) 28089s
run: gitlab-workhorse: (pid 15411) 671s; run: log: (pid 4536) 28089s
run: grafana: (pid 15420) 671s; run: log: (pid 4529) 28089s
run: logrotate: (pid 15434) 670s; run: log: (pid 4559) 28089s
down: nginx: (pid 15517) 670s; run: log: (pid 4540) 28089s
run: node-exporter: (pid 15525) 669s; run: log: (pid 4542) 28089s
run: postgres-exporter: (pid 15534) 669s; run: log: (pid 4524) 28089s
run: postgresql: (pid 15545) 669s; run: log: (pid 4550) 28089s
run: prometheus: (pid 15554) 668s; run: log: (pid 4538) 28089s
run: redis: (pid 15569) 668s; run: log: (pid 4557) 28089s
run: redis-exporter: (pid 15574) 667s; run: log: (pid 4546) 28089s
run: sidekiq: (pid 15582) 666s; run: log: (pid 4549) 28089s
run: unicorn: (pid 15595) 665s; run: log: (pid 4543) 28089s
Nginx service appears to be down, and try to manually restart nginx:
- gitlab-ctl start nginx && gitlab-ctl status
Opened successfully, but still 500 error page, look at the status found nginx and hung up, the internal service nginx there is a problem, continue troubleshooting.
Look at nginx log (complete log in here / var / log / gitlab / nginx / current, but can also see real-time situation by tail):
2) View Log
gitlab-ctl tail nginx
[root@mail gitaly]# gitlab-ctl tail nginx
==> /var/log/gitlab/nginx/current <==
==> /var/log/gitlab/nginx/error.log <==
==> /var/log/gitlab/nginx/gitlab_error.log <==
==> /var/log/gitlab/nginx/gitlab_access.log <==
127.0.0.1 - - [08/Apr/2020:21:35:47 +0800] "GET /assets/locale/zh_CN/app-3ca994c73a42fcc481725a2f0fe66f4de1e2fdd0fd8f8af65379d3c92605537e.js HTTP/1.1" 200 173387 "http://mysql.dev.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
127.0.0.1 - - [08/Apr/2020:21:35:47 +0800] "GET /assets/webpack/pages.root.810e9807.chunk.js HTTP/1.1" 200 1993 "http://mysql.dev.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
127.0.0.1 - - [08/Apr/2020:21:35:48 +0800] "GET /assets/icons-e91700f3f1ecff110fc2c35aa62aec8f2aad69d1bfb35844186a11175a79e25f.svg HTTP/1.1" 200 25998 "" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
127.0.0.1 - - [08/Apr/2020:21:35:48 +0800] "GET /assets/fontawesome-webfont-2adefcbc041e7d18fcf2d417879dc5a09997aa64d675b7a3c4b6ce33da13f3fe.woff2 HTTP/1.1" 200 77160 "http://mysql.dev.com/assets/application-70ce56efa7e602d4b127087b0eca064681ecdd49b57d86665da8b081da39408b.css" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
127.0.0.1 - - [08/Apr/2020:21:36:09 +0800] "GET /root/finally HTTP/1.1" 200 13695 "http://mysql.dev.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
127.0.0.1 - - [08/Apr/2020:21:36:09 +0800] "GET /assets/webpack/commons~pages.admin.clusters~pages.admin.clusters.destroy~pages.admin.clusters.edit~pages.admin.clus~097f0f39.9383357a.chunk.js HTTP/1.1" 200 5614 "http://mysql.dev.com/root/finally" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
127.0.0.1 - - [08/Apr/2020:21:36:09 +0800] "GET /assets/webpack/pages.projects.show.99b231eb.chunk.js HTTP/1.1" 200 11335 "http://mysql.dev.com/root/finally" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
127.0.0.1 - - [08/Apr/2020:21:36:09 +0800] "GET /assets/webpack/commons~pages.groups.epics.index~pages.groups.epics.show~pages.groups.milestones.edit~pages.groups.m~b7d7bc7f.039d71af.chunk.js HTTP/1.1" 200 11374 "http://mysql.dev.com/root/finally" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
127.0.0.1 - - [08/Apr/2020:21:36:09 +0800] "GET /assets/illustrations/cluster_popover-889b8203a86c86d1fd3f0b181e588071d00cd9c77590fd87dba4d19035e1c0df.svg HTTP/1.1" 200 748 "http://mysql.dev.com/root/finally" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
127.0.0.1 - - [08/Apr/2020:21:36:10 +0800] "GET /assets/webpack/vendors~select2.8428e8e9.chunk.js HTTP/1.1" 200 17972 "http://mysql.dev.com/root/finally" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
3) Restart
gitlab-ctl restart
