一、部署环境说明
docker服务版本：
version 18.09.0
nginx服务版本：
version: nginx/1.15.10
redis服务版本：
version: redis/5.0.3
tomcat服务版本：
version: tomcat/8.5.30
JDK服务版本：
alpine-oraclejdk8
二、安装docker服务
配置阿里yum源并安装docker
# yum install docker -y
三、项目部署
1、准备工作
拉取如下镜像
# docker images
[root@localhost ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE nginx 1.15.10 2bcb04bdb83f 3 months ago 109MB tomcat 8.5.30 38bdcf7a367c 14 months ago 546MB redis latest 82629e941a38 6 months ago 95MB huanwei/alpine-oraclejdk8 latest 28e66d63b8a4 2 years ago 167MB
拉取命令如下：
# docker pull docker.io/huanwei/alpine-oraclejdk8
# docker pull daocloud.io/library/tomcat:8.5.30
# docker pull daocloud.io/library/nginx:1.15.10
# docker pull daocloud.io/library/redis:5.0.3
镜像重命名
# docker tag daocloud.io/library/nginx:1.15.10 nginx:1.15.10
# docker tag daocloud.io/library/tomcat:8.5.30 tomcat:8.5.30
删除旧的镜像名
# docker rmi daocloud.io/library/nginx:1.15.10
# docker rmi daocloud.io/library/tomcat:8.5.30
软件版本说明：
nginx version: nginx/1.15.10 redis version: redis/5.0.3 tomcat version: tomcat/8.5.30 alpine-oraclejdk8
2、创建两个tomcat容器
创建一个测试tomcat容器，用于拷贝tomcat原文件
# docker run --name test -d -i -t tomcat:8.5.30 /bin/bash
[root@localhost docker-server]# docker run --name test -d -i -t tomcat:8.5.30 /bin/bash 1287b583d3b7b85a65ea4d80640124fc5dbd1905d0fc5c33ef05e0ba9872c681
进入测试容器熟悉tomcat安装位置
# docker exec -it test bash
[root@localhost docker-server]# docker exec -it test bash root@1287b583d3b7:/usr/local/tomcat# root@1287b583d3b7:/usr/local/tomcat# ls LICENSE NOTICE RELEASE-NOTES RUNNING.txt bin conf include lib logs native-jni-lib temp webapps work root@1287b583d3b7:/usr/local/tomcat# root@1287b583d3b7:/usr/local/tomcat# pwd /usr/local/tomcat root@1287b583d3b7:/usr/local/tomcat# root@1287b583d3b7:/usr/local/tomcat# cd .. root@1287b583d3b7:/usr/local# root@1287b583d3b7:/usr/local# ls bin etc games include lib man sbin share src tomcat root@1287b583d3b7:/usr/local# root@1287b583d3b7:/usr/local# exit exit
拷贝容器的tomcat文件用于后期的tomcat映射挂载
# docker cp test:/usr/local/tomcat /home/tjsp/
[root@localhost docker-server]# docker cp test:/usr/local/tomcat /home/tjsp/ [root@localhost docker-server]# cd .. [root@localhost tjsp]# ls docker-server tomcat [root@localhost tjsp]# [root@localhost tjsp]# cd tomcat/ [root@localhost tomcat]# [root@localhost tomcat]# [root@localhost tomcat]# ls LICENSE NOTICE RELEASE-NOTES RUNNING.txt bin conf include lib logs native-jni-lib temp webapps work [root@localhost tomcat]#
复制tomcat文件用于映射挂载tomcat1服务
[root@localhost tjsp]# cp -r tomcat/ tomcat1 [root@localhost tjsp]# [root@localhost tjsp]# [root@localhost tjsp]# ls docker-server tomcat tomcat1 [root@localhost tjsp]#
创建容器
# docker run -p 8081:8080 --name tomcat1 -h Front-end1 -v /home/tjsp/tomcat1:/usr/local/tomcat -d tomcat:8.5.30
# docker run -p 8082:8080 --name tomcat2 -h Front-end2 -v /home/tjsp/tomcat2:/usr/local/tomcat -d tomcat:8.5.30
[root@localhost tjsp]# docker run -p 8081:8080 --name tomcat1 -h Front-end1 -v /home/tjsp/tomcat1:/usr/local/tomcat -d tomcat:8.5.30 d41f7b69ebbf9717629b220907fc7c88e8d106f2b99a805c6b0465b4bc49f484 [root@localhost tjsp]# [root@localhost tjsp]# docker run -p 8082:8080 --name tomcat2 -h Front-end2 -v /home/tjsp/tomcat2:/usr/local/tomcat -d tomcat:8.5.30 78b5482a0a9546f5355fccb8f28bf4e08751f58e74b8a3b66af36a775ae31788 [root@localhost tjsp]#
查看运行状态
# docker ps
[root@localhost ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 78b5482a0a95 tomcat:8.5.30 "catalina.sh run" About an hour ago Up About an hour 0.0.0.0:8082->8080/tcp tomcat2 d41f7b69ebbf tomcat:8.5.30 "catalina.sh run" About an hour ago Up About an hour 0.0.0.0:8081->8080/tcp tomcat1 1287b583d3b7 tomcat:8.5.30 "/bin/bash" About an hour ago Up About an hour 8080/tcp test
添加office服务，运行一个tomcat跑war包
# cp -r tomcat tomcat3
[root@localhost ~]# cd /home/tjsp/ [root@localhost tjsp]# [root@localhost tjsp]# ls docker-server nginx tomcat tomcat1 tomcat2 [root@localhost tjsp]# [root@localhost tjsp]# cp -r tomcat tomcat3 [root@localhost tjsp]# [root@localhost tjsp]# ls docker-server nginx tomcat tomcat1 tomcat2 tomcat3
创建一个tomcat容器
# docker run -p 8086:8080 --name tomcat3 -h tjsp_office -v /home/tjsp/tomcat3:/usr/local/tomcat -d tomcat:8.5.30
[root@localhost tjsp]# docker run -p 8086:8080 --name tomcat3 -h tjsp_office -v /home/tjsp/tomcat3:/usr/local/tomcat -d tomcat:8.5.30 9fe494bf43bc446f9f021d19e25ec5e030e6031c8fc65dcc8f08609f3b4bc946 [root@localhost tjsp]# [root@localhost tjsp]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9fe494bf43bc tomcat:8.5.30 "catalina.sh run" 6 seconds ago Up 5 seconds 0.0.0.0:8086->8080/tcp tomcat3
3、创建nginx容器
创建一个测试nginx容器，用于拷贝nginx原文件
# docker run --name nginx_test -h nignx1.15.10 -d nginx:1.15.10
# docker cp nginx_test:/etc/nginx /home/tjsp/
[root@localhost tjsp]# docker run --name nginx_test -h nignx1.15.10 -d nginx:1.15.10 0b02f6f222ef38aa2f39fa7ac5d5a588b5df24bb6d7e28fba90aa55b0a4bf82e [root@localhost tjsp]# [root@localhost tjsp]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0b02f6f222ef nginx:1.15.10 "nginx -g 'daemon of…" 4 seconds ago Up 3 seconds 80/tcp nginx_test 78b5482a0a95 tomcat:8.5.30 "catalina.sh run" About an hour ago Up About an hour 0.0.0.0:8082->8080/tcp tomcat2 d41f7b69ebbf tomcat:8.5.30 "catalina.sh run" About an hour ago Up About an hour 0.0.0.0:8081->8080/tcp tomcat1 1287b583d3b7 tomcat:8.5.30 "/bin/bash" About an hour ago Up About an hour 8080/tcp test [root@localhost tjsp]# [root@localhost tjsp]# docker cp nginx_test:/etc/nginx /home/tjsp/ [root@localhost tjsp]# [root@localhost tjsp]# ls docker-server nginx tomcat tomcat1 tomcat2 [root@localhost tjsp]# [root@localhost tjsp]# cd nginx/ [root@localhost nginx]# [root@localhost nginx]# ls conf.d fastcgi_params koi-utf koi-win mime.types modules nginx.conf scgi_params uwsgi_params win-utf [root@localhost nginx]#
创建容器
# docker run --name nginx_tjsp -h nignx1.15.10 -p 7080:80 -v /home/tjsp/nginx:/etc/nginx/ -d nginx:1.15.10
[root@localhost nginx]# docker run --name nginx_tjsp -h nignx1.15.10 -p 7080:80 -v /home/tjsp/nginx:/etc/nginx/ -d nginx:1.15.10 39e65baa524a62e2a07ad0ccc035baf829f3f9736148529b79d18452011bf390 [root@localhost nginx]# [root@localhost nginx]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 39e65baa524a nginx:1.15.10 "nginx -g 'daemon of…" 4 seconds ago Up 3 seconds 0.0.0.0:7080->80/tcp nginx_tjsp 0b02f6f222ef nginx:1.15.10 "nginx -g 'daemon of…" 3 minutes ago Up 3 minutes 80/tcp nginx_test 78b5482a0a95 tomcat:8.5.30 "catalina.sh run" About an hour ago Up About an hour 0.0.0.0:8082->8080/tcp tomcat2 d41f7b69ebbf tomcat:8.5.30 "catalina.sh run" About an hour ago Up About an hour 0.0.0.0:8081->8080/tcp tomcat1 1287b583d3b7 tomcat:8.5.30 "/bin/bash" About an hour ago Up About an hour 8080/tcp test
修改nginx配置文件
# vi /home/tjsp/nginx/nginx.conf
配置文件如下：
user root; worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; ## Proxy upstream tjsp_client_proxy { server 192.168.2.207:8081 weight=1 max_fails=2 fail_timeout=30s; server 192.168.2.207:8082 weight=1 max_fails=2 fail_timeout=30s; } upstream tjsp_server_proxy { server 192.168.2.207:8083 weight=1 max_fails=2 fail_timeout=30s; server 192.168.2.207:8084 weight=1 max_fails=2 fail_timeout=30s; server 192.168.2.207:8087 weight=1 max_fails=2 fail_timeout=30s; } upstream etl_server_proxy { server 192.168.2.207:8085 weight=1 max_fails=2 fail_timeout=30s; #server 172.17.8.20:8086 weight=1 max_fails=2 fail_timeout=30s; } upstream exp_server_proxy { server 192.168.2.207:8086 weight=1 max_fails=2 fail_timeout=30s; } server { listen 80; server_name localhost; error_page 500 502 503 504 /50x.html; location / { root html; autoindex on; autoindex_exact_size off; autoindex_localtime on; index index.html index.htm; } location /tjsp { root html; autoindex on; autoindex_exact_size off; autoindex_localtime on; index index.html index.htm; add_header Access-Control-Allow-Origin *; add_header Access-Control-Allow-Headers "Origin, X-Requested-With, Content-Type, Accept"; add_header Access-Control-Allow-Methods "GET, POST, OPTIONS"; proxy_pass http://tjsp_client_proxy/tjsp; #try_files $uri $uri/ /main proxy_redirect off; proxy_set_header Host $http_host; proxy_set_header Cookie $http_cookie; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; client_max_body_size 300m; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; } location /a_ssets { root html; autoindex on; autoindex_exact_size off; autoindex_localtime on; index index.html index.htm; add_header Access-Control-Allow-Origin *; add_header Access-Control-Allow-Headers "Origin, X-Requested-With, Content-Type, Accept"; add_header Access-Control-Allow-Methods "GET, POST, OPTIONS"; proxy_pass http://tjsp_client_proxy/assets; #try_files $uri $uri/ /main proxy_redirect off; proxy_set_header Host $http_host; proxy_set_header Cookie $http_cookie; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; client_max_body_size 300m; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; } location /zjb { root html; index index.html index.htm; add_header Access-Control-Allow-Origin *; add_header Access-Control-Allow-Headers "Origin, X-Requested-With, Content-Type, Accept"; add_header Access-Control-Allow-Methods "GET, POST, OPTIONS"; proxy_pass http://tjsp_server_proxy/; proxy_redirect off; proxy_set_header Host $http_host; proxy_set_header Cookie $http_cookie; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; client_max_body_size 300m; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; } location /etl { root html; index index.html index.htm; add_header Access-Control-Allow-Origin *; add_header Access-Control-Allow-Headers "Origin, X-Requested-With, Content-Type, Accept"; add_header Access-Control-Allow-Methods "GET, POST, OPTIONS"; proxy_pass http://etl_server_proxy/; proxy_redirect off; proxy_set_header Host $http_host; proxy_set_header Cookie $http_cookie; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; client_max_body_size 300m; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; } location /exp { root html; index index.html index.htm; add_header Access-Control-Allow-Origin *; add_header Access-Control-Allow-Headers "Origin, X-Requested-With, Content-Type, Accept"; add_header Access-Control-Allow-Methods "GET, POST, OPTIONS"; proxy_pass http://exp_server_proxy/office; proxy_redirect off; proxy_set_header Host $http_host; proxy_set_header Cookie $http_cookie; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; client_max_body_size 300m; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; } location /exl { root html; index index.html index.htm; add_header Access-Control-Allow-Origin *; add_header Access-Control-Allow-Headers "Origin, X-Requested-With, Content-Type, Accept"; add_header Access-Control-Allow-Methods "GET, POST, OPTIONS"; proxy_pass http://etl_server_proxy/; proxy_redirect off; proxy_set_header Host $http_host; proxy_set_header Cookie $http_cookie; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; client_max_body_size 300m; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; } } }
4、创建redis容器
说明：
redis的密码为123456
保护模式已经关闭
集群模式未开启
# docker run --name redis -h redis_tjsp -p 6379:6379 -d redis redis-server --requirepass "123456" --appendonly yes --protected-mode no
进入redis查看
# docker exec -it redis bash
[root@localhost tjsp]# docker run --name redis -h redis_tjsp -p 6379:6379 -d redis redis-server --requirepass "123456" --appendonly yes --protected-mode no 7626b4b8f3c1b2f8161341209f159167876cdc3fee8b3b90c180770dc426f3bd [root@localhost tjsp]# [root@localhost tjsp]# docker exec -it redis bash root@redis_tjsp:/data# root@redis_tjsp:/data# ls appendonly.aof root@redis_tjsp:/data# root@redis_tjsp:/data# exit exit [root@localhost tjsp]#
5、构建jdk镜像
创建一个构建目录
# mkdir -p /home/tjsp/docker-server
编写Dockerfile文件
# cd /home/tjsp/docker-server/
# vi Dockerfile
FROM docker.io/huanwei/alpine-oraclejdk8 ADD app.jar / EXPOSE 8082 ENTRYPOINT ["java","-D java.security.egd=file:/dev./urandom","-jar","/app.jar","--spring.profiles.active=dev207"] VOLUME /tmp
构建镜像
# docker build -t cnbi-tjsp:latest .
[root@localhost tjsp]# cd docker-server/ [root@localhost docker-server]# [root@localhost docker-server]# ls Dockerfile app.jar [root@localhost docker-server]# [root@localhost docker-server]# docker build -t cnbi-tjsp:latest . Sending build context to Docker daemon 113MB Step 1/5 : FROM docker.io/huanwei/alpine-oraclejdk8 ---> 28e66d63b8a4 Step 2/5 : ADD /app.jar // ---> 063e68f2e01a Step 3/5 : EXPOSE 8082 ---> Running in 9ab4a3c0fc63 Removing intermediate container 9ab4a3c0fc63 ---> 7522d6d8ea7f Step 4/5 : ENTRYPOINT ["java","-D java.security.egd=file:/dev./urandom","-jar","/app.jar"] ---> Running in 3726768330c0 Removing intermediate container 3726768330c0 ---> d6ddeff92429 Step 5/5 : VOLUME /tmp ---> Running in b7ac287ab1c9 Removing intermediate container b7ac287ab1c9 ---> 692144a74536 Successfully built 692144a74536 Successfully tagged cnbi-tjsp:latest [root@localhost docker-server]#
查看构建的镜像
# docker images
[root@localhost docker-server]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE cnbi-tjsp latest 692144a74536 3 minutes ago 280MB
6、创建djk镜像容器
创建挂载目录
# mkdir -p /home/tjsp/nginx/html/file
[root@localhost tjsp]# cd nginx/ [root@localhost nginx]# pwd /home/tjsp/nginx [root@localhost nginx]# ls conf.d fastcgi_params koi-utf koi-win mime.types modules nginx.conf scgi_params uwsgi_params win-utf [root@localhost nginx]# [root@localhost nginx]# mkdir -p html/file [root@localhost nginx]# [root@localhost nginx]# ls conf.d fastcgi_params html koi-utf koi-win mime.types modules nginx.conf scgi_params uwsgi_params win-utf [root@localhost nginx]# [root@localhost nginx]# ls html/ file [root@localhost nginx]#
创建容器
# docker run -itd -p 8083:8082 -v /home/tjsp/nginx/html/file/:/home/upload/ --name tjsp1 cnbi-tjsp
# docker run -itd -p 8084:8082 -v /home/tjsp/nginx/html/file/:/home/upload/ --name tjsp2 cnbi-tjsp
解决高并发时添加
# docker run -itd -p 8087:8082 -v /home/tjsp/nginx/html/file/:/home/upload/ --name tjsp3 cnbi-tjsp
查看启动情况
注意：启动前需要运行redis，nginx配置，tomcat前端
# docker ps
[root@localhost nginx]# docker run -itd -p 8083:8082 -v /home/tjsp/nginx/html/file/:/home/upload/ --name tjsp1 cnbi-tjsp ea2df7c4d1051978e67f04ea0d79b7e14081a60bf9c0adfd24b2884bdcefa213 [root@localhost nginx]# [root@localhost nginx]# docker run -itd -p 8084:8082 -v /home/tjsp/nginx/html/file/:/home/upload/ --name tjsp2 cnbi-tjsp f0ed2dd79b9fc4797e84c8f8523660cd38b88fdd5b4ebd971b02eeaa18ce8d1e [root@localhost nginx]# [root@localhost nginx]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f0ed2dd79b9f cnbi-tjsp "java '-D java.secur…" 16 minutes ago Up 5 seconds 0.0.0.0:8084->8082/tcp tjsp2 ea2df7c4d105 cnbi-tjsp "java '-D java.secur…" 17 minutes ago Up 8 seconds 0.0.0.0:8083->8082/tcp tjsp1 7626b4b8f3c1 redis "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:6379->6379/tcp redis 39e65baa524a nginx:1.15.10 "nginx -g 'daemon of…" About an hour ago Up About a minute 0.0.0.0:7080->80/tcp nginx_tjsp 78b5482a0a95 tomcat:8.5.30 "catalina.sh run" 2 hours ago Up About a minute 0.0.0.0:8082->8080/tcp tomcat2 d41f7b69ebbf tomcat:8.5.30 "catalina.sh run" 2 hours ago Up About a minute 0.0.0.0:8081->8080/tcp tomcat1
7、构建etl镜像
创建一个构建目录
# mkdir -p /home/tjsp/docker-etl
编写Dockerfile文件
# cd /home/tjsp/docker-etl/
# vi Dockerfile
FROM docker.io/huanwei/alpine-oraclejdk8 ADD app.jar / ADD kettle5.4 /kettle5.4/ EXPOSE 8085 ENTRYPOINT ["java","-D java.security.egd=file:/dev./urandom","-jar","/app.jar"] VOLUME /tmp
# ls /home/tjsp/docker-etl/
[root@localhost docker-etl]# ls Dockerfile app.jar kettle5.4 kettle5.4.zip
构建镜像
# docker build -t tjsp-etl:latest .
查看构建的镜像
# docker images
[root@localhost ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE tjsp-etl latest 74c531083372 22 hours ago 970MB
8、创建etl镜像容器
创建容器
# docker run -itd --name tjsp-etl -p 8085:8085 tjsp-etl --network host
[root@localhost ~]# docker run -itd --name tjsp-etl -p 8085:8085 tjsp-etl --network host e14d155e53192bd1027ee2ed4c0b8b145e41a964d6de824b91c23df3997a6548 [root@localhost ~]# [root@localhost ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES e14d155e5319 tjsp-etl "java '-D java.secur…" 11 seconds ago Up 10 seconds 0.0.0.0:8085->8085/tcp tjsp-etl b4ded4008023 cnbi-tjsp "java '-D java.secur…" 21 hours ago Up 44 minutes 0.0.0.0:8087->8082/tcp tjsp3 504c7fd1eec6 cnbi-tjsp "java '-D java.secur…" 21 hours ago Up 44 minutes 0.0.0.0:8084->8082/tcp tjsp2 6b3c6ddee4c6 cnbi-tjsp "java '-D java.secur…" 21 hours ago Up 44 minutes 0.0.0.0:8083->8082/tcp tjsp1 9fe494bf43bc tomcat:8.5.30 "catalina.sh run" 25 hours ago Up About an hour 0.0.0.0:8086->8080/tcp tomcat3 7626b4b8f3c1 redis "docker-entrypoint.s…" 26 hours ago Up 26 hours 0.0.0.0:6379->6379/tcp redis 39e65baa524a nginx:1.15.10 "nginx -g 'daemon of…" 26 hours ago Up 25 hours 0.0.0.0:7080->80/tcp nginx_tjsp 78b5482a0a95 tomcat:8.5.30 "catalina.sh run" 27 hours ago Up 25 hours 0.0.0.0:8082->8080/tcp tomcat2 d41f7b69ebbf tomcat:8.5.30 "catalina.sh run" 27 hours ago Up 25 hours 0.0.0.0:8081->8080/tcp tomcat1
9、进入etl镜像容器设置kettle并启动
进入etl容器
# docker exec -it tjsp-etl /bin/sh
[root@localhost ~]# docker exec -it tjsp-etl /bin/sh / # / # ls app.jar bin dev etc home kettle5.4 lib lib64 linuxrc logs media mnt proc root run sbin srv sys tmp usr var / #
设置kettle
/ # cd /root/ ~ # ls -a . .. .ash_history .kettle .oracle_jre_usage ~ # ~ # cd .kettle/ ~/.kettle # ~/.kettle # ls kettle.properties ~/.kettle # # This file was generated by Pentaho Data Integration version 4.4.0. # # Here are a few examples of variables to set: # # PRODUCTION_SERVER = hercules # TEST_SERVER = zeus # DEVELOPMENT_SERVER = thor # # Note: lines like these with a # in front of it are comments # KETTLE_DEST_HOST=192.168.2.201 KETTLE_DEST_DATABASE=ORCL KETTLE_DEST_PORT=1521 KETTLE_DEST_USER=BRDB_TJSP_1203 KETTLE_DEST_PASSWORD=cnbi2018 KETTLE_EAS_HOST=192.168.2.206 KETTLE_EAS_DATABASE=orcl KETTLE_EAS_PORT=1521 KETTLE_EAS_USER=TJEAS KETTLE_EAS_PASSWORD=123456 ~/.kettle # pwd /root/.kettle ~/.kettle #
添加kettle流程
# docker cp eas_to_brdb/ tjsp-etl:/root/
[root@localhost tjsp]# ll total 16 drwxr-xr-x. 3 root root 77 Jul 23 16:00 docker-etl drwxr-xr-x. 2 root root 39 Jul 23 15:46 docker-server drwxr-xr-x. 4 root root 83 Jul 23 17:10 eas_to_brdb drwxr-xr-x. 2 root root 6 Jul 24 09:38 etl_jar drwxr-xr-x. 2 root root 12288 Jul 23 17:21 lib drwxr-xr-x. 4 root root 211 Jul 23 18:40 nginx drwxr-xr-x. 2 root root 21 Jul 24 11:45 tjsp_jar drwxr-sr-x. 11 root root 197 May 5 2018 tomcat drwxr-xr-x. 11 root root 197 Jul 23 09:11 tomcat1 drwxr-xr-x. 11 root root 197 Jul 23 09:20 tomcat2 drwxr-xr-x. 11 root root 197 Jul 23 11:46 tomcat3 [root@localhost tjsp]# [root@localhost tjsp]# docker cp eas_to_brdb/ tjsp-etl:/root/ [root@localhost tjsp]#
启动kettle
/ # chmod -R 777 kettle5.4/ / # / # chmod -R 777 /root/eas_to_brdb/ / # cd kettle5.4/ /kettle5.4 # /kettle5.4 # ls -l total 7632 -rwxrwxrwx 1 root root 551 Jun 14 2015 Carte.bat drwxrwxrwx 1 root root 35 Jul 23 06:09 Data Integration.app -rwxrwxrwx 1 root root 138 Jun 14 2015 Encr.bat -rwxrwxrwx 1 root root 131 Jun 14 2015 Import.bat -rwxrwxrwx 1 root root 136 Jun 14 2015 Kitchen.bat -rwxrwxrwx 1 root root 13366 Jun 14 2015 LICENSE.txt -rwxrwxrwx 1 root root 124 Jun 14 2015 Pan.bat -rwxrwxrwx 1 root root 7302319 Jun 14 2015 PentahoDataIntegration_OSS_Licenses.html -rwxrwxrwx 1 root root 537 Jun 14 2015 README_INFOBRIGHT.txt -rwxrwxrwx 1 root root 537 Jun 14 2015 README_LINUX.txt -rwxrwxrwx 1 root root 345 Jun 14 2015 README_OSX.txt -rwxrwxrwx 1 root root 341 Jun 14 2015 README_UNIX_AS400.txt -rwxrwxrwx 1 root root 3803 Jun 14 2015 Spoon.bat -rwxrwxrwx 1 root root 98 Jun 14 2015 SpoonConsole.bat -rwxrwxrwx 1 root root 1271 Jun 14 2015 SpoonDebug.bat -rwxrwxrwx 1 root root 405 Jun 14 2015 carte.sh drwxrwxrwx 1 root root 39 Jul 23 06:09 docs -rwxrwxrwx 1 root root 140 Jun 14 2015 encr.sh -rwxrwxrwx 1 root root 2354 Jun 14 2015 import-rules.xml -rwxrwxrwx 1 root root 275 Jun 14 2015 import.sh -rwxrwxrwx 1 755 root 308 Jun 14 2015 kitchen.sh drwxrwxrwx 1 root root 107 Jul 23 06:09 launcher drwxrwxrwx 2 root root 12288 Jul 23 09:21 lib drwxrwxrwx 1 root root 58 Jul 23 06:09 libswt -rwxrwxrwx 1 root root 274 Jun 14 2015 pan.sh drwxrwxrwx 1 root root 4096 Jul 23 06:09 plugins drwxrwxrwx 1 root root 176 Jul 23 06:09 pwd -rwxrwxrwx 1 root root 522 Jun 14 2015 runSamples.bat -rwxrwxrwx 1 root root 305 Jun 14 2015 runSamples.sh drwxrwxrwx 1 root root 51 Jul 23 06:09 samples -rwxrwxrwx 1 root root 4098 Jun 14 2015 set-pentaho-env.bat -rwxrwxrwx 1 root root 3710 Jun 14 2015 set-pentaho-env.sh drwxrwxrwx 1 root root 29 Jul 23 06:09 simple-jndi -rwxrwxrwx 1 root root 31 Jun 14 2015 spoon.command -rwxrwxrwx 1 root root 353118 Jun 14 2015 spoon.ico -rwxrwxrwx 1 root root 2460 Jun 14 2015 spoon.png -rwxrwxrwx 1 root root 5341 Jun 14 2015 spoon.sh drwxrwxrwx 1 root root 4096 Jul 23 06:09 ui -rwxrwxrwx 1 root root 1628 Jun 14 2015 yarn.sh /kettle5.4 # /kettle5.4 # ls -l /root/ total 0 drwxrwxrwx 4 root root 83 Jul 23 09:10 eas_to_brdb /kettle5.4 # /kettle5.4 # ./kitchen.sh -file /root/eas_to_brdb/eas_to_brdb.kjb Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0 2019/07/24 05:55:09 - Kitchen - Start of run. 2019/07/24 05:55:14 - eas_to_brdb - Start of job execution 2019/07/24 05:55:14 - eas_to_brdb - Starting entry [load_dimension_data] 2019/07/24 05:55:14 - load_dimension_data - Starting entry [set_period_variable] 2019/07/24 05:55:14 - set_period_variable - Loading transformation from XML file [file:///root/eas_to_brdb/load_dimension_data/set_period_variable.ktr] 2019/07/24 05:55:14 - set_period_variable - Dispatching started for transformation [set_period_variable] 2019/07/24 05:55:14 - 获取传递期间.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0) 2019/07/24 05:55:15 - 设置期间变量.0 - Setting environment variables... 2019/07/24 05:55:15 - 起止期间计算.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0) 2019/07/24 05:55:15 - 设置期间变量.0 - Set variable ORGPERIOD to value [201808] 2019/07/24 05:55:15 - 设置期间变量.0 - Set variable ENDPERIOD to value [201808] 2019/07/24 05:55:15 - 设置期间变量.0 - Set variable ORGTIME to value [2018-08-01 00:00:00] 2019/07/24 05:55:15 - 设置期间变量.0 - Set variable ENDTIME to value [2018-08-32 00:00:00] 2019/07/24 05:55:15 - 设置期间变量.0 - Finished after 1 rows. 2019/07/24 05:55:15 - 设置期间变量.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0) ***************************** ************************* ******************** *************** ********** ******
10、查看etl服务是否启动成功
监听端口
# lsof -i:8085
[root@localhost ~]# lsof -i:8085 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME docker-pr 11347 root 4u IPv6 20280487 0t0 TCP *:8085 (LISTEN)
查看容器启动后10行日志
# docker logs --tail 10 -tf tjsp-etl
[root@localhost ~]# docker logs --tail 10 -tf tjsp-etl 2019-07-24T04:30:02.092266671Z 2019-07-24 04:30:02.091 INFO 1 --- [isson-netty-1-2] o.r.c.pool.MasterPubSubConnectionPool : 1 connections initialized for 192.168.2.207/192.168.2.207:6379 2019-07-24T04:30:02.101305132Z 2019-07-24 04:30:02.100 DEBUG 1 --- [ main] org.redisson.connection.DNSMonitor : DNS monitoring enabled; Current masters: {redis://192.168.2.207:6379=192.168.2.207/192.168.2.207:6379}, slaves: {} 2019-07-24T04:30:02.727084940Z 2019-07-24 04:30:02.723 DEBUG 1 --- [ main] o.mybatis.spring.SqlSessionFactoryBean : Scanned package: 'com.cnbi.cloud.etl.vo' for aliases 2019-07-24T04:30:02.816255252Z 2019-07-24 04:30:02.811 INFO 1 --- [ main] c.c.cloud.etl.config.FullSQLInterceptor : mybatis intercept dialect:oracle 2019-07-24T04:30:02.816415640Z Logging initialized using 'class org.apache.ibatis.logging.stdout.StdOutImpl' adapter. 2019-07-24T04:30:02.820326603Z 2019-07-24 04:30:02.817 DEBUG 1 --- [ main] o.mybatis.spring.SqlSessionFactoryBean : Parsed configuration file: 'class path resource [mybatis/mybatis.cfg.xml]' 2019-07-24T04:30:02.936385155Z 2019-07-24 04:30:02.933 DEBUG 1 --- [ main] o.mybatis.spring.SqlSessionFactoryBean : Parsed mapper file: 'class path resource [mybatis/mapper/formulaMapper.xml]' 2019-07-24T04:30:02.948821805Z 2019-07-24 04:30:02.947 DEBUG 1 --- [ main] o.mybatis.spring.SqlSessionFactoryBean : Parsed mapper file: 'class path resource [mybatis/mapper/synchronousDataMapper.xml]' 2019-07-24T04:30:02.980705201Z 2019-07-24 04:30:02.979 DEBUG 1 --- [ main] o.mybatis.spring.SqlSessionFactoryBean : Parsed mapper file: 'class path resource [mybatis/mapper/TableCensusMapper.xml]' 2019-07-24T04:30:07.086914939Z 2019-07-24 04:30:07.086 INFO 1 --- [ main] com.cnbi.cloud.ETLApplication : Started ETLApplication in 12.023 seconds (JVM running for 13.086)
end