I deployed a shinyapps using the stack: R, docker, and shinyproxy and so far the tools have been great with users happy. However am struggling a bit with load balancing so that I can scale and stop bothering users when I run updates on the solution.
With advise from https://www.shinyproxy.io/shinyproxy-containers/, I went through the process but using https://github.com/openanalytics/shinyproxy-config-examples/tree/master/02-containerized-docker-engine the containerized docker engine. My solution runs perfectly when I run java -jar shinyproxy-2.0.3.jar not containerised. But now when I containerize it, it gets to be very very slow, infact even with internal networking active the containers crush when I try open them.
At the moment I am just trying to test if containerizing shinyproxy can work without giving me issues but to a fail.
My Nginx configuration on the server
server {
root /var/www/html;
# Add index.php to the list if you are using PHP
index index.html index.htm index.nginx-debian.html;
server_name qnumsolutions.com www.qnumsolutions.com;
location / {
proxy_pass http://127.0.0.1:8080/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 600s;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Protocol $scheme;
}
listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/qnumsolutions.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/qnumsolutions.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
Shinyproxy application.yml
proxy:
title: The Operational Intelligence Solution
logo-url: http://qnum.co.za/wp-content/uploads/2018/03/OI-Solution-Icon.png
landing-page: /
heartbeat-rate: 100000
heartbeat-timeout: 600000
port: 8080
authentication: simple
container-log-path: ./container-logs
admin-groups: admin
usage-stats-url: http://159.65.95.235:8086/write?db=shinyproxy_usagestats
# Example: 'simple' authentication configuration
users:
- name: [email protected]
password: prupr1.
groups: admin
docker:
url: http://localhost:2375
port-range-start: 20000
specs:
- id: OISolution1
description: Application is designed to help organisations use advanced analytics to manage product visibility and variance
container-cmd: ["R", "-e shiny::runApp('/root/hostedoi')"]
container-image: oisolution
container-volumes: ["/home/shiny/database:/mnt/persistent1"]
access-groups: admin
- id: OISolution2
description: Application is designed to help organisations use advanced analytics to manage product visibility and variance
container-cmd: ["R", "-e shiny::runApp('/root/hostedoi1')"]
container-image: oisolution1
container-volumes: ["/home/shiny/database:/mnt/persistent1"]
access-groups: admin
logging:
file:
shinyproxy.log
Dockerfile
FROM openjdk:8-jre
RUN mkdir -p /opt/shinyproxy/
RUN wget https://www.shinyproxy.io/downloads/shinyproxy-2.0.3.jar -O /opt/shinyproxy/shinyproxy.jar
COPY application.yml /opt/shinyproxy/application.yml
WORKDIR /opt/shinyproxy/
CMD ["java", "-jar", "/opt/shinyproxy/shinyproxy.jar"]
And lastly sudo docker ps yields
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
bc396c8bd26a ois-shinyproxy "java -jar /opt/shin…" About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp clever_beaver
Additional links I consulted: https://fly.io/articles/load-balancing-a-fleet-of-docker-containers-using-fly/
Your assistance would be greatly appreciated.
It seems like there is a problem with your usage-stats-url. Maybe a firewall or something prevents ShinyProxy to reach this destination? I could reproduce this slow behavior, but it disappeared when I commented this line.
Besides that, there are some mistakes in your application.yml regarding Docker. I altered it a bit and managed to run it on my local machine with this configuration (nginx.conf and the Dockerfile were just slightly altered to fit my needs for docker-compose):
proxy:
title: The Operational Intelligence Solution
logo-url: http://qnum.co.za/wp-content/uploads/2018/03/OI-Solution-Icon.png
landing-page: /
heartbeat-rate: 100000
heartbeat-timeout: 600000
port: 8080
authentication: simple
container-log-path: ./container-logs
admin-groups: admin
#usage-stats-url: http://159.65.95.235:8086/write?db=shinyproxy_usagestats
# Example: 'simple' authentication configuration
users:
- name: [email protected]
password: prupre35A21.
groups: admin
docker:
internal-networking: true
specs:
- id: 01_hello
display-name: Hello Application
description: Application which demonstrates the basics of a Shiny app
container-cmd: ["R", "-e", "shinyproxy::run_01_hello()"]
container-image: openanalytics/shinyproxy-demo
container-volumes: ["/tmp:/mnt/persistent1"]
container-network: shinyproxy_slow_default
access-groups: admin
- id: 06_tabsets
container-cmd: ["R", "-e", "shinyproxy::run_06_tabsets()"]
container-image: openanalytics/shinyproxy-demo
container-volumes: ["/tmp:/mnt/persistent1"]
container-network: shinyproxy_slow_default
access-groups: admin
logging:
file:
shinyproxy.log
I ran everything with docker-compose, see:
version: "3.1"
services:
web:
image: nginx:alpine
ports:
- 80:80
- 443:443
volumes:
- ./nginx.conf:/etc/nginx/conf.d/default.conf
shinyproxy:
image: sp_test
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./application.yml:/opt/shinyproxy/application.yml