nginx performs load balancing to solve the problem of exposing one interface to the outside of the multi-machine multi-gpu card service

Idea: Multiple gpu service interfaces-->ngxin for load balancing -->exposing one to the outside.

Take one machine with two cards as an example, where gunicorn deploys one card multi-process service refer to this article

1. Make nginx load balancing mirror

1. Make Dockerfie

FROM nginx:1.13.3
COPY ./ /
RUN mkdir /app
COPY /nginx.conf /etc/nginx/nginx.conf

2.nginx.conf detailed

#user  nobody;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

	#bx----------------------
	upstream algoserver{
	    server 192.168.102.200:10009;
	}
	
    server {
        listen       8082;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            #root   html;
            #index  index.html index.htm;
			#bx--------------------------------
			proxy_pass http://algoserver;
			proxy_set_header Host $host;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

        # proxy the PHP scripts to Apache listening on 127.0.0.1:80
        #
        #location ~ \.php$ {
        #    proxy_pass   http://127.0.0.1;
        #}

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        #location ~ \.php$ {
        #    root           html;
        #    fastcgi_pass   127.0.0.1:9000;
        #    fastcgi_index  index.php;
        #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
        #    include        fastcgi_params;
        #}

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #    deny  all;
        #}
    }


    # another virtual host using mix of IP-, name-, and port-based configuration
    #
    #server {
    #    listen       8000;
    #    listen       somename:8080;
    #    server_name  somename  alias  another.alias;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}


    # HTTPS server
    #
    #server {
    #    listen       443 ssl;
    #    server_name  localhost;

    #    ssl_certificate      cert.pem;
    #    ssl_certificate_key  cert.key;

    #    ssl_session_cache    shared:SSL:1m;
    #    ssl_session_timeout  5m;

    #    ssl_ciphers  HIGH:!aNULL:!MD5;
    #    ssl_prefer_server_ciphers  on;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}

}

Where server 192.168.102.200:10009;
        server 192.168.102.200:10010;

These are the two services started by the gpu, which are now mapped to 192.168.102.200:8082.

3.build mirror

docker build -t nginx/express:0.1 .

2. Start the container for load balancing

The above port 8082 is mapped to 10016, and users can call 10009 and 10010 gpu services through 10016.

docker run -it -p 10016:8082 -v /home/fanzonghao/red_detection/software/nginx.conf:/etc/nginx/nginx.conf nginx/express:0.1