Nginx + Tomcat for load balancing

Nginx + Tomcat for load balancing

1. Environment

Linux version:  5.4.0-96-generic
OS Version:     ubuntu1~20.04
Architecture:   amd64
Nginx version:  nginx/1.18.0 (Ubuntu)
JVM Version:    11.0.15+10-Ubuntu-0ubuntu0.20.04.1
Tomcat Version: Apache Tomcat/9.0.63

2. Install

2.1 Install Nginx

Enter at the command line:

sudo apt-get install nginx

Test command:

sudo nginx -t

The window shows:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Enter the server ip in the browser:

2.2 Install Java

Install jdk:

sudo apt-get -y install openjdk-11-jdk

View version:

java --version

The output is as follows:

openjdk 11.0.15 2022-04-19
OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.20.04.1)
OpenJDK 64-Bit Server VM (build 11.0.15+10-Ubuntu-0ubuntu0.20.04.1, mixed mode, sharing)

Successful installation.

Check where Java is installed:

update-alternatives --config java

The output is as follows:

There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/java-11-openjdk-amd64/bin/java

Add $JAVA_HOME to environment variables:

sudo vim $HOME/.bashrc
shift+G

Add at the end of the line:

export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
export PATH=$PATH:$JAVA_HOME

Exit vim, activate the environment variable:

source $HOME/.bashrc

Verify environment variables:

echo $JAVA_HOME

The output is as follows, the activation is successful.

/usr/lib/jvm/java-11-openjdk-amd64

2.3 Install Tomcat

Downloaded from the official website Tomcat 9.0.63, pay attention to Core/tar.gz(pgp, sha512)the version, or click here to download , upload the file to the path of the server /usr/local, and enter the following command to decompress:

cd /usr/local && tar -zxvf apache-tomcat-9.0.63.tar.gz

Configure environment variables and open Tomcatthe startup script:

cd apache-tomcat-9.0.63 && vim bin/startup.sh
shift+G

exec "$PRGDIR"/"$EXECUTABLE" start "$@"Insert the following statement one line above the statement :

JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
JRE_HOME=$JAVA_HOME/jre
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME
CLASSPATH=.:$JRE_HOME/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
TOMCAT_HOME=/usr/local/apache-tomcat-9.0.63

Save and exit.

Open server 8080port:

firewall-cmd --zone=public --add-port=8080/tcp --permanent

restart firewall

systemctl restart firewalld.service

Go to the cloud server management console to add a security group:

run:

./bin/shutdown.sh && rm -rf logs/catalina.out && ./bin/startup.sh ; tail -f logs/catalina.out

The output should look like this:

When accessing in a browser $ip:8080, it displays:

It means that your configuration is successful.

3. Deployment

3.1 Tomcat server deployment

This experiment needs to use 3 ports, respectively 8001, 8082, 8083, after opening the port in the Tencent Cloud console, shellmanually open the port in and restart the firewall:

firewall-cmd --zone=public --add-port=8081/tcp --permanent
firewall-cmd --zone=public --add-port=8082/tcp --permanent
firewall-cmd --zone=public --add-port=8083/tcp --permanent
systemctl restart firewalld.service

Pay attention to go to Tencent Cloud to open your security group

We use the Tomcatbuilt-in projects to implement different ports to access different applications. Assuming that you have completely followed the above method, then your action at this time is to open the configuration file of the server according to the root path:3WebTomcat/usr/local/apache-tomcat-9.0.63

cd /usr/local/apache-tomcat-9.0.63/conf && vim server.xml

Delete the original code and replace it with the following code:

<?xml version="1.0" encoding="UTF-8"?>
<Server port="8005" shutdown="SHUTDOWN">
     <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
     <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
     <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
     <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
     <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />
     <GlobalNamingResources>
     <Resource name="UserDatabase" auth="Container"
               type="org.apache.catalina.UserDatabase"
               description="User database that can be updated and saved"
               factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
               pathname="conf/tomcat-users.xml" />
     </GlobalNamingResources>

     <Service name="Catalina">
          <Connector port="8081" protocol="HTTP/1.1"
                    connectionTimeout="20000"
                    redirectPort="8443" />
               <Engine name="Catalina" defaultHost="localhost">
                    <Realm className="org.apache.catalina.realm.LockOutRealm">
                    <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
                         resourceName="UserDatabase"/>
                    </Realm>
                    <Host name="localhost"  appBase="webapps"
                         unpackWARs="true" autoDeploy="true">
                         <Context path="" docBase="ROOT" reloadable="true" />
                         <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
                              prefix="localhost_access_log" suffix=".txt"
                              pattern="%h %l %u %t &quot;%r&quot; %s %b" />
                    </Host>
               </Engine>
     </Service>

     <Service name="Catalina2">
          <Connector port="8082" protocol="HTTP/1.1"
                    connectionTimeout="20000"
                    redirectPort="8443" />
               <Engine name="Catalina2" defaultHost="localhost">
                    <Realm className="org.apache.catalina.realm.LockOutRealm">
                    <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
                         resourceName="UserDatabase"/>
                    </Realm>
                    <Host name="localhost"  appBase="webapps"
                         unpackWARs="true" autoDeploy="true">
                         <Context path="" docBase="examples" reloadable="true" />
                         <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
                              prefix="localhost_access_log" suffix=".txt"
                              pattern="%h %l %u %t &quot;%r&quot; %s %b" />
                    </Host>
               </Engine>
     </Service>

     <Service name="Catalina3">
          <Connector port="8083" protocol="HTTP/1.1"
                    connectionTimeout="20000"
                    redirectPort="8443" />
               <Engine name="Catalina3" defaultHost="localhost">
                    <Realm className="org.apache.catalina.realm.LockOutRealm">
                    <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
                         resourceName="UserDatabase"/>
                    </Realm>
                    <Host name="localhost"  appBase="webapps"
                         unpackWARs="true" autoDeploy="true">
                         <Context path="" docBase="examples/websocket" reloadable="true" />
                         <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
                              prefix="localhost_access_log" suffix=".txt"
                              pattern="%h %l %u %t &quot;%r&quot; %s %b" />
                    </Host>
               </Engine>
     </Service>
</Server>

In this way, different servers can be obtained by accessing $ip:8081, $ip:8082, and respectively.$ip:8083

The resources of port 8083 are incomplete, because Tomcat only provides 2 valid apps by default, and the third port will return a 404 page, which is also an application to some extent.

3.2 Nginx reverse proxy

Edit nginx.confconfiguration file:

vim /etc/nginx/nginx.conf

Insert the following statement at httpthe end of the module:

upstream tomcat_server {
    server 101.42.117.143:8081;
    server 101.42.117.143:8082;
    server 101.42.117.143:8083;
}

server {
    listen 80;
    server_name 101.42.117.143;

    location / {
        proxy_pass http://tomcat_server;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Start or restart nginx:

nginx 或 nginx -s reload

In this way, the port can be accessed directly 80, or $ipit can be proxied to the 3 servers on the back end.

nginx uses round robin by default.

4. Load balancing

4.1 Polling

The polling strategy nginx.confis configured as follows:

upstream tomcat_server {
    server 101.42.117.143:8081 weight=1;
    server 101.42.117.143:8082 weight=1;
    server 101.42.117.143:8083 weight=1;
}

server {
    listen 80;
    server_name 101.42.117.143;

    location / {
        proxy_pass http://tomcat_server;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

restart nginx:

nginx -s reload

With postmanfrequent visits, the frequency with which the page appears is ABC ABC ABC ABC.

4.2 Weighted Round Robin

The weighted round robin strategy nginx.confis configured as follows:

upstream tomcat_server {
    server 101.42.117.143:8081 weight=1;
    server 101.42.117.143:8082 weight=3;
    server 101.42.117.143:8083 weight=5;
}

server {
    listen 80;
    server_name 101.42.117.143;

    location / {
        proxy_pass http://tomcat_server;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

restart nginx:

nginx -s reload

With postmanfrequent visits, the frequency with which the page appears is CBCACBCBC CBCACBCBC.

This algorithm is called the proposer rotation selection algorithm. In each round, the selected object will be subtracted from the total weight value. In each round, the weight value of each node will be added to a fixed preset weight value. This is a fair The rotation algorithm can prevent nodes with excessive weight from continuously occupying resources. The process is as follows:

initial weight	weighted	choose	The chosen one minus the weight and
0 0 0	1 3 5	C	1 3 -4
1 3 -4	2 6 1	B	2 -3 1
2 -3 1	3 0 6	C	3 0 -3
3 0 -3	4 3 2	A	-5 3 2
-5 3 2	-4 6 7	C	-4 6 -2
-4 6 -2	-3 9 3	B	-3 0 3
-3 0 3	-2 3 8	C	-2 3 -1
-2 3 -1	-1 6 4	B	-1 -3 4
-1 -3 4	0 0 9	C	0 0 0
0 0 0	1 3 5	C	1 3 -4
1 3 -4	2 6 1	B	2 -3 1
2 -3 1	3 0 6	C	3 0 -3
…	…	…	…

4.2 IP Hash

IP HashThe policy nginx.confconfiguration is as follows:

upstream tomcat_server {
    server 101.42.117.143:8081;
    server 101.42.117.143:8082;
    server 101.42.117.143:8083;
    ip_hash;
}

server {
    listen 80;
    server_name 101.42.117.143;

    location / {
        proxy_pass http://tomcat_server;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

restart nginx:

nginx -s reload

It will $ipmap the host's hash to a random fixed value, and then take 3the modulo to get the corresponding port number;

Using postmanthe frequent access of host 1, the frequency of page appearance is AAAAAA;

Using postmanthe frequent visits of host 2, the frequency of page appearance is BBBBBB.