[Nginx+Tomcat+Session high-performance cluster construction]
With the development of the IT industry, linux servers are widely used in enterprises, and people have higher and higher requirements for application services on linux. The earlier apache server and apache had advantages and disadvantages, and apache gradually could not meet people's requirements.
At present, nginx is widely used as a high-performance web server. Whether it is a personal website or a large portal website, nginx is used as the first choice for building a web server.
Next, we will build our own high-performance web server with automatic failover cluster and load balancing server.
1. Construction of Nginx+Tomcat+Session high-performance cluster service
This article uses the current mainstream version to build, the build environment and version are as follows:
系统版本：Centos 5.3 64位系统
Nginx版本为：nginx-0.8.54.tar.gz
Jdk版本为：1.6.0_18_64
Tomcat版本为：Apache Tomcat/6.0.30
Other similar versions are also available! The packages required to install the service are as follows: Download from the integration part of this website:
Please download jdk from the following website: Please select the jdk of your own system version.
https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_Developer-Site/en_US/-/USD/ViewFilteredProducts-SingleVariationTypeFilter
[Tomcat download address]
http://blog.mgcrazy.com/download/apache-tomcat-6.0.30.tar.gz
[Nginx-0.8.54 download]
http://blog.mgcrazy.com/download/nginx-0.8.54.tar.gz
http://blog.mgcrazy.com/download/pcre-8.01.tar.gz
Second, first install Nginx
Before installing nginx, you need to install the pcre package and zlib to support rewriting, regularization and web page compression, etc.]
Download the required packages to /usr/src [the path can be changed according to your own habits]
(1) First install pcre:
cd /usr/src &&tar xzf pcre-8.01.tar.gz &&cd pcre-8.01 && ./configure --prefix=/usr/local/pcre &&make &&make install(2), and then install nginx: [Install the jvmroute module for nginx]
cd /usr/src &&svn checkout http://nginx-upstream-jvm-route.googlecode.com/svn/trunk/nginx-upstream-jvm-route-read-only && useradd www && tar xzf nginx-0.8.54.tar.gz &&cd nginx-0.8.54 && patch -p0 < ../nginx-upstream-jvm-route-read-only/jvm_route.patch && ./configure --prefix=/usr/local/nginx-0.8 --with-http_stub_status_module --with-openssl=/usr/ --with-pcre=/usr/src/pcre-8.01 --add-module=../nginx-upstream-jvm-route-read-only --user=www --group=www &&make &&make installnginx note that *
–with-pcre=/usr/src/pcre-8.01
points to the path where the source package is decompressed, not the installation path, otherwise
make[1]: *** [/usr/local/pcre/Makefile] Error 127an error
Nginx is installed! Let's not worry about the configuration of the nginx configuration file, let's modify the tomcat configuration file first:
3. Tomcat installation configuration:
(1) Download the required packages to the /usr/src directory;
First install jdk:
chmod o+x jdk* && ./jdk* ;The package will prompt to press the Enter key. We can install it according to the prompt. After decompression, the jdk1.6.0_18 folder will be generated in the current directory
mkdir -p /usr/java && mv jdk1.6.0_18 /usr/java/.
vi /etc/profile Add the following statement at the end:
export JAVA_HOME=/usr/java/jdk1.6.0_18
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH:$HOMR/bin
source /etc/profile //使环境变量马上生效
java –version //查看java版本，The display version is 1.6.0_18, and it is 64-bit to prove that the installation was successful!
Java HotSpot(TM) 64-Bit Server VM (build 16.0-b10, mixed mode)
cd /usr/src && tar xzf apache-tomcat-6.0.30.tar.gzDecompression is complete and execute:
mv apache-tomcat-6.0.30 /usr/local/tomcat_1 && cp /usr/local/tomcat_1 /usr/local/tomcat_2 -rMove to /usr/local and rename it to the tomcat_1 folder, and cp a tomcat_2
to deploy two Tomcat clusters for load balancing!
(2), set the java variable of tomcat
cd /usr/local/tomcat_1/bin/ && vi setenv.sh Create a setenv.sh script file and add the following statement
JAVA_HOME=/usr/java/jdk1.6.0_18
JAVA_JRE=/usr/java/jdk1.6.0_18/jreAnd copy the setenv.sh file to tomcat_2/bin/:
cp -p /usr/local/tomcat_1/bin/setenv.sh /usr/local/tomcat_2/bin Download it!
Save and exit
Check whether the .sh files in the bin directory under the two tomcats have executable permissions. If not, enter the corresponding directory and execute chmod o+x .sh!
Create different directories for the two tomcats as follows:
mkdir -p /usr/webapps/{www_1,www_2}(3), set the server.xml file of Tomcat:
The following is the specific content of my tomcat configuration file. I wanted to send it in the form of an attachment. It does not support the attachment function at present. The important modification is for your reference. If you paste all the configuration here, it shows that there is a problem. Two web links! In this way part of the explanation comes into play. I don't understand welcome message
This is the complete server.xml file download address:
http://blog.mgcrazy.com/download/server.xml.tgz
Configuration file download address of tomcat_2:
http://blog.mgcrazy.com/download/server2.xml.tgz
The ports in the two clusters configured by Tomcat are 4000 and 4001, which must be set to different ports; and two jvmroute names must be set separately, which will be used in nginx later!
Configure Tomcat session replication:
add the following lines to the web.xml files under the two tomcat confs:
After the configuration is complete, let's test the broadcast
java -cp tomcat-replication.jar MCaster 224.0.0.1 45564 Terminal1
java -cp tomcat-replication.jar MCaster 224.0.0.1 45564 Terminal2If no error is reported,
tomcat-replication.jar can be broadcast normally Download: http://cvs.apache.org/~fhanik/tomcat-replication.jar
If it is two machines, you can use tcpdump to capture packets to see!
After Tomcat is configured, start two tomcats, first start tomcat1, the startup log of tomcat_1 is as follows:
信息: Initializing Coyote HTTP/1.1 on http-8080
2011-3-18 19:56:21 org.apache.catalina.startup.Catalina load
信息: Initialization processed in 1088 ms
2011-3-18 19:56:21 org.apache.catalina.core.StandardService start
信息: Starting service Catalina
2011-3-18 19:56:21 org.apache.catalina.core.StandardEngine start
信息: Starting Servlet Engine: Apache Tomcat/6.0.18
2011-3-18 19:56:21 org.apache.catalina.ha.tcp.SimpleTcpCluster start
信息: Cluster is about to start
2011-3-18 19:56:21 org.apache.catalina.tribes.transport.ReceiverBase bind
信息: Receiver Server Socket bound to:/192.168.2.79:4000
2011-3-18 19:56:21 org.apache.catalina.tribes.membership.McastServiceImpl setupSocket
信息: Setting cluster mcast soTimeout to 500
2011-3-18 19:56:21 org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
信息: Sleeping for 1000 milliseconds to establish cluster membership, start level:4
2011-3-18 19:56:22 org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
信息: Done sleeping, membership established, start level:4
2011-3-18 19:56:22 org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
信息: Sleeping for 1000 milliseconds to establish cluster membership, start level:8
2011-3-18 19:56:23 org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
信息: Done sleeping, membership established, start level:8
2011-3-18 19:56:23 org.apache.catalina.ha.deploy.FarmWarDeployer start
严重: FarmWarDeployer can only work as host cluster subelement!
2011-3-18 19:56:23 org.apache.catalina.ha.session.DeltaManager start
信息: Register manager to cluster element Engine with name Catalina
2011-3-18 19:56:23 org.apache.catalina.ha.session.DeltaManager start
信息: Starting clustering manager at
2011-3-18 19:56:23 org.apache.catalina.ha.session.DeltaManager getAllClusterSessions
信息: Manager [localhost#]: skipping state transfer. No members active in cluster group.
2011-3-18 19:56:23 org.apache.catalina.ha.session.JvmRouteBinderValve start
信息: JvmRouteBinderValve started
2011-3-18 19:56:23 org.apache.coyote.http11.Http11Protocol start
信息: Starting Coyote HTTP/1.1 on http-8080
2011-3-18 19:56:24 org.apache.jk.common.ChannelSocket init
信息: JK: ajp13 listening on /0.0.0.0:8411
2011-3-18 19:56:24 org.apache.jk.server.JkMain start
信息: Jk running ID=0 time=0/42 config=null
2011-3-18 19:56:24 org.apache.catalina.startup.Catalina start
信息: Server startup in 2676 msThe log of Tomcat_2 is as follows:
000,{-64, -88, 2, 79},4000, alive=86901,id={78 42 25 -19 -102 3 64 10 -76 93 69 -43 45 76 95 -112 }, payload={}, command={}, domain={}, ]. This operation will timeout if no session state has been received within 60 seconds.
2011-3-18 19:57:48 org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor report
信息: ThroughputInterceptor Report[
Tx Msg:1 messages
Sent:0.00 MB (total)
Sent:0.00 MB (application)
Time:0.01 seconds
Tx Speed:0.04 MB/sec (total)
TxSpeed:0.04 MB/sec (application)
Error Msg:0
Rx Msg:1 messages
Rx Speed:0.00 MB/sec (since 1st msg)
Received:0.00 MB]
2011-3-18 19:57:48 org.apache.catalina.ha.session.DeltaManager waitForSendAllSessions
信息: Manager [localhost#]; session state send at 11-3-18 下午7:57 received in 125 ms.
2011-3-18 19:57:48 org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor report
信息: ThroughputInterceptor Report[
Tx Msg:2 messages
Sent:0.00 MB (total)
Sent:0.00 MB (application)
Time:0.05 seconds
Tx Speed:0.02 MB/sec (total)
TxSpeed:0.02 MB/sec (application)
Error Msg:0
Rx Msg:2 messages
Rx Speed:0.00 MB/sec (since 1st msg)
Received:0.00 MB]
2011-3-18 19:57:49 org.apache.catalina.ha.session.JvmRouteBinderValve start
信息: JvmRouteBinderValve started
2011-3-18 19:57:49 org.apache.coyote.http11.Http11Protocol start
信息: Starting Coyote HTTP/1.1 on http-8081
2011-3-18 19:57:49 org.apache.jk.common.ChannelSocket init
信息: JK: ajp13 listening on /0.0.0.0:8412
2011-3-18 19:57:49 org.apache.jk.server.JkMain start
信息: Jk running ID=0 time=0/34 config=null
2011-3-18 19:57:49 org.apache.catalina.startup.Catalina start
信息: Server startup in 2962 msIf there is an error or please check the reason!
skipping state transfer. No members active in cluster group.”mcastBindAddress not added or added incorrectly
The above error is reported. If one of the errors is reported, and mcastBindAddress has been modified, there is still an error. We can ignore it and start another tomcat to see.
The second tomcat does not have No members active in cluster group, which means that the two tomcats communicate normally.
Fourth, finally configure Nginx:
Post the content of the nginx.conf configuration file, the download address is as follows:
http://blog.mgcrazy.com/download/nginx.conf.tgz
Configuration files need to pay attention to:
http
{
upstream backend {
server 192.168.2.79:8080 srun_id=tomcat1;
#【tomcat1是我tomcat_1 server.xml里面配置的jvmroute=tomcat1】
server 192.168.2.79:8081 srun_id=tomcat2;
#【tomcat2是我tomcat_2 server.xml里面配置的jvmroute=tomcat2】
jvm_route $cookie_JSESSIONID|sessionid reverse;
#【配置jvmroute 、session会话】
}
}After the configuration is complete, create a new /usr/webapps/www_1/index.jsp test page
<%@ page language="java" import="java.util.*" pageEncoding="UTF-8"%>
http://blog.mgcrazy.com
<%out.print(request.getSession()) ;%>
<%out.println(request.getHeader("Cookie")); %>
tomcat_2的发布目录index.jsp内容为：
<%@ page language="java" import="java.util.*" pageEncoding="UTF-8"%>
http://wgkgood.gicp.net
<%out.print(request.getSession()) ;%>
<%out.println(request.getHeader("Cookie")); %>Just start nginx!
Final test! In the same browser window, ctrl+f5 refreshes, it is the same tomcat content, it will not jump to another tomcat
http://192.168.2.79:81
The result is as follows:
http://wgkgood.gicp.net
org.apache.catalina.session.StandardSessionFacade@2d7aece8
JSESSIONID=FEB19E24719ED666D1C9C50A7A09A2BA.tomcat2Another browser window is:
http://blog.mgcrazy.com
org.apache.catalina.session.StandardSessionFacade@259e215b
rtime=0; ltime=1298014951125; cnzz_eid=85744900-1298010049-; Hm_lvt_0e74098a87be059a433e5a74f1b5fecf=1298014951312; Hm_lvt_b7c035fd79b2c378053e06c86493de76=1300439158373; JSESSIONID=B2574ABB491F50F37162479BD3FE761C.tomcat1No matter how you refresh the same window, it will not change!
And when you down a tomcat, it will automatically transfer the session to another tomcat!
Users can't feel the service hangs up!
When the two services are normal, load balancing can be performed. When a service is down, it can be automatically eliminated and its session can be replicated!