Zookeeper plus solr-cloud to implement Solr cluster

True distributed zookeeper environment, non-pseudo environment cluster, additional solr cluster environment, the addition of ik word segmentation

Environment and download package preparation

Download package

1. Jdk8
2. Tomcat8
3. Solr7
4. Zookeeper3.4.10

environment

zookeeperIt is best to have more than 3 servers with odd-numbered nodes. I used two servers here. In fact, this is problematic. I will talk about it later, and then the nodes are 4 tomcats. First build zookeeperand then deploy solr-cloud.

Zookeeper cluster construction

Features of Zookeeper:
1. Cluster management: fault tolerance, load balancing.
2. Centralized management of configuration files
3. Cluster entry

All files used for building later are placed under the solr-clouddirectory

Deploy zookeeper nodes on two servers respectively

Unzip the entire Zookeeper3.4.10package , copy it to the 06 server solr-clouddirectory and name it zookeeper01, and under the 07 server solr-clouddirectory , name it aszookeeper02

Create data/myidfile storageid

06 on server

07 on server

Modify the configuration file

The modification solr-cloud/zookeeper01/conf/zoo_sample.cfgis solr-cloud/zookeeper01/conf/zoo.cfg, the modification content is as follows:
zk01Node:

zk02Node:

2181 is the port for the zk client to connect to the cluster. Custom ports such as 2881 and 3881 are the ports for the direct communication of the zk cluster, which can be defined by themselves, and each zknode can be different.

start zookeepernode

It is too troublesome to execute the cluster command every time, so configure alias, Alias ​​and the use of parameters .
For example, if the manual startup zookeeperneeds to be executed /opt/app/ctpsp/solr-cloud/zookeeper01/bin/zkServer.sh start, the zookeeperStartAllcommand , and the subsequent restart tomcatand query status commands are all commands aliasafter .

alias tomcatshutall=shutAllTomcatexec

startZookeerperexec() {
/opt/app/ctpsp/solr-cloud/zookeeper01/bin/zkServer.sh  start
}
alias zookeeperStartAll=startZookeerperexec


statusZookeerperexec() {
/opt/app/ctpsp/solr-cloud/zookeeper01/bin/zkServer.sh  status
}
alias zookeeperStatusAll=statusZookeerperexec

zookeeperStartAllStart zk on 06 and 07 zk nodes respectively , zookeeperStatusAllquery the zk status, the display is as follows:
On 06:

07:

You can see here one 07zk节点is leader, one 06zk节点is follower.

Distributed Cluster Split Brain Problem

Here is a bit of extended discussion that can be skipped and goes directly to solr-cloudthe deployment without affecting the configuration.

Here we try to stop the 07 leader zknode and see if another 06 will becomeleader节点

What is the status of 06?

It can be seen that 06 stops directly, which is the distributed cluster split-brain problem. If the zk node is greater than 3, another leaderone . Therefore, the two servers cannot do zookeeperthe clustering function, but the configuration is the same. This article is still based on two deployments.

Solr-clouddeploy

Create instances in 06and respectively07tomcat01/2tomcat/3/4

Modify the tomcatrunning port

Configuration file location:
/opt/app/ctpsp/solr-cloud/tomcat01/conf/server.xml

1 changed to 8105,8205,8305,8405

2 changed to 8180,8280,8380,8480

3 changed to8189,8289,8389,8489

Add the solr project in tomcat

Refer to the solrproject solr on Linux, and copy it solr-cloudto tomcat.

Add solrhomeproject

Copy the solrhomeproject solr-cloudto

The current project structure is like this:
06 on:

07 on:

Check it out, the Long March Road is coming to an end.

Modify the solrhomeconfiguration

Configure two servers 4 solrhomeunder the solr.xmlcorresponding server ip and corresponding port

Modify solrconfiguration associationssolrhome

The same is to configure two servers in 4 places, look at the picture to understand

Zookeeper uniformly manages and configures the solrhome file.

# 进入solr源码包 
/usr/local/src/solr-6.6.0/server/scripts/cloud-scripts

#执行
./zkcli.sh -zkhost 10.237.67.6:2181,10.237.67.7:2181 -cmd upconfig -confdir /opt/app/ctpsp/solr-cloud/solrhome01/configsets/sample_techproducts_configs/conf -confname myconf

zkhostis the cluster ip; confdiris solrhomethe confdirectory; confnameis the name of the uploaded conffile. This command only needs to be executed on any zk node to realize the unified management of 4 solrhomeconfigurations .

Check zk的confif the upload is successful, enter this directory, execute zookeepeerthe script under to enter zookeeper, use quitto exit

[prouser@b7515169318-1 conf]$ cd ../../zookeeper01/bin/
[prouser@b7515169318-1 bin]$ ls
README.txt  zkCleanup.sh  zkCli.cmd  zkCli.sh  zkEnv.cmd  zkEnv.sh  zkServer.cmd  zkServer.sh  zookeeper.out
[prouser@b7515169318-1 bin]$ ./zkCli.sh

After entering, check if there is a myconffile , next to it myconfIKis the conf where I uploaded the ik word segmentation later. If the conf configuration is modified here, you only need to perform this step again and confnameset it to the same as before to update.

associated solrandzookeeper

Modify the configuration file

The main zookeeperthing cluster ip configuration of , which can be searched umaskto locate. 4 tomcatmodifications are the same.

start tomcat

Started on two servers respectively 01-04 tomcat, the same definition is aliasconvenient for each use

startAllTomcatexec() {
bash /opt/app/ctpsp/solr-cloud/tomcat01/bin/startup.sh
bash /opt/app/ctpsp/solr-cloud/tomcat02/bin/startup.sh
}
alias tomcatstartall=startAllTomcatexec


shutAllTomcatexec() {
bash /opt/app/ctpsp/solr-cloud/tomcat01/bin/shutdown.sh
bash /opt/app/ctpsp/solr-cloud/tomcat02/bin/shutdown.sh
}
alias tomcatshutall=shutAllTomcatexec

Access the cluster

It can be seen that compared to the stand-alone version, the cluster has more red boxes.

create a collectiontry

View the cluster cloud map

At this point, the deployment of zookeeper + solrcloud is completed. Because the space is too long, the deployment of cluster ik word segmentation is in the next article.