Solr 5.0 cluster construction

One, build a zookeeper cluster
1. Download zookeeper and decompress 3 copies, rename them to zookeeper-3.4.10-42181, zookeeper-3.4.10-42182, zookeeper-3.4.10-42183 (4218* indicates the connection port)
2. Enter D:\mysoftsetup\solr-colud\zookeeper\zookeeper-3.4.10-42181\conf directory modification
(zookeeper-3.4.10-42182, zookeeper-3.4.10-42183 operations are similar)
zoo_sample.cfg is modified to: zoo.cfg
Modify as follows: #Save
data path
dataDir=D:/mysoftsetup/solr-colud/zookeeper/zookeeper-3.4.10-42181/data
### log file Save log file
dataLogDir=D:/mysoftsetup/solr-colud/zookeeper/ zookeeper-3.4.10-42181/log
# the port at which the clients will connect port number (note: if it is zookeeper-3.4.10-42182, then clientPort=42182)
clientPort=42181 #cluster

communication interface configuration (1 is serverid , corresponding to the content of the myid file, followed by ip, zk communication port, election port respectively)
server.1=127.0.0.1:4888:5888
server.2=127.0.0.1:4889:5889
server.3=127.0.0.1:4890:5890

3.在D:\mysoftsetup\solr-colud\zookeeper\zookeeper-3.4.10-42181\data目录建立myid文件，写入1。（zookeeper-3.4.10-42182，zookeeper-3.4.10-42183操作类似）（注意：如果为zookeeper-3.4.10-42182，myid文件写入2）

4.zookeeper-3.4.10-42181/conf/zoo.cfg 配置如下：
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
#dataDir=/tmp/zookeeper

dataDir=D:/mysoftsetup/solr-colud/zookeeper/zookeeper-3.4.10-42181/data
### log file 保存日志文件
dataLogDir=D:/mysoftsetup/solr-colud/zookeeper/zookeeper-3.4.10-42181/log
# the port at which the clients will connect
clientPort=42181
# the maximum number of client connections.
# increase this if you need to handle more clients
maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
autopurge.purgeInterval=1

server.1=127.0.0.1:4888:5888
server.2=127.0.0.1:4889:5889
server.3=127.0.0.1:4890:5890

zookeeper-3.4.10-42182/conf/zoo.cfg 配置如下：

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
#dataDir=/tmp/zookeeper

dataDir=D:/mysoftsetup/solr-colud/zookeeper/zookeeper-3.4.10-42182/data
### log file 保存日志文件
dataLogDir=D:/mysoftsetup/solr-colud/zookeeper/zookeeper-3.4.10-42182/log
# the port at which the clients will connect
clientPort=42182
# the maximum number of client connections.
# increase this if you need to handle more clients
maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
autopurge.purgeInterval=1

server.1=127.0.0.1:4888:5888
server.2=127.0.0.1:4889:5889
server.3=127.0.0.1:4890:5890

zookeeper-3.4.10-42183/conf/zoo.cfg 配置如下：
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
#dataDir=/tmp/zookeeper

dataDir=D:/mysoftsetup/solr-colud/zookeeper/zookeeper-3.4.10-42183/data
### log file 保存日志文件
dataLogDir=D:/mysoftsetup/solr-colud/zookeeper/zookeeper-3.4.10-42183/log
# the port at which the clients will connect
clientPort=42183
# the maximum number of client connections.
# increase this if you need to handle more clients
maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
autopurge.snapRetainCount=3
Purge task interval in hours
# Set to "0" to disable auto purge feature
autopurge.purgeInterval=1

server.1=127.0.0.1:4888:5888
server.2=127.0.0.1:4889:5889
server. 3=127.0.0.1:4890:5890

5. Start in sequence without any error.

The second solrCloud cluster is built.
2.1 Unzip the downloaded solr-5.0.0.zip.
Create D:\mysoftsetup\solr-colud\solr_home1 directory, copy all files under solr-5.0.0\server\solr to D:\mysoftsetup\solr-colud\solr_home1, and modify solr.xml.
You need to change the port in solr.xml to the container service port, such as 28983 of tomcat

<int name="hostPort">${jetty.port:28983}</int>
You can also configure a fixed ip by modifying the host
    <str name="host">127.0.0.1</str>


put solr-5.0.0\server\webapps\solr.
Copy all the jars under solr-5.0.0\server\lib\ext to the WEB-INF\lib directory under the solr folder just obtained.
Copy solr-5.0.0\server\resources\log4j.properties to the webapps\solr\WEB-INF\classes directory.
Modify webapps\solr\WEB-INF\web.xml:
   <env-entry>
       <env-entry-name>solr/home</env-entry-name>
       <env-entry-value>D:\mysoftsetup\solr -colud\solr_home1</env-entry-value>
       <env-entry-type>java.lang.String</env-entry-type>
    </env-entry>
specify solr/home as D:\mysoftsetup\solr- colud\solr_home1 directory.
Modify D:\mysoftsetup\solr-colud\solr_home1\solr.xml, and modify hostPort to the port number of the tomcat.
Copy the solr-5.0.0\server\solr\configsets\basic_configs\conf directory to D:\mysoftsetup\solr-colud\solr_home1 in


2.2. Configure tomcat

and modify tomcat's server.xml,
Modify the catalina.bat file of tomcat and add this line under

setlocal : set "JAVA_OPTS=%JAVA_OPTS% -DzkHost=127.0.0.1:28983,127.0.0.1:28986,127.0.0.1:28990" and make two copies of tomcat , pay attention to modify the port number. My three tomcat paths are: E:\Solr_Workspace\apache-tomcat-7.0.62_28983 E:\Solr_Workspace\apache-tomcat-7.0.62_28986 E:\Solr_Workspace\apache-tomcat-7.0.62_28990 Make two copies of solr_home1 :solr_home2,solr_home3 Then reconfigure the solr folder under webapps under tomcat2 and 3 according to step 3, and modify the differences between 1, 2, and 3 so that they correspond to tomcat1, 2, and 3 respectively. (Note: the collection1 directories under solr_home2 and solr_home3 are deleted, not required). The set JAVA_OPTS of catalina.bat of tomcat2 and 3 are respectively: set “JAVA_OPTS=%JAVA_OPTS% -DzkHost=127.0.0.1:28983,127.0.0.1:28986,127.0.0.1:28990” Note: catalina can also not be modified here. bat,














And directly add under the solr>solrcloud node of the solr.xml file in the directory configured by the solr/home variable:
<str name="zkHost">1127.0.0.1:28983,127.0.0.1:28986,127.0.0.1:28990< /str>

2.3 Log in to solr to build the core

http://localhost:28983/solr/

2.3.1 Use the command to upload the core configuration file (Note: You can upload multiple configuration files, which core corresponds to which configuration file needs to be created when Use the collection.configName parameter to specify)
upload configuration file command (windows version, jar contains: D:\mysoftsetup\solr-colud\solr-5.0.0\server\solr-webapp\webapp\WEB-INF\lib )
java -classpath .;D:/mysoftsetup/solr-colud/solrcould/solr-war-lib/* ;D:\mysoftsetup\solr-colud\solr-5.0.0\server\lib\ext org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 127.0.0.1:42181,127.0.0.1:42182,127.0.0.1:42183 -confdir D:/mysoftsetup/solr-colud/solrcould/solr-config -confname shop_goods
or
go to D:\mysoftsetup\solr- 5.0.0\server\scripts\cloud-scripts directory
Execute:
zkcli.bat -zkhost 127.0.0.1:42181,127.0.0.1:42182,127.0.0.1:42183 -cmd upconfig -confdir D:/mysoftsetup/solr-colud/solrcould/solr-config -confname shop_goods
Execute:



2.3. 2 Use the command to create the core, and specify which configuration file name to use
1. Create the core name as collection3; use the configuration file name as: shop_goods; the number of logical fragments: 3; the number of replicas of the fragment: 1
http:/ 2.
Create



a core name as shop_goods2; the name of the configuration file used is: shop_goods; the number of logical fragments is: 2; the number of replicas of the fragment is: 2

http://127.0.0.1:28983/solr/admin/collections?action=CREATE&name=shop_goods2&numShards=2&replicationFactor=2&maxShardsPerNode=2&collection.configName=shop_goods&createNodeSet=127.0.0.1:28983_solr,127.0.0.1:28986_solr,127.0_solr 
at 28990.0.1: http://localhost:28983/solr/ page click cloud on the left to display



3. If you want to delete the collection3 interface,
http://localhost:8983/solr/admin/collections?action=DELETE&name=collection3

4. Reload the collection3 interface
http://localhost:8983/solr/admin/collections?action=RELOAD&name=collection3


Note: Solr failed to pass the api call, there may be no associated configuration file or the associated configuration file is incorrect.





 Manage the cluster through the cluster api (Core Admin)
1). Create an interface (the first automatic allocation)
Java code  
1. http://127.0.0.1:28983/solr/admin/collections?action=CREATE&name=collection1&numShards=3&replicationFactor=1&maxShardsPerNode=1&collection.configName=myconf    
2.http://127.0.0.1:28983/solr/admin/collections ?action=CREATE&name=collection1&numShards=3&replicationFactor=1&maxShardsPerNode=1&collection.configName=myconf&createNodeSet=127.0.0.1:28986_solr,127.0.0.1:28983_solr,127.0.0.1:28990_solr  
   This will come out a collection, it has 3 shards, each shard has 1 data node, 1 backup node, that is, the collection has 6 core
 
 parameters:
name: the name of the collection to be created
numShards: the number of logical shards to be created when the collection is created
replicationFactor: the number of replicas of the shard. A replicationFactor of 3 means that each logical shard will have 3 copies.
maxShardsPerNode: The default value is 1, the maximum number of shards on each Solr server node (new in 4.2)
Pay attention to three values: numShards, replicationFactor, liveSolrNode (the current surviving solr node). A normal solrCloud cluster does not allow multiple replicas of the same shard to be deployed on the same liveSolrNode. Therefore, when maxShardsPerNode=1, numShards*replicationFactor>liveSolrNode , an error is reported. Therefore, it is correct because the following conditions are met: numShards*replicationFactor<liveSolrNode*maxShardsPerNode
createNodeSet: If this parameter is not provided, cores will be created on all active nodes. If this parameter is provided, cores will be created on the specified solr node.
For example, I am now Create 3 slices and 1 copy on 3 tomcats. If this parameter is not provided, the result is like this.


 
Provide this parameter, for example: createNodeSet=127.0.0.1:28986_solr,127.0.0.1:28983_solr,127.0.0.1:28990_solr
The result is like this



  collection.configName: The name of the configuration file used for the new collection. If this parameter is not provided, the collection name will be used as the name of the configuration file.
 
Create an instance of interface 2 (manual allocation): It is recommended to use the following multiple links (3 shards, one backup on each node), because in this way you can create as many 
Java codes   as you want
1.http://127.0.0.1:28983/solr/admin/cores?action=CREATE&name=shard1_replica1&instanceDir=shard1_replica1&dataDir=data&collection=collection1&shard=shard1&collection.configName=myconf  
2.http://127.0.0.1:28990/solr/admin/cores?action=CREATE&name=shard1_replica2&instanceDir=shard1_replica2&dataDir=data&collection=collection1&shard=shard1&collection.configName=myconf  
3.  
4.  
5.http://127.0.0.1:28990/solr/admin/cores?action=CREATE&name=shard2_replica1&instanceDir=shard2_replica1&dataDir=data&collection=collection1&shard=shard2&collection.configName=myconf  
6.http://127.0.0.1:28986/solr/admin/cores?action=CREATE&name=shard2_replica2&instanceDir=shard2_replica2&dataDir=data&collection=collection1&shard=shard2&collection.configName=myconf  
7.  
8. http://127.0.0.1:28986/solr/admin/cores?action=CREATE&name=shard3_replica1&instanceDir=shard3_replica1&dataDir=data&collection=collection1&shard=shard3&collection.configName=myconf 9.http:  
//127.0.0.1:28983/solr/admin /cores?action=CREATE&name=shard3_replica2&instanceDir=shard3_replica2&dataDir=data&collection=collection1&shard=shard3&collection.configName=myconf  
 Parameter meaning:
name: The name of the new core The naming rule of the
created core:
coreName_shardName_replicaN
For example: create a pscp collection, 2 shards, each There are two backups on each shard
and they are named as follows:
pscp_shard1_replica1
pscp_shard1_replica2
pscp_shard2_replica1
pscp_shard2_replica2
shard: Specify an allocation id, the core will hang on that shard (write it casually, if you don’t have this id, it will help you create it for the first time)
collection . configName: Specify a configuration file from zookeeper
instanceDir and dataDir: see their meaning from the following figure
Naming rules: instanceDir has the same name as name, dataDir: it is recommended to be named data

 
. Summary 1: Two ways to add a replica to a cluster
Java code  
1.http:/ /127.0.0.1:28983/solr/admin/collections?action=ADDREPLICA&collection=collection1&shard=shard2&node=192.168.66.128:8085_solr  
_ On the 192.168.66.128:8085_solr machine  
3.http://127.0.0.1:28986/solr/admin/cores?action=CREATE&name=shard3_replica1&instanceDir=shard3_replica1&dataDir=data&collection=collection1&shard=shard3&collection.configName=myconf  
 
2). Delete the interface
Java Code  
1. http://localhost:8983/solr/admin/collections?action=DELETE&name=mycollection  
   Parameters:
name: The name of the collection alias that will be created
collections: A comma-separated list of one or more collection aliases
 
3). Reload the interface, at this time, the corresponding core will reload the configuration file
Java code  
1. http://localhost:8983/solr/admin/collections?action=RELOAD&name=mycollection  
 parameter:
name: will be overloaded Collection name
 
4). Split shard interface
Java code  
1. http://localhost:8983/solr/admin/collections?action=SPLITSHARD&collection=<collection_name>&shard=shardId  
 collection:name of collection
shard:shard to be split The ID
 command cannot be used for clusters that use custom hashes, because such clusters do not have an explicit hash range. It is only used for clusters with plain or compositeid routes. This command will split the shard corresponding to the given shard index into two new shards. By dividing the fragment range into two equal partitions and splitting its documents in the parent fragment (the fragmented fragment) according to the new fragment range. The new shards will be named appending_0 and _1. For example: shard=shard1 is split, the new shards will be named shard1_0 and shard1_1. Once new shards are created, they are activated while the parent shard (forked shard) is suspended so there will be no new requests to the parent (forked shard). This feature meets the requirements for seamless segmentation and no downtime. The original fragmented data will not be deleted. Use the new API command to overload shards at the user's discretion. This feature was released from Solr 4.3. Since some bugs were found in the 4.3 release, it is recommended to wait for 4.3.1 to use this feature.
 
3. Upload files to Zookeeper for management through command line tools. The
reason why
it can be distributed is because ZooKeeper is introduced to save configuration files uniformly. Therefore, it is necessary to upload SolrCloud's configuration files to ZooKeeper. Management tools must have packages first. These packages are all jar packages under /WEB-INF/lib in solr.war.
Step 1: Create a new folder
Create two new folders on any machine that can communicate with the Zookeeper cluster. For example, the following is my directory
/usr/solrCloud/conf/files /usr/solrCloud/conf/lib
files: used to save configuration files lib: used to store jar packages
Step 2 : Upload the required jars and Configuration file
Upload the jar to the lib directory, and put the jars under the solr distribution package (solr-4.8.0\example\solr-webapp\webapp\WEB-INF\lib\ and solr-4.8.0\example\lib\ext\ below All packages must be uploaded to the above lib directory Upload
the solr configuration file to the above files directory
Step 3: Upload the file to Zookeeper for unified management of
Java code  
1.java -classpath .:/usr/solrCloud/conf/lib/* org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 192.168.27.18:2181,192.168.27.18:2182,192.168.27.18:2183 -confdir /usr/solrCloud/conf/files -confname myconf  
-cmd upconfig: upload the configuration file
-confdir: the directory of the configuration file 
-confname: specify the corresponding name
Check whether the file has been uploaded to the Zookeeper server:
Java code  
1.sh zkCli.sh - server localhost:2181  
2.ls /configs/myconf  
Step 4: Associate the configuration file uploaded to ZooKeeper with the collection
Java code  
1.java -classpath .:/usr/solrCloud/conf/lib/* org.apache.solr .cloud.ZkCLI -cmd linkconfig -collection collection1 -confname myconf -zkhost 192.168.27.18:2181,192.168.27.18:2182,192.168.27.18:2183  
-cmd linkconfig:"bind" the configuration file for the specified collection
-collection:specified above the name of the collection
-confname: the name of the configuration file
above The meaning of the above code is: the created core (collection1) will use the myconf configuration file.
For example: executing the following request will create a core as collection1, then the configuration file he uses is zookeeper The myconf configuration
Java code   in 1.http
://localhost:8983/solr/admin/collections?action=CREATE&name=collection1&numShards=3&replicationFactor=1  
Then again, if there is only one configuration on the cluster managed by zookeeper, then create All cores will use this default configuration. If there are multiple copies, if the fourth step is not performed, creating a core will throw an exception and the build will fail!
For example, execute:
Java code  
1.http://127.0.0.1:28983/solr/admin/collections?action=CREATE&name =sdf&numShards=3&replicationFactor=1  
 will throw: because there are two configurations above, but the fourth step is not performed, which associates the configuration with the core (name=sdf) that will be created.

Solr cluster operation
　　The following operations are accessed at the beginning of http://172.16.20.101:8983/solr, and then the complete path is followed by this url. For example: I wrote /admin/collections?action=DELETE&name=article, the access path is: http://172.16.20.101:8983/solr/admin/collections?action=DELETE&name=article
　　4.1 Create a collection
　　There are two ways to create a collection , depending on the routing method:
the first implicit routing method
/admin/collections?action=CREATE&name=123456&router.name=implicit&shards=shard1,shard2,shard3&maxShardsPerNode=3&collection.configName=123456
　　/admin/collections?action=CREATE&name=article&router. name=implicit&shards=shard1,shard2,shard3&maxShardsPerNode=5&collection.configName=collection3ConfThe
　　second compositeId routing method
　　/admin /collections?action=CREATE&name=article&router.name=compositeId&numShards=5&replicationFactor=1&
　　or
　　/admin/collections?action=CREATE&name=application&router.name=compositeId&numShards=1&replicationFactor=2&maxShardsPerNode=2&collection.configName=application&createNodeSet=172.16.20.102:8983_solr,172.16.20.103:8983_solr
　　4.2 delete collection
　　=admin/collections?action=DELETE&name
　　4.3 Adding shard
　　implicit routing to achieve this requirement is relatively simple, as long as you create a shard.
　　The /admin/collections?action=CREATESHARD&collection=article&shard=shard1&name=article_shard1_replica2
　　compositeId route is a little more troublesome to implement this requirement, and it is implemented through the split (SPLITSHARD) operation.
　　/admin/collections?action=SPLITSHARD&collection=article&shard=shard1
　　4.4 Delete shard shard
　　/admin/collections?action=DELETESHARD&collection=article&shard=shard1
　　4.5 Upload the SolrCloud configuration file to zookeeper, and save the configuration file uniformly
 
　　java -classpath .:/data/solr/solr-5.2.1/server/solr-webapp/webapp/WEB-INF/lib/* org.apache.solr. cloud.ZkCLI -cmd upconfig -zkhost 172.16.20.101:2181,172.16.20.102:2181,172.16.20.102:2181 -confdir /data/solr/solr-5.2.1/server/solr/configsets/sample_techproducts_configs/conf/ -confname collection1Conf
　　4.6 associates the configuration files uploaded to ZooKeeper with the Collection
　　java -classpath .:/data/solr/solr-5.2.1/server/solr-webapp/webapp/WEB-INF/lib/* org.apache.solr. cloud.ZkCLI -cmd linkconfig -collection collection1 -confname collection1Conf -zkhost 172.16.20.101:2181,172.16.20.102:2181,172.16.20.103:2181
　　4.7 Upload the configuration to zookeeper, if you want the running solr to load these files synchronously, You need to ask solr to reload the configuration file
/admin/collections?action=RELOAD&name=article