Ranger installation and deployment - solr installation

1 Overview

  Lucene is a Java language written in realization of the principle of the use of inverted text retrieval library;

  Solr Lucene is a text retrieval application services based implementation. Solr has deployed stand-alone mode, multi-machine Master-Slaver method, Cloud ways.

  SolrCloud is distributed Solr-based search program and the Zookeeper. When the index increases, a single disk system can not meet the demand, slow query speed, then you need a distributed index. In a distributed index, the original large index, will be divided into a plurality of small indexes can be combined Solr returned index results of these small, then returned to the client.

  solr installation, is currently stored as the audit log Ranger, it is used here is present inside solr Ranger-admin convenient installation, can be configured via the configuration file directly, a key deployment.

  Convenient installation source ranger solr, you first need to compile ranger, can refer https://www.cnblogs.com/swordfall/p/11887317.html

2. Standalone Standalone Installation

2.1 modify the file install.properties

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin/contrib/solr_for_audit_setup
# vim install.properties 
Configuring java home path
JAVA_HOME=/usr/lib/java/jdk1.8.0_151

false to true
SOLR_INSTALL=true

solr online download path
SOLR_DOWNLOAD_URL=http://archive.apache.org/dist/lucene/solr/8.3.0/solr-8.3.0.tgz

solr installation directory
SOLR_INSTALL_FOLDER=/opt/app/ranger-solr-8.3.0-simple

solr对接ranger的服务
SOLR_RANGER_HOME=/opt/app/ranger-solr-8.3.0-simple/ranger_audit_server
solr port of ranger SOLR_RANGER_PORT
=6083 solr deployment model SOLR_DEPLOYMENT=standalone solr data storage directory SOLR_RANGER_DATA_FOLDER=/opt/app/ranger-solr-8.3.0-simple/ranger_audit_server/data solr standalone deployment, it is empty SOLR_ZK=

2.2 initialization script to install solr

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin/contrib/solr_for_audit_setup
# ./setup.sh 

  Initialization results are as follows, indicating success:

   Start stand-alone solr

# pwd
/opt/app/ranger-solr-8.3.0-simple/ranger_audit_server/scripts
# ./start_solr.sh

  Start results are as follows, indicating a successfully:

 

  Note: The specific operation can view the following documents

# pwd
/opt/app/ranger-solr-8.3.0-simple/ranger_audit_server
# vim install_notes.txt

2.3 modify ranger-admin install.properties file

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin
# vim install.properties
审计日志存储为solr
audit_store=solr

audit_solr_urls=http://bridge1:6083/solr/ranger_audits
audit_solr_user=
audit_solr_password=
audit_solr_zookeepers=

  重新初始化ranger-admin,并重启

# ranger-admin stop
Getting pid from /var/run/ranger/rangeradmin.pid ..
Found Apache Ranger Admin Service with pid 104623, Stopping it...
Apache Ranger Admin Service with pid 104623 has been stopped.

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin
# ./setup.sh

# ranger-admin start
Starting Apache Ranger Admin Service
Apache Ranger Admin Service with pid 127607 has started.

  查看单机版solr是否已经集成到ranger-admin里面,右上角没报错,则已经集成到ranger-admin。

3. SolrCloud安装

3.1 机器准备

  solrcloud至少需要三台机器

3.2 修改install.properties文件

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin/contrib/solr_for_audit_setup
# vim install.properties 
配置java home路径
JAVA_HOME=/usr/lib/java/jdk1.8.0_151

false改为true
SOLR_INSTALL=true

solr线上下载路径
SOLR_DOWNLOAD_URL=http://archive.apache.org/dist/lucene/solr/8.3.0/solr-8.3.0.tgz

solr安装目录
SOLR_INSTALL_FOLDER=/opt/app/ranger-solr-8.3.0-cloud

solr对接ranger的服务
SOLR_RANGER_HOME=/opt/app/ranger-solr-8.3.0-cloud/ranger_audit_server

solr连接ranger的端口
SOLR_RANGER_PORT=6083

solr部署模式
SOLR_DEPLOYMENT=solrcloud

solr数据存储目录
SOLR_RANGER_DATA_FOLDER=/opt/app/ranger-solr-8.3.0-cloud/ranger_audit_server/data

solrcloud在zookeeper上的路径
SOLR_ZK=bridge1:2181,bridge2:2181,bridge3:2181/ranger_audits

ranger连接solr的路径,${SOLR_RANGER_PORT}需要改为6083,与上面的SOLR_RANGER_PORT值一样
SOLR_HOST_URL=http://`hostname -f`:6083

3.3 初始化solr安装脚本

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin/contrib/solr_for_audit_setup
# ./setup.sh 

  初始化结果如下,表明成功:

3.4 远程拷贝ranger-solr-8.3.0-cloud到其他机器

# pwd
/opt/app
# scp -r ranger-solr-8.3.0-cloud/ root@bridge2:/opt/app/
# scp -r ranger-solr-8.3.0-cloud/ root@bridge3:/opt/app/

3.5 启动ranger-solr-8.3.0-cloud

  首先,分别在三台机器启动zookeeper,zookeeper安装,参考https://www.cnblogs.com/swordfall/p/8667409.html

# pwd
/opt/app/zookeeper-3.4.5/bin
# ./zkServer.sh start

  然后,在三台机器分别执行如下命令,创建/var/log/solr/ranger_audits,并赋予solr/solr 用户名、用户组,否则启动solrcloud会报错。紧接着,在其中一台机器执行add_ranger_audits_conf_to_zk.sh文件,使solrcloud与zookeeper关联上。

# mkdir -p /var/log/solr/ranger_audits
# chown solr:solr /var/log/solr/ranger_audits

# pwd
/opt/app/ranger-solr-8.3.0-cloud/ranger_audit_server/scripts
# ./add_ranger_audits_conf_to_zk.sh

  再接着,在三台机器下分别启动solrcloud

pwd
/opt/app/ranger-solr-8.3.0-cloud/ranger_audit_server/scripts
# ./start_solr.sh

  最后,在其中一台机器执行create_ranger_audits_collection.sh文件,在zookeeper上创建ranger_audits集合

# pwd
/opt/app/ranger-solr-8.3.0-cloud/ranger_audit_server/scripts
# ./create_ranger_audits_collection.sh

  最后的最后,启动结果如下,则表明成功:

  注:具体操作可以查看如下文件

# pwd
/opt/app/ranger-solr-8.3.0-cloud/ranger_audit_server
# vim install_notes.txt

3.6 修改ranger-admin install.properties文件

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin
# vim install.properties
审计日志存储为solr
audit_store=solr

audit_solr_urls=http://bridge1:6083/solr/ranger_audits
audit_solr_user=
audit_solr_password=
audit_solr_zookeepers=bridge1:2181,bridge2:2181,bridge3:2181/ranger_audits

  重新初始化ranger-admin,并重启

# ranger-admin stop
Apache Ranger Admin Service is not running

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin
# ./setup.sh 

# ranger-admin start
Starting Apache Ranger Admin Service
Apache Ranger Admin Service with pid 11465 has started

  查看集群版solrcloud是否已经集成到ranger-admin里面,右上角没报错,则已经集成到ranger-admin。

总结

【参考资料】

https://segmentfault.com/a/1190000010836061#item-1

https://www.cnblogs.com/hellxz/p/7434588.html

https://www.cnblogs.com/yjt1993/p/11837398.html

 

 

 

 

Guess you like

Origin www.cnblogs.com/swordfall/p/11967385.html