Ranger安装部署 - solr安装

1. 概述

  Lucene是一个Java语言编写的利用倒排原理实现的文本检索类库;

  Solr是以Lucene为基础实现的文本检索应用服务。Solr部署方式有单机方式、多机Master-Slaver方法、Cloud方式。

  SolrCloud是基于Solr和Zookeeper的分布式搜索方案。当索引越来越大,一个单一的系统无法满足磁盘需求,查询速度缓慢,此时就需要分布式索引。在分布式索引中,原来的大索引,将会分成多个小索引,solr可以将这些小索引返回的结果合并,然后返回给客户端。

  solr安装,目前是作为Ranger审计日志的存储,故这里使用的是Ranger-admin里面存在的solr便捷安装方式,可以通过配置文件来直接配置,一键部署。

  ranger solr的便捷式安装来源,首先需要编译ranger,可以参考https://www.cnblogs.com/swordfall/p/11887317.html

2. 单机Standalone安装

2.1 修改install.properties文件

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin/contrib/solr_for_audit_setup
# vim install.properties 
配置java home路径
JAVA_HOME=/usr/lib/java/jdk1.8.0_151

false改为true
SOLR_INSTALL=true

solr线上下载路径
SOLR_DOWNLOAD_URL=http://archive.apache.org/dist/lucene/solr/8.3.0/solr-8.3.0.tgz

solr安装目录
SOLR_INSTALL_FOLDER=/opt/app/ranger-solr-8.3.0-simple

solr对接ranger的服务
SOLR_RANGER_HOME=/opt/app/ranger-solr-8.3.0-simple/ranger_audit_server
solr连接ranger的端口 SOLR_RANGER_PORT
=6083 solr部署模式 SOLR_DEPLOYMENT=standalone solr数据存储目录 SOLR_RANGER_DATA_FOLDER=/opt/app/ranger-solr-8.3.0-simple/ranger_audit_server/data solr单机部署,故为空 SOLR_ZK=

2.2 初始化solr安装脚本

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin/contrib/solr_for_audit_setup
# ./setup.sh 

  初始化结果如下,表明成功:

   启动单机版solr

# pwd
/opt/app/ranger-solr-8.3.0-simple/ranger_audit_server/scripts
# ./start_solr.sh

  启动结果如下,则表明成功:

 

  注:具体操作可以查看如下文件

# pwd
/opt/app/ranger-solr-8.3.0-simple/ranger_audit_server
# vim install_notes.txt

2.3 修改ranger-admin install.properties文件

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin
# vim install.properties
审计日志存储为solr
audit_store=solr

audit_solr_urls=http://bridge1:6083/solr/ranger_audits
audit_solr_user=
audit_solr_password=
audit_solr_zookeepers=

  重新初始化ranger-admin,并重启

# ranger-admin stop
Getting pid from /var/run/ranger/rangeradmin.pid ..
Found Apache Ranger Admin Service with pid 104623, Stopping it...
Apache Ranger Admin Service with pid 104623 has been stopped.

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin
# ./setup.sh

# ranger-admin start
Starting Apache Ranger Admin Service
Apache Ranger Admin Service with pid 127607 has started.

  查看单机版solr是否已经集成到ranger-admin里面,右上角没报错,则已经集成到ranger-admin。

3. SolrCloud安装

3.1 机器准备

  solrcloud至少需要三台机器

3.2 修改install.properties文件

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin/contrib/solr_for_audit_setup
# vim install.properties 
配置java home路径
JAVA_HOME=/usr/lib/java/jdk1.8.0_151

false改为true
SOLR_INSTALL=true

solr线上下载路径
SOLR_DOWNLOAD_URL=http://archive.apache.org/dist/lucene/solr/8.3.0/solr-8.3.0.tgz

solr安装目录
SOLR_INSTALL_FOLDER=/opt/app/ranger-solr-8.3.0-cloud

solr对接ranger的服务
SOLR_RANGER_HOME=/opt/app/ranger-solr-8.3.0-cloud/ranger_audit_server

solr连接ranger的端口
SOLR_RANGER_PORT=6083

solr部署模式
SOLR_DEPLOYMENT=solrcloud

solr数据存储目录
SOLR_RANGER_DATA_FOLDER=/opt/app/ranger-solr-8.3.0-cloud/ranger_audit_server/data

solrcloud在zookeeper上的路径
SOLR_ZK=bridge1:2181,bridge2:2181,bridge3:2181/ranger_audits

ranger连接solr的路径,${SOLR_RANGER_PORT}需要改为6083,与上面的SOLR_RANGER_PORT值一样
SOLR_HOST_URL=http://`hostname -f`:6083

3.3 初始化solr安装脚本

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin/contrib/solr_for_audit_setup
# ./setup.sh 

  初始化结果如下,表明成功:

3.4 远程拷贝ranger-solr-8.3.0-cloud到其他机器

# pwd
/opt/app
# scp -r ranger-solr-8.3.0-cloud/ root@bridge2:/opt/app/
# scp -r ranger-solr-8.3.0-cloud/ root@bridge3:/opt/app/

3.5 启动ranger-solr-8.3.0-cloud

  首先,分别在三台机器启动zookeeper,zookeeper安装,参考https://www.cnblogs.com/swordfall/p/8667409.html

# pwd
/opt/app/zookeeper-3.4.5/bin
# ./zkServer.sh start

  然后,在三台机器分别执行如下命令,创建/var/log/solr/ranger_audits,并赋予solr/solr 用户名、用户组,否则启动solrcloud会报错。紧接着,在其中一台机器执行add_ranger_audits_conf_to_zk.sh文件,使solrcloud与zookeeper关联上。

# mkdir -p /var/log/solr/ranger_audits
# chown solr:solr /var/log/solr/ranger_audits

# pwd
/opt/app/ranger-solr-8.3.0-cloud/ranger_audit_server/scripts
# ./add_ranger_audits_conf_to_zk.sh

  再接着,在三台机器下分别启动solrcloud

pwd
/opt/app/ranger-solr-8.3.0-cloud/ranger_audit_server/scripts
# ./start_solr.sh

  最后,在其中一台机器执行create_ranger_audits_collection.sh文件,在zookeeper上创建ranger_audits集合

# pwd
/opt/app/ranger-solr-8.3.0-cloud/ranger_audit_server/scripts
# ./create_ranger_audits_collection.sh

  最后的最后,启动结果如下,则表明成功:

  注:具体操作可以查看如下文件

# pwd
/opt/app/ranger-solr-8.3.0-cloud/ranger_audit_server
# vim install_notes.txt

3.6 修改ranger-admin install.properties文件

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin
# vim install.properties
审计日志存储为solr
audit_store=solr

audit_solr_urls=http://bridge1:6083/solr/ranger_audits
audit_solr_user=
audit_solr_password=
audit_solr_zookeepers=bridge1:2181,bridge2:2181,bridge3:2181/ranger_audits

  重新初始化ranger-admin,并重启

# ranger-admin stop
Apache Ranger Admin Service is not running

# pwd
/opt/app/apache-ranger-2.0.0/target/ranger-2.0.0-admin
# ./setup.sh 

# ranger-admin start
Starting Apache Ranger Admin Service
Apache Ranger Admin Service with pid 11465 has started

  查看集群版solrcloud是否已经集成到ranger-admin里面,右上角没报错,则已经集成到ranger-admin。

总结

【参考资料】

https://segmentfault.com/a/1190000010836061#item-1

https://www.cnblogs.com/hellxz/p/7434588.html

https://www.cnblogs.com/yjt1993/p/11837398.html

猜你喜欢

转载自www.cnblogs.com/swordfall/p/11967385.html
今日推荐