CDH offline installation of hadoop learning

1. Environmental preparation

1. Prepare three virtual machines (centos7)

2. Java installation (jdk1.8)

3, ssh password-free settings

4. Configure fixed IP address and hosts settings

5. Turn off firewall and selinux

5.1selinux is permanently valid

Modify SELINUX="" in the /etc/selinux/config file to disabled, then restart.

5.2 Permanent effect of firewall

Turn off the firewall directly

systemctl stop firewalld.service #停止firewall

systemctl disable firewalld.service #Prohibit firewall from starting

6. The time of the three servers should be set the same (all nodes are configured with NTP service https://blog.csdn.net/yjgithub/article/details/54613172 )

7. Prepare one of them to install MySQL ( https://blog.csdn.net/chichichichi/article/details/79913111 )

8. Confirm whether the following packages are installed

yum install chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb -y

2. CM installation (download address: http://archive.cloudera.com/cm5/cm/5/ )

1. Create a directory: mkdir /opt/clouderamanager

2. Upload cloudera-manager-centos7-cm5.13.3_x86_64.tar.gz and extract the tar zxvf cloudera-manager-centos7-cm5.13.3_x86_64.tar.gz

3. Modify /opt/clouderamanager/cm-5.13.3/etc/cloudera-scm-agent/config.ini and change server_host=CDH to your host name; CDH is the host name I want to use as the server server

4. Upload mysql-connector-java-5.1.44-bin.jar to /usr/share/java/ and change the name to mysql-connector-java.jar (the name should be mysql-connector-java; it cannot be changed)

5. Copy the /opt/clouderamanager folder to the CDH1 and CDH2 servers;

6. Execute on three servers:

useradd --system --home=/opt/clouderamanager/cm-5.13.3/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm 创建 cloudera-scm用户

7. Log in to mysql and execute GRANT ALL PRIVILEGES ON *.* TO 'cdh'@'%' IDENTIFIED BY 'cdh' WITH GRANT OPTION;

8. Go back to the server machine and execute: /opt/clouderamanager/cm-5.13.3/share/cmf/schema/scm_prepare_database.sh mysql cm -h 192.168.35.186 -ucdh -pcdh --scm-host 192.168.35.187 scm scm scm

Note: 192.168.35.186 is the address of the mysql server; 192.168.35.187 is the address of the server server

9. Create a parcel directory

server node

mkdir -p /opt/cloudera/parcel-repo/
chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/

agent node

mkdir -p /opt/cloudera/parcels/
chown cloudera-scm:cloudera-scm /opt/cloudera/parcels

10. Download CDH-5.13.3-1.cdh5.13.3.p0.2-el7.parcel.sha1, manifest.json, CDH-5.13.3-1.cdh5.13.3.p0.2-el7.parcel

Download address: http://archive.cloudera.com/cdh5/parcels/5.13.3.2/

11. Upload the three files to the /opt/cloudera/parcel-repo/ directory of the server machine and repair CDH-5.13.3-1.cdh5.13.3.p0.2-el7.parcel.sha1

It is CDH-5.13.3-1.cdh5.13.3.p0.2-el7.parcel.sha (the name must be changed)

12、cd /opt/clouderamanager/cm-5.13.3/etc/init.d

13. Execute the command on the server node: ./cloudera-scm-server start ./cloudera-scm-agent start (server and agent are the same)

Execute ./cloudera-scm-agent start on the agent node

14. The startup time will be longer because some data needs to be initialized; although it prompts [ok], the background log is still running; you can enter

cd /opt/clouderamanager/cm-5.13.3/log/cloudera-scm-server for log viewing

15. Log viewing: tail -f cloudera-scm-server.log