1. Installation JDK
. 1) Download jdk-8u65-linux-x64.tar.gz
2) create / soft Folder
$>sudo mkdir /soft
$>sudo chown grj:grj /soft
3) tar open
$>tar -xzvf jdk-8u65-linux-x64.tar.gz -C /soft
4) Create a symbolic link
$> -s /soft/jdk-1.8.0_65 LN / Soft / jdk
5) Verify jdk installation was successful
$>cd /soft/jdk/bin
$>./java -version
6) centos configure the environment variables
a) edit / etc / profile
$>sudo nano /etc/profile
...
export JAVA_HOME=/soft/jdk
exprot PATH=$PATH:$JAVA_HOME/bin
b) the environment variables take effect immediately
$> Source / etc / Profile
c) enter any directory, test is ok
$>cd ~
$>java -version
2. Install hadoop (requires every host in the cluster are installed)
1) Download hadoop-2.7.3.tar.gz
2) tar open
$>tar -xzvf hadoop-2.7.3.tar.gz -C /soft
3) Create a symbolic link
$>ln -s /soft/hadoop-2.7.3 /soft/hadoop
4) Verify that the installation was successful hadoop
$>cd /soft/hadoop/bin
$>./hadoop version
5) Configure hadoop environment variables
$>sudo nano /etc/profile
...
export JAVA_HOME=/soft/jdk
exprot PATH=$PATH:$JAVA_HOME/bin
export HADOOP_HOME=/soft/hadoop
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
6)生效
$>source /etc/profile
3. cluster machine configuration (modify host name)
1) Each host needs to be modified as the hosts file host name in this file take effect after restart
/etc/hostname
s201
2)/etc/hosts
127.0.0.1 localhost
192.168.24.201 s201
192.168.24.202 s202
192.168.24.203 s203
192.168.24.204 s204
3) Each host to be changed according to the hosts file in this document ip
/etc/sysconfig/network-scripts/ifcfg-exxxxx
...
IPADDR=..
Network Service Restart $> sudo service network restart
4) modify the /etc/resolv.conf file, all hosts are changed to the same nameserver
nameserver 192.168.24.2
4. Prepare fully distributed host ssh (no adhesion implement user login other hosts belonging to the cluster in S201)
1) generates a key pair on the host s201
$> ssh-keygen -t rsa -P '' -f ~ / .ssh / id_rsa
2) A public key file s201 copied to the 202 to 204 id_rsa.pub remote host.
And placed /home/centos/.ssh/authorized_keys
$> @ scp id_rsa.pub CentOS S201: /home/grj/.ssh/authorized_keys
$> @ scp id_rsa.pub CentOS S202: /home/grj/.ssh/authorized_keys
$ > scp id_rsa.pub CentOS @ S203: /home/grj/.ssh/authorized_keys
$> @ scp id_rsa.pub CentOS S204: /home/grj/.ssh/authorized_keys
5. Configure the fully distributed
1) Create a configuration directory (this time can put the original configuration folder / soft / hadoop / etc / hadoop deleted or renamed, follow symbolic links to prevent its same name)
$>cp -r /soft/hadoop/etc/hadoop /soft/hadoop/etc/full
2) create symbolic link
$> ln -s / soft / hadoop / etc / full hadoop
3) modify the configuration file ($ {hadoop_home} / etc / full /)
[core-site.xml]
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://s201/</value>
</property>
</configuration>
[hdfs-site.xml]
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
</configuration>
[mapred-site.xml]
不变
[yarn-site.xml]
<?xml version="1.0"?>
<configuration>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>s201</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
4) modify the slaves file (the file to store all data nodes host name)
[/soft/hadoop/etc/full/slaves]
s202
s203
s204
5) Modify Hadoop environment variable file [/soft/hadoop/etc/full/hadoop-env.sh]
...
export JAVA_HOME=/soft/jdk
...
6) Distribution Configuration
$>cd /soft/hadoop/
$>scp -r etc centos@s202:/soft/hadoop
$>scp -r etc centos@s203:/soft/hadoop
$>scp -r etc centos@s204:/soft/hadoop
7) file system is formatted (in the name of the node to perform this operation S201)
$> hadoop the NameNode -format
8) to start the process hadoop
$>start-all.sh