-------------------
1. Install jdk
a) Download jdk-8u65-linux-x64.tar.gz
b) tar open
$>su centos ; cd ~
$>mkdir downloads
$>cp /mnt/hdfs/downloads/bigdata/jdk-8u65-linux-x64.tar.gz ~/downloads
$>tar -xzvf jdk-8u65-linux-x64.tar.gz
c)create/ soft folder
$>sudo mkdir /soft
$>sudo chown centos:centos /soft
d) Move the tar file to /soft
$>mv ~/downloads/jdk-1.8.0_65 /soft/
e) Create a symbolic link
$ >ln -s /soft/jdk-1.8.0_65 /soft/jdk
f) Verify that the jdk installation is successful
$>cd /soft/jdk/bin
$>./java -version
centos configure environment variables
------- -----------------
1. Edit /etc/profile
$>sudo nano /etc/profile
...
export JAVA_HOME=/soft/jdk
exprot PATH=$PATH:$JAVA_HOME/bin
2. Make the environment variable take effect immediately
$>source /etc/profile 3. Enter any directory and test whether it is ok $>cd ~ $> java -version install hadoop ------------------------- 1. Install hadoop a) download hadoop-2.7.3.tar.gz b) tar open $>su centos ; cd ~ $>cp /mnt/hdfs/downloads/bigdata/hadoop-2.7.3.tar.gz ~/downloads $>tar -xzvf hadoop-2.7.3.tar.gz c) none d) Move the tar file to /soft $>mv ~/downloads/hadoop-2.7.3 /soft/ e) Create a symbolic link $>ln -s /soft/hadoop-2.7.3 /soft/hadoop f) Verify jdk Whether the installation is successful $>cd /soft/hadoop/bin $>./hadoop version 2. Configure hadoop environment variables
$>sudo nano /etc/profile
...
export JAVA_HOME=/soft/jdk
exprot PATH=$PATH:$JAVA_HOME/bin
export HADOOP_HOME=/soft/hadoop
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin 3.生效
$>source /etc/profile
Configure hadoop
--------------------
1.standalone(local)
nothing!
No need to enable a separate hadoop process. 2.Pseudodistributed mode pseudo-distributed mode. a) Enter ${HADOOP_HOME}/etc/hadoop directory b) Edit core-site.xml <?xml version="1.0"?> <configuration> <property> <name>fs.defaultFS</name> <value>hdfs ://localhost/</value> </property>
</configuration>
<?xml version="1.0"?>
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
注意:cp mapred-site.xml.template mapred-site.xml
<?xml version="1.0"?>
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
<?xml version="1.0"?>
<configuration>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>localhost</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
f) Configure SSH
1) Check if ssh related packages are installed (openssh-server + openssh-clients + openssh)
$yum list installed | grep ssh
2) Check if sshd process is started
$>ps -Af | grep sshd 3) Generate a public-private key pair on the client side. $>ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa 4) Generate ~/.ssh folder, which contains id_rsa (private key) + id_rsa.pub (public key) 5) Append public key Go to the ~/.ssh/authorized_keys file (fixed file name and location) $>cd ~/.ssh $>cat id_rsa.pub >> authorized_keys 6) Modify the permission of authorized_keys to 644. $>chmod 644 authorized_keys 7) Test $ >ssh localhost