BigData <4> _Hadoop stand-alone installation

First, the environment

1. Native Mac, set the name server ip's.

vim /etc/hosts

Increase server ip and name mapping relationship:
120.xx.xxx.145 hadoop000 easy to remember
using the ssh login to the server:

1. On Linux, set up within the network server ip and server name mapping "Ali cloud use, when to use Hadoop to build a network ip"
   is also modified in / etc / hosts, the server name hadoop000

2. Create a file folder associated hadoop mkdir
   mkdir Software storage software installation package
   installation directory mkdir app store software
   mkdir data storage using data
   mkdir lib storage jar developed
   script mkdir shell storage project
   mkdir maven_resp store to use the maven-dependent

3. Pre-installed Hadoop requires Java 1.8 (I use Java11), ssh
   the Java and ssh configuration slightly

java -version

See the installation instructions to install the complete version number.

1. ssh-free secret landing configuration

ssh-keygen -t rsa

All the way round, in ~ / .ssh / in:
id_rsa private
id_rsa.pub public key of
the public key written authorized_keys

cat id_rsa.pub >> authorized_keys
chmod 600 authorized_keys

Note:
. A visit between each node Hadoop cluster (single access node is their own) requires configuring a free secret landing, otherwise the late start NameNode, DataNode need to enter a password
b Hadoop cluster on the late Ali cloud server, on the IDEA. API programmatic access to Hadoop cluster time, also need to avoid tight landing between local and server. That needs to be written to the local machine's authorized_keys public key server

Two, Hadoop installation directory and Detailed configuration hadoop_env

1. Using CDH version, version using hadoop-2.6.0-cdh5.15.1 'attention back with the tail number of the hive version also cdh5.15.1, use the hive-1.1.0-cdh5.15.1 "
   Download: HTTP: //archive.cloudera .com / cdh5
   Download: http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.15.1.tar.gz
   downloaded to the server. And extract it to the appropriate directory

wget http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.15.1.tar.gz
mv ./hadoop-2.6.0-cdh5.15.1.tar.gz ~/hadoop/software
tar -zxvf hadoop-2.6.0-cdh5.15.1.tar.gz -C ~/hadoop/app  

1. Detailed contents of the directory

drwxr-xr-x  2 guoqiang guoqiang  4096 9月   2 01:00 bin  
drwxr-xr-x  2 guoqiang guoqiang  4096 8月  10 2018 bin-mapreduce1
drwxr-xr-x  3 guoqiang guoqiang  4096 8月  10 2018 cloudera
drwxr-xr-x  6 guoqiang guoqiang  4096 8月  10 2018 etc
drwxr-xr-x  5 guoqiang guoqiang  4096 8月  10 2018 examples
drwxr-xr-x  3 guoqiang guoqiang  4096 8月  10 2018 examples-mapreduce1
drwxr-xr-x  2 guoqiang guoqiang  4096 8月  10 2018 include
drwxr-xr-x  3 guoqiang guoqiang  4096 8月  10 2018 lib
drwxr-xr-x  3 guoqiang guoqiang  4096 8月  10 2018 libexec
-rw-r--r--  1 guoqiang guoqiang 85063 8月  10 2018 LICENSE.txt
drwxr-xr-x  2 guoqiang guoqiang  4096 9月   3 14:29 logs
-rw-r--r--  1 guoqiang guoqiang 14978 8月  10 2018 NOTICE.txt
-rw-r--r--  1 guoqiang guoqiang  1366 8月  10 2018 README.txt
drwxr-xr-x  3 guoqiang guoqiang  4096 9月   3 11:13 sbin
drwxr-xr-x  4 guoqiang guoqiang  4096 8月  10 2018 share
drwxr-xr-x 18 guoqiang guoqiang  4096 8月  10 2018 src

bin: hadoop client commands
etc / hadoop: Hadoop-related configuration files storage directory
sbin: Hadoop startup script related processes
share: common tools

1. Configuring
   a, arranged JAVA_HOME

vim ./etc/hadoop/hadoop-env.sh

b, pseudo-distributed single-node configuration
etc / hadoop / core-site.xml

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>