Hadoop operating environment is built based on CentOS6

Change to static IP

# vim /etc/sysconfig/network-scripts/ifcfg-eth0

There are 5 items that need to be modified:

IPADDR=192.168.1.101 (set fixed IP)

GATEWAY=192.168.1.2 (set gateway)

ONBOOT=yes (set auto)

BOOTPROTO=static (set static)

DNS1=192.168.1.2                     (Set DNS server)

# service network restart

Modify hostname

Modify the hosts file of linux

#vi /etc/sysconfig/network

NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME= hadoop001

Note: Do not have "_" underscore in the host name

#vim /etc/hosts

Add the following

192.168.1.101 hadoop100

After restarting the device and restarting the device, after restarting, check the host name, it has been modified successfully

turn off firewall

Check firewall startup status

#chkconfig iptables --list

turn off firewall

#chkconfig iptables off

Create a file in the opt directory

Create hadoop user

Do the following in the root user

[root@lzl opt]# adduser hadoop
[root@lzl opt]# passwd 123456

Set the atguigu user to have root privileges

Modify the /etc/sudoers file , find the following line , and add a line under root

# vi /etc/sudoers

## Allow root to run any commands anywhere
root    ALL=(ALL)     ALL
hadoop   ALL=(ALL)     ALL

After the modification is completed, you can now log in with the atguigu account, and then use the command su - to obtain root privileges to operate

Create a folder in the /opt directory

(1) Create module and software folders under the root user

    [root@lzl opt]#mkdir module

    [root@lzl opt]#mkdir software

(2) Modify the owner of the module and software folders

    [root@lzl opt]#chown hadoop:hadoop module

    [root@lzl opt]# chown hadoop : hadoop sofrware

install jdk

1) Uninstall existing jdk

(1) Check whether the java software is installed:

[root @ lzl opt] # rpm -qa | grep java

(2) If the installed version is lower than 1.7, uninstall the jdk:

[root@lzl opt]# rpm -e package

Start to download java jdk and install

 

The installation process is no longer described, and the environment variable configuration is introduced.

Open /etc/profile file

# vi /etc/profile

在profie文件末尾添加jdk路径：

##JAVA_HOME

export JAVA_HOME=/opt/module/jdk1.8.0_144

exportPATH=$PATH:$JAVA_HOME/bin

让修改后的文件生效执行# source /etc/profile

测试java是否安装成功

# java -version

安装Hadoop（下载地址）

1）进入到Hadoop安装包路径下：

[root@lzl ~]# cd /opt/software/

2）解压安装文件到/opt/module下面

[root@lzl software]# tar -zxfhadoop-2.7.2.tar.gz -C /opt/module/

3）查看是否解压成功

[root@lzl software]# ls /opt/module/

hadoop-2.7.2  

4）在/opt/module/hadoop-2.7.2/etc/hadoop路径下配置hadoop-env.sh

（1）Linux系统中获取jdk的安装路径：

[root@lzl jdk1.8.0_144]# echo $JAVA_HOME

/opt/module/jdk1.8.0_144

（2）修改hadoop-env.sh文件中JAVA_HOME 路径：

root@lzl hadoop]# vi hadoop-env.sh

        修改JAVA_HOME如下

exportJAVA_HOME=/opt/module/jdk1.8.0_144

5）将hadoop添加到环境变量

（1）获取hadoop安装路径：

[root@ lz hadoop-2.7.2]#pwd

/opt/module/hadoop-2.7.2

（2）打开/etc/profile文件：

[root@ lzl hadoop-2.7.2]# vi/etc/profile

        在profie文件末尾添加jdk路径：（shitf+g）

##HADOOP_HOME

export HADOOP_HOME=/opt/module/hadoop-2.7.2

export PATH=$PATH:$HADOOP_HOME/bin

export PATH=$PATH:$HADOOP_HOME/sbin

（3）保存后退出 :wq

（4）让修改后的文件生效：

[root@ lzl hadoop-2.7.2]#source /etc/profile

（5）重启(如果hadoop命令不能用再重启)： 

[root@ lzl hadoop-2.7.2]#sync

        [root@ lzl hadoop-2.7.2]# reboot

6）修改/opt目录下的所有文件所有者为hadoop

       [root@lzl opt]# chown hadoop:hadoop -R /opt/

7）切换到atguigu用户

       [root@lzl opt]# su hadoop

本地运行Hadoop 案例

官方grep案例

1）创建在hadoop-2.7.2文件下面创建一个input文件夹

[lzl@lzl hadoop-2.7.2]$mkdir input

2）将hadoop的xml配置文件复制到input

[lzl@lzl hadoop-2.7.2]$cpetc/hadoop/*.xml input

3）执行share目录下的mapreduce程序

[lzl@lzl hadoop-2.7.2]$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jargrep input output 'dfs[a-z.]+'

4）查看输出结果

[lzl@lzl hadoop-2.7.2]$ cat output/*

官方wordcount案例

1）创建在hadoop-2.7.2文件下面创建一个wcinput文件夹

[lzl@lzl hadoop-2.7.2]$mkdir wcinput

2）在wcinput文件下创建一个wc.input文件

[lzl@lzl hadoop-2.7.2]$cd wcinput

[lzl@lzl wcinput]$touch wc.input

3）编辑wc.input文件

	[lzl@lzl wcinput]$vim wc.input
	在文件中输入如下内容
	hadoop yarn
	hadoop mapreduce 
	linzhiling
	linzhiling
	保存退出：：wq

4）回到hadoop目录/opt/module/hadoop-2.7.2

5）执行程序：

[lzl@lzl hadoop-2.7.2]$ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar wordcount wcinput wcoutput

6）查看结果：

[lzl@lzl hadoop-2.7.2]$catwcoutput/part-r-00000

linzhiling 2

hadoop 2

mapreduce      1

yarn    1