搭建Hadoop

1.修改虚拟机内存为1GB

2.Xshell / SercureCRT 客户端连接虚拟机

ctrl+alt+t --》打开 terminal 窗口
ifconfig --》查看IP地址

3.关闭防火墙
临时关闭
service iptables stop
永久关闭
chkconfig iptables off

4.修改主机名
查看主机名
hostname

修改主机名
vim /etc/sysconfig/network

修改后立即生效
source /etc/sysconfig/network

需重启
reboot

临时修改主机名
hostname xxxx

5.配置 hosts
vim /etc/hosts

6.配置免密登录

ssh [email protected]
需输入用户密码

配置
ssh-keygen 回车一路默认

找到生成的id_rsa.pub 文件（su root , cd /root 在 root 文件夹下）
ssh-copy-id -i .ssh/id_rsa.pub [email protected]

再次执行ssh [email protected] 无需密码

7.安装配置JDK

8.上传安装Hadoop

tar -zxvf hadoop.tar.gz

cd hadoop/etc/haddoop
--------------------------------------------------------------------------
vim hadoop-env.sh
i 插入

ESC ：wq 保存

source hadoop-env.sh 立即生效

--------------------------------------------------------------------------

vim core-site.xml

<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://linux01:9000</value>
</property>

<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/software/hadoop/usr/local/software/hadoop-2.7.1/tmp</value>
</property>
</configuration>

cd ../hadoop-2.7.1
mkdir tmp

----------------------------------------------------------------------------
vim hdfs-site.xml

<configuration>
<property>
<name>dfs-replication</name>
<value>1</value>
</property>

<property>
<name>dfs.permissions</name>
<value>false</value>
</property>

<property>
<name>dfs.datanode.data.dir</name>
<value>/usr/local/software/hadoop/usr/local/software/hadoop-2.7.1/tmp</value>
</property>

</configuration>

此处：datanode.dir 配置需要与 core-site.xml 中 datanode.dir 的配置一致
否则，上传时报错

http://blog.sina.com.cn/s/blog_61d8d9640102whof.html

--------------------------------------------------------------------------

cp mapred-site.xml.template mapred-site.xml

vim mapred-site.xml

<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>

-------------------------------------------------------------------------

vim yarn-site.xml

<configuration>


<property>
<name>yarn.resourcemanager.hostname</name>
<value>linux01</value>
</property>

<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>

-------------------------------------------------------------------------

vim slaves

linux01

从属机器名称
--------------------------------------------------------------------------

配置环境变量
vim /etc/profile

source /etc/profile

9.测试
hadoop namenode -formate

cd /sbin
sh start-all.sh

jps 查看进程是否启动成功

猜你喜欢