Hadoop的伪分布式搭建需要进行的配置

Hadoop的伪分布模式

                   1、特点:具备Hadoop的所有功能,在单机上模拟一个分布式的环境

                    (1)HDFS:主:NameNode,数据节点:DataNode

                     (2)Yarn:容器,运行MapReduce程序

                       主节点:ResourceManager

                       从节点:NodeManager



2、搭建伪分布式步骤:

                   (1)hdfs-site.xml

                            <!--配置HDFS的冗余度-->

                            <property>

                              <name>dfs.replication</name>

                              <value>1</value>

                            </property>

                            <!--配置是否检查权限-->

                            <property>

                              <name>dfs.permissions</name>

                              <value>false</value>

                            </property>    

                   (2)core-site.xml

                            <!--配置HDFS的NameNode-->

                            <property>

                              <name>fs.defaultFS</name>

                              <value>hdfs://192.168.88.11:9000</value>

                            </property>

                            <!--配置DataNode保存数据的位置-->

                            <property>

                              <name>hadoop.tmp.dir</name>

                             <value>/root/training/hadoop-2.4.1/tmp</value>

                            </property>              

                           

                           

                   (3)mapred-site.xml

                            <!--配置MR运行的框架-->

                            <property>

                              <name>mapreduce.framework.name</name>

                              <value>yarn</value>

                            </property>              

                           

                   (4)yarn-site.xml

                            <!--配置ResourceManager的地址-->

                            <property>

                             <name>yarn.resourcemanager.hostname</name>

                              <value>192.168.88.11</value>

                            </property>

                            <!--配置NodeManager执行任务的方式-->

                            <property>

                             <name>yarn.nodemanager.aux-services</name>

                              <value>mapreduce_shuffle</value>

                            </property>              

                           

                   (5)格式化NameNode

                       hdfs namenode -format

                            日志:Storagedirectory /root/training/hadoop-2.4.1/tmp/dfs/name has been successfullyformatted.

                           

                           

                   (6)启动:start-all.sh

                              (*) HDFS: 存储数据

                                        (*) Yarn:执行计算

                                       

                   (7)访问:(*)命令行

                             (*)Java API

                                       (*)Web Console:

                                                        HDFS:http://192.168.88.11:50070

                                                        Yarn:http://192.168.88.11:8088


猜你喜欢

转载自blog.csdn.net/unique_cn/article/details/80787374
今日推荐