hadoop学习资料-03-安装配置续02

执行:vim hdfs-site.xml

<configuration>
 <property>
   <name>dfs.replication</name>
   <value>1</value>
 </property>


<property>
   <name>dfs.permissions</name>
   <value>false</value>
</property>
</configuration>

cp mapred-site.xml.template  mapred-site.xml

vim mapred-site.xml

<!--制定mapreduce运行到yarn上,提高算法-->

<configuration>
<property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
</configuration>

mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
<!--制定mapreduce运行到yarn上,提高算法-->
<property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
</configuration>

配置yarm-site.xml

<configuration>

<!-- Site specific YARN configuration properties -->
  <property>
    <name>yarn.resourcemanager.hostname</name>
    <value>hadoop01</value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
</configuration>

<?xml version="1.0"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->
<configuration>

<!-- Site specific YARN configuration properties -->
  <property>
    <name>yarn.resourcemanager.hostname</name>
    <value>hadoop01</value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
</configuration>

配置slaves文件

执行vim slaves

hadoop01

删除localhost   改为hadoop01

JAVA_HOME=/usr/local/src/java/jdk1.7.0_51
JAVA_BIN=/usr/local/src/java/jdk1.7.0_51/bin
HADOOP_HOME=/usr/local/src/hadoop/hadoop-2.7.1
PATH=$JAVA_HOME/bin:$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JAVA_HOME JAVA_BIN HADOOP_HOME  PATH CLASSPATH

 

source /etc/profile       echo $HADOOP_HOME

也可以这样测试  hadoop fs -ls /

hadoop namenode -format       

     

 

重启linux  :  reboot

再次重新格式化: hadoop namenode -format             输入yes

警告异常:ARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

解决连接:http://blog.csdn.net/young_kim1/article/details/50324345

log4j.logger.org.apache.hadoop.util.NativeCodeLoader=ERROR  
加以上的那句话就是让日志打印错误的级别,警告就不打印了,掩耳盗铃的作用。

cd sbin/    切换目录

先启动这个文件 start-dfs.sh  是hdfs的启动文件。    启动命令: ./start-dfs.sh       再输入yes       启动完成后输入命令jps

NameNode  相等于账本节点信息。
DataNode  相等于数据节点信息。
SecondaryNameNode  相等于nameNode节点的助理。身上有最新数据的一部分。如果用他来还原的时候会丢失一部分数据。

再启动start-yarn.sh文件  是mapreduce处理运算用的(交给yarn来给他运算)的文件 
启动命令:
./start-yarn.sh  
启动完成后输入命令jps


ResourceManager    相等于运算中的老大。
NodeManager   相等于下面节点的老大。

上面的两部是分开启动的。你也可以直接用./start-all.sh命令启动。./stop-all.sh命令关闭

http://192.168.191.135:50070     来查看启动的hadoop服务

可以用以下命令进行操作

hadoop命令大全     链接:https://blog.csdn.net/m0_38003171/article/details/79086780

猜你喜欢

转载自blog.csdn.net/qq_40680190/article/details/81532717