hadoop-1.0.3集群安装

 1、下载vmware和centos,在vmware里安装2份centos(可安装好一个复制一个)

2、一个作为hadoop的master一个作为hadoop的slave

3、在两台机器上分别配置hosts文件 vim /etc/hosts 添加:

192.168.237.128 master

192.168.237.129 slave1

 
 

4、安装jdk ,vim /etc/profile export JAVA_HOME 和 PATH,

要让配置起效要执行

source /etc/profile

 
 

5、分别给两台机器新增用户hadoop用户组hadoop

6、ssh免登陆,两台机器分别配置如下命令

ssh-keygen -t rsa

cp id_rsa.pub authorized_keys (master端可不执行)

 

   在master上执行:

ssh-copy-id -i $HOME/.ssh/id_rsa.pub hadoop@slave1 (将mater的公钥追加到slave1的authorized_keys中)

 

 如果有更多机器同理(ssh slave1若出现“Agent admitted failure to sign using the key” 错误执行ssh-add /home/hadoop/.ssh/id_rsa

)

7、安装hadoop-1.0.3

 下载hadoop-1.0.3.tar.gz ,分别在两台机器解压当前用户的目录下/home/hadoop

tar xzf hadoop-1.0.3.tar.gz

 
 

使用管理员对解压后的文件夹授权给hadoop用户及组 

sudo chown -R hadoop:hadoop hadoop-1.0.3

 
 

在/etc/profile添加hadoop环境变量

export HADOOP_INSTALL=/usr/local/hadoop/hadoop-1.0.3

 
 
 
 
 

export PATH=$HADOOP_INSTALL/bin:$PATH

 

 8、配置hadoop

    

   修改hadoop-env.sh,去掉注释

export JAVA_HOME=/usr/lib/jvm/default-java

 

 

修改

core-site.xml

  <?xml version="1.0"?>

 

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

 

<configuration>

 

<property>

 

<name>fs.default.name</name>

 

<value>hdfs://master:9000</value>

 

</property>

 

</configuration>

 
 
 
 
 
 

 修改

hdfs-site.xml

<?xml version="1.0"?>

<?xml-stylesheet type="text/xsl"href="configuration.xsl"?>

<configuration>

<property>

<name>dfs.name.dir</name>

<value>/home/hadoop/namedir</value>

<!--在master的这几个目录要自己建-->

</property>

<property>

<name>dfs.data.dir</name>

<value>/home/hadoop/datadir</value>

</property>

<property>

<name>fs.checkpoint.dir</name>

<value>/home/hadoop/ckdir</value>

</property> <property>

<name>dfs.replication</name>

<value>2</value>

</property>

</configuration>

 

  

 修改

mapred-site.xml

  

 <?xml version="1.0"?>   <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>    <configuration>    <property>    <name>mapred.job.tracker</name>    <value>namenode:9001</value>    <final>true</final>    </property>    <property>    <name>mapred.local.dir</name>    <value>/home/hadooptest/mapred/local</value>    <final>true</final>    </property>    <property>    <name>mapred.system.dir</name>    <value>/home/hadooptest/mapred/system</value>    <final>true</final>    </property>    <property>    <name>mapred.tasktracker.map.tasks.maximum</name>    <value>2</value>    <final>true</final>    </property>    <property>    <name>mapred.tasktracker.reduce.tasks.maximum</name>    <value>2</value>    <final>true</final>    </property>    <property>    <name>mapred.child.java.opts</name>    <value>-Xmx400m</value>    </property>   </configuration>

 

 

修改masters

master

 
 

修改slaves

slave1

 

 两台机器保持以上配置一致

9、格式化namenode

bin/hadoop namenode -format

 
 

10、启动hadoop

bin/start-all.sh

 

11、验证安装

jps

 

 master上显示:

2900 NameNode 3075 SecondaryNameNode 3246 Jps 3154 JobTracker

 

  

slave1上显示:

2622 TaskTracker

2516 DataNode

2695 Jps

 
 

12、日志在hadoop安装包底下的logs里有任何错误请查看里面的报错日志

 
 
 
 

 

猜你喜欢

转载自wiselyman.iteye.com/blog/1685863