2012-04-20
Super easy experience, so as not to forget
1. Install JVM
2. Open ssh
ps -ef | grep sshd to see if ssh is enabled
3. Unzip hadoop
4. Configure the following files:
hadoop-env.sh，core-site.xml，hdfs-site.xml，mapred-site.xml，
1) masters record the name of the machine running the secondary namenode (done at namenode)
2) slaves list the machine on which datanode or tasktracker is running (done at namenode)
3) hadoop-env.sh java_home and hadoop_opt
4) core-site.xml Mainly specify hadoop.tmp.dir and fs.default.name (default configuration is {hadoopInstaller}/src/core/core-default.xml)
5) hdfs-site.xml mainly specifies dfs.replication (default configuration is {hadoopInstaller} /src/hdfs/hdfs-default.xml)
6) mapred-site.xml mainly specifies mapred.job.tracker (default configuration is {hadoopInstaller}/src/mapred/mapred-default.xml)
need to execute
ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa (generate key)
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys (no password required locally)
Copy the content of authorized_keys to each node, or append the content of id_dsa.pub to authorized_keys of each node
5. Test whether ssh is connected between each node
6. Copy the entire hadoop to each node. Because the configuration is the same
7. 在name node的bin目录下执行格式化命令：
hadoop namenode -format
8. 完成所有配置
以后在name node上用start-all.sh和stop-all.sh来开启和关闭
默认name node的50030端口开放给jobtracker信息的web显示，50070给hdfs的信息web展示
【注意】
关于ssh有一点要说明。如果一台服务器已经允许其他机器以公共密钥形式ssh至本地的root权限，那么无论怎样都没办法无需密码进入非root的账户了。把密钥放到/root/.ssh/authorized_keys里，类似用法：ssh