Hadoop-1.2.1/1.0.1 install on Ubuntu

1.hadoop-1.2.1官方包：点我

Hadoop-1.0.1官方包：点我

1.1 解压到home目录中的Hadoop下

sudo tar -zxvf 包名

cd hadoop-1.2.1

2.jdk 1.6 点我

jdk 1.8 点我

jdk 移到usr/local/java中去后直接./执行（bin文件）就解压了环境变量配置：vi ~/.bashrc 或者vi /etc/profile中添加java_home啥的记得用source重置一下。ps：新手如果发现执行某些操作出现permitted等关于权限的词时，在这个操作前面加上sudo即可,加上sudo就意味着以管理员权限运行（JDK1.8的也能用）

~/.bashrc示例：

#hadoop环境变量

export HADOOP_PREFIX=/home/root1/Hadoop/hadoop-1.2.1
export PATH=$PATH:$HADOOP_PREFIX/bin:$HADOOP_PREFIX/sbin

#jdk环境变量
export JAVA_HOME=/usr/local/java/jdk1.6.0_45
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH

3.Hadoop配置

3.1 在$Hadoop_HOME/conf/hadoop-env.sh中配置

sudo gedit conf/hadoop-env.sh

去掉export JAVA_HOME=/usr/local/java/jdk1.6.0_45前面的注释号，并将路径换成你的jdk路径

3.2 配置conf/core-site.xml

sudo gedit conf/core-site.xml

<configuration>
     <property>
         <name>fs.default.name</name>
         <value>hdfs://localhost:9000</value>
     </property>
</configuration>

3.3 配置conf/hdfs-site.xml

<configuration>
     <property>
         <name>dfs.replication</name>
         <value>1</value>
     </property>
</configuration>

3.4 配置conf/mapred-site.xml

<configuration>
     <property>
         <name>mapred.job.tracker</name>
         <value>localhost:9001</value>
     </property>
</configuration>

4.1 ssh配置并将其设置为免密登陆

#安装gedit和ssh服务

sudo apt-get update
sudo apt-get install gedit

sudo apt-get install ssh
sudo apt-get install openssh-server

#安装后，可以使用如下命令登陆本机

hadoop@ubuntu:~$ ssh localhost
The authenticity of host 'localhost (127.0.0.1)' can't be established.
ECDSA key fingerprint is a6:34:ed:64:8b:7b:2d:6e:6e:0c:97:c3:dc:33:ba:ae.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
hadoop@localhost's password: //输入你Linux的登陆用户的密码
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)
* Documentation: https://help.ubuntu.com/
277 packages can be updated.
183 updates are security updates.

上一步的登录需要输入密码，我们需要设置成无密登录。
首先退出刚才的 ssh，就回到了我们原先的终端窗口，然后利用 ssh-keygen 生成密钥，并将密钥加入到授权中：

hadoop@ubuntu:~$ exit
logout
Connection to localhost closed.
hadoop@ubuntu:~$ cd ~/.ssh/
hadoop@ubuntu:~/.ssh$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): //回车
Enter passphrase (empty for no passphrase)://回车
Enter same passphrase again: //回车
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
f4:6b:33:97:43:2d:e4:f0:96:ca:e9:79:b2:f6:51:6c hadoop@ubuntu
The key's randomart image is:
+--[ RSA 2048]----+
| |
| |
| . . . |
| . . = + |
| S . B E |
| . * = |
| O = |
| oo=.o |
| .+=. |
+-----------------+
hadoop@ubuntu:~/.ssh$ cat ./id_rsa.pub >> authorized_keys
hadoop@ubuntu:~/.ssh$

#再次用ssh localhost登录，就不需要输入密码了，如下所示

hadoop@ubuntu:~/Desktop$ ssh localhost
Welcome to Ubuntu 17.10 (GNU/Linux 4.13 .0-21-generic x86_64)

5. 开始执行

格式化节点

$ bin/hadoop namenode -format（在bin外面的目录执行）

开启Hadoop进程

$ bin/start-all.sh

错误：

chown: changing ownership of '/home/root1/Hadoop/hadoop-1.2.1/libexec/../logs': Operation not permitted
starting namenode, logging to /home/root1/Hadoop/hadoop-1.2.1/libexec/../logs/hadoop-root1-namenode-ubuntu.out
/home/root1/Hadoop/hadoop-1.2.1/bin/hadoop-daemon.sh: line 137: /home/root1/Hadoop/hadoop-1.2.1/libexec/../logs/hadoop-root1-namenode-ubuntu.out: Permission denied

这是因为你的用户权限不够，此时可以使用sudo chown -hR Eddie（当前用户名） hadoop-xxx(当前版本)增加权限

也能去外面把hadoop目录下的文件夹全部变成“平民文件夹”啊哈哈sudo chmod 777 -R ./hadoop-1.2.1（/后面的就是你要改变文件夹的子文件）

再次执行bin/start-all.sh 即可成功。

jps

6847 SecondaryNameNode
6517 NameNode
6935 JobTracker
7123 TaskTracker
6310 RunJar
7282 Jps
6682 DataNode

上面是伪分布式

单机版本的：

默认情况下，Hadoop配置是以非分布式模式运行，作为一个Java进程，利于调试

The following example copies the unpacked conf directory to use as input and then finds and displays every match of the given regular expression. Output is written to the given output directory.
$ mkdir input
$ cp conf/*.xml input
$ bin/hadoop jar hadoop-examples-*.jar grep input output 'dfs[a-z.]+'
$ cat output/*

问题：

INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
18/10/17 23:34:20 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
待解决。晚安

Hadoop-1.2.1/1.0.1 install on Ubuntu

猜你喜欢