1.1 Preliminary preparations
1 ) CentOS networking
Configure CentOS to connect to the external network. Preferably with a freshly cloned virtual machine
Note: Compile with root role to reduce problems with folder permissions
2 ) jar package preparation (hadoop source code, JDK8 , maven , ant , protobuf)
（1）hadoop-2.7.6-src.tar.gz
（2）jdk-8u144-linux-x64.tar.gz
（ 3 ） apache-ant-1.9.9-bin.tar.gz
（ 4 ） apache-maven-3.0.5-bin.tar.gz
（ 5 ） protobuf-2.5.0.tar.gz
1.2 jar package installation _
0 ) Note: All operations must be done under root user
1) Unzip the JDK , configure the environment variables JAVA_HOME and PATH , and verify the java -version ( the following need to verify whether the configuration is successful )
[root@node1 software] # tar -zxf jdk-8u144-linux-x64.tar.gz -C /opt/module/
[root@node1 software]# vi /etc/profile
| #JAVA_HOME export JAVA_HOME=/opt/module/jdk1.8.0_144 export PATH=$PATH:$JAVA_HOME/bin |
[root@node1 software]#source /etc/profile
Verify command: java -version
2) Maven decompresses, configures MAVEN_HOME and PATH .
[root@node1 software]# tar -zxvf apache-maven-3.0.5-bin.tar.gz -C /opt/module/
[root@node1 apache-maven-3.0.5]# vi /etc/profile
| #MAVEN_HOME export MAVEN_HOME=/opt/module/apache-maven-3.0.5 export PATH=$PATH:$MAVEN_HOME/bin |
[root@node1 software]#source /etc/profile
Verify command: mvn -version
3) ant decompression, configure ANT _HOME and PATH .
[root@node1 software]# tar -zxvf apache-ant-1.9.9-bin.tar.gz -C /opt/module/
[root@node1 apache-ant-1.9.9]# vi /etc/profile
| #ANT_HOME export ANT_HOME=/opt/module/apache-ant-1.9.9 export PATH=$PATH:$ANT_HOME/bin |
[root@node1 software]#source /etc/profile
Verify command: ant -version
4) Install glibc-headers and g++ command as follows :
[root@node1 apache-ant-1.9.9]# yum install glibc-headers
[root@node1 apache-ant-1.9.9]# yum install gcc-c++
5) Install make and cmake
[root@node1 apache-ant-1.9.9]# yum install make
[root@node1 apache-ant-1.9.9]# yum install cmake
6) Unzip protobuf , enter the main directory of protobuf after unzipping , /opt/module/protobuf-2.5.0
Then execute the commands one after the other:
[root@node1 software]# tar -zxvf protobuf-2.5.0.tar.gz -C /opt/module/
[root@node1 opt]# cd /opt/module/protobuf-2.5.0/
[[email protected]]#./configure
[root@node1 protobuf-2.5.0]# make
[root@node1 protobuf-2.5.0]# make check
[root@node1 protobuf-2.5.0]# make install
[root@node1 protobuf-2.5.0]# ldconfig
[root@node1 hadoop-dist]# vi /etc/profile
| #LD_LIBRARY_PATH export LD_LIBRARY_PATH=/opt/module/protobuf-2.5.0 export PATH=$PATH:$LD_LIBRARY_PATH |
[root@node1 software]#source /etc/profile
Verify command: protoc --version
7) Install openssl library
[root@node1 software]#yum install openssl-devel
8) Install the ncurses-devel library:
[root@node1 software]#yum install ncurses-devel
At this point, the compilation tool installation is basically complete.
1.3 Compile the source code
1 ) Unzip the source code to the /opt/ directory
[root@node1 software]# tar -zxvf hadoop-2.7.6-src.tar.gz -C /opt/
2) Go to the main directory of hadoop source code
[root@node1 hadoop-2.7.6-src]# pwd
/opt/hadoop-2.7.6-src
3) Execute the compile command through maven
[root@node1 hadoop-2.7.6-src]#mvn package -Pdist,native -DskipTests -Dtar
The waiting time is about 30 minutes , and the final success is all SUCCESS.
4 ) The successful 64 -bit hadoop package is under /opt/hadoop-2.7.6-src/hadoop-dist/target .
[root@node1 target]# pwd
/opt/hadoop-2.7.6-src/hadoop-dist/target
1.4 Common problems and solutions _ _
1) JVM memory overflow during MAVEN install
Processing method: The heap size of MAVEN_OPT can be adjusted in both the environment configuration file and the maven execution file . (For details , see MAVEN compilation JVM tuning issues, such as: http://outofmemory.cn/code-snippet/12652/maven-outofmemoryerror-method )
2) Maven reports an error during compilation . It may be caused by the network blocking problem that the download of the dependent library is incomplete, and the command is executed multiple times (one pass is difficult):
[root@hadoop101 hadoop-2.7.6-src]#mvn package -Pdist,native -DskipTests -Dtar
3) Report errors such as ant and protobuf , the plugin download is incomplete or the plugin version is wrong. There are many special cases in the initial link, and it is recommended
Issue summary post for version 2.7.0 http://www.tuicool.com/articles/IBn63qf