Ambari 2.7.3 source code build and installation issues

Others 2021-02-28 02:30:56 views: null

table of Contents

Introduction:

Installation Environment:

Build tool installation:

Source code construction and installation: 

Build issues:

reference:

Introduction:

The Apache Ambari project mainly develops software to provide, manage and monitor Hadoop clusters to make it easier to manage Hadoop clusters. Ambari provides an intuitive and easy-to-use web UI for Hadoop management by using RESTful API. ( Official website link )

Installation Environment:

  • Centos ： 7.6
  • Mysql: 5.7 (note that it is not recommended to use the Mysql8.0  reference link )

Build tool installation:

Since the downloaded Ambari is the source code ( download link ), it needs to be installed and used after the build. The build process needs to install some tools ( refer to the link ).

  •  JDK8(Ambari 2.6 and below can be compiled with JDK 7, from Ambari 2.7, it can be compiled with at least JDK 8)

   #Check the version information after installation

# java -version   
# mvn -v
  • Python 2.7 (Ambari 2.7 or later require Python 2.7 as minimum supported version)
#  python -V
  • Python setuptools:

    for Python 2.6: Download setuptools and run:

# sh setuptools-0.6c11-py2.6.egg

   for Python 2.7:  Download setuptools and run:

# sh setuptools-0.6c11-py2.7.egg
  • rpmbuild (rpm-build package)
# yum install -y rpm-build
  • g++ (gcc-c++ package)
# yum install -y gcc-c++
  • Node.js 

    Here you need to download from the official website (link) ( Linux Binaries (x64) ) to install, and then manually configure environment variables and paths;

# vim /etc/profile

 Add the following two lines:
 export NODEJS=/***/***/node-v10.16.0-linux-x64 #Extract the path of NodeJS
 export PATH=:$NODEJS/bin:$PATH

# source /etc/profile

   After configuration, check the node version and npm version separately

# node -v

# npm -v
  • Brunch

   Install Brunch with npm

# npm install -g brunch

   Check the version information after installation:

# brunch -v
  • Modify the /etc/profile file and set the JVM parameters, otherwise OutOfMemoryError will easily occur during compilation. 

   Open /etc/profile (# vim /etc/profile) and add the following code to the last line:

export _JAVA_OPTIONS="-Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true"

Source code construction and installation: 

Reference connection

1. Download the source code

Select download mirror ( link )

# wget https://mirrors.tuna.tsinghua.edu.cn/apache/ambari/ambari-2.7.3/apache-ambari-2.7.3-src.tar.gz
# tar xfvz apache-ambari-2.7.3-src.tar.gz
# cd apache-ambari-2.7.3-src
# mvn versions:set -DnewVersion=2.7.3.0.0
 
# pushd ambari-metrics
# mvn versions:set -DnewVersion=2.7.3.0.0
# popd

2. Build the source code

# mvn -B clean install rpm:rpm -DnewVersion=2.7.3.0.0 -DbuildNumber=4295bb16c439cbc8fb0e7362f19768dde1477868 -DskipTests -Dpython.ver="python >= 2.6"

Note: This process generally takes a long time. Because downloading the dependent package is time-consuming, it may take a day or two. It is recommended to build in modules, which will also encounter many problems. If you encounter specific problems, it is recommended to refer to link 1 and link 2 . Here are a few problems encountered in the following construction problems.

3. Ambari Server installation

When the build is successful, the ambari-server/target/rpm/ambari-server/RPMS/noarch/ directory or ambari-server/target/rpm/ambari-server/RPMS/x86_64/ will generate ambari-server-2.7.3.0- 0.x86_64.rpm 

# yum install ambari-server*.rpm

4. Configure and start Ambari Server

Run the ambari-server setup command and follow the on-screen command prompts step by step to configure Ambari Server, Database, JDK, LDAP and other options:

ambari-server setup

The configuration process is as follows:

# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? n
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (3): 3
Hostname (localhost): 
Port (3306): 
Database name (ambari): 
Username (ambari): 
Enter Database Password (***): 
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
.
Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it...
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

Note: Here you need to import this file into mysql according to the prompted directory /home/my_Ambari/apache-ambari-2.7.3-src/ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql (enter After mysql creates the database, source /home/my_Ambari/apache-ambari-2.7.3-src/ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql) to complete the mysql table configuration.

After the configuration is successful, start Ambari Server:

# ambari-server start

5. Install and start Ambari Agent on all hosts 

After the build is successful, ambari-agent-2.7.3.0 will be generated under the /home/***/apache-ambari-2.7.3-src/ambari-agent/target/rpm/ambari-agent/RPMS/x86_64/ directory -0.x86_64.rpm file, copy this file to the host that needs to be installed, install Ambari Agent,

# yum install ambari-agent*.rpm

After the installation is complete, configure the host address and open the /etc/ambari-agent/conf/ambari-agent.ini (vim /etc/ambari-agent/conf/ambari-agent.ini) file:

[server]
#配置 Ambari-Server 安装的主机
hostname=172.16.18.103    
url_port=8440
secured_url_port=8441
connect_retry_delay=10
max_reconnect_retry_delay=30

Start ambari-agent after configuration 

# ambari-agent start

6. Deploy the cluster through Ambari Web UI

Enter http://<ambari-server-host>:8080 in the browser.

Enter username: admin password: admin

If the following page is displayed, the configuration is successful.

Build issues:

Question 1.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.12:check (default) on project ambari: Too many files with unapproved license: 1 See RAT report in: /home/***/Apache_Ambari/apache-ambari-2.7.3-src/target/rat.txt -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

solve:

Added in the mvn command: -Drat.skip=true

 

Question 2.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (psutils-compile) on project ambari-metrics-host-monitoring: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec failοnerrοr="true" dir="/home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/psutil" executable="/home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/../../ambari-common/src/main/unix/ambari-python-wrap">... @ 4:325 in /home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/target/antrun/build-psutils-compile.xml
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :ambari-metrics-host-monitoring
解决：

This problem usually occurs because the previous download of a certain software has timed out, so you need to manually download the jar package that was downloaded over time and put it in the path specified during download.

 

Question 3.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project ambari-server: Compilation failure
[ERROR] An unknown compilation problem occurred
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :ambari-server

solve:

This is because the compiler version and Jvm running version specified in the pom.xml file are inconsistent with the JAVA and JVM versions installed in the installation environment. The Java version installed on this machine is 1.8, so you need to modify the corresponding version in the pom.xml file to be consistent. Modify The following parts:

<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.2</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
        </configuration>
</plugin>

 

 Question 4.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-yarn (install node and yarn) on project ambari-web: The plugin com.github.eirslett:frontend-maven-plugin:1.4 requires Maven version 3.1.0 -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginIncompatibleException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :ambari-web

solve:

The maven version is too low, you need to manually download the new version of Maven and configure the environment variables.

 

Question 5.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (phoenix_download) on project ambari-metrics-timelineservice: An Ant BuildException has occured: java.net.SocketException: Connection reset
[ERROR] around Ant part ...<get usetimestamp="true" src="http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/phoenix/phoenix-5.0.0.3.0.0.0-1634.tar.gz" dest="/home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/phoenix.tar.gz"/>... @ 5:301 in /home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/antrun/build-Download Phoenix.xml Solution:
[ERROR] -> [Help 1]

This is due to the timeout of downloading phoenix-5.0.0.3.0.0.0-1634.tar.gz when building the ambari-metrics-timelineservice module.

On the one hand, download the file manually through the specified link, http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/phoenix/phoenix-5.0. 0.3.0.0.0-1634.tar.gz, and modify the name (phoenix.tar.gz) as required and put it in /home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ ambari-metrics-timelineservice/target/embedded/ path.

On the other hand, modify the mirror image of the mvn library ( reference 5 ):

Edit mvn's settings.xml file, generally in the ~/.m2/settings.xml directory or, home/Maven/apache-maven-3.6.1/conf/settings.xml under the mvn installation directory, add mirror http://repo .hortonworks.com/content/groups/public/   and  http://nexus-private.hortonworks.com/nexus/content/groups/public  .

<settings>
         <mirrors>
            <mirror>
               <id>public</id>
               <mirrorOf>*</mirrorOf>
               <url>http://nexus-private.hortonworks.com/nexus/content/groups/public</url>
            </mirror>
         </mirrors>
</settings>

 

Question 6.

[INFO] Downloading http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/hbase/hbase-2.0.0.3.0.0.0-1634-bin.tar.gz  to /home/my_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz

 

[get] Getting: http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/hbase/hbase-2.0.0.3.0.0.0-1634-bin.tar.gz 
[get] To: /home/my_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz

solve:

This is the same as problem five. It is because the downloading file is slower and stuck here. The solution is to manually specify the connection to download the file, and then modify the name and place it in the specified directory.

 

reference:

1. https://www.cnblogs.com/barneywill/p/10264135.html

2. https://blog.csdn.net/ZhouyuanLinli/article/details/88963289

3. https://blog.csdn.net/slx_2011/article/details/45038089

4.https://blog.csdn.net/guoyachuan01/article/details/39500575

5.https://community.hortonworks.com/questions/157149/apache-ambari-error-during-maven-build-missing-ins.html

Guess you like

Origin blog.csdn.net/abcdu1/article/details/90710851
Recommended
The “35-year-old curse” of Chinese coders
蘭雅 CorelDRAW 插件 2024.5.1 国际劳动节版，免费下载
Ranking
Methods and Practices of Manufacturing Data Quality Improvement
Serial port upgrade program for efficient transmission of large firmware
Use KITTI to run LIOSAM and complete the EVO evaluation
Calling methods on string literals (Java)
ActiveMQ and Spring integration (4)
You have to know the HTTPS! ! !
PAT whether Structures and Algorithms 7-4 with a binary search tree (40 rows streamlined structure)
el-upload brings request background
UVA - 1204 Fun Game-like pressure dp
2019-12-28
Daily
More
2024-05-04(18)
2024-05-03(8)
2024-05-02(0)
2024-05-01(4)
2024-04-30(36)
2024-04-29(5)
2024-04-28(12)
2024-04-27(29)
2024-04-26(22)
2024-04-25(32)

Code World

© 2018 codetd.com