Hue is an open source Apache Hadoop UI system. It was first evolved from Cloudera Desktop and contributed to the open source community by Cloudera. It is implemented based on the Python web framework Django. By using Hue, we can interact with the Hadoop cluster on the browser-side web console to analyze and process data, such as manipulating data on HDFS, running MapReduce Jobs, and so on. Feature set supported by Hue: 1. Manage session data, user authentication and authorization based on lightweight sqlite database, which can be customized to MySQL, Postgresql, and Oracle, etc.; 2. Access HDFS based on File Browser ; 3. Develop and run Hive queries based on the Hive editor; 4. Support applications based on Solr for search, and provide visual data views and dashboards (Dashboard); 5. Support applications based on Impala for interactive queries; 6. Support Spark editor and dashboard (Dashboard); 7. Support Pig editor, and be able to submit script tasks; 8. Support Oozie editor, can submit and monitor Workflow, Coordinator and Bundle through the dashboard; 9. Support HBase Browser, which can visualize data, query data, and modify HBase tables; 10. Support Metastore browser, which can access Hive metadata and HCatalog; 11. Support Job browser, which can access MapReduce Job (MR1/MR2-YARN); 12. Support Job Designer, able to create MapReduce/Streaming/Java Job; 13. Support Sqoop 2 editor and Dashboard; 14. Support ZooKeeper browser and editor; 15. Support MySql, PostGresql, Sqlite and Oracle Database query editor
The installation of Hue is not so simple, because the official version is not compiled, we need to download the source code from github, install the dependencies, compile and install it ourselves, the most difficult thing in the compilation process is that various dependencies are missing and errors are reported. This article will actually come to The complete operation process of downloading - compiling - installing - configuring - running hue
1. Environmental preparation
1. CentOS7.2 system (virtual machine)
2.hue-4.1 source package
Download address: https://codeload.github.com/cloudera/hue/zip/branch-4.1
3. maven environment
2. Compilation environment
We install the dependencies needed to compile hue in the centos system:
yum install krb5-devel cyrus-sasl-gssapi cyrus-sasl-deve libxml2-devel libxslt-devel mysql mysql-
devel openldap-devel python-devel python-simplejson sqlite-devel
The above required dependencies are installed, and we will install the required dependencies during the compilation process.
Compiling hue also requires the maven environment, so you need to install maven before compiling, and the Maven installation previously built in the << Linux development environment
Configuration >> has been practiced, please refer to the practice process here
3. Compilation process
1. Upload hue source package
This article is carried out in the centos environment of the virtual machine. We first upload the hue source package to the virtual machine system
2. Unzip the source package
Since the source package is downloaded from github, the downloaded format is zip, we use the unzip command to unzip it
Note: If there is no unzip service, use yum install unzip to install
3. Start compiling
We enter the hue source directory and use make apps to compile
4. Compilation problem solving
1>. Solve the maven permission problem
When compiling, it prompts that maven has insufficient permissions
Solution: Enter the bin directory of maven and modify the mvn permissions to chmod 777 mvn
2>.gcc command error
Compilation prompts commod gcc command error
Solution: yum install gcc
3>.ffi.h fatal error
Missing ffi.h fatal error when compiling
Solution: yum install libffi-devel openssl-devel
4>.gmp.h fatal error
Missing gmp.h fatal error when compiling
Solution: yum install gmp-devel
5>. Missing execvp error
Missing execvp error when compiling
Solution: yum install gcc-c++
5. Compilation is complete
After the above steps, hue is compiled. After the compilation is completed, there is an app.reg file and a build folder in the hue directory.
Complete dependencies:
yum install krb5-devel cyrus-sasl-gssapi cyrus-sasl-deve libxml2-devel libxslt-devel mysql mysql-
devel openldap-devel python-devel python-simplejson sqlite-devel libffi-devel openssl-devel gmp-
devel gcc gcc-c++
Fourth, start verification
1. Start
Go to the compilation directory cd build/env/bin/ and enter the startup command: ./hue runserver 0.0.0.0:8000, note that if you do not add
The 0.0.0.0:8000 parameter means that only the machine can access it, or use the ./supervisor command to start
At this point the hue service has been started and completed
2. Verify
Enter 192.168.1.50:8000 on the computer browser. If you are logging in for the first time, you need to create a user first.
V. Summary
This article mainly practices the compilation and installation process of hue. In the following articles, we will continue to in-depth hue operation and integration of hdfs, hive,
hbase, mysql, etc.