HUE3.9.0详细安装文档
友情提示，安装任何LUW软件之前，务必先阅读readme自带文档，里面有很多关键信息，不要拿着就开搞，浪费半天时间发现一大堆错误。
4.2 修改配置文件hue.ini（可参考为知笔记）... 7
5.3 删除desktopdb.json中所有 useradmin.userprofile 的对象，以下是例子... 14
5.4.10 Load any backed up data:22
5.4.11(InnoDB only) Drop the foreign key:23
6.2filelists.sqlite.bz2 from base: [Errno -1] 的解决方法... 25
6.3 Cannotretrieve repository metadata (repomd.xml) for repository:xxxxxx. 26
1 环境准备
CentOS-6.8 (Final)
JDK-1.8
maven-3.3.9
Git-1.7.1
Hue-3.9.0
Hadoop-2.7.3
Hive-1.2.1
Python-2.6.6
2 启动HIVE后台进程(确保HIVE安装OK)
hive --service metastore &
hive --service hiveserver2 &
同时用 ps -ef|grep hive检查是否启动成功
3 安装OS依赖包
yum install 以下包：
ant
asciidoc
cyrus-sasl-devel
cyrus-sasl-gssapi
cyrus-sasl-plain
gcc
gcc-c++
gmp-devel
krb5-devel
libffi-devel
libtidy
libxml2-devel
libxslt-devel
make
mvn
mysql
mysql-devel
openldap-devel
openssl-devel
python-devel
sqlite-devel
确保安装完毕，再做后续操作，否则会抱各种错误
4 编译安装HUE
4.1 编译Hue(保持服务器网络畅通)
chown -R hadoop:Hadoop hue-3.9.0
cd hue-3.9.0
make apps
不报错则表示编译OK
4.2 修改配置文件hue.ini（可参考为知笔记）
上述过程如果没有任何问题，我们就已经安装好Hue,Hue的配置文件为/***_DATA/hadoop/hue-3.9.0/desktop/conf/hue.ini，默认的配置文件不能正常运行Hue，所以需要修改其中的内容，与我们对应的Hadoop集群配置相对应。该配置文件根据整合不同的软件，将配置分成多个段，每个段下面还有子段，便于管理配置，如下所示（省略子段名称）：
· desktop
· libsaml
· libopenid
· liboauth
· librdbms
· hadoop
· filebrowser
· liboozie
· oozie
· beeswax
· impala
· pig
· sqoop
· proxy
· hbase
· search
· indexer
· jobsub
· jobbrowser
· zookeeper
· spark
· useradmin
· libsentry
4.3 hue.ini关键项说明
我们很容易根据需要来配置自己需要的内容。我们修改配置文件的情况，如下表所示：
| Hue配置段 |
Hue配置项 |
Hue配置值 |
说明 |
| desktop |
default_hdfs_superuser |
hadoop |
HDFS管理用户 |
| desktop |
http_host |
***-dev-hadoop-namenode-1.novalocal |
Hue Web Server所在主机/IP |
| desktop |
http_port |
8088 |
Hue Web Server服务端口 |
| desktop |
server_user |
hadoop |
运行Hue Web Server的进程用户 |
| desktop |
server_group |
hadoop |
运行Hue Web Server的进程用户组 |
| desktop |
default_user |
hue |
Hue管理员 |
| hadoop/hdfs_clusters |
fs_defaultfs |
hdfs://ns1:8020 |
对应core-site.xml配置项fs.defaultFS |
| hadoop/hdfs_clusters |
hadoop_conf_dir |
/***_DATA/hadoop/hadoop-2.7.3/etc/hadoop |
Hadoop配置文件目录 |
| hadoop/yarn_clusters |
resourcemanager_host |
hadoop6 |
对应yarn-site.xml配置项yarn.resourcemanager.hostname |
| hadoop/yarn_clusters |
resourcemanager_port |
8032 |
ResourceManager服务端口号 |
| hadoop/yarn_clusters |
resourcemanager_api_url |
http://***-dev-hadoop-namenode-1.novalocal:8088 |
对应于yarn-site.xml配置项yarn.resourcemanager.webapp.address |
| hadoop/yarn_clusters |
proxy_api_url |
http://***-dev-hadoop-namenode-1.novalocal:8888 |
对应yarn-site.xml配置项yarn.web-proxy.address |
| hadoop/yarn_clusters |
history_server_api_url |
http://***-dev-hadoop-namenode-1.novalocal:19888 |
对应mapred-site.xml配置项mapreduce.jobhistory.webapp.address |
| beeswax |
hive_server_host |
***-dev-hadoop-namenode-1.novalocal |
Hive所在节点主机名/IP |
| beeswax |
hive_server_port |
10000 |
HiveServer2服务端口号 |
| beeswax |
hive_conf_dir |
/usr/local/hive/conf |
Hive配置文件目录 |
4.4 启动Hue
export HUE_HOME=/***_DATA/hadoop/hue-3.9.0，确保已经配置HUE_HOME
进入Hue主目录执行：nohup supervisor &
启动无异常则OK
4.5 登录HUE
http://172.16.33.231:8000，第一次登录输入hadoop的用户名和密码，进去后会进行配置文件检查
然后用HUE账户登录后可以新增组和用户
4.6 执行HIVE查询语句
5 HUE元数据库改为Mysql
5.1 停止HUE服务
Ps -ef|grep hue 然后kill掉
5.2 备份sqllite数据库
hue <HUE_HOME>/build/env/bin/huedumpdata > desktopdb.json
5.3 删除desktopdb.json中所有 useradmin.userprofile 的对象，以下是例子
{
"pk": 1,
"model": "useradmin.userprofile",
"fields": {
"creation_method": "HUE",
"user": 1,
"home_directory": "/user/alice"
}
},
{
"pk": 2,
"model": "useradmin.userprofile",
"fields": {
"creation_method": "HUE",
"user": 1100714,
"home_directory": "/user/bob"
}
},
.....
5.4 安装配置Mysql数据库
5.4.1 安装MYSQL客户端
| OS |
Command |
| RHEL |
$ sudo yum install mysql-devel |
| SLES |
$ sudo zypper install mysql-devel |
| Ubuntu or Debian |
$ sudo apt-get install libmysqlclient-dev |
5.4.2 安装MYSQL连接器
| OS |
Command |
| RHEL |
$ sudo yum install mysql-connector-java |
| SLES |
$ sudo zypper install mysql-connector-java |
| Ubuntu or Debian |
$ sudo apt-get install libmysql-java |
5.4.3 安装MYSQL服务器
| OS |
Command |
| RHEL |
$ sudo yum install mysql-server |
| SLES |
$ sudo zypper install mysql $ sudo zypper install libmysqlclient_r17 |
| Ubuntu or Debian |
$ sudo apt-get install mysql-server |
5.4.4 配置 /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
bind-address=<ip-address>
default-storage-engine=InnoDB
sql_mode=STRICT_ALL_TABLES
5.4.5 启动MySQL服务
| OS |
Command |
| RHEL |
$ sudo service mysqld start |
| SLES and Ubuntu or Debian |
$ sudo service mysql start |
5.4.6 修改密码（此步可不做）
$ sudo /usr/bin/mysql_secure_installation
[...]
Enter current password for root (enter fornone):
OK, successfully used password, moving on...
[...]
Set root password? [Y/n] y
New password:
Re-enter new password:
Remove anonymous users? [Y/n] Y
[...]
Disallow root login remotely? [Y/n] N
[...]
Remove test database and access to it [Y/n] Y
[...]
Reload privilege tables now? [Y/n] Y
All done!
5.4.7 配置MySQL开机自启
| OS |
Command |
| RHEL |
$sudo /sbin/chkconfig mysqld on ssudo /sbin/chkconfig --list mysqld mysqld 0:off 1:off 2:on 3:on 4:on 5:on 6:off |
| SLES |
$ sudo chkconfig --add mysql |
| Ubuntu or Debian |
$ sudo chkconfig mysql on |
5.4.8 创建数据库hue
mysql> create database hue;
5.4.9 给hue授权
mysql> grant all on hue.* to'hue'@'localhost' identified by '<secretpassword>';
配置hue.ini touse MySQL. 修改以下选项
host=***-dev-hadoop-namenode-1.novalocal
port=3306
engine=mysql
user=hue
password=hue
name=hue
5.4.10 Load any backed up data:
- Ensure a logs directory exists and is writable at /opt/cloudera/parcels/CDH/lib/hue/logs.
- Ensure the logs directory has hue user and group ownership.
- Synchronize and migrate the database.
$sudo mkdir /opt/cloudera/parcels/CDH/lib/hue/logs
$sudo chown hue:hue /opt/cloudera/parcels/CDH/lib/hue/logs
$sudo -u hue <HUE_HOME>/build/env/bin/hue syncdb --noinput
$sudo -u hue <HUE_HOME>/build/env/bin/hue migrate
$mysql -u hue -p <secretpassword>
mysql> SHOW CREATE TABLE auth_permission;
5.4.11 (InnoDBonly) Drop the foreign key:
mysql> ALTER TABLE auth_permission DROP FOREIGN KEYcontent_type_id_refs_id_XXXXXX;
a. Delete the rowsin the django_content_type table:
mysql> DELETE FROM hue.django_content_type;
b. Load the data:
$<HUE_HOME>/build/env/bin/hue loaddata <some-temporary-file>.json
c. (InnoDB only)Add the foreign key:
$mysql -u hue -p <secretpassword>
mysql> ALTER TABLE auth_permission ADD FOREIGN KEY (`content_type_id`)
REFERENCES`django_content_type` (`id`);
5.4.12 再次启动HUE
nohup supervisor &
无错误信息则正常
6 FAQ帮助
6.1 Mysql报错ERROR 1045
| 错误：ERROR 1045 (28000): Access denied for user ‘root’@’localhost’ (using password: NO) |
| 解决步骤： |
| #1.停止mysql数据库 /etc/init.d/mysqld stop #2.执行如下命令 mysqld_safe --user=mysql --skip-grant-tables --skip-networking & #3.使用root登录mysql数据库 mysql -u root mysql #4.更新root密码 mysql> UPDATE user SET Password=PASSWORD('hadoop') where USER='root'; #5.刷新权限 mysql> FLUSH PRIVILEGES; #6.退出mysql mysql> quit #7.重启mysql /etc/init.d/mysqld restart #8.使用root用户重新登录mysql mysql -uroot -p Enter password: hadoop |
6.2 filelists.sqlite.bz2from base: [Errno -1] 的解决方法
Centos下更新出现 ：http://mirror.averse.net/centos/5.7/updates/i386/repodata/filelists.sqlite.bz2:[Errno -1] Metadata file does not match checksum
用下面方法解决：
yumclean all
yummakecache
6.3 Cannotretrieve repository metadata (repomd.xml) for repository:xxxxxx
这个错误其实很简单，错误信息已经提示你，就是xxxxxx.repo这个文件有问题。
1. 打开/etc/yum.repos.d/xxxxxx.repo文件
2. enabled=1改成enabled=0
