hiveMetastore元数据库初始化失败:java.io.IOException: Schema script failed, errorcode 2

版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/liaoyanyunde/article/details/83116394
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py", line 259, in <module>
    HiveMetastore().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py", line 59, in start
    self.configure(env)
  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py", line 73, in configure
    hive(name = 'metastore')
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk
    return fn(*args, **kwargs)
  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py", line 320, in hive
    user = params.hive_user
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 273, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 71, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 93, in checked_call
    tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 141, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 294, in _call
    raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'export HIVE_CONF_DIR=/usr/hdp/current/hive-metastore/conf/conf.server ; /usr/hdp/current/hive-metastore/bin/schematool -initSchema -dbType mysql -userName hive -passWord [PROTECTED] -verbose' returned 1. WARNING: Use "yarn jar" to launch YARN applications.
Metastore connection URL:	 jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :	 org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User:	 hive
Starting metastore schema initialization to 1.2.1000
Initialization script hive-schema-1.2.1000.mysql.sql
Connecting to jdbc:derby:;databaseName=metastore_db;create=true
Connected to: Apache Derby (version 10.10.2.0 - (1582446))
Driver: Apache Derby Embedded JDBC Driver (version 10.10.2.0 - (1582446))
Transaction isolation: TRANSACTION_READ_COMMITTED
0: jdbc:derby:> !autocommit on
Autocommit status: true
0: jdbc:derby:> /*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */ 
Error: Syntax error: Encountered "<EOF>" at line 1, column 64. (state=42X01,code=30000)

Closing: 0: jdbc:derby:;databaseName=metastore_db;create=true
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:285)
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:258)
	at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:508)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: Schema script failed, errorcode 2
	at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:391)
	at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:348)
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:281)
	... 8 more
*** schemaTool failed ***

分析错误可以看到,是因为执行sql脚本 hive-schema-1.2.1000.mysql.sql中的sql报语法错误。开始以为是因为sql脚本的版本跟我们所使用的mysql数据库的版本不匹配导致,结果将 hive-schema-1.2.1000.mysql.sql中/*!40101……这种开头的sql语句都删除了,重启hiveMetastore,依然报sql语法错误。此时便可排除mysql版本的原因。

继续看错误日志:
错误是执行/usr/hdp/current/hive-metastore/bin/schematool -initSchema -dbType mysql -userName hive -passWord [PROTECTED] -verbose这个指令时触发的,看了下下面打印的日志:

Metastore connection URL:	 jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :	 org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User:	 hive

在这里插入图片描述
发现连接元数据库的databaseName用的居然不是我们传入的hive,而是metastore_db;
驱动Driver也不是mysql的驱动,而是本地derby数据库的驱动org.apache.derby.jdbc.EmbeddedDriver

说明虽然我们调用schematool 指令传入的数据类型是mysql,传入的数据库名也是mysql的,但实际调用却是derby数据库,所以才会因为在derby数据库上执行mysql的sql脚本文件而一直包sql语法错误。

第一想到的就是我们的配置文件hive-site.xml有问题,检查配置正确,关于metastore的配置均是mysql。

后来发现在/etc/haddoop/conf下还有一个hive-site.xml,果然,该配置文件没有任何关于matastroe的配置项,将/etc/hive/conf/hive-site.xml拷贝过来替换这个文件,解决问题

PS:derby数据库是默认的hiveMetaStore数据库类型,如果我们不指定使用mysql或者oracle等其他数据库,则默认使用derby数据库。

所以,可以分析出,初始化hiveMetaStore数据库时,拼接执行指令/usr/hdp/current/hive-metastore/bin/schematool -initSchema -dbType mysql -userName hive -passWord [PROTECTED] -verbose用的是/etc/hive/conf/hive-site.xml,但是提交到yarn中去执行指令的时候,会再次读取/etc/haddoop/conf/hive-site.xml中的配置项。

猜你喜欢

转载自blog.csdn.net/liaoyanyunde/article/details/83116394
今日推荐