pyspark mongodb问题

问题1:
root 用户运行pyspark 没问题
hadoop 用户运行就报如下错

df1 = spark.read.format("com.mongodb.spark.sql.DefaultSource").option("collection","devices").load()
17/02/15 19:34:26 WARN MongoInferSchema: Field 'devcaps' contains conflicting types converting to StringType
17/02/15 19:34:27 ERROR PoolWatchThread: Error in trying to obtain a connection. Retrying in 7000ms
java.sql.SQLException: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.setReadOnly(Unknown Source)
at com.jolbox.bonecp.ConnectionHandle.setReadOnly(ConnectionHandle.java:1324)


求解
。。。

以hadoop用户 运行spark-submit 就不报错,原因可能是pyspark 的bug


猜你喜欢

转载自jacklin2015.iteye.com/blog/2357311
今日推荐