Spark2 SQL dynamic partitioning error

Given the information as follows:

ERROR yarn.ApplicationMaster: User class threw exception: org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: Number of dynamic partitions created is 3464, which is more than 1000. To solve this try to set hive.exec.max.dynamic.partitions to at least 3464.;
org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: Number of dynamic partitions created is 3464, which is more than 1000. To solve this try to set hive.exec.max.dynamic.partitions to at least 3464.;
	at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:107)
	at org.apache.spark.sql.hive.HiveExternalCatalog.loadDynamicPartitions(HiveExternalCatalog.scala:829)
	at org.apache.spark.sql.hive.execution.InsertIntoHiveTable.sideEffectResult$lzycompute(InsertIntoHiveTable.scala:319)
	at org.apache.spark.sql.hive.execution.InsertIntoHiveTable.sideEffectResult(InsertIntoHiveTable.scala:221)
	at org.apache.spark.sql.hive.execution.InsertIntoHiveTable.doExecute(InsertIntoHiveTable.scala:413)
	at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114)
	at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114)
	at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:135)

After Finding modify the code to add spark

hiveContext.sql("hive.exec.dynamic.partition=true")
hiveContext.sql("set hive.exec.dynamic.partition.mode=nonstrict")
hiveContext.sql("SET hive.exec.max.dynamic.partitions=100000")
hiveContext.sql("SET hive.exec.max.dynamic.partitions.pernode=100000")

Still an error. .
https://github.com/apache/spark/pull/17223
through online tips, spark2 code modification hive does not support the configuration information later. .

So attempt to modify the hive-site.xml, add the following configuration:

<property>
      <name>hive.exec.dynamic.partition</name>
      <value>true</value>
    </property>
  <property>
      <name>hive.exec.dynamic.partition.mode</name>
      <value>nonstrict</value>
  </property>
  <property>
      <name>hive.exec.max.dynamic.partitions</name>
      <value>100000</value>
   </property>
   <property>
      <name>hive.exec.max.dynamic.partitions.pernode</name>
      <value>100000</value>
  </property>

Hard work pays off, finally running success!