Hive报错, [Fatal Error] total number of created files now is 100028, which exceeds 100000.

版权声明:本文为博主原创文章,转载请说明出处 https://blog.csdn.net/u010002184/article/details/88572872

问题1:

 Hive Dynamic partition Error : [Fatal Error] total number of created files now is 100028, which exceeds 100000. Killing the job.

Solution:

So my config increases the default partitions and files created limit:

 set hive.exec.dynamic.partition=true;
 set hive.exec.max.dynamic.partitions=100000;
 SET hive.exec.max.dynamic.partitions.pernode=100000;
 set hive.exec.dynamic.partition.mode=nonstrict;
 SET hive.exec.max.created.files=900000;

转自:https://confusedcoders.com/data-engineering/etl/hive/hive-dynamix-partition-error-fatal-error-total-number-of-created-files-now-is-100000-which-exceeds-100000-killing-the-job

转自:https://blog.csdn.net/Post_Yuan/article/details/62887619

猜你喜欢

转载自blog.csdn.net/u010002184/article/details/88572872
今日推荐