Problem Description:
java.sql.SQLException: EXECUTION FAILED: Task MAPRED-SPARK error SparkException: [Error 1] Topo check failed. Mapred tasks exceed 1000000000
Root Cause:
数据量太大导致,需要设小reduce个数
Solution:
通过设置参数set mapred.reduce.tasks=xx;
Problem Description:
java.sql.SQLException: EXECUTION FAILED: Task MAPRED-SPARK error SparkException: [Error 1] Topo check failed. Mapred tasks exceed 1000000000
Root Cause:
数据量太大导致,需要设小reduce个数
Solution:
通过设置参数set mapred.reduce.tasks=xx;