Kaitian Creation
(1) Draw a picture to explain the spark workflow and the correspondence between each role on the cluster
(2) Which Spark operator operations involve shuffle
(3) What are the main methods for spark performance optimization
(4) What are the common problems when the map-reduce program is running?
(5) In the shuffle process of hadoop and spark, how do you avoid some performance problems when programming?
(6) What is the role of Hadoop's TextInputFormat and how to customize it
(7) What data is skewed and how to solve it
(8) There are order data tables server_id (server ID), role_id (role ID)/money (recharge amount) to count the top 3 role IDs and total recharge amounts in each server, using HIVE SQL and Spark operators respectively achieve