background:
An MR task scan hbase exports the hive file, but finds that the data suddenly becomes smaller in some days, and the number of data rows in it also becomes smaller (the data in hbase only increases but does not decrease)
Reason location:
The log is very hidden, you need to go to the MR task to check the log log in the mapper, and find that there is an error log for further positioning
It is found that there are too many mysql connections, you can see the following code is very critical (this is to read the configuration information of mysql every time)
If the number of mappers is large, you need to connect to mysql multiple times, resulting in too many connections
Fix:
Read the mysql configuration information in the main function, and then pass it to the mapper through parameters, so as to avoid reading mysql configuration information every time the mapper is used.
