1. Problem description:
On eclipse, the mapper, reducer, and main method classes are automatically generated through the hadoop plug-in to achieve wordcount statistics.
Pending file content:
hello tom
hello joy
hello rose
hello joy
hello jerry
hello tom
hello rose
hello joy
After testing several times, the processing results are:
hello 1
hello 1
hello 1
hello 1
hello 1
hello 1
hello 1
hello 1
jerry 1
joy 1
joy 1
joy 1
rose 1
rose 1
tom 1
tom 1
The result of counting the number of occurrences by word has not been achieved.
2. Problem Analysis
1. From the analysis of your results above, the reducer does not work at all, but the program can run without error
2. Add some corresponding print statements in the reducer, and there is no output. It is confirmed that the reducer is not executed. After analyzing for a long time, the problem cannot be found.
3. Searched the Internet and found the problem
https://blog.csdn.net/world_java/article/details/17427631
3. Problem solving
1. Problematic reducer class definition, no error is reported (but the default reduce method is actually called)
2. After adding @override to the reduce method , an error occurs
3. After checking, it is found that there is a problem with the parameters of the Reducer Iterable<LongWritable>, after modifying it to LongWritable, the error is eliminated
4. Execution results
hello 8
jerry 1
joy 3
rose 2
tom 2
It is recommended to add @override when overriding methods , so that overriding errors can be detected.