Overview:
Use wordcount to do hadoop performance test, and conduct performance analysis and evaluation according to the growth of count data scale
Version:
bin/hadoop version
Hadoop 2.3.0-cdh5.0.0
Test steps:
1. Use randomtextwriter to generate a test set of a specified size
2. Execute wordcount:
nohup bin/hadoop jar share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.3.0-cdh5.0.0.jar wordcount /home/test/mrinput50 /home/test/mroutput50 > wc.log 2>&1 &
3. Evaluation content:
Total time spent by all maps in occupied slots (ms)=1504892 Total time spent by all reduces in occupied slots (ms)=84038 Total time spent by all map tasks (ms)=1504892 Total time spent by all reduce tasks (ms)=84038 GC time elapsed (ms)=17285 CPU time spent (ms)=1812107