需求:
需要测试机器mr的执行性能,那么可以通过mrbench进行测试
语法:
bin/hadoop jar share/hadoop/mapreduceX/hadoop-test-XXX.jar mrbench [-baseDir <base DFS path for output/input, default is /benchmarks/MRBench>] [-jar <local path to job jar file containing Mapper and Reducer implementations, default is current jar file>] [-numRuns <number of times to run the job, default is 1>] [-maps <number of maps for each run, default is 2>] [-reduces <number of reduces for each run, default is 1>] [-inputLines <number of input lines to generate, default is 1>] [-inputType <type of input to generate, one of ascending (default), descending, random>] [-verbose]
需求:
生成每个文件100000行,1000个mapper,200个reducer,执行200次:
bin/hadoop jar share/hadoop/mapreduceX/hadoop-test-XXX.jar mrbench -numRuns 200 -inputLines 100000 -maps 1000 -reduces 200
汇总信息:
DataLines Maps Reduces AvgTime (milliseconds) 100000 1000 200 28522