A, TestDFSIO write performance test
switch to the directory: / opt / hadoop / share / hadoop / mapreduce /
Clearing the data
Hadoop-MapReduce Hadoop JAR-Client-JobClient-2.6.4.jar -clean
Hadoop -ls FS / Benchmarks /
Write performance test (10 Mapper, 10G)
hadoop hadoop-MapReduce JAR-Client-JobClient-2.6.4.jar TestDFSIO -write 10 -size 10240MB
Read performance test (10 Mapper, 1OG)
Hadoop-MapReduce Hadoop JAR-Client-JobClient -nrFiles 10 -read-2.6.4.jar TestDFSIO -size 10240MB
namenode load test
to see nnbench options
hadoop hadoop-MapReduce JAR-Client-JobClient-2.6.4.jar nnbench
nnbench NameNode for load testing, it will generate a lot of requests associated with HDFS, NameNode to exert greater pressure. This test can be created on the HDFS, read, rename, and delete file operation, using the following example 5 and 10 mapper reducer 1000 to create a file
hadoop jar hadoop-mapreduce-client- jobclient-2.6.4.jar nnbench - operation create_write -maps 10 -reduces 5 -numberOfFiles 1000 -replicationFactorPerFile 3 -readFileAfterOpen true
mrbench test
mrbench will repeatedly perform a small job, operation for checking whether the job can be small and repeated run efficiently on whether the cluster.
The following example will run a small job twice
# hadoop jar hadoop-mapreduce-client -jobclient-2.6.4.jar mrbench -numRuns 2
Two, Terasort sort test
three basic steps sort test: generating random data -> Sort -> verify the sort result
on a more detailed schematic Terasort see http://blog.csdn.net/yuesichiu/article/details/ 17,298,563
1, generates random data (10 Maps)
Hadoop JAR examples Hadoop-MapReduce-10-2.6.4.jar teragen -Dmapreduce.job.maps = 10000000 / tmp / Hadoop / Terasort
2、运行测试(10个maps)
hadoop jar hadoop-mapreduce-examples-2.6.4.jar terasort -Dmapreduce.job.maps=10 /tmp/hadoop/terasort /tmp/hadoop/terasort_out
3、验证结果
hadoop jar hadoop-mapreduce-examples-2.6.4.jar teravalidate /tmp/hadoop/terasort_out /tmp/hadoop/terasort_report