Configuration conf=new Configuration(); //map结果压缩 conf.setBoolean("mapred.compress.map.output", true); conf.setClass("mapred.output.compression.codec", GzipCodec.class, CompressionCodec.class); Job job=new Job(conf,"ufoshapestimes"); job.setJarByClass(Run.class); job.setMapperClass(ShapeTimeMapper.class); job.setReducerClass(ShapeTimeReducer.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(Text.class); FileInputFormat.addInputPath(job, new Path(args[0])); FileOutputFormat.setOutputPath(job, new Path(args[1])); //reduce结果压缩 FileOutputFormat.setCompressOutput(job, true); FileOutputFormat.setOutputCompressorClass(job, GzipCodec.class); System.exit(job.waitForCompletion(true)?1:0);
Hadoop学习笔记--对输出文件进行压缩
猜你喜欢
转载自blog.csdn.net/xqz583722585/article/details/80632746
今日推荐
周排行