2018年安徽省大数据比赛MapReduce题目解答第三题

版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。

本文链接： https://blog.csdn.net/qq_41479464/article/details/101922867

3、求每年最高气温（10分）

根据/opt/ahdsjjs/Temperature.txt编写MapperReduce脚本 获得每年的最高气温，并输出

数据格式如下：  

2014010114
2014010216
2014010317
2014010410
2014010506
2012010609
2012010732
2012010812
2012010919
2012011023
2001010116
2001010212
2001010310
2001010411
2001010529
2013010619
2013010722
2013010812
2013010929
2013011023
2008010105
2008010216
2008010337
2008010414
2008010516
2007010619
2007010712
2007010812
2007010999
2007011023
2010010114
2010010216
2010010317
2010010410
2010010506
2015010649
2015010722
2015010812
2015010999
2015011023

比如：2010012325表示在2010年01月23日的气温为25度。

现在要求使用MapReduce，计算每一年出现过的最大气温

思路：其实就是字符串的截取，那到年，以年为key进行分组，天气为value

主函数：

import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.TreeMap;

import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;

public class Myred extends Reducer<Text, IntWritable, Text, IntWritable> {
	@Override
	protected void reduce(Text key, Iterable<IntWritable> values,
			Reducer<Text, IntWritable, Text, IntWritable>.Context context) throws IOException, InterruptedException {
		/**
		 *  2014 14
			2014 16
			2014 17
			2014 10
			2014 06
		 */
		int maxval = Integer.MIN_VALUE;
		int max2val = -10000000;
		for (IntWritable val : values) {		
			maxval = Math.max(maxval, val.get());
		}
		context.write(new Text(key+" "+maxval), new IntWritable(max2val));
	}
}

Map函数：

import java.io.IOException;

import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;

import tq.TQ;

public class Mymap extends Mapper<LongWritable, Text, Text, IntWritable>{
	@Override
	protected void map(LongWritable key, Text value, Mapper<LongWritable, Text, Text, IntWritable>.Context context)
			throws IOException, InterruptedException {
//		2014010506
//		2012010609
		String line = value.toString();
		String yera = line.substring(0,6);
		int Tem = Integer.parseInt(line.substring(8,10));
		context.write(new Text(yera), new IntWritable(Tem));
	}
	

}

Reduce函数：

import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.TreeMap;

import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;

public class Myred extends Reducer<Text, IntWritable, Text, IntWritable> {
	@Override
	protected void reduce(Text key, Iterable<IntWritable> values,
			Reducer<Text, IntWritable, Text, IntWritable>.Context context) throws IOException, InterruptedException {
		/**
		 *  2014 14
			2014 16
			2014 17
			2014 10
			2014 06
		 */
		int maxval = Integer.MIN_VALUE;
		int max2val = -10000000;
		for (IntWritable val : values) {		
			maxval = Math.max(maxval, val.get());
		}
		context.write(new Text(key+" "+maxval), new IntWritable(max2val));
	}
}