準備
- 次のように/入力HDFSディレクトリfruit.tsvというファイルを作成します。
- 次のようにHBaseの中に果物という名前のテーブルを作成します。
マッパーを作成します
public class FruitMapper extends Mapper<LongWritable, Text, ImmutableBytesWritable, Put> {
@Override
protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
String lineValue = value.toString();
String[] values = lineValue.split("\t");
String rowKey = values[0];
String name = values[1];
String color = values[2];
ImmutableBytesWritable rowKeyWritable = new ImmutableBytesWritable(Bytes.toBytes(rowKey));
Put put = new Put(Bytes.toBytes(rowKey));
put.addColumn(Bytes.toBytes("info"), Bytes.toBytes("name"), Bytes.toBytes(name));
put.addColumn(Bytes.toBytes("info"), Bytes.toBytes("color"), Bytes.toBytes(color));
context.write(rowKeyWritable, put);
}
}
レデューサーを作成します
public class FruitReducer extends TableReducer<ImmutableBytesWritable, Put, NullWritable> {
@Override
protected void reduce(ImmutableBytesWritable key, Iterable<Put> values, Context context) throws IOException, InterruptedException {
for(Put put: values){
context.write(NullWritable.get(), put);
}
}
}
ランナーの作成
public class FruitRunner {
public static void main(String[] arg) throws Exception {
Configuration conf = HBaseConfiguration.create();
String[] args = {"hdfs://hcmaster:8020/input/fruit.tsv"};
Job job = Job.getInstance(conf, FruitRunner.class.getSimpleName());
job.setJarByClass(FruitRunner.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
job.setMapperClass(FruitMapper.class);
job.setMapOutputKeyClass(ImmutableBytesWritable.class);
job.setMapOutputValueClass(Put.class);
TableMapReduceUtil.initTableReducerJob("fruit", FruitReducer.class, job);
job.setNumReduceTasks(1);
boolean isSuccess = job.waitForCompletion(true);
if (!isSuccess) {
throw new IOException("Job running with error");
}
int status = isSuccess ? 0 : 1;
System.exit(status);
}
}
ラン
- 瓶のパッケージとLinuxにアップロード
- JARパッケージを実行
3.結果の表示