hdfs追加文件

hdfs追加文件测试代表如下:
Configuration conf=new Configuration();
conf.setBoolean("dfs.support.append", true);
conf.set("dfs.client.block.write.replace-datanode-on-failure.policy","NEVER");
conf.set("dfs.client.block.write.replace-datanode-on-failure.enable","true");
FileSystem fs = FileSystem.get(new URI("hdfs://192.168.1.46:8020"), conf,"hdfs");

boolean isExist=fs.exists(new Path("/test/test.txt"));
if(isExist){
FSDataOutputStream out = fs.append(new Path("/test/test.txt"));
FileInputStream in = new FileInputStream(new File("d:/test.txt"));
IOUtils.copyBytes(in, out, 2048, true);
}else{
FSDataOutputStream out = fs.create(new Path("/test/test.txt"));
FileInputStream in = new FileInputStream(new File("d:/test.txt"));
IOUtils.copyBytes(in, out, 2048, true);
}

    conf.set("dfs.client.block.write.replace-datanode-on-failure.policy","NEVER");
    conf.set("dfs.client.block.write.replace-datanode-on-failure.enable","true");

    增加以止两个属性是为解决报错: Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try

猜你喜欢

转载自qun715715.iteye.com/blog/2207432