The access method of HDFS API operation and the use of JUnit test class

Access methods for HDFS API operations: mainly divided into file system access methods and URL access methods

package com.wyg.hdfs;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URI;
import java.net.URISyntaxException;

import org.apache.commons.io.IOUtils;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.FsUrlStreamHandlerFactory;
import org.junit.Test;

public class HDFSAPIdemo {
    
    
	// 文件系统访问  方式一
@Test
public  void getFileSystem1() throws IOException {
    
    
	//1.实例化对象
	Configuration conf=new Configuration();
	//2、设置文件系统类型
	conf.set( "fs.defaultFS", "hdfs://192.168.2.101:9000");
	//3、获取指定的文件系统
	FileSystem fileSystem=FileSystem.get(conf);
	//4、输出
	System.out.println(fileSystem);
}
	
	//方式二
	@Test
	public void getFileSystem2() throws IOException {
    
    
		FileSystem fileSystem=FileSystem.get(new Configuration());
		System.out.println(fileSystem);
	}
	
	//方式三
	@Test
	public void getFileSystem3() throws IOException {
    
    
		//实例化文件对象
		Configuration conf=new Configuration();
		//指定文件系统类型
		FileSystem fileSystem=FileSystem.newInstance(conf);
		
		System.out.println(fileSystem);
	}	
	//方式四
	@Test
	public void getFileSystem4() throws IOException, URISyntaxException {
    
    
		FileSystem fileSystem=FileSystem.newInstance(new URI("hdfs://192.168.2.101:9000"), new Configuration());
		System.out.println(fileSystem);
	}
	
	
    //url访问   方式三
	@Test
	public void urlHdfs() throws IOException {
    
    
		// 注册url
		URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());
		// 获取hdfs文件的输入流
		InputStream inputStream = new URL("hdfs://192.168.2.101:9000/xiaohua.txt").openStream();
		// 获取本地文件的输出流
		FileOutputStream outputStream = new FileOutputStream(new File("E:\\banzhang.txt"));
		// 实现文件的拷贝
		IOUtils.copy(inputStream, outputStream);
		// 关闭流
		IOUtils.closeQuietly(inputStream);
		IOUtils.closeQuietly(outputStream);
	}
}

In the process of writing code, it is important to note that the guide package cannot lead to errors.

The use of JUnit test class can also be called unit test

Step 1: Right-click the maven project

Step 2:

You can run a single unit or all units. If you want to run a single unit, right-click that unit to run