Hadoop2.6+IntellijIDEA2017+win10开发

1、首先下载windows x64的hadoop客户端依赖包
hadoop客户端依赖包
2、配置环境变量
需要将依赖包解压缩，配置为HADOOP_HOME，同时将bin文件夹设置到PATH中

3、使用IntellijIDEA2017选择maven工程
下面是一个参考maven配置

    <repositories>
        <repository>
            <id>nexus-aliyun</id>
            <name>Nexus aliyun</name>
            <url>http://maven.aliyun.com/nexus/content/groups/public</url>
        </repository>
        <repository>
            <id>cloudera</id>
            <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
        </repository>
    </repositories>

    <dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>${hadoop.version}</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

其中，阿里云不是必须的，只是可以加速。cloudera是必须的，因为我用的是CDH版本的hadoop。
4、hdfs测试代码

public class HDFSApp {
    public static final String HDFS_PATH="hdfs://192.168.190.133:8020";
    FileSystem fileSystem=null;
    Configuration configuration=null;
    @Before
    public void setUp() throws Exception{
        configuration=new Configuration();
        fileSystem=FileSystem.get(new URI(HDFS_PATH),configuration);
        System.out.println("HDFS APP SETUP");
    }
    @Test
    public void create() throws IOException {
        FSDataOutputStream outputStream=fileSystem.create(new Path("/hdfsapi/test/a.txt"));
        outputStream.write("hello hadoop".getBytes());
        outputStream.flush();
        outputStream.close();
    }
    @Test
    public void cat() throws IOException {
        FSDataInputStream inputStream=fileSystem.open(new Path("/hdfsapi/test/a.txt"));
        IOUtils.copyBytes(inputStream,System.out,1024);
        inputStream.close();
    }
    @Test
    public void mkdir() throws IOException {
        fileSystem.mkdirs(new Path("/hdfsapi/test"));
    }
    @After
    public void tearDown() throws Exception{
        configuration=null;
        configuration=null;
        System.out.println("HDFS APP SHUTDOWN");
    }
}

5、运行测试代码，unit test pass即为配置完成