体验avro

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/shursulei/article/details/79711186

一、下载avro
创建maven项目(使用eclipse或者idea)

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
    <dependency>
          <groupId>org.apache.avro</groupId>
          <artifactId>avro</artifactId>
          <version>1.8.2</version>
    </dependency>
  </dependencies>

补充:
<groupId>com.shursulei</groupId>
  <artifactId>myavro</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>myavro</name>
  <url>http://maven.apache.org</url>

二、创建schema文件user.avsc

E:\BigDate\avro\user.avsc
{"namespace": "example.avro",
 "type": "record",
 "name": "User",
 "fields": [
     {"name": "name", "type": "string"},
     {"name": "favorite_number",  "type": ["int", "null"]},
     {"name": "favorite_color", "type": ["string", "null"]}
 ]
}

三、编译schema,生成java code

java -jar /path/to/avro-tools-1.8.2.jar compile schema <schema file> <destination>
E:\BigDate\avro>java -jar avro-tools-1.8.2.jar compile schema user.avsc .
Input files to compile:
  user.avsc
log4j:WARN No appenders could be found for logger (AvroVelocityLogChute).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

四、使用java代码

public class Serialize 
{
    public static void main( String[] args ) throws IOException
    {
        //创建User
        User user=new User();
        user.setName("tom");
        user.setFavoriteNumber(888);
        user.setFavoriteColor("red");
        //通过build创建对象
        User user2=User.newBuilder()
                .setName("tom2")
                .setFavoriteNumber(999)
                .setFavoriteColor("blue")
                .build();
     // Serialize user1, user2 and user3 to disk
     //通过Writer写入文件磁盘
        DatumWriter<User> userDatumWriter = new SpecificDatumWriter<User>(User.class);
        DataFileWriter<User> dataFileWriter = new DataFileWriter<User>(userDatumWriter);
        //这是数据文件
        dataFileWriter.create(user.getSchema(), new File("E:\\BigDate\\avro\\users.avro"));
        dataFileWriter.append(user);
        dataFileWriter.append(user2);
        dataFileWriter.close();
        System.out.println("over");
    }
}