数据序列化框架——Kryo

数据序列化框架——Kryo

1. 概览

• 简介
      Kryo是一款专用于Java的快速、高效的数据序列化的框架，同时也支持Scala、Clojure、Objective-C等。目前已被多种大型项目使用，包括：Apache Hive、Apache Spark 、Storm、Apache Dubbo 等。

• 特点

  • 快速、高效、占用空间小
  • 使用简单（不用像protobuf一样编写协议文件）
  • 支持浅拷贝、深拷贝功能
  • 专用于Java（如果你的服务器全是Java端，那么采用Kryo是一个非常好的选择）

• GitHub地址：https://github.com/EsotericSoftware/kryo

  注: 如果你觉得下载依赖包麻烦，也可以使用本博客提供的jar包

2. Java中使用Kryo

• 添加maven依赖

  <dependency>
      <groupId>com.esotericsoftware</groupId>
      <artifactId>kryo</artifactId>
      <version>3.0.3</version>
  </dependency>
  

• 导包

  import com.esotericsoftware.kryo.Kryo;
  import com.esotericsoftware.kryo.io.Input;
  import com.esotericsoftware.kryo.io.Output;
  
  import java.io.FileInputStream;
  import java.io.FileNotFoundException;
  import java.io.FileOutputStream;
  

• 待使用的对象Person

  public class Person {
  
      private String name;
  
      private int age;
  
      private String address;
  
      private Person parent;
  
      public Person() {
      }
  
      public Person(String name, int age, String address) {
          this.name = name;
          this.age = age;
          this.address = address;
      }
  
      public Person(String name, int age, String address, Person parent) {
          this.name = name;
          this.age = age;
          this.address = address;
          this.parent = parent;
      }
  
      public String getName() {
          return name;
      }
  
      public void setName(String name) {
          this.name = name;
      }
  
      public Person getParent() {
          return parent;
      }
  
      public void setParent(Person parent) {
          this.parent = parent;
      }
  
      @Override
      public String toString() {
          return "Person{" +
                  "name='" + name + '\'' +
                  ", age=" + age +
                  ", address='" + address + '\'' +
                  ", parent=" + parent +
                  '}';
      }
  }
  

• 文件序列化与反序列化

  Kryo kryo = new Kryo();
  kryo.register(Person.class);
  
  Person parent = new Person("老王", 38, "BeiJing");
  
  System.out.println("--------> 序列化到文件中");
  Output output = new Output(new FileOutputStream("file.bin"));
  kryo.writeObject(output, parent);
  output.close();
  
  Input input = new Input(new FileInputStream("file.bin"));
  Person object1 = kryo.readObject(input, Person.class);
  input.close();
  

• byte序列化与反序列化

  Kryo kryo = new Kryo();
  kryo.register(Person.class);
  
  Person parent = new Person("老王", 38, "BeiJing");
  
  System.out.println("--------> byte方式序列化");
  
  // -1 表示maxBufferSize可以无限制，源码中是给的Integer.MAX_VALUE
  Output output2 = new Output(1024, -1);
  kryo.writeObject(output2, parent);
  byte[] bytes = output2.getBuffer();
  
  Input input2 = new Input(bytes);
  Person object2 = kryo.readObject(input2, Person.class);
  System.out.println("object2 = " + object2);
  

• 深拷贝与浅拷贝

  Kryo kryo = new Kryo();
  kryo.register(Person.class);
  
  Person parent = new Person("老王", 38, "BeiJing");
  Person person = new Person("小明", 18, "BeiJing", parent);
  
  System.out.println("--------> 深拷贝");
  Person copyPerson2 = kryo.copy(person);
  copyPerson2.getParent().setName("老李");
  System.out.println("copyPerson2 = " + copyPerson2);
  System.out.println("person = " + person);
  
  System.out.println("--------> 浅拷贝");
  Person copyPerson = kryo.copyShallow(person);
  copyPerson.getParent().setName("老李");
  System.out.println("copyPerson = " + copyPerson);
  System.out.println("person = " + person);
  

3. Scala中使用Kryo

注: 我所使用的scala是2.11.8版本

• Scala中使用Kryo非常简单，但还需要额外再引入几个Twitter的工具包（文章顶部有下载）
  • twitter/chill : chill_2.11-0.8.0.jar, chill-java-0.8.0.jar
  • twitter/bijection: bijection-core_2.11-0.9.1.jar, chill-bijection_2.11-0.9.1.jar
  • jar包也可以从maven仓库下载，pom.xml依赖如下

  <dependency>
    <groupId>com.twitter</groupId>
      <artifactId>chill_2.11</artifactId>
      <version>0.8.0</version>
  </dependency>
  <dependency>
      <groupId>com.twitter</groupId>
      <artifactId>chill-java</artifactId>
      <version>0.8.0</version>
  </dependency>
  <dependency>
      <groupId>com.twitter</groupId>
      <artifactId>bijection-core_2.11</artifactId>
      <version>0.9.1</version>
  </dependency>
  <dependency>
      <groupId>com.twitter</groupId>
      <artifactId>chill-bijection_2.11</artifactId>
      <version>0.9.1</version>
  </dependency>
  

• 使用示例

  import com.twitter.chill.KryoInjection
  
  /**
    * Descr: Scala Kryo序列化的Injection示例
    * Date: 2019/8/21 14:50
    *
    * @author A Lion~
    */
  object ScalaDemo {
  
    def main(args: Array[String]): Unit = {
      // 构建对象
      case class Person(name: String, age: Int, address: String)
      val person = Person("小明", 18, "BeiJing")
  
      // 将对象转为byte数组
      val bytes: Array[Byte] = KryoInjection(person)
      
      // 解析byte数组，转为对象
      val tryDecode: scala.util.Try[Any] = KryoInjection.invert(bytes)
  
      println(s"tryDecode = ${tryDecode.get}")
    }
  
  }