Data sequence framework --Kryo

Data sequence framework --Kryo

1. Overview

• About
      Kryo is a dedicated Java is fast, efficient data serialization framework also supports Scala, Clojure, Objective-C and so on. It has been using a variety of large-scale projects, including: Apache Hive, Apache Spark, Storm , Apache Dubbo and so on.

• Feature

  • Fast, efficient, small footprint
  • Use simple (not the same as protobuf written protocol file)
  • Support shallow copy and deep copy function
  • Dedicated to Java (if your server all the Java side, then the use of Kryo is a very good choice)

• GitHub address: https: //github.com/EsotericSoftware/kryo

  Note: If you feel downloading dependencies trouble, you can also use the jar package offered this blog

2. Java is used Kryo

• Add maven dependence

  <dependency>
      <groupId>com.esotericsoftware</groupId>
      <artifactId>kryo</artifactId>
      <version>3.0.3</version>
  </dependency>
  

• Guide package

  import com.esotericsoftware.kryo.Kryo;
  import com.esotericsoftware.kryo.io.Input;
  import com.esotericsoftware.kryo.io.Output;
  
  import java.io.FileInputStream;
  import java.io.FileNotFoundException;
  import java.io.FileOutputStream;
  

• Person objects to be used

  public class Person {
  
      private String name;
  
      private int age;
  
      private String address;
  
      private Person parent;
  
      public Person() {
      }
  
      public Person(String name, int age, String address) {
          this.name = name;
          this.age = age;
          this.address = address;
      }
  
      public Person(String name, int age, String address, Person parent) {
          this.name = name;
          this.age = age;
          this.address = address;
          this.parent = parent;
      }
  
      public String getName() {
          return name;
      }
  
      public void setName(String name) {
          this.name = name;
      }
  
      public Person getParent() {
          return parent;
      }
  
      public void setParent(Person parent) {
          this.parent = parent;
      }
  
      @Override
      public String toString() {
          return "Person{" +
                  "name='" + name + '\'' +
                  ", age=" + age +
                  ", address='" + address + '\'' +
                  ", parent=" + parent +
                  '}';
      }
  }
  

• File serialization and de-serialization

  Kryo kryo = new Kryo();
  kryo.register(Person.class);
  
  Person parent = new Person("老王", 38, "BeiJing");
  
  System.out.println("--------> 序列化到文件中");
  Output output = new Output(new FileOutputStream("file.bin"));
  kryo.writeObject(output, parent);
  output.close();
  
  Input input = new Input(new FileInputStream("file.bin"));
  Person object1 = kryo.readObject(input, Person.class);
  input.close();
  

• byte serialization and deserialization

  Kryo kryo = new Kryo();
  kryo.register(Person.class);
  
  Person parent = new Person("老王", 38, "BeiJing");
  
  System.out.println("--------> byte方式序列化");
  
  // -1 表示maxBufferSize可以无限制，源码中是给的Integer.MAX_VALUE
  Output output2 = new Output(1024, -1);
  kryo.writeObject(output2, parent);
  byte[] bytes = output2.getBuffer();
  
  Input input2 = new Input(bytes);
  Person object2 = kryo.readObject(input2, Person.class);
  System.out.println("object2 = " + object2);
  

• Deep copy and shallow copy

  Kryo kryo = new Kryo();
  kryo.register(Person.class);
  
  Person parent = new Person("老王", 38, "BeiJing");
  Person person = new Person("小明", 18, "BeiJing", parent);
  
  System.out.println("--------> 深拷贝");
  Person copyPerson2 = kryo.copy(person);
  copyPerson2.getParent().setName("老李");
  System.out.println("copyPerson2 = " + copyPerson2);
  System.out.println("person = " + person);
  
  System.out.println("--------> 浅拷贝");
  Person copyPerson = kryo.copyShallow(person);
  copyPerson.getParent().setName("老李");
  System.out.println("copyPerson = " + copyPerson);
  System.out.println("person = " + person);
  

3. Scala in use Kryo

Note: scala I used was version 2.11.8

• Scala used Kryo very simple, but it requires additional reintroduction (top of the article has a download) a few Twitter Toolkit
  • twitter/chill : chill_2.11-0.8.0.jar, chill-java-0.8.0.jar
  • twitter/bijection: bijection-core_2.11-0.9.1.jar, chill-bijection_2.11-0.9.1.jar
  • jar package can also be downloaded from maven repository, pom.xml dependency follows

  <dependency>
    <groupId>com.twitter</groupId>
      <artifactId>chill_2.11</artifactId>
      <version>0.8.0</version>
  </dependency>
  <dependency>
      <groupId>com.twitter</groupId>
      <artifactId>chill-java</artifactId>
      <version>0.8.0</version>
  </dependency>
  <dependency>
      <groupId>com.twitter</groupId>
      <artifactId>bijection-core_2.11</artifactId>
      <version>0.9.1</version>
  </dependency>
  <dependency>
      <groupId>com.twitter</groupId>
      <artifactId>chill-bijection_2.11</artifactId>
      <version>0.9.1</version>
  </dependency>
  

• Examples of Use

  import com.twitter.chill.KryoInjection
  
  /**
    * Descr: Scala Kryo序列化的Injection示例
    * Date: 2019/8/21 14:50
    *
    * @author A Lion~
    */
  object ScalaDemo {
  
    def main(args: Array[String]): Unit = {
      // 构建对象
      case class Person(name: String, age: Int, address: String)
      val person = Person("小明", 18, "BeiJing")
  
      // 将对象转为byte数组
      val bytes: Array[Byte] = KryoInjection(person)
      
      // 解析byte数组，转为对象
      val tryDecode: scala.util.Try[Any] = KryoInjection.invert(bytes)
  
      println(s"tryDecode = ${tryDecode.get}")
    }
  
  }