序列化与反序列化框架介绍(二)

继序列化与反序列化框架介绍(一)

1.6 Hessian序列化

支持跨语言的二进制序列化协议，相对于Java默认序列化，具有更好的性能与易用性

1.6.1 核心类

AbstractSerializerFactory、AbstractHessianOutput、AbstcactSerializer、AbstractHessianInput、AbstractDeserializer，和本质输入输出的ByteArrayOutputStream、ByteArrayInputStream

1.6.2 抽取通用序列化/反序列化方法

/**
 * @author pdc
 */
public class HessianSerializer implements ISerializer {

    public byte[] serialize(Object obj) {
        if (obj == null)
            throw new NullPointerException();

        try {
            ByteArrayOutputStream os = new ByteArrayOutputStream();
            HessianOutput ho = new HessianOutput(os);
            ho.writeObject(obj);
            return os.toByteArray();
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    public <T> T deserialize(byte[] data, Class<T> clazz) {
        if (data == null) throw new NullPointerException();
        try {
            ByteArrayInputStream is = new ByteArrayInputStream(data);
            HessianInput hi = new HessianInput(is);
            return (T) hi.readObject();
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

1.7 protobuf/protostuff序列化

1.7.1 protobuf

是Google的一种数据交换格式，独立于语言、平台；是一个纯粹的应用层协议，可以和各种运输层协议一起使用

优缺点：

1.7.1.1 开发步骤：

1.省略，根据自己的OS百度即可

2.编写addressbook.proto

3.执行命令，生成相关的序列化工具类：

./protoc --java_out=./ ./addressbook.proto

4.将生成的AddressBookProtos复制到工程里面，编写应用的序列化与反序列化工具：

/**
 * @author pdc
 */
public class ProtoBufSerializer{

    public static void main(String[] args) throws Exception {
        //构建一个Person对象
        AddressBookProtos.Person person = AddressBookProtos.Person
                .newBuilder()
                .setEmail("[email protected]")
                .setId(10000)
                .setName("kongxuan")
                .addPhone(
                        AddressBookProtos.Person.PhoneNumber.newBuilder().setNumber("13300000000")
            .setType(AddressBookProtos.Person.PhoneType.HOME).build()).build();
        //序列化
        System.out.println(person.toByteString());
        System.out.println(Arrays.toString(person.toByteArray()));
        // 反序列化方法一
        AddressBookProtos.Person newPerson = AddressBookProtos.Person.parseFrom(person.toByteString());
        System.out.println(newPerson);
        // 反序列化方法二
        newPerson = AddressBookProtos.Person.parseFrom(person.toByteArray());
        System.out.println(newPerson);
    }
}

在此基础上可以抽取出通用的序列化/反序列化方法：

/**
 * @author pdc
 */
public class ProtoBufSerializer implements ISerializer {

    public <T> byte[] serialize(T obj) {
        try {
            if (!(obj instanceof GeneratedMessageV3)) {
                throw new UnsupportedOperationException("not supported obj type");
            }
            return (byte[]) MethodUtils.invokeMethod(obj, "toByteArray");
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }

    public <T> T deserialize(byte[] data, Class<T> cls) {
        try {
            if (!GeneratedMessageV3.class.isAssignableFrom(cls)) {
                throw new UnsupportedOperationException("not supported obj type");
            }
            Object o = MethodUtils.invokeStaticMethod(cls, "getDefaultInstance");
            return (T) MethodUtils.invokeMethod(o, "parseFrom", new Object[]{data});
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

1.7.2 protostuff

protobuf需要预先编写.proto IDL文件，再通过protobuf提供的编译器生成对应于各种语言的代码
但是对于Java来说，Java具有反射和动态代码生成的能力，这个预编译过程不是必需的，可以在代码执行时实现
protostuff就是实现此功能的框架，基于protobuf，实现了无须预编译就能对JavaBean进行序列化/反序列化能力

1.7.2.1 抽取通用序列化/反序列化方法

/**
 * @author pdc
 */
public class ProtoStuffSerializer implements ISerializer {

    private static Map<Class<?>, Schema<?>> cachedSchema = new ConcurrentHashMap<>();
    /**
     * 反序列化实例化对象时，使用Objenesis
     */
    private static Objenesis objenesis = new ObjenesisStd(true);

    /**
     * 线程安全
     * @param cls
     * @param <T>
     * @return
     */
    private static <T> Schema<T> getSchema(Class<T> cls) {
        Schema<T> schema = (Schema<T>) cachedSchema.get(cls);
        if (schema == null) {
            schema = RuntimeSchema.createFrom(cls);
            cachedSchema.put(cls, schema);
        }
        return schema;
    }

    public <T> byte[] serialize(T obj) {
        Class<T> cls = (Class<T>) obj.getClass();
        LinkedBuffer buffer = LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);
        try {
            Schema<T> schema = getSchema(cls);
            return ProtostuffIOUtil.toByteArray(obj, schema, buffer);
        } catch (Exception e) {
            throw new RuntimeException(e);
        } finally {
            buffer.clear();
        }
    }

    public <T> T deserialize(byte[] data, Class<T> cls) {
        try {
            //如果使用Java反射，则代码为
            //T message = (T)cls.getConstructors()[0].newInstance();
            //这种做法要求对象保留无参构造，且调用构造函数不会造成额外的作用
            //影响了通用性和健壮性
            T message = (T) objenesis.newInstance(cls);
            Schema<T> schema = getSchema(cls);
            ProtostuffIOUtil.mergeFrom(data, message, schema);
            return message;
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

1.8 Thrift序列化

虽有Thrift RPC框架，但也可以单独使用Thrift进行序列化/反序列化操作

1.8.1 开发步骤

1.编写数据结构的 .thrift文件

2.使用Thrift提供的代码生成工具，生成.thrift文件对应的Java类

3.使用TSerializer和TDeserializer对类对象进行序列化/反序列化操作

/**
 * @author pdc
 * User为自定义对象
 */
public class ThriftSerializer implements ISerializer {
    
    public <T> byte[] serialize(T obj) {
        try {
            if (!(obj instanceof TBase)) {
                throw new UnsupportedOperationException("not supported obj type");
            }
            TSerializer serializer = new TSerializer(new TCompactProtocol.Factory());
            return serializer.serialize((TBase) obj);
        } catch (TException e) {
            throw new RuntimeException(e);
        }
    }


    public <T> T deserialize(byte[] data, Class<T> cls) {
        try {
            if (!TBase.class.isAssignableFrom(cls)) {
                throw new UnsupportedOperationException("not supported obj type");
            }
            //可以指定如下序列化协议
            //TBinaryProtocol  TJSONProtocol  TCompactProtocol
            TBase o = (TBase) cls.newInstance();
            TDeserializer tDeserializer = new TDeserializer(new TCompactProtocol.Factory());
            tDeserializer.deserialize(o, data);
            return (T) o;
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

1.9 Avro序列化

虽可以用作RPC框架，但也可以单独用来进行序列化/反序列化操作

1.9.1 主要不同点：

动态类型、无标记数据、不用手动分配的字段ID

1.9.2 特点

性能高、基本代码少、产出数据量精简

1.9.3 特性

1.9.4 序列化编码方式

1.二进制编码

性能更好，序列化后产生的码流更小

2.JSON编码

可读性好，适合在开发调试阶段使用

1.9.5 IDL

1.9.6 Schema

如果仅仅使用Avro的序列化/反序列化功能，则可以使用JSON格式来定义所需要序列化的数据结构Schama

定义文件以.avsc后缀结尾，JSON相关键如下：

1.9.7 序列化/反序列化实现

• 方式1

  编写序列化类的Schame并生成对应的对象；使用此对象，序列化生成byte[]，再反序列化生成新的对象

/**
 * @author pdc
 */
public class AvroSerializer implements ISerializer {

    public static void main(String[] args) throws IOException {
        User userAvro = new User();
        userAvro.setAge(28);
        userAvro.setEmail("[email protected]");
        userAvro.setName("pdc");

        //1.先自动生成代码,再序列化方式
        //1.1 序列化
        DatumWriter<User> userDatumWriter = new SpecificDatumWriter<User>(User.class);
        ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
        BinaryEncoder binaryEncoder = EncoderFactory.get().directBinaryEncoder(outputStream, null);
        userDatumWriter.write(userAvro, binaryEncoder);
        byte[] data = outputStream.toByteArray();

        //1.2 反序列化
        DatumReader<User> userDatumReader = new SpecificDatumReader<User>(User.class);
        BinaryDecoder binaryDecoder = DecoderFactory.get().directBinaryDecoder(new ByteArrayInputStream(data), null);
        User userAvroCopy = userDatumReader.read(new User(), binaryDecoder);
        System.out.println("age:" + userAvroCopy.getAge() + " email:" + userAvroCopy.getEmail() + " name:" + userAvroCopy.getName());
    }
}

• 方式2

  直接使用Schema文件进行序列化与反序列化

/**
 * @author pdc
 */
public class AvroSerializer implements ISerializer {

    public static void main(String[] args) throws IOException {
        //2.直接根据avsc文件进行序列化的操作方式
        Schema schema = new Schema.Parser().parse(new File("src/main/avro/user.avsc"));
        GenericRecord userAvro2 = new GenericData.Record(schema);
        userAvro2.put("age", 18);
        userAvro2.put("email", "[email protected]");
        userAvro2.put("name", "pdc");
        //2.1 序列化
        DatumWriter<GenericRecord> datumWriter = new GenericDatumWriter<GenericRecord>(schema);
        ByteArrayOutputStream outputStream2 = new ByteArrayOutputStream();
        BinaryEncoder binaryEncoder2 = EncoderFactory.get().directBinaryEncoder(outputStream2, null);
        datumWriter.write(userAvro2, binaryEncoder2);
        byte[] data2 = outputStream2.toByteArray();
        //2.2 反序列化
        DatumReader<User> userDatumReader2 = new SpecificDatumReader<User>(User.class);
        BinaryDecoder binaryDecoder2 = DecoderFactory.get().directBinaryDecoder(new ByteArrayInputStream(data2), null);
        User userAvroCopy2 = userDatumReader2.read(new User(), binaryDecoder2);
        System.out.println("age:" + userAvroCopy2.getAge() + " email:" + userAvroCopy2.getEmail() + " name:" + userAvroCopy2.getName());
    }
}

• 方式3

  将序列化的结果保存到文件中

/**
 * @author pdc
 */
public class AvroSerializer implements ISerializer {

    public static void main(String[] args) throws IOException {
        //3.以文件的形式将序列化
        //3.1 序列化
        File file = new File("users.avro");
        DataFileWriter<User> dataFileWriter = new DataFileWriter<User>(userDatumWriter);
        dataFileWriter.create(userAvro.getSchema(), file);
        dataFileWriter.append(userAvro);
        dataFileWriter.append(userAvroCopy2);
        dataFileWriter.close();
        //3.2 从文件中反序列化
        DataFileReader<User> dataFileReader = new DataFileReader<User>(file, userDatumReader);
        User user = null;
        while (dataFileReader.hasNext()) {
            user = dataFileReader.next(user);
            System.out.println("age:" + user.getAge() + " email:" + user.getEmail() + " name:" + user.getName());
        }
       
    }
}

1.9.8 抽取通用序列化/反序列化方法

/**
 * @author pdc
 */
public class AvroSerializer implements ISerializer {

    public static void main(String[] args) throws IOException {
        byte[] data_1 = new AvroSerializer().serialize(userAvro);
        User user1 = new AvroSerializer().deserialize(data_1, User.class);
        System.out.println("age:" + user1.getAge() + " email:" + user1.getEmail() + " name:" + user1.getName());
    }

    public <T> byte[] serialize(T obj) {
        try {
            if (!(obj instanceof SpecificRecordBase)) {
                throw new UnsupportedOperationException("not supported obj type");
            }
            DatumWriter userDatumWriter = new SpecificDatumWriter(obj.getClass());
            ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
            BinaryEncoder binaryEncoder = EncoderFactory.get().directBinaryEncoder(outputStream, null);
            userDatumWriter.write(obj, binaryEncoder);
            return outputStream.toByteArray();
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    public <T> T deserialize(byte[] data, Class<T> clazz) {
        try {
            if (!SpecificRecordBase.class.isAssignableFrom(clazz)) {
                throw new UnsupportedOperationException("not supported clazz type");
            }
            DatumReader userDatumReader = new SpecificDatumReader(clazz);
            BinaryDecoder binaryDecoder = DecoderFactory.get().directBinaryDecoder(new ByteArrayInputStream(data), null);
            return (T) userDatumReader.read(clazz.newInstance(), binaryDecoder);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

1.10 JBoss Marshalling序列化框架介绍

Java对象序列包，兼容Java原生的序列化机制，对Java原生序列化机制进行了优化，提升了性能

在保持跟Serializble接口兼容的同时增加了一些可调的参数和附加特性，可通过工厂类进行配置

是一个很好的替代原生Java序列化的框架

1.10.1 抽取通用序列化/反序列化方法

/**
 * @author pdc
 */
public class MarshallingSerializer implements ISerializer {

    final static MarshallingConfiguration configuration = new MarshallingConfiguration();
    //获取序列化工厂对象,参数serial标识创建的是java序列化工厂对象
    final static MarshallerFactory marshallerFactory = Marshalling.getProvidedMarshallerFactory("serial");

    static {
        configuration.setVersion(5);
    }


    public byte[] serialize(Object obj) {
        final ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
        try {
            final Marshaller marshaller = marshallerFactory.createMarshaller(configuration);
            marshaller.start(Marshalling.createByteOutput(byteArrayOutputStream));
            marshaller.writeObject(obj);
            marshaller.finish();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return byteArrayOutputStream.toByteArray();
    }


    public <T> T deserialize(byte[] data, Class<T> clazz) {
        try {
            ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(data);
            final Unmarshaller unmarshaller = marshallerFactory.createUnmarshaller(configuration);
            unmarshaller.start(Marshalling.createByteInput(byteArrayInputStream));
            Object object = unmarshaller.readObject();
            unmarshaller.finish();
            return (T) object;
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

1.11 序列化框架的选型

1.技术层面考虑

2.其他层面考虑

3.选型建议：