Previous article Interpretation of Communication Protocol Serialization (1) : http://www.cnblogs.com/tohxyblog/p/8974641.html
Foreword : In the previous article, we introduced java serialization and Google protobuf, but because the use of protobuf is not as simple as other serialization (first write the .proto file, then compile the .proto file to generate the corresponding .java file ), so no matter how good he is, he still failed to seize the share of json.
In this article, we are going to introduce a protobuf-based java serialization protocol - prorostuff, which can be very easy to use on the java side, and deserialization can be done by protobuf (then the front end can be decoded by protobuf in other languages) .
1. Introduction to protostuff
protostuff is based on Google protobuf, but provides more functionality and easier usage. Among them, protostuff-runtime realizes the ability of protobuf serialization/deserialization of java beans without precompiling. The limitation of protostuff-runtime is that schema needs to be passed in before serialization, and deserialization is not responsible for object creation but only for copying, so a default constructor must be provided. In addition, protostuff can also be serialized into json/yaml/xml formats according to the configuration of protobuf.
In terms of performance, protostuff does not lose to native protobuf, and even has the potential to surpass it.
2. Protostuff features
-
Support for messages generated by protostuff-compiler
-
Support for existing POJOs
-
Support for Java messages generated by existing protoc
-
Interoperability with various mobile platforms (Android, Kindle, j2me)
-
Support transcoding
3. Tool class implementation
Guide package:
<!-- //protostuff serialization-->
<dependency>
<groupId>com.dyuproject.protostuff</groupId>
<artifactId>protostuff-core</artifactId>
<version>1.0.8</version>
</dependency>
<dependency>
<groupId>com.dyuproject.protostuff</groupId>
<artifactId>protostuff-runtime</artifactId>
<version>1.0.8</version>
</dependency>
<!-- Objenesis -->
<dependency>
<groupId>org.objenesis</groupId>
<artifactId>objenesis</artifactId>
<version>2.1</version>
</dependency>
Tools:
package com.result.base.tools;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import org.objenesis.Objenesis;
import org.objenesis.ObjenesisStd;
import com.dyuproject.protostuff.LinkedBuffer;
import com.dyuproject.protostuff.ProtobufIOUtil;
import com.dyuproject.protostuff.ProtostuffIOUtil;
import com.dyuproject.protostuff.Schema;
import com.dyuproject.protostuff.runtime.RuntimeSchema;
/**
* @author by huangxinyu
* @version created: Jan 9, 2018 7:41:24 PM
* Protostuff serialization tool
*/
public class SerializationUtil {
private static Map<Class<?>, Schema<?>> cachedSchema = new ConcurrentHashMap<>();
private static Objenesis objenesis = new ObjenesisStd(true);
private SerializationUtil() {
}
@SuppressWarnings("unchecked")
private static <T> Schema<T> getSchema(Class<T> cls) {
Schema<T> schema = (Schema<T>) cachedSchema.get(cls);
if (schema == null) {
schema = RuntimeSchema.createFrom(cls);
if (schema != null) {
cachedSchema.put(cls, schema);
}
}
return schema;
}
@SuppressWarnings("unchecked")
public static <T> String serializeToString(T obj) {
Class<T> cls = (Class<T>) obj.getClass();
LinkedBuffer buffer = LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);
try {
Schema<T> schema = getSchema(cls);
return new String(ProtobufIOUtil.toByteArray(obj, schema, buffer), "ISO8859-1");
} catch (Exception e) {
throw new IllegalStateException(e.getMessage(), e);
} finally {
buffer.clear();
}
}
public static <T> T deserializeFromString(String data, Class<T> cls) {
try {
T message = (T) objenesis.newInstance(cls);
Schema<T> schema = getSchema(cls);
ProtobufIOUtil.mergeFrom(data.getBytes("ISO8859-1"), message, schema);
return message;
} catch (Exception e) {
throw new IllegalStateException(e.getMessage(), e);
}
}
@SuppressWarnings("unchecked")
public static <T> byte[] serializeToByte(T obj) {
Class<T> cls = (Class<T>) obj.getClass();
LinkedBuffer buffer = LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);
try {
Schema<T> schema = getSchema(cls);
return ProtobufIOUtil.toByteArray(obj, schema, buffer);
} catch (Exception e) {
throw new IllegalStateException(e.getMessage(), e);
} finally {
buffer.clear();
}
}
public static <T> T deserializeFromByte(byte[] data, Class<T> cls) {
try {
T message = (T) objenesis.newInstance(cls);
Schema<T> schema = getSchema(cls);
ProtobufIOUtil.mergeFrom(data, message, schema);
return message;
} catch (Exception e) {
throw new IllegalStateException(e.getMessage(), e);
}
}
}
4. Performance test
4.1 Test Environment
xstraem version: 1.3.1
protobuf-java version: 3.0.0-alpha-2
java version: 1.7
-Xms2048m
-Xmx2048m
4.2 Test Tools
Time: console output time
CPU & Memory: jconsole
file size: file properties
4.3 Description
In the test, the JavaBeans used in the three tests, xml, protoBuf and protostuff, have the same field type, the same number of fields (about 28), the same value attached to the field, and all contain a List<String> field. The size of the field controls the size of the JavaBean object. In this test size=100
4.4 Results
Test A: 10000 objects
| xstream |
protobuf |
protostuff |
||
| Serialization |
Time (ms) |
2399 |
648 |
261 |
| Occupied CPU (%) |
24.2 |
12.3 |
3.4 |
|
| Occupied memory (M) |
154 |
235 |
92 |
|
| Size of each file (byte) |
2822 |
574 |
574 |
|
|
|
||||
| deserialize |
Time (ms) |
3378 |
167 |
224 |
| CPU usage (%) |
15.9 |
14.2 |
6.1 |
|
| Occupied memory (M) |
248 |
307 |
164 |
|
| Remarks: 10000 objects |
||||
Test B: 25000 objects
| xstream |
protobuf |
protostuff |
||
| Serialization |
Time (ms) |
4161 |
767 |
293 |
| Occupied CPU (%) |
31.2 |
14.6 |
4.7 |
|
| Occupied memory (M) |
495 |
228 |
194 |
|
| Size of each file (byte) |
2822 |
574 |
574 |
|
|
|
||||
| deserialize |
Time (ms) |
6941 |
252 |
393 |
| CPU usage (%) |
31.9 |
21.9 |
8.1 |
|
| Occupied memory (M) |
411 |
382 |
348 |
|
| Remarks: 25000 objects |
||||
Test C: 100000 objects
| xstream |
protobuf |
protostuff |
||
| Serialization |
Time (ms) |
12867 |
3070 |
704 |
| Occupied CPU (%) |
42.5 |
44.9 |
22.3 |
|
| Occupied memory (M) |
1098 |
1058 |
572 |
|
| Size of each file (byte) |
2822 |
574 |
574 |
|
|
|
||||
| deserialize |
Time (ms) |
24442 |
4540 |
1522 |
| CPU usage (%) |
38.8 |
68.2 |
24.1 |
|
| Occupied memory (M) |
2215 |
597 |
870 |
|
| Remarks: 50000 objects |
||||
Quoting the histogram of the last set of data:
4.5 Conclusion
1. Serialization:
1.1. In terms of speed: protostuff is about 3 times faster than protobuf, and protobuf is 4-5 times faster than xml. The multiple remains basically unchanged with the increase of serialized objects.
1.2. On the CPU: protostuff occupies the least, protobuf second, and xml last.
1.3. In terms of memory: protostuff occupies the least, protobuf is the second, and xml is the last.
1.4. Generated file size: protostuff occupies the least, protobuf second, xml last, the first two are about 1/4 of the latter.
2. Deserialization
2.1. Speed: In the case of a small number of deserialized objects, protobuf is about 1/4 faster than protostuff and 10+ times faster than xml. But as the number of objects increases, protobuf happens to noticeably slow down! Thus being overtaken by protostuff.
2.2. On the CPU: protostuff occupies the least, protobuf second, and xml last.
2.3. In terms of memory: protostuff occupies the least, protobuf is the second, and xml is the last.
3. Summary
In all aspects, the advantages of protostuff are very interview, and protobuf is not weak, consider using it instead of xml.