Serialization: Comparison of ProtoBuf and JSON!
Introduction
ProtoBuf is a tool developed by the google team to store and read structured data efficiently. What is structured data, as it is literally expressed, is data with a certain structure. For example, there are many records in the phone book, and each record contains name, ID, email, phone, etc. This structure is repeated.
similar
XML and JSON can also be used to store such structured data, but the data represented by ProtoBuf can be more efficient and compress the data smaller.
principle
ProtoBuf uses the ProtoBuf compiler to compile the unique .proto suffix data structure files that are not related to the programming language into specific class files for each programming language (Java, C/C++, Python), and then through the support of each programming language provided by Google The library lib can call the API. (For how to write the proto structure, you can consult the documentation yourself)
ProtoBuf compiler installation
Mac : brew install protobuf
for example
1. Create a proto file first
message.proto
syntax = "proto3";
message Person {
int32 id = 1;
string name = 2;
repeated Phone phone = 4;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message Phone {
string number = 1;
PhoneType type = 2;
}
}
Copy the code
2. Create a Java project
and place the proto file in the src/main/proto folder
3. Compile the proto file to the Java version and
use the command line to cd to the src/main directory
Terminal execution command: protoc --java_out=./java ./proto/*.proto
You will find that the corresponding Java class has been generated in your src/main/java
4. Rely on the Java version of the ProtoBuf support library.
Here is just a chestnut that uses Gradle to use dependencies.
implementation'com.google.protobuf:protobuf-java: 3.9.1'Copy
code
5. Convert Java objects to ProtoBuf data
Message.Person.Phone.Builder phoneBuilder = Message.Person.Phone.newBuilder();
Message.Person.Phone phone1 = phoneBuilder
.setNumber("100860")
.setType(Message.Person.PhoneType.HOME)
.build();
Message.Person.Phone phone2 = phoneBuilder
.setNumber("100100")
.setType(Message.Person.PhoneType.MOBILE)
.build();
Message.Person.Builder personBuilder = Message.Person.newBuilder();
personBuilder.setId(1994);
personBuilder.setName("XIAOLEI");
personBuilder.addPhone(phone1);
personBuilder.addPhone(phone2);
Message.Person person = personBuilder.build();
long old = System.currentTimeMillis();
byte[] buff = person.toByteArray();
System.out.println("ProtoBuf 编码耗时：" + (System.currentTimeMillis() - old));
System.out.println(Arrays.toString(buff));
System.out.println("ProtoBuf 数据长度:" + buff.length);
Copy the code
6. Convert the ProtoBuf data back to a Java object
System.out.println("-开始解码-");
old = System.currentTimeMillis();
Message.Person personOut = Message.Person.parseFrom(buff);
System.out.println("ProtoBuf 解码耗时：" + (System.currentTimeMillis() - old));
System.out.printf("Id:%d, Name:%s\n", personOut.getId(), personOut.getName());
List<Message.Person.Phone> phoneList = personOut.getPhoneList();
for (Message.Person.Phone phone : phoneList)
{
System.out.printf("手机号:%s (%s)\n", phone.getNumber(), phone.getType());
}
Copy code
Compare
In order to reflect the advantages of ProtoBuf, I wrote a Java class with the same structure and converted Java objects into JSON data to compare with ProtoBuf. The JSON compilation library uses the GSON library provided by Google. Part of the JSON code will not be posted, and the results will be displayed directly.
Comparing results
Run 1 time
[JSON start encoding]
JSON encoding once, time-consuming: 22ms
JSON data length: 106
-start decoding-
JSON decoding once, time-consuming: 1ms
[ProtoBuf start encoding]
ProtoBuf encoding once, time-consuming: 32ms
ProtoBuf data length: 34
-start decoding-
ProtoBuf decoding once, time-consuming: 3ms
copy code
run 10 times
[JSON start encoding]
JSON encoding 10 times, time: 22ms
JSON data length: 106
-start decoding-
JSON decoding 10 times, time: 4ms
[ProtoBuf start encoding]
ProtoBuf encoding 10 times, time-consuming: 29ms
ProtoBuf data length: 34
-start decoding-
ProtoBuf decoding 10 times, time-consuming: 3ms
copy the code and
run 100 times
[JSON start encoding]
JSON encoding 100 times, time-consuming: 32ms
JSON data length: 106
-start decoding-
JSON decoding 100 times, time-consuming: 8ms
[ProtoBuf start encoding]
ProtoBuf encoding 100 times, time-consuming: 31ms
ProtoBuf data length: 34 -start decoding
-
ProtoBuf decoding 100 times, time-consuming: 4ms
copy code
run 1000 times
[JSON start encoding]
JSON encoding 1000 times, time-consuming: 39ms
JSON data length: 106
-start decoding-
JSON decoding 1000 times, time-consuming: 21ms
[ProtoBuf start encoding]
ProtoBuf encoding 1000 times, time-consuming: 37ms
ProtoBuf data length: 34 -start decoding
-
ProtoBuf decoding 1000 times, time-consuming: 8ms
copy code
run 10,000 times
[JSON start encoding]
JSON encoding 10000 times, time-consuming: 126ms
JSON data length: 106
-start decoding-
JSON decoding 10000 times, time-consuming: 93ms
] [Protobuf start encoding
protobuf encoding 10000, Processed: 49ms
protobuf data length: 34
- Start decoding -
protobuf decoding 10,000, Processed: 23ms
copy the code
to run 100,000
[Begin JSON encoding]
JSON encoding 100000 times, time-consuming: 248ms
JSON data length: 106 -Start
decoding-
JSON decoding 100000 times, time-consuming: 180ms
] [Protobuf start encoding
protobuf coding 100,000, Processed: 51ms
protobuf data length: 34
- Start decoding -
protobuf decoding 100,000, Processed: 58ms
copy the code
summary
Encoding and decoding performance The
above chestnuts are just simple sampling. In fact, according to my experiments,
When the frequency is less than 1,000, the encoding and decoding performance of ProtoBuf is comparable to JSON, and even worse than JSON.
The number of times is more than 2,000, and the encoding and decoding performance of ProtoBuf is much higher than that of JSON.
When the number is over 100,000, the codec performance of ProtoBuf is obvious, which is far higher than the performance of JSON.
Memory occupies
34 of ProtoBuf's memory, while JSON reaches 106, ProtoBuf's memory occupies only 1/3 of JSON.
end
In fact, this experiment has a lot to be optimized. Even this kind of rough test can see the advantages of ProtoBuf.
compatible
New field
Add the nickname field to the proto file
Generate Java files
Use the old proto byte array data, convert it into an object
Id: 1994, Name: XIAOLEI Mobile number: 100860 (HOME) Mobile number: 100100 (MOBILE) getNickname=
As a result, the conversion can be successful.
Delete field
Delete the name field in the proto file
Generate Java files
Use the old proto byte array data, convert it into an object
Id: 1994, Name: null Mobile number: 100860 (HOME) Mobile number: 100100 (MOBILE)
As a result, the conversion can be successful.