Article Directory
Data sequence framework --protobuf
1. Overview
- Introduction
protobuf full name Protocol Buffers, Google is a data exchange format (protocol) for serializing the data, for efficient transmission between different servers. If you are not very understanding, to be an analogy, we usually used the http, xml, json, etc., it is a data exchange protocol. - Feature
- Cross-platform, cross-language
- Scalability
- A small amount of data is serialized, the transmission speed
- High resolution efficiency
- GitHub Address: https://github.com/protocolbuffers/protobuf
2. The number of issues of common transmission
- Different languages, to transfer data among different servers
Clearly, the Java server side Java serialization their Person object obtained byte stream transmitted to the server side Python, Python side to resolve their own way is not feasible. Therefore, the general use xml, json, http and other data exchange protocol. Because byte character stream flow, we are the same way, only you need a good agreement both ends of the character parsed format on the line. - Slow transmission efficiency, the amount of data, serialization and deserialization inefficient
figure, Java first end of the Person object in accordance with the agreed format encoded json format string is then serialized, and then transmitted to the byte stream data Python side. After Python receives the byte stream data is deserialized and then decoded to the Person object according to the agreed format. This is possible.
When we use the json transmission format, our data exchange already has a cross-platform, cross-language capabilities, but because of the increasing amount of data exchange network, the drawbacks of the byte stream into a character stream and then analytically also exposed : the large space, transmission efficiency is not enough, not enough encoding and decoding performance.
What if a uniform definition of analytical specification byte stream is not good enough? However, each written in a different language every byte stream parsing code (byte stream operation too complicated), every change once data format protocols have to re-write the code once parse byte stream, it is estimated programmers to do analytical I will curse "WTF"! ! This time, we are very distressed, Google's protobuf this to solve your problem! - protobuf did what?
First, Google unified platform for each language defines a uniform data exchange format, you only need to write message.proto file in accordance with the specification (object-data exchange). Secondly, Google provides protoc utility for message.proto parsed into each language code file. Finally, different language platform programmers only need to hold each file language code, the code calls the method of transmission over the byte stream serialization, deserialization can!
3. Use protobuf example
Note: There are all top of the article Download
- Download proroc parser
- GitHub address: https: //github.com/protocolbuffers/protobuf/releases
- Download protoc-3.9.1-win64.zip, you can extract
- Written agreement document city_message.proto
// 指定使用哪个版本的语法 syntax = "proto3"; package pbexample; // 指定 package路径 option java_package = "com.skey.pbexample.protocol"; // 生成的类名 option java_outer_classname = "CityProtocols"; message CityMessage { // 后面的 1 表示字段名，一个message中所有的字段名都是唯一的 string name = 1; // Enum 类型 enum Season { SPRING = 0; SUMMER = 1; AUTUMN = 2; WINTER = 3; } Season season = 2; // repeated: 声明一个List<Report> repeated Report news = 3; } message Report { // String 类型 int32 id = 1; // Map 类型 map<string, string> kv = 2; } - Using the following command generating CityProtocols.java
bin\protoc.exe -I=D:\test --java_out=D:\test D:\test\city_message.proto 第一个参数: -I 用于指定--proto_path （协议文件的父目录） 第二个参数: --java_out 用于指定生成的文件的路径 第三个参数: 用于指定待解析的协议文件（也可以写成相对父目录的相对路径.\city_message.proto） - The resulting package path corresponding to the item in the copy-in CityProtocols.java
- java code incorporated in dependence protobuf-java.jar (Protoc program version corresponding to the previous version of the note)
<dependencies> <dependency> <groupId>com.google.protobuf</groupId> <artifactId>protobuf-java</artifactId> <version>3.9.1</version> </dependency> </dependencies>
- Examples of basic tests
import com.google.protobuf.InvalidProtocolBufferException; import com.skey.pbexample.protocol.CityProtocols; /** * Descr: ProtocolBuffer测试 * Date: 2019/8/20 19:10 * * @author A Lion~ */ public class TestPB { public static void main(String[] args) throws InvalidProtocolBufferException { // 构建2个简单的新闻报道 CityProtocols.Report report1 = CityProtocols.Report.newBuilder() .setId(1) .putKv("Football", "value1") .putKv("Basketball", "value2") .build(); CityProtocols.Report report2 = CityProtocols.Report.newBuilder() .setId(2) .putKv("ThreeGorges", "value3") .putKv("Taoyuan", "value4") .build(); // 构建整个城市的新闻 CityProtocols.CityMessage cityMessage = CityProtocols.CityMessage.newBuilder() .setName("ChongQing") .setSeason(CityProtocols.CityMessage.Season.SUMMER) .addNews(report1) .addNews(report2) .build(); // 转为字节数组 byte[] byteArray = cityMessage.toByteArray(); // 解析，将字节数组转为对象 CityProtocols.CityMessage message = CityProtocols.CityMessage.parseFrom(byteArray); System.out.println(message.getName()); System.out.println(message.getSeason()); System.out.println(message.getNewsList()); System.out.println("---------"); System.out.println(message); } } - Example network transmission
- Server code
import com.skey.pbexample.protocol.CityProtocols; import java.io.IOException; import java.io.InputStream; import java.net.ServerSocket; import java.net.Socket; /** * Descr: ProtocolBuffer服务端 * Date: 2019/8/21 16:17 * * @author A Lion~ */ public class ServerDemo { public static void main(String[] args) throws IOException { // 建立Socket服务端 ServerSocket server = new ServerSocket(23333); // 循环，一直接收消息 while (true) { // 接收数据 Socket socket = server.accept(); InputStream is = socket.getInputStream(); // 解析InputStream，转为CityMessage对象 CityProtocols.CityMessage message = CityProtocols.CityMessage.parseFrom(is); // 打印信息 System.out.println("---------------------------------------------------------"); System.out.println(message.getName()); System.out.println(message.getSeason()); System.out.println(message.getNewsList()); is.close(); socket.close(); } // server.close(); } }- Client code
import com.skey.pbexample.protocol.CityProtocols; import java.io.IOException; import java.io.OutputStream; import java.net.Socket; /** * Descr: ProtocolBuffer客户端 * Date: 2019/8/21 16:22 * * @author A Lion~ */ public class ClientDemo { public static void main(String[] args) throws IOException { // 构建2个简单的新闻报道 CityProtocols.Report report1 = CityProtocols.Report.newBuilder() .setId(1) .putKv("Football", "value1") .putKv("Basketball", "value2") .build(); CityProtocols.Report report2 = CityProtocols.Report.newBuilder() .setId(2) .putKv("ThreeGorges", "value3") .putKv("Taoyuan", "value4") .build(); // 构建整个城市的新闻 CityProtocols.CityMessage cityMessage = CityProtocols.CityMessage.newBuilder() .setName("ChongQing") .setSeason(CityProtocols.CityMessage.Season.SUMMER) .addNews(report1) .addNews(report2) .build(); // 建立Socket连接 Socket socket = new Socket("127.0.0.1", 23333); // 建立连接后获得输出流 OutputStream outputStream = socket.getOutputStream(); // 转为字节数组 byte[] byteArray = cityMessage.toByteArray(); // 将数据传输过去 outputStream.write(byteArray); outputStream.close(); socket.close(); } }
