Network Game Protocol: Serialization and Deserialization Based on Protobuf

This section will explain the serialization and deserialization in online game development. When we want to store/transmit the data of a data object, we need to encode the data object instance into binary data, so that these binary data can be stored and transmitted. When we receive and read these binary data, we can reconstruct the corresponding data object instance according to the data. This process is called serialization and deserialization. This section will explain the serialization and deserialization based on Protobuf in detail from the following three points. as follows:

right! There is a unity learning exchange group that gathers a group of zero-based beginners who love to learn unity, and there are also some technical leaders who are engaged in unity development. You are welcome to exchange and learn.

Detailed explanation of the basic principles of serialization and deserialization based on binary data

Let's first explain the basic principles of binary serialization and deserialization through a simple case. The data of any complex object can be composed of simple data , such as the Vector3 structure object, which is composed of three float components x, y, and z. The basic data types of data objects include: Int, float, double, string, boolean, etc. Any object can embed sub-objects, or use an array to store multiple sub-objects. Sub-objects and arrays make the composition of our data objects very complex .

Next, let's take an object as an example to explain:

struct Vector {
    int x;
    int y;
    int z;
}

struct Size {
    int w;
    int h;
}
class Rect {
    Vector org;
    Size s;
}

The Rect object contains the Vector class and Size class sub-objects, and these sub-objects are composed of basic data. If we want to encode the Rect object, we must first encode the Vector object, and then encode the Size object, so that we can get the encoding of the Rect object. To solve it, we write the encoding function for the Rect object, as follows:

The basic data type has an encoding function. This does not need to be changed. It can be written once and used repeatedly later;

byte[] Float_encode(float) {…}

byte[] Int_encode(int) {…}

byte[] string_encode(string) {…}

byte[] bool_encode(bool) {…}

The internal structure of each complex data object is defined by the developer, so it is necessary to use how many objects there are to write the function implementation, and these functions can call the encoding function of the basic data type, and finally combine them into byte[ ];

byte[] Vector_encode(Vector v) { 
    byte[] b1 = int_encode(v.x);
    byte[] b2 = int_encode(v.y);
    byte[] b3 = int_encode(v.z);
    return b1 + b2 + b3;
}
byte[] Size_encode(Size s) { 
    byte[] b1 = int_encode(s.w);
    byte[] b2 = int_encode(s.h);
    return b1 + b2;
}
byte[] Rect_encode(Rect r) {
    byte[] b1 = Vector_encode(r.org);
    byte[] b2 = Size_encode(r.s);
    return b1 + b2;
}

Decoding is also similar. For basic data types, a decoding function can be implemented once. For complex objects, developers call the decoding of basic data types by themselves, and decode them one by one.

After the above analysis, we conclude as follows:

Design architecture and corresponding tools of Protobuf

The above four points are summarized and analyzed. In order to solve the above problems, Google has launched a protobuf open source project. At present, many of the serialization and deserialization of online games are based on it. Protobuf solves the four problems in the above sequence, and the main design is as follows:

(1) Encode/decode the binary data of the basic data type into a fixed function to form a "library", which we call the runtime library;

(2) We will implement multiple language versions of the runtime library to support mainstream programming languages;

(5) The developer only needs to define the data object to be serialized/deserialized into the protocol file, and then run the compiler to generate the code of the target programming language corresponding to the protocol file, and then the classes and objects in it can be used.

The Protoc protocol compiler is developed in C/C++ and can generate multiple programming languages. When compiling protobuf, this tool will also be compiled at the same time. The Runtime library code can be copied directly. After the protocol code is generated, it can be directly used in the engineering project. Use these generated data types directly in your code.

How Unity integrates with Protobuf

How does a Unity project use Protobuf? Follow the steps below to do it.

(1) The runtime library code of the built-in C# protobuf is transferred to the Unity project, copied over, and placed directly in Unity. I usually create a 3rd folder under Scripts to store it, as shown below:

(2) Use the cmake source code to compile protobuf to generate the protoc compilation tool. If you are too lazy to compile, you can directly download the compiled protoc on github.

(3) Define a protocol generation folder outside the Unity Assets folder, which is specially used to define the protocol file .proto, then run the compiler to generate c# code, and then copy it to the specified code directory of the project Assets.

That's all for today's sharing. Follow me to learn more about the development of online games.