I have some issue with getting values from TimeDate and Guid types from protobuf-net:
I have .Net Client and .Net Server and they are communicating via protobuf-net. And now I have to implement a communication from a java client to this .Net Server and I can not change already existed .Net Server communication logic so I have to use already existed protobuf communication and the issue is following:
protobuf-net understands two .net types: DateTime and Guid but I'm not able to parse it via google protobuf:
.Net Server class example:
[ProtoContract]
public class SomeClass
{
[ProtoMember(1)]
public DateTime? CurrentDateTime { get; set; }
[ProtoMember(2)]
public Guid CurrentGiud { get; set; }
}
I'm not able to parse it via google protobuf because it knows nothing about DateTime and Guid types so I can only get byte[] from these fields, .proto example:
message SomeClass
{
bytes CurrentDateTime = 1;
bytes CurrentGiud = 2;
}
So after serialization/deserialization a stream I can get byte[] from those fields and now I need somehow convert it to appropriate values, so I need something like this:
var customDateTime = ConvertByteArrayToCustomDateTime(byteArray);
byte[] byteArray = ConvertCustomDateTimeToByteArray(customDateTime);
var customGuid = ConvertByteArrayToCustomGuid(byteArray);
byte[] byteArray = ConvertCustomGuidToByteArray(customGuid);
or this:
string strDateTime = ConvertByteArrayToStringDateTime(byteArray); //e.g. "13.08.2019 17:42:31"
byte[] byteArray = ConvertStringDateTimeToByteArray(strDateTime);
string strGuid = ConvertByteArrayToStringGuid(byteArray); // e.g. "{7bb7cdac-ebad-4acf-90ff-a5525be3caac}"
byte[] byteArray = ConvertStringGuidToByteArray(strGuid);
DateTime real example:
Example N1:
DateTime = 13.08.2019 17:42:31
after serialization/deserialization
byte[] = { 8, 142, 218, 151, 213, 11, 16, 3 }
Example N2:
DateTime = 25.06.2019 20:15:10
after serialization/deserialization
byte[] = { 8, 156, 131, 148, 209, 11, 16, 3 }
Guid real example:
Example N1:
Guid = {7bb7cdac-ebad-4acf-90ff-a5525be3caac}
after serialization/deserialization
byte[] = { 9, 172, 205, 183, 123, 173, 235, 207, 74, 17, 144, 255, 165, 82, 91, 227, 202, 172 }
Example N2:
Guid = {900246bb-3a7b-44d4-9b2f-1da035ca51f4}
after serialization/deserialization
byte[] = { 9, 187, 70, 2, 144, 123, 58, 212, 68, 17, 155, 47, 29, 160, 53, 202, 81, 244 }
Solution:
Add following messages into your .proto:
message CustomDateTime
{
sint64 value = 1; // the offset (in units of the selected scale) from 1970/01/01
CustomTimeSpanScale scale = 2; // the scale of the timespan [default = DAYS]
CustomDateTimeKind kind = 3; // the kind of date/time being represented [default = UNSPECIFIED]
enum CustomTimeSpanScale
{
DAYS = 0;
HOURS = 1;
MINUTES = 2;
SECONDS = 3;
MILLISECONDS = 4;
TICKS = 5;
MINMAX = 15; // dubious
}
enum CustomDateTimeKind
{
// The time represented is not specified as either local time or Coordinated Universal Time (UTC).
UNSPECIFIED = 0;
// The time represented is UTC.
UTC = 1;
// The time represented is local time.
LOCAL = 2;
}
}
message CustomGuid
{
fixed64 lo = 1; // the first 8 bytes of the guid (note:crazy-endian)
fixed64 hi = 2; // the second 8 bytes of the guid (note:crazy-endian)
}
Now your .proto class should look like this:
message SomeClass
{
CustomDateTime CurrentDateTime = 1;
CustomGuid CurrentGiud = 2;
}
Simple DateTime parser:
public static string ConvertCustomDateTimeToString(CustomDateTime customDateTime)
{
var dateTime = DateTime.Parse("01.01.1970 00:00:00");
if (customDateTime.Scale == CustomDateTime.Types.CustomTimeSpanScale.Seconds)
{
dateTime = dateTime.AddSeconds(customDateTime.Value);
}
else
{
throw new Exception("CustomDateTime supports only seconds");
}
return dateTime.ToString();
}
public static CustomDateTime ConvertStringToCustomDateTime(string strDateTime)
{
var defaultTime = DateTime.Parse("01.01.1970 00:00:00");
var dateTime = DateTime.Parse(strDateTime);
var customDateTime = new CustomDateTime
{
Kind = CustomDateTime.Types.CustomDateTimeKind.Unspecified,
Scale = CustomDateTime.Types.CustomTimeSpanScale.Seconds,
Value = (long) (dateTime - defaultTime).TotalSeconds
};
return customDateTime;
}
Simple Guid parser:
public static string ConvertCustomGuidToString(CustomGuid customGuid)
{
var str = string.Empty;
var array = BitConverter.GetBytes(customGuid.Lo);
var newArray = new byte[8];
newArray[0] = array[3];
newArray[1] = array[2];
newArray[2] = array[1];
newArray[3] = array[0];
newArray[4] = array[5];
newArray[5] = array[4];
newArray[6] = array[7];
newArray[7] = array[6];
str += BitConverter.ToString(newArray).Replace("-", "");
str += BitConverter.ToString(BitConverter.GetBytes(customGuid.Hi)).Replace("-", "");
return str;
}
public static CustomGuid ConvertStringToCustomGuid(string strGuid)
{
strGuid = strGuid.Replace(" ", "");
strGuid = strGuid.Replace("-", "");
strGuid = strGuid.Replace("{", "");
strGuid = strGuid.Replace("}", "");
if (strGuid.Length != 32)
{
throw new Exception("Wrong Guid format");
}
byte[] array = new byte[16];
for (int i = 0; i < 32; i += 2)
array[i / 2] = Convert.ToByte(strGuid.Substring(i, 2), 16);
var newArrayLo = new byte[8];
newArrayLo[0] = array[3];
newArrayLo[1] = array[2];
newArrayLo[2] = array[1];
newArrayLo[3] = array[0];
newArrayLo[4] = array[5];
newArrayLo[5] = array[4];
newArrayLo[6] = array[7];
newArrayLo[7] = array[6];
var newArrayHi = new byte[8];
newArrayHi[0] = array[8];
newArrayHi[1] = array[9];
newArrayHi[2] = array[10];
newArrayHi[3] = array[11];
newArrayHi[4] = array[12];
newArrayHi[5] = array[13];
newArrayHi[6] = array[14];
newArrayHi[7] = array[15];
var customGuid = new CustomGuid
{
Lo = BitConverter.ToUInt64(newArrayLo, 0),
Hi = BitConverter.ToUInt64(newArrayHi, 0)
};
return customGuid;
}
We need to talk about the two types separately. There's back-story to each!
DateTime / TimeSpan - so: way back in history, .NET folks kept wanting protobuf-net to round-trip DateTime / TimeSpan. There was nothing defined by Google for this kind of purpose, so protobuf-net made something up. The details are in bcl.proto, but I don't recommend worrying about that. The short version would be: "they're kinda awkward to work with if you're not protobuf-net".
Roll forward 5+ years, and Google finally defined the well-known Duration and Timestamp types. Unfortunately, they're not 1:1 matches for how protobuf-net decided to implement them, and I can't change the default layout without breaking existing consumers. But! For new code, or for cross-platform purposes, protobuf-net knows how to talk Duration / Timestamp, and if it is even remotely possible, I strongly recommend changing your layout. The good news is: this is really simple:
[ProtoMember(1, DataFormat = DataFormat.WellKnown)]
public DateTime? CurrentDateTime { get; set; }
This will now use .google.protobuf.Timestamp instead of .bcl.DateTime; this also works with TimeSpan / .google.protobuf.Duration.
The key point here: there is a simple option that you can switch to that will make this "just work"; the default is for compatibility with something that protobuf-net had to invent before Google had decided on a layout.
Note that changing to DataFormat.WellKnown is a data breaking change; the layout is different. If there was a way to automatically detect and compensate, it already would; there isn't.
Guid - this should have been much simpler; the sensible idea here would have been to just to serialize it as bytes in .proto terms, but ... and I regret this, I did a stupid and tried to do something clever. It backfired and I regret it. What it does is ... kinda silly, although it does make sense internally. It accesses the Guid as two consecutive fixed64 fields (again, look in bcl.proto) where these are the low/high bytes in Microsoft's craz-endian layout. By crazy-endian, I mean where the guid 00112233-4455-6677-8899-AABBCCDDEEFF is represented by the bytes 33-22-11-00-55-44-77-66-88-99-AA-BB-CC-DD-EE-FF (emphasis: this bit isn't me; this is what Microsoft and .NET do internally with guids). So; to take your N1 example, the two half fragments you're seeing there are:
- 0x09 = "field 1, type fixed64"
- 0x AC-CD-B7-7B-AD-EB-CF-4A - first half in crazy-endian
- 0x11 (decimal 17) = "field 2, type fixed64"
- 0x 90-FF-A5-52-5B-E3-CA-AC - second half in crazy-endian
Frankly, for cross-platform work, I would suggest for Guid, expose it as string or byte[] instead, and accept my sincere apologies for the inconvenience!
In both cases: if you can't change the layout, look in bcl.proto for what is actually happening. If you use Serializer.GetProto<T>(), it should generate a schema that imports bcl.proto automatically.