When serializing and deserializing values between JavaScript and C# using SignalR with MessagePack I am seeing a bit of precision loss in C# on the receiving end.
As an example I am sending the value 0.005 from JavaScript to C#. When the deserialized value appears on the C# side I am getting the value 0.004999999888241291, which is close, but not 0.005 exactly. The value on the JavaScript side is Number and on the C# side I am using double.
I have read that JavaScript can't represent floating point numbers exactly which can lead to results like 0.1 + 0.2 == 0.30000000000000004. I suspect the issue I am seeing is related to this feature of JavaScript.
The interesting part is that I am not seeing the same issue going the other way. Sending 0.005 from C# to JavaScript results in the value 0.005 in JavaScript.
Edit: The value from C# is just shortened in the JS debugger window. As @Pete mentioned it does expand to something that is not 0.5 exactly (0.005000000000000000104083408558). This means the discrepancy happens on both sides at least.
JSON serialization does not have the same issue since I am assuming it goes via string which leaves the receiving environment in control wrt parsing the value into its native numerical type.
I am wondering if there is a way using binary serialization to have matching values on both sides.
If not, does this mean that there is no way to have 100% accurate binary conversions between JavaScript and C#?
Technology used:
- JavaScript
- .Net Core with SignalR and msgpack5
My code is based on this post.
The only difference is that I am using ContractlessStandardResolver.Instance.
I tested float and double, and interestingly in this particular case, only double had the problem, whereas float seems to be working (i.e. 0.005 is read on server).
Inspecting on the message bytes suggested that 0.005 is sent as type Float32Double which is a 4-byte / 32-bit IEEE 754 single precision floating point number despite Number is 64 bit floating point.
Run the following code in console confirmed the above:
msgpack5().encode(Number(0.005))
// Output
Uint8Array(5) [202, 59, 163, 215, 10]
mspack5 does provide an option to force 64 bit floating point:
msgpack5({forceFloat64:true}).encode(Number(0.005))
// Output
Uint8Array(9) [203, 63, 116, 122, 225, 71, 174, 20, 123]
However, the forceFloat64 option is not used by signalr-protocol-msgpack.
Though that explains why float works on the server side, but there isn't really a fix for that as of now. Let's wait what Microsoft says.
Possible workarounds
- Hack msgpack5 options? Fork and compile your own msgpack5 with
forceFloat64default to true?? I don't know. - Switch to
floaton server side - Use
stringon both sides - Switch to
decimalon server side and write customIFormatterProvider.decimalis not primitive type, andIFormatterProvider<decimal>is called for complex type properties - Provide method to retrieve
doubleproperty value and do thedouble->float->decimal->doubletrick - Other unrealistic solutions you could think of
TL;DR
The problem with JS client sending single floating point number to C# backend causes a known floating point issue:
// value = 0.00499999988824129, crazy C# :)
var value = (double)0.005f;
For direct uses of double in methods, the issue could be solved by a custom MessagePack.IFormatterResolver:
public class MyDoubleFormatterResolver : IFormatterResolver
{
public static MyDoubleFormatterResolver Instance = new MyDoubleFormatterResolver();
private MyDoubleFormatterResolver()
{ }
public IMessagePackFormatter<T> GetFormatter<T>()
{
return MyDoubleFormatter.Instance as IMessagePackFormatter<T>;
}
}
public sealed class MyDoubleFormatter : IMessagePackFormatter<double>, IMessagePackFormatter
{
public static readonly MyDoubleFormatter Instance = new MyDoubleFormatter();
private MyDoubleFormatter()
{
}
public int Serialize(
ref byte[] bytes,
int offset,
double value,
IFormatterResolver formatterResolver)
{
return MessagePackBinary.WriteDouble(ref bytes, offset, value);
}
public double Deserialize(
byte[] bytes,
int offset,
IFormatterResolver formatterResolver,
out int readSize)
{
double value;
if (bytes[offset] == 0xca)
{
// 4 bytes single
// cast to decimal then double will fix precision issue
value = (double)(decimal)MessagePackBinary.ReadSingle(bytes, offset, out readSize);
return value;
}
value = MessagePackBinary.ReadDouble(bytes, offset, out readSize);
return value;
}
}
And use the resolver:
services.AddSignalR()
.AddMessagePackProtocol(options =>
{
options.FormatterResolvers = new List<MessagePack.IFormatterResolver>()
{
MyDoubleFormatterResolver.Instance,
ContractlessStandardResolver.Instance,
};
});
The resolver is not perfect, as casting to decimal then to double slows the process down and it could be dangerous.
However
As per the OP pointed out in the comments, this cannot solve the issue if using complex types having double returning properties.
Further investigation revealed the cause of the problem in MessagePack-CSharp:
// Type: MessagePack.MessagePackBinary
// Assembly: MessagePack, Version=1.9.0.0, Culture=neutral, PublicKeyToken=b4a0369545f0a1be
// MVID: B72E7BA0-FA95-4EB9-9083-858959938BCE
// Assembly location: ...\.nuget\packages\messagepack\1.9.11\lib\netstandard2.0\MessagePack.dll
namespace MessagePack.Decoders
{
internal sealed class Float32Double : IDoubleDecoder
{
internal static readonly IDoubleDecoder Instance = (IDoubleDecoder) new Float32Double();
private Float32Double()
{
}
public double Read(byte[] bytes, int offset, out int readSize)
{
readSize = 5;
// The problem is here
// Cast a float value to double like this causes precision loss
return (double) new Float32Bits(bytes, checked (offset + 1)).Value;
}
}
}
The above decoder is used when needing to convert a single float number to double:
// From MessagePackBinary class
MessagePackBinary.doubleDecoders[202] = Float32Double.Instance;
v2
This issue exists in v2 versions of MessagePack-CSharp. I have filed an issue on github, though the issue is not going to be fixed.