NewtonSoft Json deserialize decimal numbers with more then 8 decimals - json.net

I get the following json
"location": {
"x": 3693779.702309093,
"y": 500061.05750159378
}
And the following class
Class Location
public property x as string
public property y as string
end class
I need the json to be deserialised with all decimal positions
But when I deserialise with NewtonSoft Json, it is rounded to 8 decimals only.
Initially I had the class defined with x, y as decimals but I thought the rounding is caused by the data type, that's why I changed to string.
Still, it seems the actual Json deserialization is performed with only 8 decimals.
Any idea how can I make it to deserialize using all decimals?
Thank you

You may have found a bug in Json.NET itself. When reading a numeric value into a string, the value is first parsed to a numeric type (double, long or BigInteger), then back-converted to a string in JsonReader.ReadAsStringInternal():
string s;
if (Value is IFormattable)
s = ((IFormattable)Value).ToString(null, Culture);
else
s = Value.ToString();
SetToken(JsonToken.String, s, false);
return s;
The problem seems to be that Double.ToString(String, IFormatProvider) does not use round-trip precision unless the format string is explicitly specified to be "G17". If instead I use
System.ComponentModel.TypeDescriptor.GetConverter(Value).ConvertToString(null, Culture, Value)
there is no precision loss.
You might want to report an issue to NewtonSoft about this.
There are a couple of workarounds for this:
Deserialize your class with JsonSerializerSettings.FloatParseHandling = FloatParseHandling.Decimal. In this case the numeric value will be temporarily parsed to a decimal rather than a double.
Write a converter that loads the value into a JValue, checks for the value being of type JTokenType.Float, and returns the correct string:
public class StringNumericConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return objectType == typeof(string);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
if (reader.TokenType == JsonToken.Null)
return null;
var value = JToken.Load(reader);
if (value.Type == JTokenType.Float)
return System.ComponentModel.TypeDescriptor.GetConverter(((JValue)value).Value).ConvertToInvariantString(((JValue)value).Value);
return (string)value;
}
public override bool CanWrite { get { return false; } }
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
Or in VB.NET:
Public Class StringNumericConverter
Inherits JsonConverter
Public Overrides Function CanConvert(objectType As Type) As Boolean
Return objectType = GetType(String)
End Function
Public Overrides Function ReadJson(reader As JsonReader, objectType As Type, existingValue As Object, serializer As JsonSerializer) As Object
If reader.TokenType = JsonToken.Null Then
Return Nothing
End If
Dim value = JToken.Load(reader)
If value.Type = JTokenType.Float Then
Return System.ComponentModel.TypeDescriptor.GetConverter(DirectCast(value, JValue).Value).ConvertToInvariantString(DirectCast(value, JValue).Value)
End If
Return CType(value, String)
End Function
Public Overrides ReadOnly Property CanWrite() As Boolean
Get
Return False
End Get
End Property
Public Overrides Sub WriteJson(writer As JsonWriter, value As Object, serializer As JsonSerializer)
Throw New NotImplementedException()
End Sub
End Class
Note that for integral strings JValue.Value will be an Int64 or a BigInteger; you can't assume it will be a double.
Prototype fiddle.

I figured out the solution myself.
First I created the convertor below
public Class JsonGeometryConverter
Inherits JsonConverter
Public Overrides Function CanConvert(objectType As Type) As Boolean
Return True
End Function
Public Overrides Function ReadJson(reader As JsonReader, objectType As Type, existingValue As Object, serializer As JsonSerializer) As Object
Dim valF As double = reader.Value
Dim valFS = String.Format("{0:G17}", valF)
Return valFS
End Function
Public Overrides Sub WriteJson(writer As JsonWriter, value As Object, serializer As JsonSerializer)
Throw New NotImplementedException()
End Sub
End Class
Then I apply the converter attribute to the properties to extract values to
Public Class RestGIS_LocationInfo
<JsonConverter(GetType(JsonGeometryConverter))>
Public Property X As string = ""
<JsonConverter(GetType(JsonGeometryConverter))>
Public Property Y As String = ""
End Class
The converter reads the actual value from json data as double, and returns it formatted with up to 17 digits of precision (see https://msdn.microsoft.com/en-us/library/kfsatb94.aspx for details)

Related

How to declare the put method of Hash Table?

I don´t know how to resolve this error about the declaration of put method...
private Hashtable<Acronimo, Acronimo> lista;
public Dicionario() {
lista = new Hashtable<Acronimo, Acronimo>();
}
//adiciona
public boolean juntaAcronimo(Acronimo aAcronimo) {
if( lista != null) {
return lista.put(aAcronimo.getChave(), aAcronimo);
}
return false;
}
The method put(Acronimo, Acronimo) in the type Hashtable is not applicable for the arguments (String, Acronimo)
You have used invalid generic type in the lista field declaration, it should be:
private Hashtable<String, Acronimo> lista;
Consider using a HashMap instead of Hashtable. Hashtable is an old class from early Java version and due to backward compatibility it's needlessly synchronized.
private Map<String, Acronimo> lista = new HashMap<>();
Method aAcronimo.getChave() returns String while you defined your hashtable as <Acronimo, Acronimo> so all you have to do is to change this:
private Hashtable<String, Acronimo> lista;

Using JsonConverter to keep all datetime values formatted the same

I have a use case in which ALL date/time values must be saved in JSON documents using the following format:
yyyy-MM-ddTHH:mm:ss.fffffff
In other words, all date/time values will be exactly 27 characters in length regardless of the date/time value, for example:
System.DateTime.MinValue = "0001-01-01T00:00:00.0000000"
System.DateTime.MaxValue = "9999-12-31T23:59:59.9999999"
19-May-2018 10:35:12 am = "2018-05-19T10:35:12.0000000"
Unfortunately, Json.Net decides to format date/times in various formats depending on the date/time value. I have tried to use the JsonConverter attribute on date/time properties, ie:
[JsonConverter( typeof( datetime_iso8601_converter ) )]
public System.DateTime created_utc;
My converter class is as follows:
public class datetime_iso8601_converter : Newtonsoft.Json.Converters.IsoDateTimeConverter
{
public datetime_iso8601_converter()
{
// Set format that all DateTime values will use ...
base.DateTimeFormat = "yyyy-MM-ddTHH:mm:ss.fffffff";
}
}
If I run a test using the debugger, for example:
created_utc = System.DateTime.MinValue;
I can see the following line is executed:
base.DateTimeFormat = "yyyy-MM-ddTHH:mm:ss.fffffff";
However, the resulting JSON date/time value that gets written is:
"0001-01-01T00:00:00"
SECOND ATTEMPT:
I tried the following converter ...
public class datetime_iso8601_converter : Newtonsoft.Json.Converters.DateTimeConverterBase
{
// The format that all datetime values will use ...
private const String k_fmt_datetime_iso8601 = "yyyy-MM-ddTHH:mm:ss.fffffff";
// Write the given datetime value using the format above ...
public override void WriteJson( JsonWriter writer, object value, JsonSerializer serializer )
{
if ( value is System.DateTime )
{
String dt_str;
System.DateTime dt;
dt = ( System.DateTime ) value;
dt_str = dt.ToString( k_fmt_datetime_iso8601 );
writer.WriteRawValue( dt_str );
}
else
{
throw new ArgumentException( "value is not System.DateTime" );
}
}
// Return a datetime value ...
public override object ReadJson( JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer )
{
if ( reader.TokenType != Newtonsoft.Json.JsonToken.Date )
{
throw new Exception( "Invalid token. Expected Date" );
}
return reader.Value;
}
}
Again using the debugger I can see that dt_str is set to a value of:
"0001-01-01T00:00:00.0000000"
but the value that actually gets written is:
"0001-01-01T00:00:00"
I do not understand why.
HOW can I get all date/time values saved in the format required?
Thanks in advance.
The above issue was present in an environment that was using Azure Functions. The current release of Azure functions 1.0.13 does not seem to support the binding redirects, i.e.:
[JsonConverter( typeof( datetime_iso8601_converter ) )]
However, release 1.22.0 will support JsonSettings and binding redirects (hopefully).
Just ran into the same thing. The solution to is to override Newtonsoft's time format string and have it use fffffff instead of FFFFFFF 🙂
See https://github.com/JamesNK/Newtonsoft.Json/issues/2780#issuecomment-1363704872 for sample code and details.

How to Compare nullable objects

I need to compare a current List of Objects from a DB against a new List of Objects. I want to compare them and highlight for the user those which have changed (in this case, return TRUE that they are different).
Since some of my objects are Nullable, this involves a lot of IF NOT IS Nothing on the side of the NewObj and the CurrentObj...I've trying to find a more efficient way of writing the below as I have to use it to compare about 30 objects of different types, IE Date, Decimal, Int, etc..
The below works until say either of the Obj has no Rank and is therefore Nothing
Suggestions?
Dim Rank As Boolean = CompareData(NewObj, CurrentObj, "Rank")
Dim Regiment As Boolean = CompareData(NewObj, CurrentObj, "Rank")
Dim DateofBirth As Boolean = CompareData(NewObj, CurrentObj, "DoB")
Private Function CompareData(NewObj As Business.Casualty, CurrentObj As Business.Casualty, FieldToComapre As String) As Boolean
Select Case FieldToComapre
Case "DateOfBirth"
Return (Nullable.Equals(NewCasualty.DateOfBirth, CurrentCasualty.DateOfBirth))
Case "Age"
Return (Nullable.Equals(NewCasualty.Age, CurrentCasualty.Age))
Case "Rank"
Return (Nullable.Equals(NewCasualty.Rank.ID, CurrentCasualty.Rank.ID))
Case "Regiment"
Return (Nullable.Equals(NewCasualty.Regiment.ID, CurrentCasualty.Regiment.ID))
Case Else
Return True
End Select
End Function
My first suggestion is to change the name of your function to something like AreFieldValuesTheSame. My second suggestion is to negate the purpose of the function. In other words, return True when the values are the same.
If you follow my first two suggestions, then for each one of your cases, instead of simply
Return (Nullable.Equals(a.ID, b.ID))
you need something more like
Return ((a Is Nothing And b Is Nothing) Or (a.ID Is Nothing And b.ID Is Nothing) Or (Nullable.Equals(a.ID, b.ID)))
For example, a represents NewCasualty.Rank and b represents CurrentCasualty.Rank. You need to check for the object being Nothing (null in other words) before you try to check properties of the object.
Also, at the beginning of the function, you need to check whether NewObj Is Nothing and whether CurrentObj Is Nothing:
If (NewObj Is Nothing) And (CurrentObj Is Nothing) Then
Return True
Else If (NewObj Is Nothing) Or (CurrentObj Is Nothing)
Return False
Please forgive me if my VB syntax is not perfect. (I'm much more fluent in C#.)
I actually ended up just going with an Extension method..it was simpler and cleaner IMO.
Dim Rank As Boolean = CompareData(NewObj, CurrentObj, "Rank")
Dim Regiment As Boolean = CompareData(NewObj, CurrentObj, "Regiment")
Dim Trade As Boolean = CompareData(NewObj, CurrentObj, "Trade")
Private Function CompareData(NewObj As Business.Casualty, CurrentObj As Business.Casualty, FieldToComapre As String) As Boolean
Select Case FieldToComapre
Case "Trade"
Return NewCasualty.Trade.NullableEquals(CurrentCasualty.Trade)
Case "Rank"
Return NewCasualty.Rank.NullableEquals(CurrentCasualty.Rank)
Case "Regiment"
Return NewCasualty.Regiment.NullableEquals(CurrentCasualty.Regiment)
Case Else
Return True
End Select
End Function
public static class NullableCompare
{
public static bool NullableEquals<T>(this T s1, T s2)
where T : class
{
if (s1 == null)
{
return s2 == null;
}
return s1.Equals(s2);
}
}
public partial class Rank
{
public override bool Equals(object obj)
{
var p2 = obj as Rank;
if (p2 == null)
{
return false;
}
if (this.ID != p2.ID)
{
return false;
}
return this.ID == p2.ID;
}
public override int GetHashCode()
{
return this.ID.GetHashCode();
}
}

Sorting a list with Objects and A primitive String

I am trying to sort a list as given below.
List dg = new ArrayList();
dg.add(new Dog("one"));
dg.add(new Dog("two"));
dg.add(new Dog("three"));
dg.add(new Dog("four"));
dg.add(new String("Ankit"));
I have tried to implement comparable for Dog class below (not sure if it is useful here)
//Dog class below//
class Dog implements Comparable<Dog>{
String name;
public Dog(String name) {
this.name = name;
}
public String getName() {
return name;
}
public int compareTo(Dog d){
return this.getName().compareTo(d.getName());
}
The issue is however I write the comparator as given below I get ClassCastException. How to solve this?
class StringComparator implements Comparator {
public int compare(Object d, Object s) {
String p=((Dog)d).getName();
return (p.compareTo(s.toString()));
}
}
//usage of the above comparator
Arrays.sort(odog,sc);
Issue is:
See that to a list you are mixing String object with Dog objects using dg.add(new String("Ankit")); Do you really need string object.
Then you try to sort it using your comparator and your comparator blindly type cast values to Dog object. So when it encounters your String object it tries to type cast it to String object which is not possible (due to Dog not a subclass of String) and hence you get ClassCastException.
I would remove String object from Dog list to resolve the issue.

Correctly convert DateTime property with Dapper on SQLite

I'm using Dapper to insert and get objects to/from SQLite: one object have a property of type DateTime (and DateTimeOffset) that I have to persist and retrieve with milliseconds precision. I can't find a way to correctly retrieve the value because Dapper fail with:
System.FormatException : String was not recognized as a valid DateTime.
in System.DateTimeParse.ParseExactMultiple(String s, String[] formats, DateTimeFormatInfo dtfi, DateTimeStyles style)
in System.DateTime.ParseExact(String s, String[] formats, IFormatProvider provider, DateTimeStyles style)
in System.Data.SQLite.SQLiteConvert.ToDateTime(String dateText, SQLiteDateFormats format, DateTimeKind kind, String formatString)
in System.Data.SQLite.SQLite3.GetDateTime(SQLiteStatement stmt, Int32 index)
in System.Data.SQLite.SQLite3.GetValue(SQLiteStatement stmt, SQLiteConnectionFlags flags, Int32 index, SQLiteType typ)
in System.Data.SQLite.SQLiteDataReader.GetValue(Int32 i)
in System.Data.SQLite.SQLiteDataReader.GetValues(Object[] values)
in Dapper.SqlMapper.<>c__DisplayClass5d.<GetDapperRowDeserializer>b__5c(IDataReader r) in SqlMapper.cs: line 2587
in Dapper.SqlMapper.<QueryImpl>d__11`1.MoveNext() in SqlMapper.cs: line 1572
in System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
in System.Linq.Enumerable.ToList(IEnumerable`1 source)
in Dapper.SqlMapper.Query(IDbConnection cnn, String sql, Object param, IDbTransaction transaction, Boolean buffered, Nullable`1 commandTimeout, Nullable`1 commandType) in SqlMapper.cs: line 1443
in Dapper.SqlMapper.Query(IDbConnection cnn, String sql, Object param, IDbTransaction transaction, Boolean buffered, Nullable`1 commandTimeout, Nullable`1 commandType) in SqlMapper.cs: line 1382
What do I have to try? Column is of type DATETIME.
Do I have to create a custom TypeHandler and convert DateTime to and from a SQLite string in format "o"?
Dapper version 1.38
I know it's old, but I have found the solution.
After a lot of digging and analyzing Dapper code I came up with this (notice that this is 2019 year):
First you will have to create date time handler:
public class DateTimeHandler : SqlMapper.TypeHandler<DateTimeOffset>
{
private readonly TimeZoneInfo databaseTimeZone = TimeZoneInfo.Local;
public static readonly DateTimeHandler Default = new DateTimeHandler();
public DateTimeHandler()
{
}
public override DateTimeOffset Parse(object value)
{
DateTime storedDateTime;
if (value == null)
storedDateTime = DateTime.MinValue;
else
storedDateTime = (DateTime)value;
if (storedDateTime.ToUniversalTime() <= DateTimeOffset.MinValue.UtcDateTime)
return DateTimeOffset.MinValue;
else
return new DateTimeOffset(storedDateTime, databaseTimeZone.BaseUtcOffset);
}
public override void SetValue(IDbDataParameter parameter, DateTimeOffset value)
{
DateTime paramVal = value.ToOffset(this.databaseTimeZone.BaseUtcOffset).DateTime;
parameter.Value = paramVal;
}
}
Now, notice that Dapper translates .Net's type DateTimeOffset to dbType - DateTimeOffset. You need to remove this mapping and add your own like this:
SqlMapper.RemoveTypeMap(typeof(DateTimeOffset));
SqlMapper.AddTypeHandler(DateTimeHandler.Default);
That's all. Now everytime Dapper will see DateTimeOffset property in your model, it will run your DateTimeHandler to manage this.
I have found that custom TypeHandler for base types can't be used because of default typeMap that is choosen before looking for TypeHandler.
I have opened an issue dapper-dot-net but in the mean time I have solved replacing via reflection the default typeMap with a new one like the previous minus the four key DateTime, DateTime?, DateTimeOffset, DateTimeOffset?
I've made a slight modification to Adam Jachocki's solution as it didn't work for me. I am storing a date as TEXT in Sqlite and Dapper was giving me a string instead of a DateTime as the object value to parse. Apparently, Sqlite stores datetime values using three different data types: INTEGER (unix epoch), TEXT (ISO 8601 YYYY-MM-DD HH:MM:SS.SSS), and REAL ("number of days since noon in Greenwich on November 24, 4741 B.C."). That last one is really out there, so it isn't supported in the code below.
See the sqlite docs and this page for more info.
Below is my implementation of the DateTimeOffset TypeHandler. The rest of Adam's solution remains the same.
internal class DateTimeOffsetHandler : SqlMapper.TypeHandler<DateTimeOffset>
{
private static readonly TimeZoneInfo databaseTimeZone = TimeZoneInfo.Local;
private static readonly DateTime unixOrigin = new DateTime(1970, 1, 1, 0, 0, 0, 0);
public static DateTimeOffsetHandler Default { get; } = new DateTimeOffsetHandler();
public DateTimeOffsetHandler() {}
public override DateTimeOffset Parse(object value)
{
if (!TryGetDateTime(value, out DateTime storedDateValue))
{
throw new InvalidOperationException($"Unable to parse value {value} as DateTimeOffset");
}
if (storedDateValue.ToUniversalTime() <= DateTimeOffset.MinValue.UtcDateTime)
{
return DateTimeOffset.MinValue;
}
else
{
return new DateTimeOffset(storedDateValue, databaseTimeZone.BaseUtcOffset);
}
}
public override void SetValue(IDbDataParameter parameter, DateTimeOffset value)
{
DateTime paramVal = value.ToOffset(databaseTimeZone.BaseUtcOffset).DateTime;
parameter.Value = paramVal;
}
private bool TryGetDateTime(object value, out DateTime dateTimeValue)
{
dateTimeValue = default;
if (value is DateTime d)
{
dateTimeValue = d;
return true;
}
if (value is string v)
{
dateTimeValue = DateTime.Parse(v);
return true;
}
if (long.TryParse(value?.ToString() ?? string.Empty, out long l))
{
dateTimeValue = unixOrigin.AddSeconds(l);
return true;
}
if (float.TryParse(value?.ToString() ?? string.Empty, out float f))
{
throw new InvalidOperationException("Unsupported Sqlite datetime type, REAL.");
}
return false;
}
}

Resources