In order to avoid losing JSON properties when deserializating to a POCO that is missing members, I use the [JsonExtensionData] attribute. Ex:
public class Foo
{
public int Y { get; set; }
[JsonExtensionData]
private IDictionary<string, JToken> _extraStuff;
}
That way, if I try to deserialize the following, I won't lose the z property:
{
"y": 1,
"z": "added in foo"
}
So far so good.
But in reality, I have a really deep object graph. So every POCO in the graph must use the [JsonExtensionData] attribute. This is a little dangerous. As soon as I forget to add this in one of the class, doing a deserialization followed by a serialization will lose data. (the real use case is doing a HTTP GET followed by a HTTP POST and I want to be sure that I don't lose anything)
So, to be sure that I haven't forgotten any [JsonExtensionData] in my whole POCO object tree, I thought about using the following deserializer setting:
var serializerSettings = new JsonSerializerSettings
{
MissingMemberHandling = MissingMemberHandling.Error
};
But then, if I try to deserialize the previous JSON, I get the following exception:
Could not find member 'z' on object of type 'Foo'. Path 'z', line 3, position 6.
This is a quite annoying, it complains about a field that has no member in the POCO but that is covered by the [JsonExtensionData] attribute.
Is there a way to only raise errors when data is actually data being lost during the deserialization?
You may mark your object with [JsonObject(MissingMemberHandling = MissingMemberHandling.Ignore)]. This will override the serializer setting:
[JsonObject(MissingMemberHandling = MissingMemberHandling.Ignore)]
public class Foo
{
public int Y { get; set; }
[JsonExtensionData]
private IDictionary<string, JToken> _extraStuff;
}
Demo fiddle #1 here.
Alternatively, you could create a custom contract resolver that does this automatically:
public class MissingMemberContractResolver : DefaultContractResolver
{
protected override JsonObjectContract CreateObjectContract(Type objectType)
{
var contract = base.CreateObjectContract(objectType);
if (contract.ExtensionDataSetter != null && contract.MissingMemberHandling == null)
{
contract.MissingMemberHandling = MissingMemberHandling.Ignore;
}
return contract;
}
}
Then use it as follow. First cache a copy somewhere for performance:
static IContractResolver contractResolver = new MissingMemberContractResolver
{
// Modify settings such as the naming strategy if required.
NamingStrategy = new CamelCaseNamingStrategy(),
};
And then set in settings as follows:
var serializerSettings = new JsonSerializerSettings
{
ContractResolver = contractResolver,
MissingMemberHandling = MissingMemberHandling.Error
};
var foo = JsonConvert.DeserializeObject<Foo>(json, serializerSettings);
var json2 = JsonConvert.SerializeObject(foo, Formatting.Indented, serializerSettings);
Demo fiddle #2 here.
Note that MissingMemberHandling was added to to JsonObjectAttribute and JsonObjectContract in Json.NET release 12.0.2. On earlier versions neither of the above solutions are available.
Honestly I'm a bit surprised this is necessary.
As an aside, if you're creating a custom contract resolver anyway, you could make DefaultContractResolver.CreateObjectContract() throw for any object in your assembly or namespace that lacks an ExtensionDataGetter and ExtensionDataSetter. If you do that you'll be able to discover any types that lack a [JsonExtensionData] during unit testing.
Related
tl;dr; In Newtonsoft JSON.NET, how do you ignore default values for some types (enums), and not others (ints)?
My team is consuming a library that uses protocol buffers for their business entities. Every enumeration in this library/protobuf has a default value of 0, "ValueNotSet". My team is using Newtonsoft JSON.NET to serialize these entities. Here's a diluted example for a bakery's inventory:
public enum Flavor { ValueNotSet, Cherry, Blueberry, Cheese };
public class DanishInventory { public int QtyInStock; public Flavor; }
In order to conserve resources we do not want to serialize ValueNotSet (the real world scenario has many enumerations), but having zero cheese danishes in stock is valid and we do want to serialize zero. Because of this, we cannot use DefaultValueHandling = Ignore in settings.
I created a custom JsonConverter, but by the time WriteJson(...) is called the key is already in the JsonWriter. So if I write nothing the JSON is invalid, and I don't see an obvious method to back-track the writer to overwrite a key. So what is the best way to ignore default values for some types (e.g. enums), but not others (e.g. ints)?
Note that the enums are in a NuGet package and cannot be modified, e.g. by adding attributes.
You can use a variation of the DefaultValueContractResolver from this answer to Json.NET: How to make DefaultValueHandling only apply to certain types? to exclude all enum-valued properties with default values:
public class EnumDefaultValueContractResolver : DefaultContractResolver
{
protected override JsonProperty CreateProperty(MemberInfo member, MemberSerialization memberSerialization)
{
var property = base.CreateProperty(member, memberSerialization);
if (property.DefaultValueHandling == null)
{
if (property.PropertyType.IsEnum)
{
//For safety you could check here if the default value is named ValueNotSet and only set IgnoreAndPopulate in that case.
//var defaultValue = Enum.ToObject(property.PropertyType, 0);
//if (defaultValue.ToString() == "ValueNotSet")
//{
property.DefaultValueHandling = DefaultValueHandling.IgnoreAndPopulate; // Or DefaultValueHandling.Ignore if you prefer
//}
}
}
return property;
}
}
Then use it as follows:
var resolver = new EnumDefaultValueContractResolver();
var settings = new JsonSerializerSettings { ContractResolver = resolver };
var json = JsonConvert.SerializeObject(inventory, settings);
You may want to cache the contract resolver for best performance.
Demo fiddle here.
Many serializers, (including Json.NET I believe) support the ShouldSerialize*() pattern; if you don't mind doing this on a per-usage basis, you could do:
public class DanishInventory {
public int QtyInStock;
public Flavor;
public bool ShouldSerializeFlavor() => Flavor != 0;
}
I have a third-party class (lets call it Class1) which I need to serialize to JSON. If I try to do this as is, I either receive StackOverflowException or JsonSerializationException with message "Self referencing loop detected with type". I've tried to apply the following settings for the JsonConvert but it didn't help me to avoid StackOverflowException
var settings = new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.None,
PreserveReferencesHandling = PreserveReferencesHandling.None,
ReferenceLoopHandling = ReferenceLoopHandling.Ignore
};
After decompiling the Class1 I found out that a lot of properties of the Class1 are marked with [ScriptIgnore] attribute which is an analogue of [JsonIgnore] and is used by System.Web.Script.Serialization.JavaScriptSerializer but I need to use Newtonsoft serializer.
As far as Class1 is a third-party class I can't add [JsonIgnore] attribute to the needed properties.
I know that I can develop my own implementation of IContractResolver, and handle the problematic properties there, but I'd like to avoid this option.
Maybe there is a way somehow configure Newtonsoft serializer to take into consideration [ScriptIgnore] attribute as well as [JsonIgnore]. And do this configuration like it is done with ReferenceLoopHandling?
I would appreciate for any ideas.
There's no configuration option for this. If you search on github, ScriptIgnoreAttribute doesn't even appear in the Json.NET source tree.
Even though you don't want to implement your own IContractResolver, this would be the straightforward solution and very easy. First, define the following subclass of DefaultContractResolver as follows:
public class ScriptIgnoreContractResolver : DefaultContractResolver
{
protected override JsonProperty CreateProperty(MemberInfo member, MemberSerialization memberSerialization)
{
var property = base.CreateProperty(member, memberSerialization);
if (!property.Ignored)
{
if (property.AttributeProvider.GetAttributes(true).Any(p => p.GetType().FullName == "System.Web.Script.Serialization.ScriptIgnoreAttribute"))
{
property.Ignored = true;
}
}
return property;
}
}
Then serialize as follows:
// Define a static member
static readonly IContractResolver myResolver = new ScriptIgnoreContractResolver();
// And use it in your serialization method
var settings = new JsonSerializerSettings
{
ContractResolver = myResolver,
};
var json = JsonConvert.SerializeObject(rootObject, settings);
You may want to cache the contract resolver for best performance.
In the following code, I serialize an object using Json.Net. This Json has type names embedded. I then change one of the type names to induce an error (this is a test, I am dealing with a real issue in an existing project). When I deserialize the Json, I expect to get an object back that has a null value for the property with the fiddled type name. Instead the serializer craps out and returns null. Are my expectations correct? Can I change the settings somehow so that I will get a non-null object for my root object? Note that the second error that I get suggests that there is a bug in the serializer.
static public class JsonTest
{
static public void Test()
{
// Create test object
A a = new A
{
MyTest = new MyTest(),
};
// Serialize it.
string json = JsonConvert.SerializeObject(a, new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.Auto
});
// Fiddle class name to induce error
json = json.Replace("+MyTest", "+MyTest2");
// Before: {"MyTest":{"$type":"<Namespace>.JsonTest+MyTest, <Assembly>"}}
// After: {"MyTest":{"$type":"<Namespace>.JsonTest+MyTest2, <Assembly>"}}
// Deserialize
A a2 = JsonConvert.DeserializeObject<A>(json, new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.Auto,
Error = (object sender, ErrorEventArgs e) =>
{
e.ErrorContext.Handled = true; // Should have only one error: the unrecognized Type
}
});
// A second error occurs: Error = {Newtonsoft.Json.JsonSerializationException: Additional text found in JSON string after finishing deserializing object....
// a2 is null
}
public class A
{
public ITest MyTest { get; set; }
}
public interface ITest { }
public class MyTest : ITest { }
}
Update
This issue has been fixed in Json.NET 10.0.2 in this submission.
Original Answer
This looks to be a bug in Json.NET. If I set JsonSerializerSettings.MetadataPropertyHandling = MetadataPropertyHandling.ReadAhead then the problem goes away:
// Deserialize
A a2 = JsonConvert.DeserializeObject<A>(json, new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.Auto,
MetadataPropertyHandling = MetadataPropertyHandling.ReadAhead,
Error = (object sender, Newtonsoft.Json.Serialization.ErrorEventArgs e) =>
{
Debug.WriteLine(e.ErrorContext.Path);
e.ErrorContext.Handled = true; // Should have only one error: the unrecognized Type
}
});
Debug.Assert(a2 != null); // No assert.
However, it should not be necessary to turn on this setting, which enables reading metadata properties including "$type" located anywhere in a JSON object, rather than just as the first property. Most likely it coincidentally fixes the bug since it requires pre-loading the entire JSON object before beginning deserialization.
You could report an issue if you want.
Debugging a bit, the problem seems to be that, because the inner MyTest object cannot be constructed, the exception is caught and handled by JsonSerializerInternalReader.PopulateObject() while populating the outer object A. Because of this, the JsonReader does not get advanced past the inner, nested object, leaving the reader and serializer in an inconsistent state. This accounts for the second exception and the eventual Additional text found in JSON string after finishing deserializing object. Path '' exception.
We're using ASP.NET WebAPI with Entity Framework (with lazy loading) and using Json.NET for serializing the data to JSON before returning the data to the client.
We are experiencing intermittent sudden spikes in memory usage which we suspect might originate with Json.NET not recognizing reference loops when serializing data (since Entity Framework might be doing some lazy loading voodoo with proxy classes which goes under the radar of Json.NET).
I thought I'd limit how deep Json.NET was allowed to go to serialize data (at least then we'd get a sensible exception when this happens so we could fix it in the data model), but I soon discovered that the MaxDepth property of JsonSerializerSettings only kicks in when DEserializing objects.
Is there any known way of imposing a limit on Json.NET when serializing?
I can't think of a way to do this out-of-the-box with Json.NET, since (as you correctly observe) MaxDepth is ignored when serializing. What you could do is to subclass JsonTextWriter and do the checks yourself:
public class MaxDepthJsonTextWriter : JsonTextWriter
{
public int? MaxDepth { get; set; }
public int MaxObservedDepth { get; private set; }
public MaxDepthJsonTextWriter(TextWriter writer, JsonSerializerSettings settings)
: base(writer)
{
this.MaxDepth = (settings == null ? null : settings.MaxDepth);
this.MaxObservedDepth = 0;
}
public MaxDepthJsonTextWriter(TextWriter writer, int? maxDepth)
: base(writer)
{
this.MaxDepth = maxDepth;
}
public override void WriteStartArray()
{
base.WriteStartArray();
CheckDepth();
}
public override void WriteStartConstructor(string name)
{
base.WriteStartConstructor(name);
CheckDepth();
}
public override void WriteStartObject()
{
base.WriteStartObject();
CheckDepth();
}
private void CheckDepth()
{
MaxObservedDepth = Math.Max(MaxObservedDepth, Top);
if (Top > MaxDepth)
throw new JsonSerializationException(string.Format("Depth {0} Exceeds MaxDepth {1} at path \"{2}\"", Top, MaxDepth, Path));
}
}
Then, to manually generate a JSON string, you would use it like this:
var settings = new JsonSerializerSettings { MaxDepth = 10 };
string json;
try
{
using (var writer = new StringWriter())
{
using (var jsonWriter = new MaxDepthJsonTextWriter(writer, settings))
{
JsonSerializer.Create(settings).Serialize(jsonWriter, myClass);
// Log the MaxObservedDepth here, if you want to.
}
json = writer.ToString();
}
Debug.WriteLine(json);
}
catch (Exception ex)
{
Debug.WriteLine(ex);
throw;
}
Demo fiddle here.
Since your tags include web-api, if you want to do this check inside web API calls, you could follow Rick Strahl's instructions to create a custom MediaTypeFormatter for JSON: Using an alternate JSON Serializer in ASP.NET Web API; then use the code above in the OnWriteToStreamAsync method when generating the json string.
I'm working on a ASP.NET WebApi (Release Candidate) project where I'm consuming several DTOs that are marked with the [Serializable] attribute. These DTOs are outside of my control so I'm not able to modify them in any way. When I return any of these from a get method the resulting JSON contains a bunch of k__BackingFields like this:
<Name>k__BackingField=Bobby
<DateCreated>k__BackingField=2012-06-19T12:35:18.6762652-05:00
Based on the searching I've done this seems like a problem with JSON.NET's IgnoreSerializableAttribute setting and to resolve my issue I just need to set it globally as the article suggests. How do I change this setting globally in a ASP.NET Web api project?
I found easy way to get rid of k__BackingField in the names.
This fragment should be somewhere in the Application_Start() in Global.asax.cs:
JsonSerializerSettings jSettings = new Newtonsoft.Json.JsonSerializerSettings();
GlobalConfiguration.Configuration.Formatters.JsonFormatter.SerializerSettings = jSettings;
Looks like the default setting takes care of it.
Since the library does not expose a static setter for the DefaultContractResolver, I suggest you create a static wrapper over JsonConvert and it's Serialize*/Deserialize* methods (at least the ones you use).
In your static wrapper you can define a static contract resolver:
private static readonly DefaultContractResolver Resolver = new DefaultContractResolver
{
IgnoreSerializableAttribute = true
};
This you can pass to each serialization method in the JsonSerializerSettings, inside your wrapper.
Then you call your class throughout your project.
The alternative would be to get the JSON.NET source code and adjust it yourself to use that attribute by default.
For me, the following fixed the issue with circular references and k__BackingField.
In your WebApiConfig add the following to the Register() method:
JsonSerializerSettings jSettings = new Newtonsoft.Json.JsonSerializerSettings {
ContractResolver = new DefaultContractResolver {
IgnoreSerializableAttribute = true
},
ReferenceLoopHandling = ReferenceLoopHandling.Ignore
};
GlobalConfiguration.Configuration.Formatters.JsonFormatter.SerializerSettings = jSettings;
Friends, don't declare properties like this:
public String DiscretionCode { get; set; }
But, create auxiliar vars, like old....
private String discretionCode;
public String DiscretionCode
{
get { return discretionCode;}
set { discretionCode = value; }
}