I'm trying to deserialize some JSON objects using Json.NET. I've found however that when I deserialize an object that doesn't have the properties I'm looking for that no error is thrown up but a default value is returned for the properties when I access them. It's important that I'm able to detect when I've deserialized the wrong type of object. Example code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Newtonsoft.Json;
namespace Json_Fail_Test
{
class Program
{
[JsonObject(MemberSerialization.OptOut)]
private class MyJsonObjView
{
[JsonProperty("MyJsonInt")]
public int MyJsonInt { get; set; }
}
const string correctData = #"
{
'MyJsonInt': 42
}";
const string wrongData = #"
{
'SomeOtherProperty': 'fbe8c20b'
}";
static void Main(string[] args)
{
var goodObj = JsonConvert.DeserializeObject<MyJsonObjView>(correctData);
System.Console.Out.WriteLine(goodObj.MyJsonInt.ToString());
var badObj = JsonConvert.DeserializeObject<MyJsonObjView>(wrongData);
System.Console.Out.WriteLine(badObj.MyJsonInt.ToString());
}
}
}
The output of this program is:
42
0
I would prefer an exception be thrown to failing silently. Short of that is there a way to detect if the serialization failed to find a parameter?
I know I can parse the data with a Json object and then check for the parameter with a key value lookup but the codebase I'm in uses the pattern above and I'd like keep that consistent if it's possible.
The Json.Net serializer has a MissingMemberHandling setting which you can set to Error. (The default is Ignore.) This will cause the serializer to throw a JsonSerializationException during deserialization whenever it encounters a JSON property for which there is no corresponding property in the target class.
static void Main(string[] args)
{
try
{
JsonSerializerSettings settings = new JsonSerializerSettings();
settings.MissingMemberHandling = MissingMemberHandling.Error;
var goodObj = JsonConvert.DeserializeObject<MyJsonObjView>(correctData, settings);
System.Console.Out.WriteLine(goodObj.MyJsonInt.ToString());
var badObj = JsonConvert.DeserializeObject<MyJsonObjView>(wrongData, settings);
System.Console.Out.WriteLine(badObj.MyJsonInt.ToString());
}
catch (Exception ex)
{
Console.WriteLine(ex.GetType().Name + ": " + ex.Message);
}
}
Result:
42
JsonSerializationException: Could not find member 'SomeOtherProperty' on object
of type 'MyJsonObjView'. Path 'SomeOtherProperty', line 3, position 33.
See: MissingMemberHandling setting.
Just add [JsonProperty(Required = Required.Always)] to the required properties and it'll throw exception if the property is not there while deserializing.
[JsonProperty(Required = Required.Always)]
public int MyJsonInt { get; set; }
Put the following attribute on required properties:
[DataMember(IsRequired = true)]
If the member is not present, it will throw a Newtonsoft.Json.JsonSerializationException.
As Brian suggested below, you will also need this attribute on your class:
[DataContract]
As #dbc tells in comments:
At deserialization:
If your Model has a property that your JSON does not, and you want that to be an error, use [JsonProperty(Required = Required.Always)].
At serialization:
If your JSON has a property that your Model does not, and you want that to be an error, use MissingMemberHandling = MissingMemberHandling.Error.
also using [DataMember(IsRequired = true)] for error at deserialization is true when proeprty type is nullable.
Just define your members in your definition class with a question mark '?' int?:
private class MyJsonObjView
{
[JsonProperty("MyJsonInt")]
public int? MyJsonInt { get; set; }
}
When it is not initialized, it will just be null, otherwise it will be a valid value. This allows you to have settings optional and evaluate them on a per-setting basis.
Related
In order to avoid losing JSON properties when deserializating to a POCO that is missing members, I use the [JsonExtensionData] attribute. Ex:
public class Foo
{
public int Y { get; set; }
[JsonExtensionData]
private IDictionary<string, JToken> _extraStuff;
}
That way, if I try to deserialize the following, I won't lose the z property:
{
"y": 1,
"z": "added in foo"
}
So far so good.
But in reality, I have a really deep object graph. So every POCO in the graph must use the [JsonExtensionData] attribute. This is a little dangerous. As soon as I forget to add this in one of the class, doing a deserialization followed by a serialization will lose data. (the real use case is doing a HTTP GET followed by a HTTP POST and I want to be sure that I don't lose anything)
So, to be sure that I haven't forgotten any [JsonExtensionData] in my whole POCO object tree, I thought about using the following deserializer setting:
var serializerSettings = new JsonSerializerSettings
{
MissingMemberHandling = MissingMemberHandling.Error
};
But then, if I try to deserialize the previous JSON, I get the following exception:
Could not find member 'z' on object of type 'Foo'. Path 'z', line 3, position 6.
This is a quite annoying, it complains about a field that has no member in the POCO but that is covered by the [JsonExtensionData] attribute.
Is there a way to only raise errors when data is actually data being lost during the deserialization?
You may mark your object with [JsonObject(MissingMemberHandling = MissingMemberHandling.Ignore)]. This will override the serializer setting:
[JsonObject(MissingMemberHandling = MissingMemberHandling.Ignore)]
public class Foo
{
public int Y { get; set; }
[JsonExtensionData]
private IDictionary<string, JToken> _extraStuff;
}
Demo fiddle #1 here.
Alternatively, you could create a custom contract resolver that does this automatically:
public class MissingMemberContractResolver : DefaultContractResolver
{
protected override JsonObjectContract CreateObjectContract(Type objectType)
{
var contract = base.CreateObjectContract(objectType);
if (contract.ExtensionDataSetter != null && contract.MissingMemberHandling == null)
{
contract.MissingMemberHandling = MissingMemberHandling.Ignore;
}
return contract;
}
}
Then use it as follow. First cache a copy somewhere for performance:
static IContractResolver contractResolver = new MissingMemberContractResolver
{
// Modify settings such as the naming strategy if required.
NamingStrategy = new CamelCaseNamingStrategy(),
};
And then set in settings as follows:
var serializerSettings = new JsonSerializerSettings
{
ContractResolver = contractResolver,
MissingMemberHandling = MissingMemberHandling.Error
};
var foo = JsonConvert.DeserializeObject<Foo>(json, serializerSettings);
var json2 = JsonConvert.SerializeObject(foo, Formatting.Indented, serializerSettings);
Demo fiddle #2 here.
Note that MissingMemberHandling was added to to JsonObjectAttribute and JsonObjectContract in Json.NET release 12.0.2. On earlier versions neither of the above solutions are available.
Honestly I'm a bit surprised this is necessary.
As an aside, if you're creating a custom contract resolver anyway, you could make DefaultContractResolver.CreateObjectContract() throw for any object in your assembly or namespace that lacks an ExtensionDataGetter and ExtensionDataSetter. If you do that you'll be able to discover any types that lack a [JsonExtensionData] during unit testing.
I have a controller that accepts a model UpdateProductCommand like this:
public IHttpActionResult UpdateProduct(UpdateProductCommand command)
{
command.AuditUserName = this.RequestContext.Principal.Identity.Name;
// ....
}
For security issues, the AuditUserName field should never be set outside (from the API call).
How can I remove (or truncate) the value of this field from JSON request?
It can be achieved by a following ModelBinder:
using Newtonsoft.Json.Linq;
public class FieldRemoverModelBinder : IModelBinder
{
public bool BindModel(HttpActionContext actionContext, ModelBindingContext bindingContext)
{
string content = actionContext.Request.Content.ReadAsStringAsync().Result;
JObject json = JObject.Parse(content);
JToken property = json.GetValue(bindingContext.ModelName, StringComparison.OrdinalIgnoreCase);
property?.Parent.Remove();
bindingContext.Model = json.ToObject(bindingContext.ModelType);
return true;
}
}
Use it like this:
public IHttpActionResult UpdateProduct(([ModelBinder(typeof(FieldRemoverModelBinder), Name = nameof(UpdateProductCommand.AuditUserName))]UpdateProductCommand command)
{
// here command.AuditUserName will always be empty, no matter what's in json
That's what DTOs are for.
You can just create another class (UpdateProductCommandDto for example) that has only the properties you need / want to be used as the input, and then you can just use something like Automapper to map it to a new instance of UpdateProductCommand.
In the following code, I serialize an object using Json.Net. This Json has type names embedded. I then change one of the type names to induce an error (this is a test, I am dealing with a real issue in an existing project). When I deserialize the Json, I expect to get an object back that has a null value for the property with the fiddled type name. Instead the serializer craps out and returns null. Are my expectations correct? Can I change the settings somehow so that I will get a non-null object for my root object? Note that the second error that I get suggests that there is a bug in the serializer.
static public class JsonTest
{
static public void Test()
{
// Create test object
A a = new A
{
MyTest = new MyTest(),
};
// Serialize it.
string json = JsonConvert.SerializeObject(a, new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.Auto
});
// Fiddle class name to induce error
json = json.Replace("+MyTest", "+MyTest2");
// Before: {"MyTest":{"$type":"<Namespace>.JsonTest+MyTest, <Assembly>"}}
// After: {"MyTest":{"$type":"<Namespace>.JsonTest+MyTest2, <Assembly>"}}
// Deserialize
A a2 = JsonConvert.DeserializeObject<A>(json, new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.Auto,
Error = (object sender, ErrorEventArgs e) =>
{
e.ErrorContext.Handled = true; // Should have only one error: the unrecognized Type
}
});
// A second error occurs: Error = {Newtonsoft.Json.JsonSerializationException: Additional text found in JSON string after finishing deserializing object....
// a2 is null
}
public class A
{
public ITest MyTest { get; set; }
}
public interface ITest { }
public class MyTest : ITest { }
}
Update
This issue has been fixed in Json.NET 10.0.2 in this submission.
Original Answer
This looks to be a bug in Json.NET. If I set JsonSerializerSettings.MetadataPropertyHandling = MetadataPropertyHandling.ReadAhead then the problem goes away:
// Deserialize
A a2 = JsonConvert.DeserializeObject<A>(json, new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.Auto,
MetadataPropertyHandling = MetadataPropertyHandling.ReadAhead,
Error = (object sender, Newtonsoft.Json.Serialization.ErrorEventArgs e) =>
{
Debug.WriteLine(e.ErrorContext.Path);
e.ErrorContext.Handled = true; // Should have only one error: the unrecognized Type
}
});
Debug.Assert(a2 != null); // No assert.
However, it should not be necessary to turn on this setting, which enables reading metadata properties including "$type" located anywhere in a JSON object, rather than just as the first property. Most likely it coincidentally fixes the bug since it requires pre-loading the entire JSON object before beginning deserialization.
You could report an issue if you want.
Debugging a bit, the problem seems to be that, because the inner MyTest object cannot be constructed, the exception is caught and handled by JsonSerializerInternalReader.PopulateObject() while populating the outer object A. Because of this, the JsonReader does not get advanced past the inner, nested object, leaving the reader and serializer in an inconsistent state. This accounts for the second exception and the eventual Additional text found in JSON string after finishing deserializing object. Path '' exception.
We're using ASP.NET WebAPI with Entity Framework (with lazy loading) and using Json.NET for serializing the data to JSON before returning the data to the client.
We are experiencing intermittent sudden spikes in memory usage which we suspect might originate with Json.NET not recognizing reference loops when serializing data (since Entity Framework might be doing some lazy loading voodoo with proxy classes which goes under the radar of Json.NET).
I thought I'd limit how deep Json.NET was allowed to go to serialize data (at least then we'd get a sensible exception when this happens so we could fix it in the data model), but I soon discovered that the MaxDepth property of JsonSerializerSettings only kicks in when DEserializing objects.
Is there any known way of imposing a limit on Json.NET when serializing?
I can't think of a way to do this out-of-the-box with Json.NET, since (as you correctly observe) MaxDepth is ignored when serializing. What you could do is to subclass JsonTextWriter and do the checks yourself:
public class MaxDepthJsonTextWriter : JsonTextWriter
{
public int? MaxDepth { get; set; }
public int MaxObservedDepth { get; private set; }
public MaxDepthJsonTextWriter(TextWriter writer, JsonSerializerSettings settings)
: base(writer)
{
this.MaxDepth = (settings == null ? null : settings.MaxDepth);
this.MaxObservedDepth = 0;
}
public MaxDepthJsonTextWriter(TextWriter writer, int? maxDepth)
: base(writer)
{
this.MaxDepth = maxDepth;
}
public override void WriteStartArray()
{
base.WriteStartArray();
CheckDepth();
}
public override void WriteStartConstructor(string name)
{
base.WriteStartConstructor(name);
CheckDepth();
}
public override void WriteStartObject()
{
base.WriteStartObject();
CheckDepth();
}
private void CheckDepth()
{
MaxObservedDepth = Math.Max(MaxObservedDepth, Top);
if (Top > MaxDepth)
throw new JsonSerializationException(string.Format("Depth {0} Exceeds MaxDepth {1} at path \"{2}\"", Top, MaxDepth, Path));
}
}
Then, to manually generate a JSON string, you would use it like this:
var settings = new JsonSerializerSettings { MaxDepth = 10 };
string json;
try
{
using (var writer = new StringWriter())
{
using (var jsonWriter = new MaxDepthJsonTextWriter(writer, settings))
{
JsonSerializer.Create(settings).Serialize(jsonWriter, myClass);
// Log the MaxObservedDepth here, if you want to.
}
json = writer.ToString();
}
Debug.WriteLine(json);
}
catch (Exception ex)
{
Debug.WriteLine(ex);
throw;
}
Demo fiddle here.
Since your tags include web-api, if you want to do this check inside web API calls, you could follow Rick Strahl's instructions to create a custom MediaTypeFormatter for JSON: Using an alternate JSON Serializer in ASP.NET Web API; then use the code above in the OnWriteToStreamAsync method when generating the json string.
I am looking at implementing a logging mechanism in a site of mine, I wish to do basic user action logging. I don't want to log every single button they click on, but I do want to log actions they do which makes changes.
Are there any libraries or articles / tutorials out there which can help me implement a good and efficient logging mechanism for my asp.net site. Im not sure if there are any changes in MVC5 that might come in use for logging as I know user Authentication and Authorization has changed a fair amount from 4 to 5.
I'm sure that there is a dynamic library out there that will work in many different situations.
Nice to haves:
Async capability
Scalable
Simple to use
I'm thinking along the lines of creating a custom filter or attribute that then logs the suers action, but that's just my Idea, Im here to ask what the standard / industry way to do it is.
There isn't an industry standard.
I've used filters or I've overridden the "onActionExecuting" method on the base controller class to record controller / action events.
EDIT ::
Trying to be more helpful but this is really vague.
If you're worried about errors and things like that use elmah.
For other logging use Nlog or Log4Net.
If you're trying to do extra logging like auditing or something like that you can use any combination of those, or something custom. In my site I created a table that stores every click on the site by creating an object sort of like this :
public class audit
{
public int ID { get; set; }
public DateTime AuditDate { get; set; }
public string ControllerName { get; set; }
public string ActionName { get; set; }
public Dictionary<string, object> values
}
In my base constructor, I overrode the OnActionExecuting event :
protected override void OnActionExecuting(ActionExecutingContext ctx)
{
checkForLogging(ctx);
//do not omit this line
base.OnActionExecuting(ctx);
}
Lets say I want to log all Get Requests using my new audit object
private void checkForLogging(ActionExecutingContext ctx)
{
//we leave logging of posts up to the actual methods because they're more complex...
if (ctx.HttpContext.Request.RequestType == "GET")
{
logging(ctx.ActionDescriptor.ActionName, ctx.ActionDescriptor.ControllerDescriptor.ControllerName, ctx.ActionParameters);
}
}
Thats all the info I need to fill my logging object with the action name, the controller name and all the params passed into the method. You can either save this to a db, or a logfile or whatever you want really.
The point is just its a pretty big thing. This is just one way to do it and it may or may not help you. Maybe define a bit more what exactly you want to log and when you want to do it?
You can create a custom attribute and decorate methods with it and then check if that attribute is present when the OnActionExecuting method fires. You can then get that filter if present and read from it and use that to drive your logging if you want...
Maybe this example will help.
My focus on logging is in the CREATE, EDIT, DELETE actions.
I am using MVC 5 Code-first EF 6.1 (VS 2013) ,
and for this example I are referring to the Create action for an entity called "WorkFlow"
I actually view these logs from SSRS, but you could add a controller and Views for WriteUsageLog and view them from the MVC application
MODEL: Create a MODEL Entity called "WriteUsageLog" which will be where the log records are kept
CONTROLLER: Extract, or refactor, the HttpPost overload of the "Create" action from the WorkFlowController into a Partial Class called "WorkFlowController" (These partials are to avoid being deleted and rebuilt when I use the wizard to create Controllers)
Other Classes in the CONTROLLER folder: Then there are some helper functions that are required in a class called "General_Object_Extensions" and "General_ActiveDirectory_Extensions" (NOTE: these are not really 'extensions')
Add the following line to the DBContext:
public DbSet WriteUsageLogs { get; set; }
The advantage of this example is:
I am recording the following for the record:
User Name from Active Directory
The DateTime that the log record is being created
The computer name
And a Note that consists of the values for all the entity properties
I am recording the log in a table from which I can access it either using an MVC controller, or preferably from SQL Server Report Server. Where I can monitor all my MVC applications
/Models/WriteUsageLog.cs
using System;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;
namespace MileageReimbursement.Models
{
public class WriteUsageLog
{
public WriteUsageLog()
{
this.DateTimeCreated = DateTime.Now; // auto-populates DateTimeCreated field
}
[Key]
public int WriteUsageLogID { get; set; }
[Column(TypeName = "nvarchar(max)")]
public string Note { get; set; }
public string UserLogIn { get; set; }
public string ComputerName { get; set; }
public DateTime DateTimeCreated { get; private set; } //private set to for auto-populates DateTimeCreated field
}
}
/Controllers/ControllerPartials.cs
using System.Linq;
using System.Web.Mvc;
using MileageReimbursement.Models;
//These partials are to avoid be deleted and rebuilt when I use the wizard to create Controllers
namespace MileageReimbursement.Controllers
{
public partial class WorkFlowController : Controller
{
[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult Create([Bind(Include = "whatever")] WorkFlow workFlow)
{
...
if (ModelState.IsValid)
{
db.WorkFlows.Add(workFlow);
db.SaveChanges();
//===================================================================================================================
string sX = workFlow.GetStringWith_RecordProperties();
//===================================================================================================================
var logRecord = new WriteUsageLog();
logRecord.Note = "New WorkFlow Record Added - " + sX;
logRecord.UserLogIn = General_ActiveDirectory_Extensions.fn_sUser();
string IP = Request.UserHostName;
logRecord.ComputerName = General_functions.fn_ComputerName(IP);
db.WriteUsageLogs.Add(logRecord);
db.SaveChanges();
//===================================================================================================================
return RedirectToAction("Index");
}
else // OR the user is directed back to the validation error messages and given an opportunity to correct them
{
...
return View(workFlow); //This sends the user back to the CREATE view to deal with their errors
}
}
}
}
/Controllers/ControllerExtensions.cs
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.DirectoryServices.AccountManagement;
using System.Linq;
using System.Net;
using System.Net.Mail;
using System.Reflection;
using System.Security.Cryptography;
using System.Text;
using System.Web;
namespace MileageReimbursement.Controllers
{
public static class General_ActiveDirectory_Extensions
{
public static string fn_sUser()
{
char cX = '\\';
string sUser = General_functions.fn_ReturnPortionOfStringAfterLastOccuranceOfACharacter(HttpContext.Current.User.Identity.Name, cX);
return sUser; //returns just the short logon name Example for 'accessiicarewnc\ggarson', it returns 'ggarson'
}
} //General_ActiveDirectory_Extensions
public static class General_Object_Extensions
{
public static string GetStringWith_RecordProperties(this object Record)
{
string sX = null;
Dictionary<string, object> _record = GetDictionary_WithPropertiesForOneRecord(Record);
int iPropertyCounter = 0;
foreach (var KeyValuePair in _record)
{
iPropertyCounter += 1;
object thePropertyValue = _record[KeyValuePair.Key];
if (thePropertyValue != null)
{
sX = sX + iPropertyCounter + ") Property: {" + KeyValuePair.Key + "} = [" + thePropertyValue + "] \r\n";
}
else
{
sX = sX + iPropertyCounter + ") Property: {" + KeyValuePair.Key + "} = [{NULL}] \r\n";
}
}
return sX;
}
public static Dictionary<string, object> GetDictionary_WithPropertiesForOneRecord(object atype)
{
if (atype == null) return new Dictionary<string, object>();
Type t = atype.GetType();
PropertyInfo[] props = t.GetProperties();
Dictionary<string, object> dict = new Dictionary<string, object>();
foreach (PropertyInfo prp in props)
{
object value = prp.GetValue(atype, new object[] { });
dict.Add(prp.Name, value);
}
return dict;
}
} //General_Object_Extensions
public static class General_functions
{
public static string fn_ComputerName(string IP)
{
//USAGE
//From: http://stackoverflow.com/questions/1444592/determine-clients-computer-name
//string IP = Request.UserHostName;
//string compName = CompNameHelper.DetermineCompName(IP);
IPAddress myIP = IPAddress.Parse(IP);
IPHostEntry GetIPHost = Dns.GetHostEntry(myIP);
List<string> compName = GetIPHost.HostName.ToString().Split('.').ToList();
return compName.First();
}
static public string fn_ReturnPortionOfStringAfterLastOccuranceOfACharacter(string strInput, char cBreakCharacter)
{
// NOTE: for path backslash "/", set cBreakCharacter = '\\'
string strX = null;
//1] how long is the string
int iStrLenth = strInput.Length;
//2] How far from the end does the last occurance of the character occur
int iLenthFromTheLeftOfTheLastOccurance = strInput.LastIndexOf(cBreakCharacter);
int iLenthFromTheRightToUse = 0;
iLenthFromTheRightToUse = iStrLenth - iLenthFromTheLeftOfTheLastOccurance;
//3] Get the Portion of the string, that occurs after the last occurance of the character
strX = fn_ReturnLastXLettersOfString(iLenthFromTheRightToUse, strInput);
return strX;
}
static private string fn_ReturnLastXLettersOfString(int iNoOfCharToReturn, string strIn)
{
int iLenth = 0;
string strX = null;
int iNoOfCharacters = iNoOfCharToReturn;
iLenth = strIn.Length;
if (iLenth >= iNoOfCharacters)
{
strX = strIn.Substring(iLenth - iNoOfCharacters + 1);
}
else
{
strX = strIn;
}
return strX;
}
} //General_functions
}
I would agree that Log4Net and NLog seem to be the two most commonly used products on the different projects I have been a member.
If you are looking for a great tool that you can use for logging, error handling and anything else where AOP would be beneficial I would highly recommend PostSharp (http://www.postsharp.net/). You set your logging/error handling up centrally and then just decorate methods. It is a well documented and supported product. They have a community license, which is free - and it is free for individuals. They also have professional and ultimate versions of the products, which would make more sense if you're using it as a team.
I don't work at PostSharp :-) I've just used it in the past and really like it.