This question already has an answer here:
How to change all keys to lowercase when parsing JSON to a JToken
(1 answer)
Closed 5 years ago.
I have a JSON resulting from a mix of system data and user entries, like this :
{
"Properties": [{
"Type": "A",
"Name": "aaa",
"lorem ipsum": 7.1
}, {
"Type": "B",
"Name": "bbb",
"sit amet": "XYZ"
}, {
"Type": "C",
"Name": "ccc",
"abcd": false
}]
}
I need to load it, process it, and save it to MongoDB. I deserialize it to this class :
public class EntityProperty {
public string Name { get; set; }
[JsonExtensionData]
public IDictionary<string, JToken> OtherProperties { get; set; }
public string Type { get; set; }
}
The problem is that MongoDB does not allow dots in key names, but the users can do whatever they want.
So I need a way to save this additional JSON data but I also need to change the key name as it's being processed.
I tried to add [JsonConverter(typeof(CustomValuesConverter))] to the OtherProperties attribute but it seems to ignore it.
Update/Clarification: since the serialization is done by Mongo (I send the objects to the library), I need the extension data names to be fixed during deserialization.
Update
Since the fixing of names must be done during deserialization, you could generalize the LowerCasePropertyNameJsonReader from How to change all keys to lowercase when parsing JSON to a JToken by Brian Rogers to perform the necessary transformation.
First, define the following:
public class PropertyNameMappingJsonReader : JsonTextReader
{
readonly Func<string, string> nameMapper;
public PropertyNameMappingJsonReader(TextReader textReader, Func<string, string> nameMapper)
: base(textReader)
{
if (nameMapper == null)
throw new ArgumentNullException();
this.nameMapper = nameMapper;
}
public override object Value
{
get
{
if (TokenType == JsonToken.PropertyName)
return nameMapper((string)base.Value);
return base.Value;
}
}
}
public static class JsonExtensions
{
public static T DeserializeObject<T>(string json, Func<string, string> nameMapper, JsonSerializerSettings settings = null)
{
using (var textReader = new StringReader(json))
using (var jsonReader = new PropertyNameMappingJsonReader(textReader, nameMapper))
{
return JsonSerializer.CreateDefault(settings).Deserialize<T>(jsonReader);
}
}
}
Then deserialize as follows:
var root = JsonExtensions.DeserializeObject<RootObject>(json, (s) => s.Replace(".", ""));
Or, if you are deserializing from a Stream via a StreamReader you can construct your PropertyNameMappingJsonReader directly from it.
Sample fiddle.
Alternatively, you could also fix the extension data in an [OnDeserialized] callback, but I think this solution is neater because it avoids adding logic to the objects themselves.
Original Answer
Assuming you are using Json.NET 10.0.1 or later, you can create your own custom NamingStrategy, override NamingStrategy.GetExtensionDataName(), and implement the necessary fix.
First, define MongoExtensionDataSettingsNamingStrategy as follows:
public class MongoExtensionDataSettingsNamingStrategy : DefaultNamingStrategy
{
public MongoExtensionDataSettingsNamingStrategy()
: base()
{
this.ProcessExtensionDataNames = true;
}
protected string FixName(string name)
{
return name.Replace(".", "");
}
public override string GetExtensionDataName(string name)
{
if (!ProcessExtensionDataNames)
{
return name;
}
return name.Replace(".", "");
}
}
Then serialize your root object as follows:
var settings = new JsonSerializerSettings
{
ContractResolver = new DefaultContractResolver { NamingStrategy = new MongoExtensionDataSettingsNamingStrategy() },
};
var outputJson = JsonConvert.SerializeObject(root, settings);
Notes:
Here I am inheriting from DefaultNamingStrategy but you could inherit from CamelCaseNamingStrategy if you prefer.
The naming strategy is only invoked to remap extension data names (and dictionary keys) during serialization, not deserialization.
You may want to cache the contract resolver for best performance.
There is no built-in attribute to specify a converter for dictionary keys, as noted in this question. And in any event Json.NET would not use the JsonConverter applied to OtherProperties since the presence of the JsonExtensionData attribute supersedes the converter property.
Alternatively, if it would be more convenient to specify the naming strategy using Json.NET serialization attributes, you will need a slightly different naming strategy. First create:
public class MongoExtensionDataAttributeNamingStrategy : MongoExtensionDataSettingsNamingStrategy
{
public MongoExtensionDataAttributeNamingStrategy()
: base()
{
this.ProcessDictionaryKeys = true;
}
public override string GetDictionaryKey(string key)
{
if (!ProcessDictionaryKeys)
{
return key;
}
return FixName(key);
}
}
And modify EntityProperty as follows:
[JsonObject(NamingStrategyType = typeof(MongoExtensionDataAttributeNamingStrategy))]
public class EntityProperty
{
public string Name { get; set; }
[JsonExtensionData]
public IDictionary<string, JToken> OtherProperties { get; set; }
public string Type { get; set; }
}
The reason for the inconsistency is that, as of Json.NET 10.0.3, DefaultContractResolver uses GetDictionaryKey() when remapping extension data names using a naming strategy that is set via attributes here, but uses GetExtensionDataName() when the naming strategy is set via settings here. I have no explanation for the inconsistency; it feels like a bug.
Related
I have .Net Core 3.1 Web Api. I am using System.Text.Json serializer since it became a default for .Net Core 3.x applications. I have set global enum to string converter as follows:
public void ConfigureServices(IServiceCollection services)
{
...
services.AddControllers()
.AddJsonOptions(options =>
options.JsonSerializerOptions.Converters.Add(new JsonStringEnumConverter())));
...
}
Enums are converted to string for controller responses (which makes sense since I configure on AddControllers() method). But if I try to manually serialize an object it still serializes enum as int. Sample below:
public class TestClass
{
public void Test()
{
var data = JsonSerializer.Serialize(new MyObject { Enum1 = MyEnum.Value1, Enum2 = MyEnum.Value2 });
}
public class MyObject
{
public MyEnum Enum1 { get; set; }
public MyEnum Enum2 { get; set; }
}
public enum MyEnum
{
Value1 = 1,
Value2 = 2
}
}
OUTPUT:
data [string]: "{\"Enum1\":1,\"Enum2\":2}"
If I add conversion attribute manually then it serializes as desired:
public class MyObject
{
[JsonConverter(typeof(JsonStringEnumConverter))]
public MyEnum Enum1 { get; set; }
public MyEnum Enum2 { get; set; }
}
OUTPUT
data [string]: "{\"Enum1\":\"Value1\",\"Enum2\":2}"
Is there a way to set global json serialization options (enum to string conversion) that would apply for manual serialization as well?
Or maybe I should just always stick to the explicit attributes?
Currently you can't. JsonSerializer being a static class with only purely static methods, default options cannot be set. You can check this open issue on Github. It seems that it has been designed this way for better performance.
As an alternative, you can create your own static class
public static class CustomJsonSerializer
{
private static JsonSerializerOptions serializerSettings =
new JsonSerializerOptions { /* whatever you need */};
public static T Deserialize<T>(this string json)
{
return JsonSerializer.Deserialize<T>(json, serializerSettings );
}
// etc.
}
I managed to get this to work by adding NewtonsoftJson and then register StringEnumConverter:
services
.AddControllers(jsonOptions => jsonOptions.ReturnHttpNotAcceptable = true)
.AddNewtonsoftJson(jsonOptions =>
{
jsonOptions.SerializerSettings.Converters.Add((JsonConverter) new StringEnumConverter());
}
I have a Web API method:
public List<Task> GetTasks([FromUri] TaskFilter filter)
{
}
The method has parameter with list of nullable identifiers:
public class TaskFilter
{
public IList<int?> Assignees { get; set; }
}
When I call it:
GET /tasks?assignees=null
Server returns an error:
{
"message":"The request is invalid.",
"modelState": {
"assignees": [ "The value 'null' is not valid for Nullable`1." ]
}
}
It works only if I pass empty string:
GET /tasks?assignees=
But standard query string converters (from JQuery, Angular, etc) do not work with nulls in such way.
How to make ASP.NET to interpret 'null' as null?
Upd: The query string can contain several identifiers, e.g.:
GET /tasks?assignees=1&assignees=2&assignees=null
Upd2: JQuery converts nulls in array to empty strings, and ASP.NET interprets them as null. So the question is about calling WebAPI from Angular 1.6 ($HttpParamSerializerProvider)
Upd3: I know about workarounds, but I do not ask for them. I want a solution for specific problem:
It is a GET method
Method accepts a list from Uri
A list can contain null values
It should be List<int?> because API docs are generated automatically, and I do not want to see text array as parameter type
By default ASP.NET expects empty strings for null values (JQuery.param works in that way)
But some client libraries (e.g. Angular) does not convert null array items to empty strings
You can create a custom model bind for this specific type, inherithing from DefaultModelBinder, for sample:
using System.ComponentModel.DataAnnotations;
using System.Web.Mvc;
public class TaskFilterBinder : IModelBinder
{
public object BindModel(ControllerContext controllerContext, System.Web.Mvc.ModelBindingContext bindingContext)
{
var request = controllerContext.HttpContext.Request;
var assignees = request.QueryString["assignees"];
if (assignees == "null") // check if assignees is null (string) then return NULL
return null;
return assignees;
}
}
Finally we need to inform the controller as to the binding we want it to use. This we can specify using attributes
[ModelBinder(typeof(TaskFilterBinder))]
as below:
public List<Task> GetTasks([FromUri(ModelBinder=typeof(TaskFilterBinder))] TaskFilter filter)
{
// Do your stuff.
}
For more reference check this link on Custom Model Binders.
Hope, this solves your problem . Thanks
Finally, I found a solution using custom value provider:
using System;
using System.Collections.Generic;
using System.Web.Http;
using System.Web.Http.Controllers;
using System.Web.Http.ValueProviders;
using System.Web.Http.ValueProviders.Providers;
using System.Globalization;
using System.Net.Http;
using System.Web.Http.ModelBinding;
public sealed class NullableValueProviderAttribute : ModelBinderAttribute
{
private readonly string[] _nullableColumns;
public NullableValueProviderAttribute(params string[] nullableColumns)
{
_nullableColumns = nullableColumns;
}
public override IEnumerable<ValueProviderFactory> GetValueProviderFactories(HttpConfiguration configuration)
{
return new ValueProviderFactory[] { new NullableValueProviderFactory(_nullableColumns) };
}
}
public class NullableValueProviderFactory : ValueProviderFactory, IUriValueProviderFactory
{
private readonly string[] _nullableColumns;
public NullableValueProviderFactory(string[] nullableColumns)
{
_nullableColumns = nullableColumns;
}
public override IValueProvider GetValueProvider(HttpActionContext actionContext)
{
return new NullableQueryStringValueProvider(actionContext, CultureInfo.InvariantCulture, _nullableColumns);
}
}
public class NullableQueryStringValueProvider : NameValuePairsValueProvider
{
private static readonly string[] _nullValues = new string[] { "null", "undefined" };
private static IEnumerable<KeyValuePair<string, string>> GetQueryNameValuePairs(HttpRequestMessage request, string[] nullableColumns)
{
foreach (var pair in request.GetQueryNameValuePairs())
{
var isNull = Array.IndexOf(nullableColumns, pair.Key) >= 0 && Array.IndexOf(_nullValues, pair.Value) >= 0;
yield return isNull ? new KeyValuePair<string, string>(pair.Key, "") : pair;
};
}
public NullableQueryStringValueProvider(HttpActionContext actionContext, CultureInfo culture, string[] nullableColumns) :
base(GetQueryNameValuePairs(actionContext.ControllerContext.Request, nullableColumns), culture)
{ }
}
And specify it in Web API action:
public List<Task> GetTasks([NullableValueProvider("assignees")] TaskFilter filter)
{
}
I have a situation in which I would like to do custom parameter binding for an api controller in ASP.NET core.In WebAPI 2.0 it was possible to perform custom binding to primitive types by implementing various interfaces such as IValueProvider and providing a ValueProviderFactory. This does not seem the case with ASP.NET core in as far as what I understand from the documentation I found here.
I did notice this SO post which lead me to this article which overrides the behavior for the MutableObjectModelBinder. It would appear I could do something along those lines such as:
[HttpGet]
[Route("api/{domain}/[controller]")]
public IEnumerable<string> Get([ModelBinder(BinderType = typeof(MyCustomBinder))]string orderby)
{
//Do stuff here
}
This doesn't necessarily seem right to me since I am just dealing with a primitive type however I cannot seem to find any documentation for another way of doing this.
Create a binder provider class for your custom type
public class MyCustomBinderProvider : IModelBinderProvider
{
public IModelBinder GetBinder(ModelBinderProviderContext context)
{
if (context == null)
{
throw new ArgumentNullException(nameof(context));
}
if (context.Metadata.ModelType == typeof(MyCustom))
{
return new BinderTypeModelBinder(typeof(MyCustomBinder));
}
return null;
}
}
and register it in the services
services.AddMvc(c =>
{
c.ModelBinderProviders.Insert(0, new MyCustomBinderProvider());
});
And the custom binder can go like
public class MyCustomBinder : IModelBinder
{
public Task BindModelAsync(ModelBindingContext bindingContext)
{
if (bindingContext.ModelType != typeof(MyCustom))
{
return TaskCache.CompletedTask;
}
var parameters = new Dictionary<string, string>();
foreach (var parameter in bindingContext.ModelType.GetProperties())
{
var valueProviderResult = bindingContext.ValueProvider.GetValue(parameter.Name);
if (valueProviderResult.FirstValue != null)
{
parameters.Add(parameter.Name, valueProviderResult.FirstValue);
}
}
var result = Activator.CreateInstance(bindingContext.ModelType);
//write your custom code to map the result with the parameters
bindingContext.Result = ModelBindingResult.Success(result);
return TaskCache.CompletedTask;
}
}
Your custom type class
[ModelBinder(BinderType = typeof(MyCustomBinder))]
public class MyCustom
{
public int Page { get; set; }
public int Rows { get; set; }
}
and your controller can take the custom class as query string parameter
[HttpGet("data")]
public DataTransferObject GetData(MyCustom query)
{
}
Migrating OP's solution from the question to an answer, with meta commentary trimmed:
I just decided to go with a helper class to parse the parameter due to having to meet deadlines.
By default convention, strings properties in an entity model that are not explicitly given a max length are set to nvarchar(max) in the database. We wish to override this convention and give strings a max length of nvarchar(100) if they are not already explicitly set otherwise.
I discovered the PropertyMaxLengthConvention built-in convention, which by its description and documentation would seem to be what I am looking for. However, it either doesn't work or I'm using it wrong or it just doesn't do what I think it does.
I've tried simply adding the convention:
modelBuilder.Conventions.Add(new PropertyMaxLengthConvention(100));
Then I thought maybe the default one is already being used, so I tried removing it first:
modelBuilder.Conventions.Remove<PropertyMaxLengthConvention>();
modelBuilder.Conventions.Add(new PropertyMaxLengthConvention(100));
I even tried explictly adding the convention before and after the default one:
modelBuilder.Conventions.AddBefore<PropertyMaxLengthConvention>(new PropertyMaxLengthConvention(100));
modelBuilder.Conventions.AddAfter<PropertyMaxLengthConvention>(new PropertyMaxLengthConvention(100));
No joy. When I add migrations, the columns are still created as nvarchar(max).
Is there a way to use that convention to do what I want? If not, can I write a custom convention that will default string properties to nvarchar(100) but will still allow me to explicitly set them to a different value including maxlength?
After tracking down the source code for the aforementioned convention, I discovered that it only sets the default max length for properties that are specified to have fixed length. (Bizarre!)
So I took the source code and modified it to create my own convention. Now string properties with unspecified max length will have a default max length instead of being nvarchar(max). The only downside is there doesn't appear to be a way to detect when the IsMaxLength() configuration is explicitly applied. So if I have a column that I do want to have created as nvarchar(max) I can't use IsMaxLength() to do it.
To address this, I created an extension method for StringPropertyConfiguration called ForceMaxLength() that configures the property with HasMaxLength(int.MaxValue) - ordinarily an invalid value, but one for which I can easily test in my custom convention. When I detect it, I simply set the MaxLength back to null and set the IsMaxLength to true and let the property configuration continue as normal.
Here's the custom convention:
using System;
using System.Collections.Generic;
using System.Data.Entity.Core.Metadata.Edm;
using System.Data.Entity.Infrastructure;
using System.Data.Entity.ModelConfiguration.Conventions;
namespace MyProject.CustomConventions
{
public class CustomPropertyMaxLengthConvention : IConceptualModelConvention<EntityType>, IConceptualModelConvention<ComplexType>
{
private const int DefaultLength = 128;
private readonly int length;
public CustomPropertyMaxLengthConvention()
: this(DefaultLength)
{
}
public CustomPropertyMaxLengthConvention(int length)
{
if (length <= 0)
{
throw new ArgumentOutOfRangeException("length", "Invalid Max Length Size");
}
this.length = length;
}
public virtual void Apply(EntityType item, DbModel model)
{
SetLength(item.DeclaredProperties);
}
public virtual void Apply(ComplexType item, DbModel model)
{
SetLength(item.Properties);
}
private void SetLength(IEnumerable<EdmProperty> properties)
{
foreach (EdmProperty current in properties)
{
if (current.IsPrimitiveType)
{
if (current.PrimitiveType == PrimitiveType.GetEdmPrimitiveType(PrimitiveTypeKind.String))
{
SetStringDefaults(current);
}
if (current.PrimitiveType == PrimitiveType.GetEdmPrimitiveType(PrimitiveTypeKind.Binary))
{
SetBinaryDefaults(current);
}
}
}
}
private void SetStringDefaults(EdmProperty property)
{
if (property.IsUnicode == null)
{
property.IsUnicode = true;
}
SetBinaryDefaults(property);
}
private void SetBinaryDefaults(EdmProperty property)
{
if (property.MaxLength == int.MaxValue)
{
property.MaxLength = null;
property.IsMaxLength = true;
}
else if (property.MaxLength == null || !property.IsMaxLength)
{
property.MaxLength = length;
}
}
}
}
Here's the extension method:
using System.Data.Entity.ModelConfiguration.Configuration;
namespace MyProject.Model.Mapping
{
public static class MappingExtensions
{
public static void ForceMaxLength(this StringPropertyConfiguration obj)
{
obj.HasMaxLength(int.MaxValue);
}
}
}
Here's how it's used:
using System.Data.Entity.ModelConfiguration;
namespace MyProject.Model.Mapping
{
public class MyEntityMap : EntityTypeConfiguration<MyEntity>
{
public MyEntityMap()
{
Property(v => v.StringValue).ForceMaxLength();
}
}
}
Or just
public class StringConventions : Convention
{
public StringConventions()
{
this.Properties<string>().Configure(x => x.HasMaxLength(100));
}
}
In EF6 you can use a custom code first convention, but you will also need to have a way to specify nvarchar(max) data type to a string property. So, I came up with the following solution.
/// <summary>
/// Set this attribute to string property to have nvarchar(max) type for db table column.
/// </summary>
[AttributeUsage(AttributeTargets.Property, AllowMultiple = false)]
public sealed class TextAttribute : Attribute
{
}
/// <summary>
/// Changes all string properties without System.ComponentModel.DataAnnotations.StringLength or
/// Text attributes to use string length 16 (i.e nvarchar(16) instead of nvarchar(max) by default).
/// Use TextAttribute to a property to have nvarchar(max) data type.
/// </summary>
public class StringLength16Convention : Convention
{
public StringLength16Convention()
{
Properties<string>()
.Where(p => !p.GetCustomAttributes(false).OfType<DatabaseGeneratedAttribute>().Any())
.Configure(p => p.HasMaxLength(16));
Properties()
.Where(p => p.GetCustomAttributes(false).OfType<TextAttribute>().Any())
.Configure(p => p.IsMaxLength());
}
}
public class CoreContext : DbContext, ICoreContext
{
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
//Change string length default behavior.
modelBuilder.Conventions.Add(new StringLength16Convention());
}
}
public class LogMessage
{
[Key]
public Guid Id { get; set; }
[StringLength(25)] // Explicit data length. Result data type is nvarchar(25)
public string Computer { get; set; }
//[StringLength(25)] // Implicit data length. Result data type is nvarchar(16)
public string AgencyName { get; set; }
[Text] // Explicit max data length. Result data type is nvarchar(max)
public string Message { get; set; }
}
I have a web method that accepts object
[WebMethod]
public static void GetObject(object data)
{
}
Also, I have 2 classes:
class ConnectionString
{
public string ConnectionString { get; set; }
public DatabaseType DatabaseType { get; set; }
}
class Path
{
public string Path { get; set; }
public bool IsNetwork { get; set; }
}
On client side, using javascript, i defined 2 similar classes as well:
function ConnectionString() {
this.ConnectionString = '';
this.DatabaseType = 0;
};
function Path() {
this.Path = '';
this.IsNetwork = false;
};
Now, according to user decision, he can ether choose to create log in database or file system. When I send data to the method, my object resulted as null. If I create method
for each object, it works. Is there a way to unbox or desirialize from OBJECT type to ?
You need to create two method overloads that each take in one of the possible classes. In the current implementation the engine does not know what classes should be put in the WSDL...
If you are using WCF you could use [KnownType] attribute to specify which classes your method supports.