Recently discovered that existing serialization code fails when the object to convert is too large. I know how to increase the maximum length in .NET's JavascriptSerializer. But how do I do the same thing in JsonConvert.SerializeObject?
References to web.config setting:
<jsonSerialization maxJsonLength="50000000"/>
Say that it is only respected by WCF. We aren't using WCF. This is a simple .NET library that requires serialization of objects.
The relevant code is:
public static string Serialize(object value)
{
return JsonConvert.SerializeObject(value, SerializerSettings);
}
This is how I create the settings:
private static JsonSerializerSettings SerializerSettings
{
get
{
var settings = new JsonSerializerSettings();
settings.Converters.Add(new StringEnumConverter());
settings.ContractResolver = new CamelCasePropertyNamesContractResolver();
settings.NullValueHandling = NullValueHandling.Ignore;
return settings;
}
}
I don't see a property I need in JsonSerializerSettings. My object is multilayered but mostly of string types. I convert an uploaded file and store it as a string. The size of that file is the element that I am changing when I see the failire.
Unfortunately the library I am using doesn't log so I can't find the exact error. But the code works with smaller files (<150KB) and fails with larger ones >500KB.
Related
does anybody know a way to configure NJsonSchema to use CamelCase property naming durching code generation? I've a JSON schema with property names like message_id which lead to C# property name 'Message_id' where i.e. 'MessageId' whould be a more C#-like way.
With an attribute like '[JsonProperty("message_id"]' it would be no problem to specified the connection between the different names.
So, you asked about code generation. I was having trouble with the schema it generated not matching what was getting sent to my Angular app. So, while this isn't exactly what you were looking for, perhaps it helps you find an answer (maybe?).
To generate the schema with the camel case property names, I'm setting the Default Property Name Handling to CamelCase, but this is using the deprecated call to set these settings directly. There should be some way to use the SerializerSettings directly, but I wasn't quite able to make that work. This isn't production code for me, so it will do.
internal class SchemaFileBuilder<T>
{
public static void CreateSchemaFile()
{
CreateSchemaFile(typeof(T).Name);
}
public static void CreateSchemaFile(string fileName)
{
JsonSchemaGeneratorSettings settings = new JsonSchemaGeneratorSettings();
settings.DefaultPropertyNameHandling = PropertyNameHandling.CamelCase;
var schema = NJsonSchema.JsonSchema.FromType<T>(settings);
var json = schema.ToJson();
Directory.CreateDirectory("Schemas");
File.WriteAllText($"Schemas\\{fileName}.schema.json", json);
}
}
I set this up as a generic function so I could pass multiple schemas in to either createSchemaFile functions. Here's are some example calls which would generate a Person.schema.json file and a Persons.schema.json file:
SchemaFileBuilder<Person>.CreateSchemaFile();
SchemaFileBuilder<Dictionary<string, Person>>.CreateSchemaFile("Persons");
Given the following json configuration file some_settings.json
{
"AnEmptyArray": [],
"APopulatedArray": [1, 2, 3]
}
When adding as a source to the asp.net core configuration system and requesting all the keys
var configuration = new ConfigurationBuilder()
.AddJsonFile("some_settings.json")
.Build();
var keys = configuration
.AsEnumerable()
.Select(c => c.Key);
The results are
APopulatedArray
APopulatedArray:2
APopulatedArray:1
APopulatedArray:0
For some reason, there is no key for AnEmptyArray
The results are the same if I remove the AnEmptyArray key completly from the json
{
"APopulatedArray": [1, 2, 3]
}
This means there is no way of detecting the difference between an empty array and the array not being provided at all.
Is this correct?
This means there is no way of detecting the difference between an
empty array and the array not being provided at all.
Is this correct?
If shortly - yes.
To understand the reason, we should dig into the .Net Core configuration loading mechanism. Adding every new source, like AddJsonFile() or AddEnvironmentVariables(), actually ads another instance of IConfigurationSource to IConfigurationBuilder. When you finally call IConfigurationBuilder.Build() method, you get instance of IConfigurationRoot, which is essentially a holder for a list of loaded configuration providers:
public interface IConfigurationRoot : IConfiguration
{
IEnumerable<IConfigurationProvider> Providers { get; }
void Reload();
}
Goring further, IConfigurationProvider operates only with configuration key-values pairs:
public interface IConfigurationProvider
{
IEnumerable<string> GetChildKeys(IEnumerable<string> earlierKeys, string parentPath);
IChangeToken GetReloadToken();
void Load();
void Set(string key, string value);
bool TryGet(string key, out string value);
}
IConfigurationProvider does not care at all about any initial structure of your settings, like arrays or sub-sections:
Such mechanism has several advantages:
You could have different parts of configuration (even for one section) in different sources.
You could override specific key values of a section (keeping other values for a section) in a different source.
So answer to your question is "Yes", you can't detect the difference between an empty array and missing array through standard configuration mechanism. Initial JSON structure is just lost and configuration is stored as key-value pairs. Empty arrays does not produce any key-value pair in IConfigurationProvider.
I have an ASP.NET app which needs to have multiple layouts for the views. There's a normal web layout, and a "static" layout for a self-contained page with no dependencies (to be used as a display template in a doc management system).
It's easy enough to have a view switch its layout.
So the all-in-one-page layout needs to include all of its CSS in a <style> tag inside the page. I found a way to find out what files are in a bundle and wrote a method like so:
#MyCode.IncludeCSS("~/StaticLayout/css")
It reads the CSS files in the bundle and emits their content into the page inside a <style> tag.
But I'd like to minify the CSS.
I haven't been able to find any documentation for System.Web.Optimization.CssMinify. You need to call the Process method, which needs a BundleContext. So... find one, create one? MSDN isn't much use:
public BundleContext(
HttpContextBase context,
BundleCollection collection,
string bundleVirtualPath
)
Parameters
context
Type: System.Web.HttpContextBase
The context.
Well, that's definitely useful. I had no way of knowing that context (of type context) was the context until some bulging-brained colossus of the intellect in Redmond took some time out from his busy nap to push a definite article in front of it with his fat little paws before passing out in a puddle of warm drool.
Nevertheless, I still don't know what it is or where you get one from, and neither does Google. HttpContext.Current is not derived from it and has no properties (that I can find) which are.
This page has some good information about normal bundling, but it assumes that the framework will be doing the bundling and serving the bundle content in the conventional way, as a separately requested resource.
Has anybody done this?
Ed Plunkett,
If you are using this as a HTML helper the HttpContextBase can be grabbed from the ViewContext.HttpContext property. This can be created such as.
public static IHtmlString BundleCss(this HtmlHelper html, string outputPath, params string[] cssFilePaths)
{
var collection = new BundleCollection();
foreach (var cssFilePath in cssFilePaths)
collection.Add(new Bundle(cssFilePath));
var bundler = new BundleContext(html.ViewContext.HttpContext, collection, outputPath);
//... ommitted code
System.Web.Optimization.CssMinify minify = new CssMinify();
minify.Process(bundler, response);
//... TODO: Grab the response content and return
}
Yes this is a basic example using a HTML helper. Let me know if this does't answer your question.
EDIT: After re-reading the question I am clarifying my response. So the above helps find the HttpContextBase property from the question. However I think the question is how you can actually read the content of the files, minify them and out put them into a <style> tag on the page.
I took a stab at writing my own interpretation of your requirement and came up with the following set of classes
My CssOutputBundler class:
public class CssOutputBundler
{
static readonly Dictionary<string, string> cachedOutput = new Dictionary<string, string>();
readonly string tempFileOutputPath;
readonly HtmlHelper helper;
readonly IList<string> virtualFilePaths;
public CssOutputBundler(HtmlHelper helper, string tempFileOutputPath)
{
if (helper == null)
throw new ArgumentNullException("helper null");
if (string.IsNullOrWhiteSpace(tempFileOutputPath))
this.tempFileOutputPath = tempFileOutputPath;
this.helper = helper;
this.virtualFilePaths = new List<string>();
this.tempFileOutputPath = tempFileOutputPath;
}
public CssOutputBundler Add(string cssFilePath)
{
if (!this.virtualFilePaths.Contains(cssFilePath))
this.virtualFilePaths.Add(cssFilePath);
return this;
}
public IHtmlString Minify()
{
if (helper == null)
throw new ArgumentNullException("helper null");
string cache_string = string.Join(",", this.virtualFilePaths);
if(cachedOutput.ContainsKey(cache_string))
return formatResponse(File.ReadAllText(cachedOutput[cache_string]));
var bundle = new StyleBundle(this.tempFileOutputPath).Include(this.virtualFilePaths.ToArray());
var collection = new BundleCollection();
collection.Add(bundle);
var context = new BundleContext(helper.ViewContext.HttpContext, collection, "");
var response = bundle.GenerateBundleResponse(context);
System.Web.Optimization.CssMinify minify = new CssMinify();
minify.Process(context, response);
string serverPath = helper.ViewContext.HttpContext.Server.MapPath(this.tempFileOutputPath);
string outputPath = Guid.NewGuid().ToString() + ".css";
while(File.Exists(Path.Combine(serverPath, outputPath)))
outputPath = Guid.NewGuid().ToString() + ".css";
File.WriteAllText(outputPath, response.Content);
cachedOutput[cache_string] = outputPath;
return formatResponse(response.Content);
}
IHtmlString formatResponse(string responseContent)
{
StringBuilder responseBuilder = new StringBuilder();
responseBuilder.AppendLine("<style type=\"text/css\">");
responseBuilder.Append(responseContent);
responseBuilder.AppendLine("</style>");
return helper.Raw(responseBuilder.ToString());
}
}
Ok the above class looks complex but all it does is creates an internal list of paths for the bundler to to bundle and minify. When the Minify method is called it formats the virtual paths into a string which is used as a cache key. This is how we determine if we have already bundled and minified these scripts.
If we have we read from the temporary bundled script that is stored on the HDD.
Note: I used hard drive storage as opposed to in memory storage. This can be easily changed by stuffing the full content into a static dictionary or other cacheing medium.
If this file set has not been bundled, minified etc then we proceed to bundle and minify the set. Finally we store the file onto the HDD and return the response. So this is pretty easy.
Now to implement the class I just wrote a quick HTML Helper as the Minification class supports chaining for Add files methods can be implemented nicely.
The HTML Helper:
public static CssOutputBundler BundleCss(this HtmlHelper helper, string outputVirtualPath)
{
return new CssOutputBundler(helper, outputVirtualPath);
}
Ok basic implementation for the bundler. Now this helper takes the string outputVirtualPath this is the virtual PATH to store the temporary files. If you do decide to eliminate saving to HDD and use a cache system you could remove this parameter from the helper and the class.
Finally this can be used in your view as...
#Html.BundleCss("~/Content/").Add("~/Content/Site.css").Add("~/Content/themes/base/jquery-ui.css").Minify();
Where the ~/Content/ is the virtual path on the HDD to store the minified files. Then we add two files "~/Content/Site.css" and ~/Content/themes/base/jquery-ui.css. Finally call the minify method which returns the html string. Such as...
<style type="text/css">
.ui-helper-hidden{display:none}.ui-helper-hidden-accessible{position:absolute!important;clip:rect(1px 1px 1px 1px);clip:rect(1px,1px,1px,1px)} ....more css
</style>
Cheers.
I have a webforms project, and am attempting to run some code that allows me to make a call to an MVC route and then render the result within the body of the web forms page.
There are a couple of HttpResponse/Request/Context wrappers which I use to execute a call to an MVC route, e.g.:
private static string RenderInternal(string path)
{
var responseWriter = new StringWriter();
var mvcResponse = new MvcPlayerHttpResponseWrapper(responseWriter, PageRenderer.CurrentPageId);
var mvcRequest = new MvcPlayerHttpRequestWrapper(Request, path);
var mvcContext = new MvcPlayerHttpContextWrapper(Context, mvcResponse, mvcRequest);
lock (HttpContext.Current)
{
new MvcHttpHandlerWrapper().PublicProcessRequest(mvcContext);
}
...
The code works fine for executing simple MVC routes, for e.g. "/Home/Index". But I can't specify any query string parameters (e.g. "/Home/Index?foo=bar") as they simply get ignored. I have tried to set the QueryString directly within the RequestWrapper instance, like so:
public class MvcPlayerHttpRequestWrapper : HttpRequestWrapper
{
private readonly string _path;
private readonly NameValueCollection query = new NameValueCollection();
public MvcPlayerHttpRequestWrapper(HttpRequest httpRequest, string path)
: base(httpRequest)
{
var parts = path.Split('?');
if (parts.Length > 1)
{
query = ExtractQueryString(parts[1]);
}
_path = parts[0];
}
public override string Path
{
get
{
return _path;
}
}
public override NameValueCollection QueryString
{
get
{
return query;
}
}
...
When debugging I can see the correct values are in the "request.QueryString", but the values never get bound to the method parameter.
Does anyone know how QueryString values are used and bound from an http request to an MVC controller action?
It seems like the handling of the QueryString value is more complex than I anticipated. I have a limited knowledge of the internals of the MVC Request pipeline.
I have been trying to research the internals myself and will continue to do so. If I find anything I will update this post appropriately.
I have also created a very simple web forms project containing only the code needed to produce this problem and have shared it via dropbox: https://www.dropbox.com/s/vi6erzw24813zq1/StackMvcGetQuestion.zip
The project simply contains one Default.aspx page, a Controller, and the MvcWrapper class used to render out the result of an MVC path. If you look at the Default.aspx.cs you will see a route path containing a querystring parameter is passed in, but it never binds against the parameter on the action.
As a quick reference, here are some extracts from that web project.
The controller:
public class HomeController : Controller
{
public ActionResult Index(string foo)
{
return Content(string.Format("<p>foo = {0}</p>", foo));
}
}
The Default.aspx page:
protected void Page_Load(object sender, EventArgs e)
{
string path = "/Home/Index?foo=baz";
divMvcOutput.InnerHtml = MvcWrapper.MvcPlayerFunctions.Render(path);
}
I have been struggling with this for quite a while now, so would appreciate any advice in any form. :)
MVC framework will try to fill the values of the parameters of the action method from the query string (and other available data such as posted form fields, etc.), that part you got right. The part you missed is that it does so by matching the name of the parameter with the value names passed in. So if you have a method MyMethod in Controller MyController with the signature:
public ActionResult MyMethod(string Path)
{
//Some code goes here
}
The query string (or one of the other sources of variables) must contain a variable named "Path" for the framework to be able to detect it. The query string should be /MyController/MyMethod?Path=Baz
Ok. This was a long debugging session :) and this will be a long response, so bear with me :)
First how MVC works. When you call an action method with input parameters, the framework will call a class called "DefaultModelBinder" that will try and provide a value for each basic type (int, long, etc.) and instance of complex types (objects). This model binder will depend on something called the ValueProvider collection to look for variable names in query string, submitted forms, etc. One of the ValueProviders that interests us the most is the QueryStringValueProvider. As you can guess, it gets the variables defined in the query string. Deep inside the framework, this class calls HttpContext.Current to retrieve the values of the query string instead of relying on the ones being passed to it. In your setup this is causing it to see the original request with localhost:xxxx/Default.aspx as the underlying request causing it to see an empty query string. In fact inside the Action method (Bar in your case) you can get the value this.QueryString["variable"] and it will have the right value.
I modified the Player.cs file to use a web client to make a call to an MVC application running in a separate copy of VS and it worked perfectly. So I suggest you run your mvc application separately and call into it and it should work fine.
I am attempting to develop a generic BizTalk application for configuring dynamic ports. I have an orchestration that pulls back all the configuration settings for each port and I want to loop through these settings and configure the ports. The settings are held in MSSQL and, for instance, two of the properties are PortName and Address. So from within the orchestration I would like to reference the port by the string variable PortName. So is there some way to get a collection of all the ports in an orchestration or reference a port via a string variable i.e. Port['MyPortName'](Microsoft.XLANGs.BaseTypes.Address) = "file://c:\test\out\%MessageId%.xml" Thanks
In order to dynamically configure Dynamic Logical Send Ports from within an orchestration, one has to store the settings into a persistent datastore (e.g. a database or configuration file) and implement a way to assign those properties dynamically at runtime.
But first, we need to understand what is happening when configurating a Dynamic Send Port.
How to Configure a Dynamic Logical Send Port
Configuring the properties of a dynamic logical send port from within an orchestration involves two steps:
First, the TransportType and target Address properties must be specified on the Send Port. This is usually done in an Expression Shape with code similar to this:
DynamicSendPort(Microsoft.XLANGs.BaseTypes.TransportType) = "FILE";
DynamicSendPort(Microsoft.XLANGs.BaseTypes.Address) = "C:\Temp\Folder\%SourceFileName%";
Second, any additional transport properties must be specified on the context of the outgoing message itself. Virtually all BizTalk adapters have additional properties that are used for the communication between the Messaging Engine and the XLANG/s Orchestration Engine. For instance, the ReceivedFileName context property is used to dynamically set a specific name for when the FILE adapter will save the outgoing message at its target location. This is best performed inside an Assignment Shape, as part of constructing the outgoing message:
OutgoingMessage(FILE.ReceiveFileName) = "HardCodedFileName.xml"
You'll notice that most configuration properties must be specified on the context of the outgoing messages, specifying a namespace prefix (e.g. FILE), a property name (e.g. ReceiveFileName) and, obviously, the value that gets assigned to the corresponding property.
In fact, all the context properties are classes that live Inside the well-known Microsoft.BizTalk.GlobalPropertySchemas.dll assembly. This is confirmed by looking up this assembly in Visual Studio's object explorer.
Even though most context properties that are necessary to configure Dynamic Logical Send Ports live Inside this specific assembly, not all of them do. For instance, the MSMQ BizTalk adapter uses a separate assembly to store its context properties. Obviously, third-party or custom adapters come with additionnal assemblies as well.
Therefore, in order to setup a context property on a Dynamic Send Port using a flexible approach like the one describe below, four pieces of information are necessary:
The fully qualified name of the assembly containing the context property classes.
The namespace prefix.
The property name.
The property value.
Storing Port Settings in a Persistent Medium
The following .XSD schema illustrate one possible structure for serializing port settings.
Once serialized, the specified context properties can then be stored in a SQL database or a configuration file very easily. For instance, here are the settings used as an example in this post:
A Flexible Approach to Configuring Dynamic Logical Send Ports
With a simple helper Library, setting up the dynamic port configuration is very easy. First, you have to retrieve the serialized settings from the persistent medium. This can easily be achieved using the WCF-SQL Adapter and a simple stored procedure.
Once retrieved, those properties can then be deserialized into a strongly-typed C# object graph. For this, first create a C# representation of the ContextProperties schema shown above, using the following command-line utility:
xsd.exe /classes /language:cs /namespace:Helper.Schemas .\ContextProperties.xsd
This generates a partial class that can be improved with the following method:
namespace Helper.Schemas
{
public partial class ContextProperties
{
public static ContextProperties Deserialize(string text)
{
using (MemoryStream stream = new MemoryStream())
{
byte[] buffer = Encoding.UTF8.GetBytes(text);
stream.Write(buffer, 0, buffer.Length);
stream.Seek(0, SeekOrigin.Begin);
return (ContextProperties)
Deserialize(
stream
, typeof(ContextProperties));
}
}
public static Object Deserialize(Stream stream, Type type)
{
XmlSerializer xmlSerializer = new XmlSerializer(type);
return xmlSerializer.Deserialize(stream);
}
}
}
Second, applying this configuration involves creating an XLANG/s message from code and setting up the context properties dynamically using reflection, based upon the description of the context property classes specified in the deserialized ContextProperties object graph.
For this, I use a technique borrowed from Paolo Salvatori's series of articles regarding dynamic transformations, which consists in creating a custom BTXMessage-derived class, used internally by the BizTalk XLANG/s engine.
namespace Helper.Schemas
{
using Microsoft.BizTalk.XLANGs.BTXEngine; // Found in Microsoft.XLANGs.BizTalk.Engine
using Microsoft.XLANGs.Core; // Found in Microsoft.XLANGs.Engine
[Serializable]
public sealed class CustomBTXMessage : BTXMessage
{
public CustomBTXMessage(string messageName, Context context)
: base(messageName, context)
{
context.RefMessage(this);
}
public void SetContextProperty(string assembly, string ns, string name, object value)
{
if (String.IsNullOrEmpty(ns))
ns = "Microsoft.XLANGs.BaseTypes";
if (String.IsNullOrEmpty(assembly))
assembly = "Microsoft.BizTalk.GlobalPropertySchemas";
StringBuilder assemblyQualifiedName = new StringBuilder();
assemblyQualifiedName.AppendFormat("{0}.{1}, {2}", ns, name, assembly);
Type type = Type.GetType(assemblyQualifiedName.ToString(), true, true);
SetContextProperty(type, value);
}
internal void SetContextProperty(string property, object value)
{
int index = property.IndexOf('.');
if (index != -1)
SetContextProperty(String.Empty, property.Substring(0, index), property.Substring(index + 1), value);
else
SetContextProperty(String.Empty, String.Empty, property, value);
}
}
}
Now, the last piece of the puzzle is how to make use of this custom class from within an Orchestration. This is easily done in an Assignment Shape using the following helper code:
namespace Helper.Schemas
{
using Microsoft.XLANGs.BaseTypes;
using Microsoft.XLANGs.Core; // Found in Microsoft.XLANGs.Engine
public static class Message
{
public static XLANGMessage SetContext(XLANGMessage message, ContextProperties properties)
{
try
{
// create a new XLANGMessage
CustomBTXMessage customBTXMessage = new CustomBTXMessage(message.Name, Service.RootService.XlangStore.OwningContext);
// add parts of the original message to it
for (int index = 0; index < message.Count; index++)
customBTXMessage.AddPart(message[index]);
// set the specified context properties
foreach (ContextPropertiesContextProperty property in properties.ContextProperty)
customBTXMessage.SetContextProperty(property.assembly, property.#namespace, property.name, property.Value);
return customBTXMessage.GetMessageWrapperForUserCode();
}
finally
{
message.Dispose();
}
}
}
}
You can use this static method inside your Assignment Shape like the code shown hereafter, where OutboundMessage represents the message which you want to set the context:
OutboundMessage = Helper.Schemas.Message.SetContext(OutboundMessage, contextProperties);
In the first place you shouldn't attempt to do configuration changes like this using an Orchestration. Technically it's feasible to do what you are attempting to do, but as a practice you shouldn't mix up your business process with administration.
The best way to do such things will be by either writing some normal scripts or PowerShell.
To answer you question, you can get the data you want from BtsOrchestration class in ExplorerOM
http://msdn.microsoft.com/en-us/library/microsoft.biztalk.explorerom.btsorchestration_members(v=bts.20)