Problem: I have a Mongo document that includes two arrays. One of the arrays is large, with subdocuments. This one serializes with no problem. Another is a simple array of this type:
staffgroups {"Tech","Sales"}
This one will not serialize. I get errors saying it's a BsonArray. The closest I've been able to get to serializing it produces a string. I need a JSON object.
Code time:
public class specialtyGroup
{
public ObjectId _id { get; set; }
public string name { get; set; }
public string location { get; set; }
public coachConfig config { get; set; }
public schedules[] coaches { get; set; }
public BsonArray staffgroups { get; set; }
}
And the webservice:
public void GetGroups()
{
var client = new MongoClient();
var db = client.GetDatabase("MongoTul");
var coll = db.GetCollection<specialtyGroup>("specialtyGroups");
string cname = HttpContext.Current.Request.Params["loc"];
var creatures = coll.Find(b => b.location == cname)
.ToListAsync()
.Result;
JavaScriptSerializer js = new JavaScriptSerializer();
Context.Response.Write(js.Serialize(creatures));
}
I've tried using aggregation and projecting. I've tried creating an additional class for staffgroups (which works for my complex array). And a few other things. All no good.
Most common error looks like this: Unable to cast object of type 'MongoDB.Bson.BsonString' to type 'MongoDB.Bson.BsonBoolean'.
I spent hours on this before posting here, then after posting I figured it out in 30 mins. Sorry.
The answer is staffgroups should be "public string[] staffgroups {get; set;}
So if any other rubes like me have that question, there's the answer.
Related
I am running into issues after adding a sub-object to my mongo documents. The query no longer returns results, even though I've added an object to my model to store the new sub-object.
I believe the issue is in adding the class for the sub-object to the object model. I can't seem to find any references anywhere online, so perhaps I'm searching for the wrong thing?
Mongo elements look as so:
{
_id: [id],
Name: "Paul",
Phone1: {
Name: "Work",
Number: "15551234567"
},
Phone2: {
Name: "Work",
Number: "15551234567"
}
}
In C# my model looks as so:
public class PersonModel {
[BsonId]
public ObjectId _Id { get; set; }
public string Name { get; set; }
public Phone Phone1 { get; set; }
public Phone Phone2 { get; set; }
}
public class Phone {
public string Name { get; set; }
public string Number { get; set; }
}
My query looks as so:
public async Task<List<PersonModel>> GetPerson(string name)
{
var people = new List<PersonModel>();
var allDocuments = await PersonCollection.FindAsync(
ds => ds.Name == name);
await allDocuments.ForEachAsync(doc => people.Add(doc));
return people;
}
Any references to a working example would be appreciated.
Thank you for looking.
The above implementation is correct. After many hours of trouble shooting it turned out I didn't have the datapoint in my database that I was querying against. Unbelievable.
If anyone else is struggling, I also found this guide that confirmed I was dealing with the subobject correctly: https://www.codementor.io/pmbanugo/working-with-mongodb-in-net-1-basics-g4frivcvz
I have two tables (NPG_Chemical and NPG_Chemical_Synonym):
public partial class NPG_Chemical
{
[Key]
[Column(TypeName = "numeric")]
public decimal NPG_Chemical_ID { get; set; }
[StringLength(256)]
public string Chemical { get; set; }
}
public partial class NPG_Chemical_Synonym
{
[Key]
[Column(TypeName = "numeric")]
public decimal NPG_Chemical_Synonym_ID { get; set; }
[ForeignKey("NPG_Chemical_ID")]
[Column(TypeName = "numeric")]
public decimal NPG_Chemical_ID { get; set; }
[StringLength(512)]
public string Synonym { get; set; }
}
In the NPG_ChemicalController I have something like:
[HttpPost]
public ActionResult Create(NPG_ChemicalViewModel model)
{
using (var context = new NPG_Model())
{
var chemical = new NPG_Chemical();
chemical.Chemical = model.NPG_Chemical.Chemical;
context.NPG_Chemical.Add(chemical);
var synonym = new NPG_Chemical_Synonym();
synonym.Synonym = model.NPG_Chemical_Synonym.Synonym;
synonym.NPG_Chemical_ID = chemical.NPG_Chemical_ID;
context.NPG_Chemical_Synonym.Add(synonym);
context.SaveChanges();
}
return View();
}
and NPG_ChemicalViewModel:
namespace NPG_Administrative_Utility.Models
{
public class NPG_ChemicalViewModel
{
public NPG_ChemicalViewModel()
{
NPG_Chemical = new NPG_Chemical();
NPG_Chemical_Synonym = new NPG_Chemical_Synonym();
}
public NPG_Chemical NPG_Chemical { get; set; }
public NPG_Chemical_Synonym NPG_Chemical_Synonym { get; set; }
}
}
When I try to create a view based on NPG_ChemicalViewModel, it shows:
Can any one help me on this?
You'll need a view model. At the simplest, you can just do something like:
public class NPG_ChemicalViewModel
{
public NPG_ChemicalViewModel()
{
NPG_Chemical = new NPG_Chemical();
NPG_Chemical_Synonym = new NPG_Chemical_Synonym();
}
public NPG_Chemical NPG_Chemical { get; set; }
public NPG_Chemical_Synonym NPG_Chemical_Synonym { get; set; }
}
Then, change your action to accept this:
public ActionResult Create(NPG_ChemicalViewModel model)
In your view, you would generate the individual properties like:
#Html.EditorFor(m => m.NPG_Chemical.Chemical)
However, it's far better to only include the properties on your view model that you want to be edited:
public class ChemicalViewModel
{
public string Chemical { get; set; }
public string Synonym { get; set; }
}
Then, in your action, you just map this posted data where it should go:
var chemical = new NPG_Chemical();
chemical.Chemical = model.Chemical;
context.NPG_Chemical.Add(chemical);
var synonym = new NPG_Chemical_Synonym();
synonym.Synonym = model.Synonym;
synonym.NPG_Chemical_ID = chemical.NPG_Chemical_ID;
context.NPG_Chemical_Synonym.Add(synonym);
That said, there's some significant issues with your code here. First, it looks like you're dealing with a one-to-one or one-to-many relationship here between Chemical and Synonym, but right now, you have no foreign keys being utilized. You should add a navigation property to your synonym class:
[ForeignKey("NPG_Chemical_ID")]
public NPG_Chemical Chemical { get; set; }
That tells Entity Framework that you have a relationship and among other things allows it to automatically fill in IDs as necessary. For example, with that, you could now simply do:
synonym.Chemical = chemical;
Instead of directly referencing the ID. That way, if the id is autogenerated or otherwise unknown before saving, the relationship will still be preserved. Whereas, without it, you'd have to save chemical first, set the autogenerated id on synonym and then save the synonym in a separate transaction.
Second, if you're going to use keys typed as "numeric". Then, you're going to be responsible for generating a unique numeric string for each record. That's a huge pain, as it's going to require checking a proposed id against other existing record ids before actually saving. Otherwise, you run the risk of a primary key collision. It's far better to use a standard autoincrementing PK or barring that, at least a GUID, where you're assured a reasonably low risk of collisions occurring.
Third, you should absolute not use using with your context. Here it's not a big deal, since, you're only saving and not reading data from the database, but in a typical view, lazy-loading will kick you in the posterior quick doing that. Your context should be request-scoped, either as an instance variable on your controller (since the controller is newed up and disposed with each request) or using dependency injection. You never want to create an instance of your context anywhere else, including an action method.
First of all, this is not exactly a duplication of the dozens of other posts and I have tried all of them and none of them work.
I have a model that contains many more values than my web api consumers need.
public class Publication
{
[Key]
public int PublicationID { get; set; }
public string PublicationTitle { get; set; }
public string Frequency { get; set; }
public DateTime NextIssueDate { get; set; }
public DateTime SpaceDeadline { get; set; }
public DateTime MaterialsDeadline { get; set; }
public DateTime CreatedDt { get; set; }
public string CreatedBy { get; set; }
public DateTime UpdatedDt { get; set; }
public string UpdatedBy { get; set; }
}
I only want say a few of the fields to be passed in the API. I've tried this code but instead of leaving out say UpdateBy in the Json result, it returns it with a null value. How do I get rid of that? I've tried several dozen variations but they either fail to compile or fail to return results.
public IQueryable<Publication> GetPublications()
{
return db.Publications
.ToList()
.Select(p => new Publication {
PublicationID = p.PublicationID,
PublicationTitle = p.PublicationTitle,
Frequency = p.Frequency,
NextIssueDate = p.NextIssueDate
})
.AsQueryable();
}
Don't serialize your DAO. Create a complete contract and then serialize it selectively. To creating different contracts for different cases, you could simplify it using Json.Net; you could just create a custom contract resolver and use it as a parameter of SerializeObject() like so
static void Main(string[] args)
{
var person = new TestContract {FirstName = "John", LastName = "Doe", Age = 36};
var firstNameContract = new SelectiveSerializer("firstname");
var allPropertiesContract = new SelectiveSerializer("firstname, lastname, age");
var allJson = JsonConvert.SerializeObject(
person,
Formatting.Indented,
new JsonSerializerSettings {ContractResolver = allPropertiesContract});
var firstNameJson = JsonConvert.SerializeObject(
person,
Formatting.Indented,
new JsonSerializerSettings {ContractResolver = firstNameContract});
Console.WriteLine(allJson);
// {
// "FirstName": "John",
// "LastName": "Doe",
// "Age": 36
// }
Console.WriteLine(firstNameJson);
// {
// "FirstName": "John",
// }
}
public class SelectiveSerializer : DefaultContractResolver
{
private readonly string[] _fields;
public SelectiveSerializer(string fields)
{
var fieldColl = fields.Split(',');
_fields = fieldColl
.Select(f => f.ToLower().Trim())
.ToArray();
}
protected override JsonProperty CreateProperty(MemberInfo member, MemberSerialization memberSerialization)
{
var property = base.CreateProperty(member, memberSerialization);
property.ShouldSerialize = o => _fields.Contains(member.Name.ToLower());
return property;
}
}
public class TestContract
{
public string FirstName { get; set; }
public string LastName { get; set; }
public int Age { get; set; }
}
Without much effort, you could probably work this into your default mediatype formatter (in the pipeline) to look for a parameter in the request called 'fields' or whatever and then use the custom contract resolver if present, and then it would be seamless default behavior to limit fields if specified or serialize the entire object if not specified.
On the academic side, here is the justification:
Any modification to the data is considered a "view concern" which means, in an API, it should controlled by query parameters and accept header. In this case, the "representation" of the data is application/json and you've chose to "filter" the returned fields. All of this can (and should be, imo) be handled during serialization. So your "model" in this case will always be the full model vs. some subset of the model. The full model in this example contains first name, last name, and age. In reality, this could be hundreds of properties. If you want to allow the client to choose a subset of the complete model, this is how you could do it with selective serialization.
You can similar behaviors in graph apis. There, the default for large models is that you get an empty object if you don't specify fields, forcing the client to be very specific about what it asks for, which is great when payload size matters (e.g. mobile applications). And, there's nothing stopping from creating field presets like 'name' which could mean 'firstname, lastname' or 'all' which includes all properties.
I've never been a fan of having hundreds of data objects that all serve some ad hoc requirement for a data set that is used in 20 different contexts where some cases require more data while others require less. IMO if you have to go through the same process to get the data, whether it complete or not, you shouldn't waste your time creating additional objects to frame the data for the sake of the client, and this should help you achieve that.
It's because you're returning a collection of Publication objects so you will get every property that is contained in that class, whether you populate it or not. If you want to return a subset of the properties then create a class that has only the properties you want to return and create an instance of that class in your query.
public IQueryable<WhatIReallyWantToReturn> GetPublications()
{
return db.Publications
.ToList()
.Select(p => new WhatIReallyWantToReturn {
PublicationID = p.PublicationID,
PublicationTitle = p.PublicationTitle,
Frequency = p.Frequency,
NextIssueDate = p.NextIssueDate
})
.AsQueryable();
}
private class WhatIReallyWantToReturn
{
public int PublicationID { get; set; }
public string PublicationTitle { get; set; }
public string Frequency { get; set; }
public DateTime NextIssueDate { get; set; }
}
using Newtonsoft.Json;
public class Publication
{
[Key]
public int PublicationID { get; set; }
public string PublicationTitle { get; set; }
public string Frequency { get; set; }
public DateTime NextIssueDate { get; set; }
public DateTime SpaceDeadline { get; set; }
public DateTime MaterialsDeadline { get; set; }
[JsonIgnore]
public DateTime CreatedDt { get; set; }
[JsonIgnore]
public string CreatedBy { get; set; }
[JsonIgnore]
public DateTime UpdatedDt { get; set; }
[JsonIgnore]
public string UpdatedBy { get; set; }
}
as Craig W. said you can use viewmodel ,also you can use anonymous type
(notice viewmodel is better way because you can use some utilities like automapper for mapping your property automatically)
JsonIgnore annotation has worked for me
[JsonIgnore]
public int Ranking { get; set; }
Here is a great article (Dec 2019) on the subject. It offers a solution for data shaping by making use of ExpandoObject and Type Reflection. The properties that the client requires can then be passed through the request as a query parameter (i.e. separated by a comma). The article also offers solution to the JSON Serialization problem.
Startup.cs file:
services.AddControllers(config =>
{
config.RespectBrowserAcceptHeader = true;
config.ReturnHttpNotAcceptable = true;
})
.AddXmlDataContractSerializerFormatters()
.AddNewtonsoftJson();
+1 for Sinaesthetic's answer.
I just finished reading an article, about GraphQL which solves exactly this problem. You can define exactly which fields do you need in the same request. No need for creating new endpoints every single time, when the caller needs just a specific subset of the properties.
If you can do this in .NET WEB API too without creating new models and endpoints, with just a very little extra effort, why wouldn't you (instead of exchanging Web Api for GraphQL).
Actually his SelectiveSerializer could be upgarded with reflection, so if you want to define which props you need in
C#, you can do this by providing property expressions, so you don't have to worry about misstyping prop names.
I bet there are other solutions for this, but the basic concept is the most important that we can define which fields we need in our json without creating new models.
public class ListKeywords
{
public int ID { set; get; }
public string Keyword { set; get; } //关键词
public string Language { set; get; } //语种
public int WordCount { set; get; } //单词数
public int WordLength { set; get; } // 字符数
public int Status { set; get; } //采集状态 0-未采集 1-采集成功 2-保存失败 3-保存成功 4-发布失败 5-发布成功
public bool Taken { set; get; }
public bool FTPStatus { set; get; }
public bool DBStatus { set; get; }
public string UrlName { set; get; }
public ListKeywords()
{
}
public ListKeywords(string keyword)
{
this.Keyword = keyword;
}
}
List<string> lines = new List<string>();
List<ListKeywords> keywordsList = new List<ListKeywords>();
using (StreamReader sr = File.OpenText(filePath))
{
string s = String.Empty;
while ((s = sr.ReadLine()) != null)
{
//lines.Add(s); //Operating normally
eywordsList.Add(new ListKeywords("some keywords")); // Operating normally
keywordsList.Add(new ListKeywords(s)); // it will be out of memeory
}
}
In text file, have 1,000,000 line data, if i use above code to load the large data to list< keywordsList >, it will raises an OutOfMemoryException, but if i load it to list< string >, it run normally. How to solved it ?
Instead of using a List maybe try using an IEnumerable w/ yield?
static IEnumerable<ListKeywords> Keywords()
{
using (StreamReader sr = File.OpenText(path))
{
string s = String.Empty;
while ((s = sr.ReadLine()) != null)
{
yield return new ListKeywords(s);
}
}
}
Note that Jon Skeet's C# in Depth offers a great explanation about this in Chapter 6. I imagine he also has some articles or posts on StackOverflow about this topic. As he points out, you want to be careful about modifying this method to pass in a StreamReader (or TextReader as is used in his example) as you would want to take ownership of the reader so it will be properly disposed of. Rather, you would want to pass in a Func<StreamReader> if you have such a need. Another interesting note he adds here - which I will point out because there are some edge cases where the reader will not actually be properly disposed of even if you don't allow the reader to be provided by the caller - it's possible for the caller to abuse the IEnumerable<ListKeywords> by doing something like Keywords().GetEnumerator() - this could result in a memory leak and could even potentially cause security issues if you have security-related code which relies on the using statement to clean up the resource.
I'm trying to write an OpenSearch Suggestion service that complies with the OpenSearch spec.
http://www.opensearch.org/Specifications/OpenSearch/Extensions/Suggestions
This spec requires the service to return a JSON array with the first element being a string and the following elements being arrays of strings. I'm able to get it almost there by returning an array of strings (string[][]) and having WCF serialize this into JSON. However, in order to comply with the spec, I tried to return an array of objects (object[]), with the first one being a string, and the rest being arrays of strings (string[]).
Whenever I try to return the array of objects, it doesn't work, such as this:
From service:
[AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
public class SuggestionService : ISuggestionService
{
public object[] Search(string searchTerms)
{
SearchSuggestions = new object[4];
SearchText = searchTerms;
SearchSuggestions[0] = SearchText;
Text = new string[10];
Urls = new string[10];
Descriptions = new string[10];
// Removed irrelevant ADO.NET code
while (searchResultReader.Read() && index < 10)
{
Text[index] = searchResultReader["Company"].ToString();
Descriptions[index] = searchResultReader["Company"].ToString();
Urls[index] = "http://dev.localhost/Customers/EditCustomer.aspx?id=" +
searchResultReader["idCustomer"];
index++;
}
SearchSuggestions[1] = Text;
SearchSuggestions[2] = Descriptions;
SearchSuggestions[3] = Urls;
return SearchSuggestions;
}
[DataMember]
public string SearchText { get; set; }
[DataMember]
public string[] Text { get; set; }
[DataMember]
public string[] Descriptions { get; set; }
[DataMember]
public string[] Urls { get; set; }
[DataMember]
public object[] SearchSuggestions { get; set; }
}
Here's the entire interface:
[ServiceContract]
public interface ISuggestionService
{
[OperationContract]
[WebGet(UriTemplate = "/Search?q={searchTerms}",
BodyStyle = WebMessageBodyStyle.Bare,
ResponseFormat = WebMessageFormat.Json)]
object[] Search(string searchTerms);
}
This causes the service to return "Error 324 (net::ERR_EMPTY_RESPONSE): Unknown error." This is the only error I've been able to get.
Am I not able to use an array of objects to store one string and three arrays? What else could I do in order to use WCF to return the proper JSON that complies with this spec?
EDIT: Added lots more of the code
You posted a bunch of [DataMember]'s. Please post the entire [DataContract]. Also show us the JSON returned when you return that DataContract.
A Data Contract should never include behavior. Try the following (I haven't had a chance to test it, and will need to fake up the data to do so):
[AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
public class SuggestionService : ISuggestionService
{
public SearchResults Search(string searchTerms)
{
var results = new SearchResults
{
SearchText = searchTerms,
Text = new string[10],
Urls = new string[10],
Descriptions = new string[10]
};
// Removed irrelevant ADO.NET code
int index = 0;
while (searchResultReader.Read() && index < 10)
{
results.Text[index] = searchResultReader["Company"].ToString();
results.Descriptions[index] = searchResultReader["Company"].ToString();
results.Urls[index] = "http://dev.localhost/Customers/EditCustomer.aspx?id=" +
searchResultReader["idCustomer"];
index++;
}
return results;
}
}
[DataContract]
public class SearchResults
{
[DataMember]
public string SearchText { get; set; }
[DataMember]
public string[] Text { get; set; }
[DataMember]
public string[] Descriptions { get; set; }
[DataMember]
public string[] Urls { get; set; }
}
Ok, I read enough of that spec and of the JSON spec, to convince myself you really do need to return an array of objects, and not an instance of a class that contains an array of objects. Of course, that didn't quite work. Here's what you needed:
[ServiceContract]
[ServiceKnownType(typeof(string))]
[ServiceKnownType(typeof(string[]))]
public interface ISuggestionService
{
[OperationContract]
[WebGet(UriTemplate = "/Search?q={searchTerms}",
BodyStyle = WebMessageBodyStyle.Bare,
ResponseFormat = WebMessageFormat.Json)]
object[] Search(string searchTerms);
}
I just tried it, and it worked. Here's the JSON (indentation added):
[
"abc",
["Company1","Company2","Company3",...],
["Company1 Description","Company2 Description","Company3 Description",...],
["http:\/\/dev.localhost\/Customers\/EditCustomer.aspx?id=1",
"http:\/\/dev.localhost\/Customers\/EditCustomer.aspx?id=2",
"http:\/\/dev.localhost\/Customers\/EditCustomer.aspx?id=3",...]
]
This is a problem that had me stumped for a while as well - there's a complete end-to-end walkthrough of how to do this, including how to support both JSON and XML opensearch (including XML attribute serialization), with downloadable code, at "Building Labs – Writing an OpenSearch Suggestions provider in C# with WCF".