Has anyone tried (and succeeded) to get subscriptions out of RavenDB with the Raven DB .Net client?
Having some Json serialisation issues that when the following runs, it throws with a
"Error converting value "Subscriber.Messages.Events.MyEvent, Version=1.0.0.0" to type 'NServiceBus.Unicast.Subscriptions.MessageType'. message
The code is simple :
var documentStore = new DocumentStore
{
Url = "http://localhost:8080/",
DefaultDatabase = "publisher",
};
documentStore.Initialize();
using (var session = documentStore.OpenSession())
{
return session.Query<NServiceBus.Unicast.Subscriptions.Raven.Subscription>("Raven/DocumentsByEntityName").ToArray();
}
It's definitely a serialisation issue as the retrieval works. As it does using the alternative below:
session.Advanced.LuceneQuery<Subscription>("Raven/DocumentsByEntityName").QueryResult.Results[0]
In the RaveDB studio I can see the following document in the publisher database.
{
"MessageType": "Subscriber.Messages.Events.MyEvent, Version=1.0.0.0",
"Clients": [
{
"Queue": "samplesubscriber",
"Machine": "myDesktopHere"
}
]
}
Error converting value "Subscriber.Messages.Events.MyEvent, Version=1.0.0.0" to type 'NServiceBus.Unicast.Subscriptions.MessageType'.
Anyone have a clue why the serialization fails?
I'm using NServiveBus.Host 4.2, Raven-DB client 1.0.616, and Newtonsoft.json 4.0.5.
Incidentally I've pulled up the types using dotpeek and created local versions. I created my own subcription, MessageType, MessageTypeConvertor from the NSB dll's. I then managed to deserialise the strings without issue. Any thoughts?
EDIT
As per suggestions the advance Lucene query does a great job of retrieving the results. But then deserialization fails. For example, search results are returned in the first line, but fail to deserlize in the return statement. I've pulled up a local version of the Subscription type from the NSB dll's, and implemented the type converter, again pulled up from the NSB libraries, and using those in place of NServiceBus.Unicast.Subscriptions.Raven.Subscription works fine. Inevitably, that's not a stable choice.
var searchResults = session.Advanced.LuceneQuery<NServiceBus.Unicast.Subscriptions.Raven.Subscription>("Raven/DocumentsByEntityName").WhereEquals("Tag", "Subscription").QueryResult.Results;
return searchResults.Select(subscriptionJsonObject => JsonConvert.DeserializeObject<NServiceBus.Unicast.Subscriptions.Raven.Subscription>(subscriptionJsonObject.ToString())).ToList();
Any further thoughts?
When you query with this form:
session.Query<Entity>("IndexName")
You are asking for all items of that index to be returned from the query, and then telling the RavenDB client to deserialize them all as the type you specified.
Normally, that works out quite well because a specific index would be built for the types you're working with. For example, you would usually build your own index of subscriptions for a specific purpose, and query that:
session.Query<Subscription>("Subscriptions/ByWhatever")
or if you built the index from C#, you might like this syntax instead to avoid the string:
session.Query<Subscription, Subscriptions_ByWhatever>()
You could also just let Raven build an index for you automatically:
session.Query<Subscription>()
Since you used the built-in Raven/DocumentsByEntityName index, you are returning all documents in the database, not just those that are subscriptions. Since only some can be deserialized as a Subscription type, the others are failing with serialization errors.
If you'd like to continue to use that index, you would need to filter it to just those of that type:
session.Advanced.LuceneQuery<Subscription>("Raven/DocumentsByEntityName")
.WhereEquals("Tag", "Subscriptions")
The LuceneQuery form is easier here, since you are filtering a field by its string name.
You could use a regular LINQ query, but you'd have to create a type that contains the Tag field, like this:
class RDBENIndexEntry
{
public string Tag { get; set; }
}
...
session.Query<RDBENIndexEntry>("Raven/DocumentsByEntityName")
.Where(x => x.Tag == "Subscriptions").OfType<Subscription>()
Related
Here is my Enum:
public enum AdvertStatus
{
Active,
Archived
}
And my entity type:
public record Advertisement
{
...
public AdvertStatus Status { get; set; }
...
}
In database it's stored as int, Database is Postgree
When I try to compare it like so:
data = data.Where(x => x.Status == searchValues.Status);
Entity Framework throws an exception sayng:
.Status == (int)__searchValues_Status_3)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'.
I tried solutions from this question: LINQ TO ENTITY cannot compare to enumeration types but it did't work.
EDIT 1:
data is database table context IQueryable<AdvertisementDTO>
searchValues.Status is type of AdvertStatus from search filter
The issue may be higher up in your Linq query, such as you are attempting to project with a Select or ProjectTo before filtering. For simple types like int/string this should work, but depending on how your DTO is declared you might be introducing problems for mpgsql.
For instance if your query is something like:
var query = _context.Advertisements
.Select(x => new AdvertisementDTO
{
// populate DTO
}).Where(x => x.Status == searchValues.Status)
// ....
then npgsql may be having issues attempting to resolve the types between what is in the DTO and the enumeration in your searchValues. From what the exception detail looks like, npgsql is trying to be "safe" with the enum and casting to intbut feeding that to PostgreSQL that results in invalid SQL. I did some quick checks and the DTO would need to be using the same Enum type (C# complains if the DTO cast the value to int, cannot use == between AdvertStatus and int fortunately) The project may have something like a value converter or other hook trying to translate enumerations which is getting brought into the mix and gunking up the works.
Try performing the Where conditions prior to projection:
var query = _context.Advertisements
.Where(x => x.Status == searchValues.Status)
.Select(x => new AdvertisementDTO
{
// populate DTO
})
// ....
If the data value is stored as an Int then this should work out of the box. npgsql does support mapping to string (which would require a ValueConverter) as well as database declared enumerations. (https://www.postgresql.org/docs/current/datatype-enum.html) However, Int columns should work fine /w enums.
If that doesn't work, I'd try with a new DbContext instance pointed at the DB and a simple entity with that Enum to load a row from that table to eliminate whether npgsql is translating the enum correctly, just to eliminate any possible converters or other code that the main DbContext/models/DTOs may be contributing.
It was all my mistake in higher repo Select projection.
Thanks you all for help. Cheers.
I've migrated concepts from a couple of CQRS frameworks I've seen and just started facing some issues.
I have a common EntityDbContext subclass which I use in any consuming project without further extension to suit the domain of the application, rather I provide interfaces, IReadEntities and IWriteEntities which have methods like Query() and Get() which behind the scenes call Set() returning the DbSet() then allowing the standard LINQ expressions to be chained on as for any EF query. I'm facing issues around using Include() on my IQueryables as I'm using LinqKit with AsExpandable() at the end of all my calls. This is what my context Query methods look like
public new IQueryable<TEntity> Query<TEntity>() where TEntity : class, IEntity
{
// AsNoTracking returns entities that are not attached to the DbContext
return QueryUnfiltered<TEntity>().Where(_recordAuthority.Clause<TEntity>());
}
public IQueryable<TEntity> QueryUnfiltered<TEntity>() where TEntity : class, IEntity
{
// AsNoTracking returns entities that are not attached to the DbContext
return Set<TEntity>().AsNoTracking().AsExpandable();
}
A typical query handler looks like this:
public async Task<IEnumerable<GetCustomerView>> Handle(CustomersBy query, CancellationToken cancellationToken)
{
var customers = _db.Query<Customer>();
// Apply filters
if (!string.IsNullOrEmpty(query.FirstName))
customers = customers.Where(x => x.FirstName.Contains(query.FirstName));
if (!string.IsNullOrEmpty(query.LastName))
customers = customers.Where(x => x.LastName.Contains(query.LastName));
// Execute the query and return the results
var view = await customers.Select(x => new GetCustomerView
{
Id = x.Id,
FirstName = x.FirstName,
LastName = x.LastName,
EmailAddress = x.EmailAddress
}).ToListAsync(cancellationToken).ConfigureAwait(false) as IEnumerable<GetCustomerView>;
return view;
}
This scenario works fine if I wanted to pull address details from a related table as I use projection on the database serve given I'm using the Select prior to execution. There are scenarios though where it makes sense to pull an object graph back and specify Include(...) statements but as it stands specifying _db.Query<Customer>().Include(c => c.Address) doesn't hydrate the Address navigation property. I've tried leaving the AsExpandable() off and then the results come back.
The question is, does anyone see a way to allow the Include statements to be provided maybe as a parameter to the method and then I loop through them and tack them on before calling AsExpandable()? I can't quite get my head around how to do it, if it's possible.
Maybe there's another approach?
Interestingly this apparently works fine on a version of this pattern a colleague uses where they are using EF 6. He says they specify Include after the AsExpandable without a problem.
This is known issue with EF Core and LinqKit AsExpandable (and in general with any extension library which uses custom IQueryProvider to perform its query expression tree preprocessing like LinqKit), because EF Core ignores all EF Core specific IQueryable extensions (Include / ThenInclude, AsNoTracking etc.) it the query provider is different (or does not inherit) the EF Core one (EF6 has no such requirements).
With that being said, currently there is no other solution than applying all EF Core specific extensions before calling AsExpandable.
Ok this works. I created an overload:
public IQueryable<TEntity> Query<TEntity, TProperty>(IEnumerable<Expression<Func<TEntity, TProperty>>> includes) where TEntity : class, IEntity
{
var query = Set<TEntity>().AsNoTracking();
foreach (var expression in includes)
{
query = query.Include(expression);
}
return query.AsExpandable();
}
From my handler I create a list of include expressions and pass to the Query:
var includes = new List<Expression<Func<Customer, object>>>
{
c => c.Address
};
var customers = _db.Query(includes);
var result = await customers.ToListAsync(cancellationToken).ConfigureAwait(false);
Execution of the query has the navigation property populated:
Means I'm not 'Fluent'ly chaining them from the client code's perspective, but I don't think it's terrible.
Thoughts?
I am trying to set up a Web API to be used for capturing the Clocking In and Clocking Out times of Employees after scanning a QR code.
I have set up this SQL Server database to test out the logic for Web API and IONIC 3 project.
My SQL Server code are as follows:
-- Creating table 1 'FarmWorker'
CREATE TABLE [dbo].[FarmWorker]
(
[FarmWorkerNum] int primary key identity(1,1),
[FarmWorkerFName] varchar (15) not null,
[FarmWorkerLName] varchar (15) not null,
[FarmWorkerIDNum] char (13) not null
);
GO
-- Creating table 2 'AttendenceSheet'
CREATE TABLE [dbo].[AttendenceSheet]
(
[AttendenceSheetID] int primary key identity(1,1),
[ClockInTime] datetime not null,
[ClockOutTime] datetime not null,
[FarmWorkerNum] int not null
FOREIGN KEY REFERENCES [FarmWorker](FarmWorkerNum)
);
GO
My Web API has been set up using Visual Studio 2015 and ASP.Net Web API 2.
My controller code for the Web API are as follows:
namespace AttendanceService.Controllers
{
public class FarmWorkerController : ApiController
{
public IEnumerable<FarmWorker> Get()
{
using (AttDBEntities entities = new AttDBEntities())
{
return entities.FarmWorkers.ToList();
}
}
public FarmWorker Get(int id)
{
using (AttDBEntities entities = new AttDBEntities())
{
return entities.FarmWorkers.FirstOrDefault(e =>Convert.ToInt32(e.FarmWorkerIDNum) == id);
}
}
}
}
When I run my Web API in the browser and attempt to run the first Get (to return a list of all FarmWorkers), I get the following error:
An error has occurred
System.InvalidOperationException: the 'ObjectContent`1' type failed to serialize the response body for content type 'application/xml; charset=utf-8'.
Type 'System.Data.Entity.DynamicProxies.FarmWorker_DA4B06B144222E86714953DE48C1952FFB 59C5CE216BAE8D8D506D8AEE9CEEBB' with data contract name 'FarmWorker_DA4B06B144222E86714953DE48C1952FFB59C5CE216BAE8D8D506D8AEE9CEEBB:htt p://schemas.datacontract.org/2004/07/System.Data.Entity.DynamicProxies' is not expected. Consider using a DataContractResolver if you are using DataContractSerializer or add any types not known statically to the list of known types - for example, by using the KnownTypeAttribute attribute or by adding them to the list of known types passed to the serializer.
Does anyone know how to resolve this error? Any help will greatly be appreciated.
Thank you in advance.
[EDIT / UPDATE 1] - 2 hour after posting
I have managed to resolve the error message by adding in this code into the WebApiConfig.cs file:
config.Formatters.Remove(config.Formatters.XmlFormatter);
config.Formatters.Add(config.Formatters.JsonFormatter);
config.Formatters.JsonFormatter.SerializerSettings.Formatting = Newtonsoft.Json.Formatting.Indented;
QUESTION: Why do I receive an error when it is using XML format over JSON format? If possible, I would like to use the XML format, is there any way to make this possible ?
Thanks in advance again.
My guess is that whatever you're using to call the endpoint (e.g. your browser, Fiddler, Postman), is specifying a Content-Type of application/json in the request. Try inspecting the request to see if that is the value of the header being sent.
If you really do wish to use XML, add the XML formatter again and make a request using a Content-Type of application/xml, which should return the content in XML format for you.
Solving the issue of 'ObjectContent1', I had to insert code into the WebApiConfig.cs
config.Formatters.Remove(config.Formatters.XmlFormatter);
config.Formatters.Add(config.Formatters.JsonFormatter);
config.Formatters.JsonFormatter.SerializerSettings.Formatting = Newtonsoft.Json.Formatting.Indented;
I am currently building a web api service using MVC and I am creating the endpoints. For example, my GET endpoint will execute a stored procedure and return the data in JSON format. The model of the data returned can vary in the future and it seems like using a dynamic return type would remove the need of having to change the model and mapping every time that happens. Basically, here is some sample code, do you notice any malpractices in my implementation?
[System.Web.Mvc.HttpGet]
[Route("companies/{id}")]
public dynamic GetCompany([FromUri] int id, string userId)
{
var parameters = new Hashtable
{
{"UserID", userId},
{"CompanyID", id}
};
var result = MyDB.ExecuteSp(CompanyReadByIdSp, parameters);
return result;
}
In fact, this would enable me to transform the object and add whatever I want to it without needing to worry about the model. Is this a bad way of doing things? Thanks ahead.
So, there are a wealth of Flex articles online about how to handle a .NET WebMethod that returns a DataSet or DataTable. Here is an example:
Handling web service results that contain .NET DataSets or DataTables
So, I know how to use result.Tables.<tablename>.Rows and the like. But what I cannot seem to figure out or find online is how to go the other direction - a method to pass objects or tables back to the .NET Webservice from Flex, without stooping to passing XML as a string, or making huge web service methods that have one parameter for each property/column of the object being stored. Surely others, smarter than I, have tackled this issue.
I am using ASP.NET 2.0 Typed DataSets, and it would be really nice if I could just pass one object or array of objects from Flex to the web service, populate my Typed DataTable, and do an Update() through the corresponding typed TableAdapter. My dream would be a [WebMethod] something like one of these:
public void SaveObject(TypedDataTable objToSave) { ... }
public void SaveObject(TypedDataSet objToSave) { ... }
I've had the typed datatables saving to the database, I know how to do that part and even a few tricks, but we had XML being passed back-and-forth as a string - eww. I'm trying to get to a more object-based approach.
The best object based approach is AMF. I assume its probably a bit late in your your development cycle to change your integration layer, but otherwise I dont know of a way to get around marshalling your object(s) back into XML or separating them out into their primitive components.
For .NET implementations of AMF check out:
FlourineFX(FOSS)
WebORB for .NET
Its amazing how easy things become once AMF is used, for example using the Mate MVC framework and an AMF call passing a complex object to the server looks something like this:
<mate:RemoteObjectInvoker instance="yourWebservice" method="saveComplexObject" showBusyCursor="true" >
<mate:resultHandlers>
<mate:CallBack method="saveComplexObjectSuccess" arguments="{[resultObject]}" />
</mate:resultHandlers>
<mate:faultHandlers>
<mate:MethodInvoker generator="{DataManager}" method="presentFault" arguments="{fault}" />
</mate:faultHandlers>
</mate:RemoteObjectInvoker>
With result and fault handlers being optional.
The direction I ended up going was close to what I hoped was possible, but is "hack-ish" enough that I would consider SuperSaiyen's suggestion to use AMF/ORM a better solution for new/greenfield projects.
For sake of example/discussion, let's say I am working with a Person table in a database, and have a typed DataSet called PeopleDataSet that has PersonTableAdapter and PersonDataTable with it.
READ would look like this in .NET web service:
[WebMethod]
public PeopleDataSet.PersonDataTable GetAllPeople() {
var adapter = new PersonTableAdapter();
return adapter.GetData();
}
... which in Flex would give you a result Object that you can use like this:
// FLEX (AS3)
something.dataProvider = result.Tables.Person.Rows;
Check out the link I put in the question for more details on how Flex handles that.
CREATE/UPDATE - This is the part I had to figure out, and why I asked this question. The Flex first this time:
// FLEX (AS3)
var person:Object = {
PersonID: -1, // -1 for CREATE, actual ID for UPDATE
FirstName: "John",
LastName: "Smith",
BirthDate: "07/19/1983",
CreationDate: "1997-07-16T19:20+01:00" // need W3C DTF for Date WITH Time
};
_pplWebSvcInstance.SavePerson(person); // do the web method call
(For handling those W3C datetimes, see How to parse an ISO formatted date in Flex (AS3)?)
On the .NET web service side then, the trick was figuring out the correct Type on the web method's parameter. If you go with just Object, then step into a call with a debugger, you'll see .NET figures it is a XmlNode[]. Here is what I figured out to do:
[WebMethod]
public int SavePerson(PeopleDataSet p) {
// Now 'p' will be a PeopleDataSet with a Table called 'p' that has our data
// row(s) (just row, in this case) as string columns in random order.
// It WILL NOT WORK to use PeopleDataSet.PersonDataTable as the type for the
// parameter, that will always result in an empty table. That is why the
// LoadFlexDataTable utility method below is necessary.
var adapter = new PersonTableAdapter();
var tbl = new PeopleDataSet.PersonDataTable();
tbl.LoadFlexDataTable(p.Tables[0]); // see below
// the rest of this might be familiar territory for working with DataSets
PeopleDataSet.PersonRow row = tbl.FirstOrDefault();
if (row != null) {
if (row.PersonID > 0) { // doing UPDATE
row.AcceptChanges();
row.SetModified();
}
else { // doing CREATE
row.CreationDate = DateTime.UtcNow; // set defaults here
row.IsDeleted = false;
}
adapter.Update(row); // database call
return row.PersonID;
}
return -1;
}
Now, the kluge utility method you saw called above. I did it as extension method, that is optional:
// for getting the Un-Typed datatable Flex gives us into our Typed DataTable
public static void LoadFlexDataTable(this DataTable tbl, DataTable flexDataTable)
{
tbl.BeginLoadData();
tbl.Load(flexDataTable.CreateDataReader(), LoadOption.OverwriteChanges);
tbl.EndLoadData();
// Probably a bug, but all of the ampersand (&) in string columns will be
// unecessarily escaped (&) - kluge to fix it.
foreach (DataRow row in tbl.Rows)
{
row.SetAdded(); // default to "Added" state for each row
foreach (DataColumn col in tbl.Columns) // fix & to & on string columns
{
if (col.DataType == typeof(String) && !row.IsNull(col))
row[col] = (row[col] as string).Replace("&", "&");
}
}
}