I'm making a web service with OData + Entity Framework for my application.
After projecting members from Database Entities to Application Model (using AutoMapper ProjectTo), Entity Framework print thousands of rows instead of limiting the rows with $top attribute in OData URL.
[HttpGet]
[EnableQuery(MaxExpansionDepth = 7, EnsureStableOrdering = false)]
public virtual SingleResult<TModel> Get([FromODataUri] TKeyType key, ODataQueryOptions<TModel> opts)
{
IQueryable<TEntity> data = _db.Set<TEntity>()
.Where(_db.BuildKeyPredicate<TEntity>(key));
return SingleResult.Create(GetProjected(opts, data));
}
private IQueryable<TModel> GetProjected(ODataQueryOptions<TModel> opts, IQueryable data)
{
string[] expandedProperties = Helpers.Expand.GetMembersToExpandNames(opts);
IQueryable<TModel> projected;
if (expandedProperties.Count() > 0)
{
projected = data.ProjectTo<TModel>(null, expandedProperties);
// expandedProperties is an array of string with the names of properties to expand specified in $expand
}
else
{
projected = data.ProjectTo<TModel>();
}
return projected;
}
example URL: http://WEBSERVER/odata/Entity(416)?$expand=Child($top=100)
in the debugger, Child lenght is more than 100... in my case 57k. This is why I use $top :)
I was thinking that Entity Framework automatically translates $top parameter in OData with TOP keyword in query, but this is not happening.
If you want more code/ more documentation please tell me!
To solve this instead of using $expand you can use a Related Entity Route like this:
http://WEBSERVER/odata/Entity(416)/Child?$top=100
In order to make route works you need to implement inside the EntityController a GetChild action like this:
[EnableQuery]
[ODataRoute("({key})/Child")]
public IQueryable<Child> GetChild([FromODataUri] int key)
{
return db.Childs.Where(m => m.EntityId == key);
}
For more information about OData v4 Related Entities visit this Entity Relations in OData v4
Related
I used .NET 3.1 until I decided to upgrade my app to .NET 6. I did it successfully but some of my modules broke, one of them is OData. I used OData and I got #odata.context, #odata.count and value returned back in 3.1. Now in .NET 6, I get only the value which means that #odata.context and #odata.count aren't present in the return object. In order to make it run, I added this line in my code
services.AddControllers().AddOData(options => options.AddRouteComponents("v1", GetEdmModel()).Select().Filter().OrderBy()); in my Startup.cs where
private static IEdmModel GetEdmModel()
{
ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
builder.EntitySet<Project>("Project");
return builder.GetEdmModel();
}
My endpoint is as
[HttpGet]
[EnableQuery(PageSize = 20)]
public async Task<ActionResult> GetAsync()
{
var projects = await _projectRepository.GetAllAsync();
return Ok(projects);
}
Do you know what can I change and how should I change it in order to get the context and count from OData in .NET 6? I use the usual OData library with 8.0.0 version
EDIT to add more info about #Tiny Wang's answer:
This seems to be working and thank you very much for it! However, I stumbled upon another problem. I tried your example and the working version was on https://localhost:44327/odata/project$count=true&$skip=0&$orderby=CreateDate%20desc. My api prefix is api/[ControllerName] and I changed
options => options.EnableQueryFeatures().AddRouteComponents("odata", GetEdmModel()).Select().Filter().OrderBy()
to
options => options.EnableQueryFeatures().AddRouteComponents("api", GetEdmModel()).Select().Filter().OrderBy()
but when I access the endpoint, I get the following error:
"The request matched multiple endpoints. Matches: MyApp.Api.Controllers.ProjectController.GetAsync (MyApp.Api) MyApp.Api.Controllers.ProjectController.GetAsync (MyApp.Api)"
even tho GetAsync() is defined only once. Do you know how can I fix this and what causes it?
The change was in the Microsoft.AspNetCore.OData package, not .NET 6. You can use that package in .NET 5 if you want.
First of all, you always had to specify $count=true. That's an expensive operation. In a paging query it meant you had to execute two queries, one to receive a single page of data and another to count all possible results.
To do that you have to enable counting, just like any other OData operation, in the AddOData() call that adds the OData middleware.
Finally, for this to have any real effect, the controller action should return an IQueryable that will be used to construct the final LINQ and by extension SQL query. Otherwise your code will load everything in memory. You'd load a 100K row table in memory only to return 10 rows
OData failed to gain traction because it allowed clients to execute inefficient and unoptimized queries. Service developers had no idea what to optimize because client developers were free to execute anything. In later versions of the protocol, all query capabilites are off by default and have to be explicitly enabled. Server developers now can restrict the maximum size, whether expensive sort operations are allowed etc. They can prevent client code from filtering or sorting by unindexed fields for example.
In my own application I add the OData middleware with :
var edmModel=ODataModels.GetEdmModel();
services.AddControllersWithViews()
.AddOData(opt => opt.Select()
.OrderBy()
.Filter()
.Count()
.Expand()
.SetMaxTop(250)
.AddRouteComponents("odata", edmModel)
);
This enables Count and sets the maximum result size to a fairly large 250 - I have some grids users tend to scroll through.
To use the OData endpoints, a Controller that inherits from ODataController is needed. Query methods should return an IQueryable. If that IQueryable comes from a DbContet, the OData query will be used to construct a LINQ query and by extension the final SQL query. This will ensure that only the necessary data will be loaded.
[EnableQuery]
public IQueryable<Customers> Get()
{
return _db.Customers.AsNoTracking();
}
An OData controller that works on top of EF Core could look like this :
public class CustomersController:ODataController
{
private readonly ILogger<CustomersController> _logger;
private readonly SalesContext _db;
public CustomersController(SalesContext db, ILogger<CustomersController> logger)
{
_logger = logger;
_db = db;
}
[EnableQuery]
public IQueryable<Customers> Get()
{
return _db.Customers.AsNoTracking();
}
[EnableQuery]
[HttpGet]
public IActionResult Get(long key)
{
var cust = _db.Customers
.AsNoTracking()
.FirstOrDefault(t => t.ID == key);
if (cust == null)
{
return NotFound($"Not found: Customer ID = {key}");
}
return Ok(cust);
}
...
The Get(key) action is necessary to allow retrieving items by ID in OData eg using customers(123).
==================================
Per my test(created a .net 6 web api project and install Microsoft.AspNetCore.OData 8.0.10), I need to add ?$count=true behind my url then it can appear in my response.
My program.cs
builder.Services.AddControllers().AddOData(options => options.EnableQueryFeatures().AddRouteComponents("odata", GetEdmModel()).Select().Filter().OrderBy());
IEdmModel GetEdmModel()
{
ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
builder.EntitySet<WeatherForecast>("Hello");
return builder.GetEdmModel();
}
My test controller:
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.OData.Query;
using Microsoft.AspNetCore.OData.Routing.Controllers;
namespace WebApi.Controllers
{
[Route("odata/[Controller]")]
public class HelloController : ODataController
{
private static readonly string[] Summaries = new[]
{
"Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
};
[EnableQuery]
public IEnumerable<WeatherForecast> Get()
{
return Enumerable.Range(1, 5).Select(index => new WeatherForecast
{
Id = index,
Date = DateTime.Now.AddDays(index),
TemperatureC = Random.Shared.Next(-20, 55),
Summary = Summaries[Random.Shared.Next(Summaries.Length)]
})
.ToArray();
}
}
}
I'm trying to update an entity using Entity Framework version 6.
I'm selecting the entity from the database like so...
public T Find<T>(object id) where T : class
{
return this._dbContext.Set<T>().Find(id);
}
And updating the entity like so..
public T Update<T>(T entity) where T : class
{
// get the primary key of the entity
object id = this.GetPrimaryKeyValue(entity);
// get the original entry
T original = this._dbContext.Set<T>().Find(id);
if (original != null)
{
// do some automatic stuff here (taken out for example)
// overwrite original property values with new values
this._dbContext.Entry(original).CurrentValues.SetValues(entity);
this._dbContext.Entry(original).State = EntityState.Modified;
// commit changes to database
this.Save();
// return entity with new property values
return entity;
}
return default(T);
}
The GetPrimaryKeyValue function is as so...
private object GetPrimaryKeyValue<T>(T entity) where T : class
{
var objectStateEntry = ((IObjectContextAdapter)this._dbContext).ObjectContext
.ObjectStateManager
.GetObjectStateEntry(entity);
return objectStateEntry.EntityKey.EntityKeyValues[0].Value;
}
Just for clarity. I'm selecting the original entry out as I need to perform some concurrency logic (that Ive taken out). I'm not posting that data with the entity and need to select it manually out of the DB again to perform the checks.
I know the GetPrimaryKeyValue function is not ideal if there's more than one primary key on the entity. I just want it to work for now.
When updating, entity framework coughs up the error below when trying to execute the GetPrimaryKeyValue function.
The ObjectStateManager does not contain an ObjectStateEntry with a reference to an object of type 'NAME_OF_ENTITY_IT_CANNOT_FIND'
I've written many repositories before and I've never had this issue, I cannot seem to find why its not working (hence the post).
Any help would be much appreciated.
Thanks guys!
Steve
It seems like you are having issues getting the PK from the entity being passed in. Instead of trying to go through EF to get this data you could either use their Key attribute or create your own and just use reflection to collect what the key names are. This will also allow you to retrieve multiple keys if it is needed. Below is an example I created inside of LinqPad, you should be able to set it to "Program" mode and paste this in and see it work. Hack the code up and use what you may. I implemented an IEntity but it is not required, and you can change the attribute to anything really.
Here are the results:
Keys found:
CustomIdentifier
LookASecondKey
Here is the code:
// this is just a usage demo
void Main()
{
// create your object from wherever
var car = new Car(){ CustomIdentifier= 1, LookASecondKey="SecretKey", Doors=4, Make="Nissan", Model="Altima" };
// pass the object in
var keys = GetPrimaryKeys<Car>(car);
// you have the list of keys now so work with them however
Console.WriteLine("Keys found: ");
foreach(var k in keys)
Console.WriteLine(k);
}
// you probably want to use this method, add whatever custom logic or checking you want, maybe put
private IEnumerable<string> GetPrimaryKeys<T>(T entity) where T : class, IEntity
{
// place to store keys
var keys = new List<string>();
// loop through each propery on the entity
foreach(var prop in typeof(T).GetProperties())
{
// check for the custom attribute you created, replace "EntityKey" with your own
if(prop.CustomAttributes.Any(p => p.AttributeType.Equals(typeof(EntityKey))))
keys.Add(prop.Name);
}
// check for key and throw if not found (up to you)
if(!keys.Any())
throw new Exception("No EntityKey attribute was found, please make sure the entity includes this attribute on at least on property.");
// return all the keys
return keys;
}
// example of the custom attribute you could use
[AttributeUsage(AttributeTargets.Property)]
public class EntityKey : Attribute
{
}
// this interface is not NEEDED but I like to restrict dal to interface
public interface IEntity { }
// example of your model
public class Car : IEntity
{
[EntityKey] // add the attribure to property
public int CustomIdentifier {get;set;}
[EntityKey] // i am demonstrating multiple keys but you can have just one
public string LookASecondKey {get;set;}
public int Doors {get;set;}
public string Make {get;set;}
public string Model {get;set;}
}
I have a MVC4 WebAPI controller that returns an IQueryable, and therefore I can use $filter and friends in the URL to manipulate the result from the REST endpoint. Here's my controller:
public class EnrollmentController : ApiController
{
[Queryable]
public IQueryable<tblEnrollment> Get()
{
var context = new ProjectEntities();
context.ContextOptions.LazyLoadingEnabled = false;
return context.tblEnrollment.AsQueryable();
}
}
But, like this poster, I'm wanting to make the JSON output format a little different to be friendlier with Ember Data's expected format. So I'd like to return this instead:
return new { enrollment = context.tblEnrollment.AsQueryable() };
However, that breaks OData capability because I'm not returning the IQueryable to the WebAPI layer. So, I'm wondering if there's a way to do something like this:
return new { enrollment = context.tblEnrollment.AsQueryable().ApplyOData() };
Which I'm sure would be way to good to be true...but is there some way to explicitly process the OData parameters against an IQueryable instead of letting the WebAPI layer do it implicitly on the result set returned from a Get method? Or is there another way to accomplish what I want here?
Incidentally, I'm stuck on EF4 for the time being, because I can't upgrade to VS2012 (and hence to .NET4.5 and hence EF5). I could theoretically upgrade to EF 4.3.1, if it would help.
Instead of marking your action as [Queryable], you can add a parameter of type ODataQueryOptions and apply it manually. Here's what it might look like:
public class EnrollmentController : ApiController
{
public object Get(ODataQueryOptions<tblEnrollment> query)
{
var context = new ProjectEntities();
context.ContextOptions.LazyLoadingEnabled = false;
var queryResults = query.ApplyTo(context.tblEnrollment.AsQueryable());
return new { enrollment = queryResults };
}
}
I have an EF object called SportDivision. For simplicity's sake, I won't include every field, just the ones that are relevant:
[Table("SportDivision", Schema = "dbo")]
public class SportDivision: BaseReferenceEntity
{
public int Id { get; set; }
public string Name { get; set; }
public int SportId { get; set; }
[ForeignKey("SportId")]
public virtual Sport Sport { get; set; }
}
So it has a SportId and it's a foreign key that points to the table Sport.
Now, I can't just use an EF object in my views, so I have a model class that's mapped to SportDivision called SportDivisionModel:
public class SportDivisionModel: BaseReferenceModel
{
public int Id { get; set; }
public string Name { get; set; }
public int SportId { get; set; }
//Read only fields
public string Sport { get; set; }
}
I use automapper to transfer data from SportDivision to SportDivisionModel and vice versa. The mapping looks like this:
Mapper.CreateMap<SportDivision, SportDivisionModel>()
.ForMember(x => x.Sport, c => c.MapFrom(e => e.Sport.Name));
Mapper.CreateMap<SportDivisionModel, SportDivision>();
And I have a genericized service that CRUDs and translates data from entity to model or model to entity. Everything works fine except on Create, of which the function is shown below:
public TModel Create<TModel, TEntity>(TModel entry)
where TModel : BaseReferenceModel
where TEntity : BaseReferenceEntity
{
var dm = ServiceLocator.Current.GetInstance<ICrudService<TEntity>>();
var raw = Mapper.Map<TModel, TEntity>(entry);
var created = dm.CreateOrUpdate(raw);
return Mapper.Map<TEntity, TModel>(dm.FindById(created.Id));
}
In the very last line, where you see dm.FindById(created.Id), it returns a SportDivisionModel object with no Sport name. A null reference exception is found in .ForMember(x => x.Sport, c => c.MapFrom(e => e.Sport.Name));. It didn't load Sport after the entry was just created in the database.
I've debugged the code, and I see that the entry with a valid SportId is entered into the SportDivision table of my database, but when I try and bring it over to my MVC application, it doesn't get all the information.
This only is an issue on create. If I simply get data from the database without creating it beforehand, or if I edit the information, then the Sport field in my model object does get populated. I don't know why this is happening, and I can't use the .Include in my generic service call (because not all BaseReferenceEntity classes have a foreign key pointing to Sport).
Please advise. Thanks in advance.
I must play Sherlock Holmes and try to derive what could be the content of CreateOrUpdate and FindById from the indications in your question:
You say that you don't use Include because of the generic service. I assume that you also don't use explicit loading (Load) because you would face the same problem that you cannot really make it generic.
Conclusion: Because the Sport navigation property in the SportDivision gets loaded in certain scenarios (Edit) this can only happen due to lazy loading. The conclusion is backed by the fact that the Sport property is marked as virtual.
Lazy loading relies on proxies. If your SportDivision entity is a proxy then
either loading the Sport entity works
or you get an exception telling you that the context is already disposed (if you have disposed the context)
Number 2 is not the case -> Conclusion: Number 1 must be the case if the pre-condition is fulfilled
But Number 1 also isn't the case (loading Sport does not work)
Conclusion: The pre-condition that your SportDivision entity is a proxy is not true.
So: SportDivision is not a proxy. Could this mean that you have lazy loading in the context disabled? No: Because you are saying that editing works it means that when you load entities from the database they are loaded as proxies and support lazy loading.
Editing works, lazy loading isn't disabled but creating a new entity does not work in the way that the Sport entity is loaded when you proceed to use the newly created entity.
Conclusion: Your newly created entity (returned from CreateOrUpdate) is not a proxy and CreateOrUpdate looks similar to this:
public TEntity CreateOrUpdate(TEntity raw) where TEntity : class
{
if (blabla)
; //update
else
{
context.Set<TEntity>().Add(raw);
context.SaveChanges();
return raw;
}
}
and FindById is just:
public TEntity FindById(int id)
{
return context.Set<TEntity>().Find(id);
}
Since you are passing raw directly into the Add method of the DbSet<T> the question raises where does raw come from and how is it created.
Obviously AutoMapper creates the entity after this line: var raw = Mapper.Map<TModel, TEntity>(entry);
How does Automapper create an entity? Probably by calling new TEntity or by using some reflection code like Activator.CreateInstance or...
It doesn't really matter how, but for sure AutoMapper doesn't instantiate an Entity Framework proxy which had to be created by:
var entity = context.Set<TEntity>().Create();
If all this is true, I feel totally screwed by AutoMapper and generic excesses. If all this wouldn't be generic we could solve the problem by:
context.Set<SportDivision>().Add(raw);
context.SaveChanges();
context.Entry(raw).Reference(r => r.Sport).Load();
Instead we must try some ugly tricks now:
context.Set<TEntity>().Add(raw);
context.SaveChanges();
context.Entry(raw).State = EntityState.Detached;
// We hope that raw is now really out of the context
raw = context.Set<TEntity>().Find(raw.Id);
// raw must be materialized as a new object -> Hurray! We have a proxy!
return raw;
(I'm really not sure if the Detached trick above does work. Aside from that you are forced to reload an entity from the database you just have created and saved which is stupid somehow.)
Potential trick number 2 (without reloading from DB but for the price of being a further step more ugly):
context.Set<TEntity>().Add(raw);
context.SaveChanges();
context.Entry(raw).State = EntityState.Detached;
// We hope that raw is now really out of the context
var anotherRaw = context.Set<TEntity>().Create(); // Proxy!
anotherRaw.Id = raw.Id;
context.Set<TEntity>().Attach(anotherRaw);
context.Entry(anotherRaw).CurrentValues.SetValues(raw);
context.Entry(anotherRaw).State = EntityState.Unchanged;
return anotherRaw; // Proxy! Lazy loading will work!
Does AutoMapper have a feature of a "custom allocator or instantiator" and can custom user data (a context) be supplied? Then there would be a chance to let AutoMapper call context.Set<TEntity>().Create();. Or is it possible to instantiate the object by hand, pass it to AutoMapper and AutoMapper just updates the object's properties?
BTW: The line...
context.Entry(anotherRaw).CurrentValues.SetValues(raw);
...is kind of EF's built-in "AutoMapper". The parameter of SetValues is a general System.Object (could be your ...Model object) and the method maps property values from the supplied object to properties of attached entities by identical property names. Maybe you can leverage this feature somehow instead of using the mapping from model to entity done by AutoMapper.
I'm working to move my simple asp.net website to a three layer architecture. Currently I have Linq queries like the one below in my code-behind files. Basically this code snippet retrieves a collection of customer data from the database and then binds it to a grid control.
I'm wondering if someone can guide me on how to rewrite this in order to move it to my newly-created data access layer. I'm thinking I will turn it into a class (e.g. GetUserBoxesByStatus()) that can be reused throughout the site.
var boxes = from p in sbm.Packages
where p.UserID == CurrentUserId && p.StatusID > 1 && p.StatusID < 3
select new { p.PackageTag, p.PackageName, p.DateReceived, p.DateShipped };
GridView1.DataSource = boxes;
DataBind();
Some of the options that I've investigated but have not had success implementing are the following:
DataTable --- returning a DataTable seems like the best solution but it also appears to require a lot of potentially unecessarry code to define a table (isn't the data source already mapped in my Linq 2 Entities dbml?)
IEneuerable --- I think I could pass an IEnumerable list between the layers but after reading many tutorials about Linq I'm still a little lost
DTO --- Conceptually I think I understand what a DTO is but I am not clear on how to begin implementing this approach
POCO --- Again, the concept seems logical enough but I don't know how to put this into practice
I'm hoping someone here can look at my code example and propose how they would tackle this using one of the above or some other solution.
Create a class with the properties you need. Select into that class. Return a strongly-typed List (so that the query is actually performed in the DAL, not in your view). Bind your data source to the list.
public class PackageViewModel
{
public string Tag { get; set; }
public string Name { get; set; }
public DateTime Received { get; set; }
public DateTime Shipped { get; set; }
}
DAL
public List<PackageViewModel> GetUserBoxesByStatus( int userID, int minStatus, int maxStatus )
{
return sbm.Packages
.Where( p => p.UserID == userID
&& p.StatusID > minStatus
&& p.StatusID < maxStatus )
.Select( p => new PackageViewModel
{
Tag = p.PackageTag,
Name = p.PackageName,
Received = p.DateReceived,
Shipped = p.DateShipped
})
.ToList();
}