We have a GraphQL dotnet implementation sitting on top of our existing app database.
<PackageReference Include="GraphQL" Version="3.3.2" />
<PackageReference Include="GraphQL.SystemTextJson" Version="3.3.2" />
<PackageReference Include="GraphQL.Server.Transports.AspNetCore" Version="4.4.1" />
<PackageReference Include="GraphQL.Server.Transports.WebSockets" Version="4.4.1" />
<PackageReference Include="GraphQL.Server.Transports.AspNetCore.SystemTextJson" Version="4.4.1" />
<PackageReference Include="GraphQL.Server.Ui.Playground" Version="4.4.1" />
<PackageReference Include="GraphQL.Server.Authorization.AspNetCore" Version="4.4.1" />
We have a fairly intertwined data structure, so when querying from some of our top level fields, and including some of their sub fields, we could be joining on a large number of tables. However, not every query would need all those joins.
I was hoping to be able to manually parse the context.Arguments in our Query ObjectGraphType when I am resolving it to eliminate the joins if a particular sub-field is not queried.
A very simple high level version of this would be as follows:
Field<ListGraphType<OrganisationType>>(
"organisations",
resolve: context =>
{
var retVal = database.Organisations();
//Are we joining on employers?
if(context.SubFields.ContainsKey("employers"))
{
retVal.Include(x => x.Employers.Where(x => x.Deleted != true))
.ThenInclude(x => x.Departments.Where(x => x.Deleted != true));
}
return retVal;
}
Where we only join on employers, if the user has queried that. However, the issue is that employers can have departments, employees, managers etc etc... and those themselves can have a large number of sub-properties.
Currently our query joins on pretty much every permutation of the query, producing a really hefty SQL query. Which is a lot of heavy lifting if the user only wants, the organisation name, and each employers name.
Filtering on the very top level is easy enough (as shown above) but I can't figure out how to query from that point onwards, I just seem to end up in an infinite loop of Children... for example:
var employerSubField = context.SubFields["employers"];
var otherJoins = employerSubField.SelectionSet.Children.Where(x => x.Children)
I can't seem to get a where name == "Employees" or anything like that. Do I need to cast the GraphQL.Language.AST.INode to something at some point? When I inspect the values at debug time it looks like I should be able to see the values I am after. But this won't compile.
var deptsField = employerSubField.SelectionSet.Selections.Where(x => x.Name == "departments");
I have found the following way to get the basic information out that I can then easily parse myself and add my own custom logic.
var query = context.Document.OriginalQuery.Substring(context.Document.OriginalQuery.IndexOf("{")).RemoveWhitespace();
returns:
{organisations{id,name,employers{id,name,clientNotes,departments{id,name}}}}
Related
The bounty expires in 1 hour. Answers to this question are eligible for a +100 reputation bounty.
eia92 wants to draw more attention to this question.
I recently upgraded my project to .NET Core 6 and now removing records from my look up tables is not working. I have a Risk object that has a collection of Users. Removing users from the risk object no longer works. Any ideas what I'm doing wrong?
My lookup table is called RiskItemUser, and it has two columns, RiskItemId and UserId.
Code:
var postSavedRisk = _riskService.Queryable().Include(c => c.AssignedTo).Where(w => w.Id == riskitem.Id).FirstOrDefault();
List<User> usersToRemove = postSavedRisk.AssignedTo.Where(c => userNamesToRemove.Contains(c.UserName)).ToList();
using (var db = new ApplicationDbContext())
{
var postSavedAssginedTo = db.RiskItemUser
.Where(w => w.RiskItemId == riskitem.Id)
.ToList();
foreach (var userToRemove in usersToRemove)
{
foreach (var riskAssignedTo in postSavedAssginedTo)
{
if(userToRemove.Id == riskAssignedTo.UserId)
db.RiskItemUser.Remove(riskAssignedTo);
await db.SaveChangesAsync().ConfigureAwait(false);
}
}
}
The code, as you show it, looks like it should work, although some parts are hidden. Therefore, it's hard to tell how to make it work. But there's room for simplification, which should result in working code.
You want to remove users whose names are specified by userNamesToRemove from a risk that's specified by riskitem.Id. Assuming that there's a navigation property RiskItemUser.User, removing these data could be done by essentially one line of code:
db.RiskItemUser.RemoveRange(
db.RiskItemUser.Where(ru => ru.RiskItemId == riskitem.Id
&& userNamesToRemove.Contains(ru.User.Name)));
await db.SaveChangesAsync().ConfigureAwait(false);
You tagged EFC 6, but as of EFC 7.0, there's support for bulk delete (and update) functions, allowing for single-statement deletion of multiple database records:
db.RiskItemUser
.Where(db.RiskItemUser.Where(ru => ru.RiskItemId == riskitem.Id
&& userNamesToRemove.Contains(ru.User.Name)))
.ExecuteDelete();
This will execute one delete statement, whereas the previous method will execute one statement per row.
Note that this bulk method is like executing raw SQL. There's no communication with EF's change tracker and EF can't coordinate the correct order of statements. I think the general advice should be to not mix these bulk methods with regular SaveChanges calls.
I am trying to get Departments with last DepartmentLog, but ef core executes n+1 query. And works so slow. I have 10 000 rows on Department table and 12 000 rows on DepartmentLogs. I am not sure it must work 1.5s - 2s for executing like this query.
var query2 = _unitOfWork.Departments.GetDbSet()
.Include(p => p.CreatedUser)
.ThenInclude(p => p.Employee)
.OrderByDescending(p => p.CreatedAt)
.Select(p => new
{
Department = p,
DepartmentLog = _unitOfWork.DepartmentLogs.GetDbSet()
.Include(m => m.CreatedUser)
.ThenInclude(m => m.Employee)
.OrderByDescending(m => m.CreatedAt)
.FirstOrDefault(m => m.DepartmentId == p.Id)
})
.Take(10).ToList();
The lambda you're passing to Select is executed for each item in the result set. Since you're making a query in that lambda, you're going to have N queries issued, one for each item in the result set.
The reason it only generates one extra query when you remove FirstOrDefault, is that you're returning the all the department logs, which then allows EF to add them to the object cache, where it can pull them out from later, instead of issuing a new query. With FirstOrDefault, you're only returning one, so EF caches that, but on the next run of the lambda, you're selecting a different one that it doesn't have cached.
To use FirstOrDefault and still have just one extra query, you should run this query first:
var departmentLogs = await _unitOfWork.DepartmentLogs.GetDbSet()
.Include(m => m.CreatedUser)
.ThenInclude(m => m.Employee)
.OrderByDescending(m => m.CreatedAt)
.ToListAsync();
Then, in your Select:
.Select(p => new
{
Department = p,
DepartmentLog = departmentLogs.FirstOrDefault(m => m.DepartmentId == p.Id)
})
Then, it will run the FirstOrDefault on the result set already in memory. However, this requires returning all the logs, which could be more problematic performance-wise, if there's a ton of them. You'll have to profile each way and see which ends up being better in your particular circumstance.
Alternatively, a probably preferably, you can simply create this query as a stored procedure, and then just call the stored procedure to get your results, instead of using LINQ to build a query. There's only so much EF can do to optimize the query, and this is a highly inefficient query, no matter what you do.
I'm currently building a ASP.NET web app search result page. I've implemented different types of sorting for users but some of them are giving unexpected results.
These two line of code are executed regardless of the sort type or any other factors:
var resources = ctx.Resource.Where(p => p.CityAlias == city.ToLower() && p.Company.Hidden != true && p.State == 1);
FilterResources(ref resources, resourceTypeId, serviceId);
FilterResources:
private void FilterResources(ref IQueryable<Resource> allRes, int resourceType, int selectedService)
{
allRes = allRes.Where(p => p.ResourceType_Id == resourceType && p.ResourceService.Any(x => x.Service_Id == selectedService));
}
Resource.ResourceService is of type ICollection that is mapped to a database table of the same name with a foreign key pointing to Resource.Id
So far so good. First a working sort example:
private void SortResourcesByName(ref IQueryable<Resource> resources)
{
resources = resources.OrderBy(p => p.Name);
}
Then we handle paging:
int count = resources.Count();
var resourceList = resources.Skip((page - 1) * 10).Take(10).ToList();
This set from entities seems to be correct and in the right order, but here's when things get weird:
If we use a different sort, some of the pages have duplicate results and in some cases if I go through the pages, I can see duplicate (or more) Resources scattered across the list.
For example:
private void SortResourcesByPrice(ref IQueryable<Resource> resources, int serviceId)
{
resources = resources.OrderByDescending(p => p.ResourceService.FirstOrDefault(s => s.Service_Id == serviceId).Price.Value)
.ThenByDescending(p => p.Company.CompanyService.FirstOrDefault(c => c.ServiceId == serviceId).Price.Value);
}
If I use this sort, the first page of the view is correct, but when I start navigating to other pages, I start to see duplicate results.
The query returns just over 200 results, which are divided among 21 pages. Pages 3 and 4 are exactly the same for some reason, pages 5 and 6 also yield same results. In these cases, ResourceService.Price = null.
Bonus weirdness:
using OrderBy() instead of OrderByDescending does not give duplicate results
private void SortResourcesByPrice(ref IQueryable<Resource> resources, int serviceId)
{
resources = resources.OrderBy(p => p.ResourceService.FirstOrDefault(s => s.Service_Id == serviceId).Price.Value)
.ThenBy(p => p.Company.CompanyService.FirstOrDefault(c => c.ServiceId == serviceId).Price.Value);
}
I'm at a total loss here, so I'd really appreciate help. If you can't make sense of my example code or can't understand the question in some other way, I'd happily try to provide more information.
Edit:
I'd like to add that the amount of results/Resources is always the same, regardless of what sort I'm using (just as it's supposed to).
Edit 2:
Fixed some numbers & typos
You can use https://expressprofiler.codeplex.com/ to view the SQL generate by EF copy and paste the sql query directly in SQL Server Management studio and look where is the duplicate origine, you can change query to remove duplicate entry and after search Linq responsible.
You can put the SQL here too
After hours of trying and searching, I think its time to share my problem with you right now.
Problem Definition :
I have a Dictionary of KeyValuePairs(named filterPool) which includes an integer (PropertyID) and a string(McValue). What I am trying to do is filtering products depending on those KeyValuePairs and return them as a DataTable/List.
You may consider this as building dynamic "Where ... And .." clauses as SQL.
Here is the code that I am using :
foreach (KeyValuePair<int, string> filter in filterPool)
{
products = products.Where(i => i.PROPERTYID == filter.Key && i.MCVALUE.Equals(filter.Value));
}
return products.ToDataTable();
The problem is the foreach loop above seems to work only once, for the latest KeyValuePair available in the Dictionary.
As far as I could find on Stackoverflow, the closest solution to my problem was : this one, also using a Dictionary of values for filtering
There must be a way to achieve the goal of filtering using Dictionary and LINQ; or there's a huge thing that I am missing/ignoring to see somehow.
Hope the problem given is clear enough for all,
Thanks
^^
This is a closure issue. You can solve it by making a temporary:
foreach (KeyValuePair<int, string> filterTmp in filterPool)
{
var filter = filterTmp; // Make a temporary
products = products.Where(i => i.PROPERTYID == filter.Key && i.MCVALUE.Equals(filter.Value));
}
return products.ToDataTable();
For details on what's happening, see Eric Lippert's post Closing over the loop variable considered harmful.
Also note that this behavior has changed for C# 5. In C# 5/VS2012, this code would work as expected.
You're overwriting your products collection on every iteration of your foreach. I'm not sure what the data type on your collection is, but you'll want to do something like this in your foreach instead:
products.AddRange(products.Where(i => i.PROPERTYID == filter.Key && i.MCVALUE.Equals(filter.Value)));
I'm not sure if that makes sense, but it seems like you're trying to create a collection full of products that match your filterPool.
I think that it's better solved with aggregate:
return filter
.Aggregate(products, (acc, filter) => acc.Where(i => i.PROPERTYID == filter.Key && i.MCVALUE.Equals(filter.Value)));
.ToDataTable();
I have a request to create an auto complete that will search an data table. Is this achieveable quickly and simply or is it a case of writing a reasonable amount of code?
Originally, I have been using a webservice and linq to point at a single column's worth of data (IDDesc) and pull back the list of products:
Product.FinalProductsDataContext dbac = new Product.FinalProductsDataContext();
return dbac.tblProduct
.Where(r => r.Account== HttpContext.Current.Session["AccountKey"].ToString() && r.IDDesc.Contains(prefixText))
.Distinct()
.OrderBy(r => r.IDDesc)
.Select(r => r.IDDesc)
.Take(count)
.ToArray();
However, if I wish the autocomplete to look at all the columns, is it a case of repeating similar LINQ statements for each of the columns contained within the datatable or is there a 'quick fix'?
I personally don't think this is an ideal scenario but it is a request I must work towards.
Any help or advice, greatly appreciated.
Rather than trying to solve this entirely with LINQ (and repeating all those statements for each column in the table, as well as hitting the database repeatedly), I think I'd look to put something in the database to do the heavy lifting here.
You could create a view that takes the fields from the table and amalgamates them into one column e.g.
CREATE VIEW [dbo].[ProductView]
AS
SELECT CAST(ProductName AS NVARCHAR(50)) AS 'ProductColumn'
FROM dbo.Products
UNION
SELECT CAST(SupplierName AS NVARCHAR(50))
FROM dbo.Products
UNION
...
which if you added the view to your context would then allow you to modify your existing LINQ query and point it at that view e.g.:
Product.FinalProductsDataContext dbac = new Product.FinalProductsDataContext();
return dbac.ProductView
.Where(r => r.Account== HttpContext.Current.Session["AccountKey"].ToString() && r.ProductColumn.Contains(prefixText))
.Distinct()
.OrderBy(r => r.ProductColumn)
.Select(r => r.ProductColumn)
.Take(count)
.ToArray();