I was having an issue with a LINQ query today and after some debugging I was able to resolve the issue but it seems like a bug to me in the way LINQ works. Here is what I had:
var myitem = context
.items
.OrderByDescending(x => x.DateEdited)
.FirstOrDefault(x => x.fkId == myFkId && x.DateEdited < someDate);
In my database I have a table with some records and I want to retrieve the most recent record that is older than "someDate" and who has a particular foreign key in a column. The above query did not work however. It was returning the oldest record with the matching foreign key. I ended up having to rewrite my query like this to get it to work:
var myitem = context
.items
.Where(x => x.fkId == myFkId && x.DateEdited < someDate)
.OrderByDescending(x => x.DateEdited)
.FirstOrDefault();
In my debugging I found out that the "x.DateEdited < someDate" was re-ordering the IEnumerable so I ended up having to put my OrderByDescending clause after the date check.
Has anybody else run into this issue? Is it a bug or expected functionality?
Even though .OrderByDescending() returns an IOrderedEnumerable, the .FirstOrDefault() is a shortcut to .Where() which only returns an IEnumerable which does not guarantee order.
Basically, adding a filter does not guarantee the order of the data. If you look at the generated SQL from the first, you will get a nested subresult from the orderby that is then filtered again.
Generally, if an operation does not explicitly define the output order, you can't depend on the result being in any particular order until you specify/apply it yourself.
Unless you know that ordering an intermediate result will yield performance improvements in the next step of an algorithm, there's no reason to so. Just apply the ordering as a final processing step.
Related
I"m trying to understand why the Microsoft.Azure.Documents.Client makes multiple calls when running a query.
var option = new FeedOptions { EnableCrossPartitionQuery = true, MaxItemCount = 100};
var myobj = cosmosClient.CreateDocumentQuery<myCosmosObj>(documentUri, option)
.Where(x => x.ID == request.Id);
while (myobj.AsDocumentQuery().HasMoreResults)
{
var results = await myobj.AsDocumentQuery().ExecuteNextAsync<myCosmosObj>();
resultList.AddRange(results);
}
A Fiddler trace shows 5 calls to the cosmos collection dbs/mycollectionname/colls/docs (the while loop above runs 5 times)
My question is 1 network hop would improve performance, so I would like to understand why it is making 5 network calls, and If there is something I need to do with the configuration to adjust this. I have already tried adjusting the ResultSize. This is roughly a 3GB collection.
David's answer is theoretically correct however it is missing a crucial point.
Your code is wrong. The way your create the document query inside the loop means that you will always query the result of the first execution 5 times.
The code should actually be like this:
var query = cosmosClient.CreateDocumentQuery<myCosmosObj>(documentUri, option)
.Where(x => x.ID == request.Id).AsDocumentQuery();
while (query.HasMoreResults)
{
var results = await query.ExecuteNextAsync<myCosmosObj>();
resultList.AddRange(results);
}
This will now properly run your query and it will use the continuation properties of the query object in order to read the next page in ExecuteNextAsync.
With a partitioned collection, the most efficient way to find a document by id is by also specifying the partition key (which then directs your query to a single partition). Without PK, there's really no way to know, up front, which partition your documents will reside in. And that's likely why you're seeing 5 calls (you likely have 5 partitions).
The alternative, which your code shows, is to do a cross-partition query, which has to do one query per partition, to seek the document you're looking for.
One more thing to note: A query will have higher RU cost than a Read. And if you already know the partition key and id, there's no need to invoke the query engine (as you can only retrieve a single document anyway, for a given partition key + row key combination).
I'm querying firestore with:
this.todosCollection = this.afs.collection('todos', ref => ref.where('owner', '==', this.userId). orderBy('due', 'asc'));
The todo items is ordered in ascending order as I want, the problem is that todos without a due date (null) comes first. I want them to come last. Is this possible within the orderBy or other technique?
That's the way the ordering of nulls is intended to work. The documentation makes this clear, and it can't be changed in the query itself.
If you want to change the ordering, then you will have to manually re-sort the documents on the client to support your preferred ordering. Or, you will have to make two queries, one that includes null and another that does not, then merge the results yourself.
I am using SQLite and NHibernate and Im storing my date as Strings on SQLite since I cant store as Date.
Everything was just fine until a need to compare dates.. I tried the following codes:
var initialDate = DateTime.Parse(_InitialDate);
var finalDate = DateTime.Parse(_FinalDate);
return session.QueryOver<Locacoes>()
.Where(c => DateTime.Parse(c.InitialDate) >= initialDate )
.Where(c => DateTime.Parse(c.FinalDate) <= finalDate).List();
but I got an exception on the first "Where": "a variable 'c' of type 'Locacoes' is referenced on scope '', but it is not defined".
How can I compare date on SQLite using NHibernate?
I've tried many things with above code, but didnt work.
Edit: Its not duplicate, the error can be the same, but the result is different
First, while Sqlite likes to pretend everything is a string, that should be considered a storage format and NO REASON why your object model should use strings to hold date values. Your object model should of course type the properties as DateTime or DateTimeOffset. Then configure your NHibernate mappings properly to map the values to what SQLite can handle (actually I think NHibernate would handle that automatically if you just tell it to use the SQLiteDialect).
Second, I don't think QueryOver() can handle things like DateTime.Parse(). Don't confuse QueryOver() with Linq2NHibernate (the Query() method), which have more advanced expression interpretation abilities. On the other hand, you no longer need to use Parse() when you begin to use correct types in your object model.
At least if you use LINQ, it should be able to handle DateTime.Date for SQLite, if you need it:
session.Query<Locacoes>()
.Where(l => c.InitialDate.Date >= initialDate)
Of course, you would only need to put Date in there if there is a non-zero time-of-day component that you need to ignore.
If I have a statement like this:
return _sqlconnection.Table<Student>().FirstOrDefault(t => t.Id == id);
Will the first retrieve all the Students and then select through then to find where the Id matches or will it know to send a SQL statement to the database with a WHERE clause?
In SQLite.Net PCL it looks like FirstOrDefault() gets converted to a limit 1 query. However, in your case you are applying a predicate to the FirstOrDefault. That would cause the entire table to get loaded since SQLite.Net doesn't handle complex Linq very gracefully.
If you want to make sure the whole table is not loaded, go for something like this:
return _sqlconnection.Table<Student>().Where(t => t.Id == id).FirstOrDefault();
There's a good thread in the Xamarin forums about this topic.
I am working on a report, and for my data set i need to use an or statement. in SQL my query would look like:
SELECT recId, etc... FROM CustTrans
WHERE (CustTrans.Closed IS NULL AND CustTrans.Invoice IS NULL)
OR (CustTrans.Invoice IS NOT NULL)
I would translate this then into range like the following (stuck on the RecId field)
Value:
((CustTrans.Invoice == ‘’) && (CustTrans.Closed == ‘’) || (CustTrans.Invoice != ‘’))
I have found numerous places explaining that this is the proper syntax, although all of them are using the programmatic method of creating a query. I need to keep mine in the AOT so the customer can modify it later on if they want, and every combination of quotes, parens, and order still does not work. Sometimes AX will accept my range value string, but it doesn't do anything, and sometimes it will give me a random syntax error.
Is there a way to accomplish what I'm looking to do? have I missed some syntax, or does AX really not let you use OR on a query without a union and another view?
Thanks!
You can create a Query in the AOT and then over-ride the init method for the Query. Where you can put these complex SQL statements and get your work done.