I need to write a query to get values from the table for pagination so I'm using WHERE and LIMIT, OFFSET condition but I get an error or empty set
SELECT * FROM v WHERE v._ts BETWEEN {} AND {}".format(value, value1)
and
SELECT * FROM v ORDER BY v._ts ASC LIMIT {} OFFSET{}".format(value, value1)
I need records from value to value1. example I need no.of rows from a table with some limits
Pagination in CosmosDb does not work like in a SQL/relational databases.
Generally, for the first request, you need run your query with filters and with a prespecified maxItemsCount (e.g. 10). The result is a collection of your entities and a continuationToken. For any consecutive requests, only need to send the continuationToken and the next 10 entites would be returned.
Here is a sample implemented in C#: https://github.com/pdhimate/WebApiBoilerPlate/blob/635d587759107dd795d1cc4221a31ca1ab960efa/Api.Data/CosmosDb/Repositories/CosmosRepoBase.cs#L76
Related
I am trying to select data based on a status which is a string. What I want is that status 'draft' comes first, so I tried this:
SELECT *
FROM c
ORDER BY c.status = "draft" ? 0:1
I get an error:
Unsupported ORDER BY clause. ORDER BY item expression could not be mapped to a document path
I checked Microsoft site and I see this:
The ORDER BY clause requires that the indexing policy include an index for the fields being sorted. The Azure Cosmos DB query runtime supports sorting against a property name and not against computed properties.
Which I guess makes what I want to do impossible with queries... How could I achieve this? Using a stored procedure?
Edit:
About stored procedure: actually, I am just thinking about this, that would mean, I need to retrieve all data before ordering, that would be bad as I take max 100 value from my database... IS there any way I can do it so I don t have to retrieve all data first? Thanks
Thanks!
ORDER BY item expression could not be mapped to a document path.
Basically, we are told we can only sort with properties of document, not derived values. c.status = "draft" ? 0:1 is derived value.
My idea:
Two parts of query sql: The first one select c.* from c where c.status ='draft',second one select c.* from c where c.status <> 'draft' order by c.status. Finally, combine them.
Or you could try to use stored procedure you mentioned in your question to process the data from the result of select * from c order by c.status. Put draft data in front of others by if-else condition.
In what order are documents returned for a Select query without an Order by clause?
Example query - SELECT * FROM c WHERE c.type=someType
Is it based on id of the documents or last modified timestamp(_ts) or the timestamp of creation or some random order?
If this info helps - This query is performed in a collection with only one partition whose partitionKey is null, and there are atmost 3 documents for a 'type'
Is it based on id of the documents or last modified timestamp(_ts) or
the timestamp of creation or some random order?
Based on my test, if you do not set any sort rule, it will be sorted as default based on the time created in the database,whatever it is partitioned or not.
In above sample documents, the sort will not be changed if I change the id,partition key(that's name) or ts.
I have a table Customer which has the following columns,
user_name,current_id,id,params,display,store.
I am writing a query like this,
SELECT * FROM Customer WHERE user_name='Mike' AND current_id='9845' AND id='Get_Owner' AND params='owner=1' order by(display) limit 6 offset 0
Now there are times when I want to fetch a particular value which is not there in the first six and I want to fetch that particular value and rest 5 values in the same way like above how can I do that?
For example I want something like this
SELECT * FROM Customer WHERE user_name='Mike' AND current_id='9845' and id='Get_Owner' AND params='owner=1' AND stored='Shelly.Am'
I want Shelly.Am and other 5 value like my first query
You can combine two queries by using a compound query.
The ORDER BY/LIMIT clauses would apply to the entire compound query, so the second query must be moved into a subquery:
SELECT *
FROM Customer
WHERE user_name='Mike'
AND current_id='9845'
AND id='Get_Owner'
AND params='owner=1'
AND stored='Shelly.Am'
UNION ALL
SELECT *
FROM (SELECT *
FROM Customer
WHERE user_name='Mike'
AND current_id='9845'
AND id='Get_Owner'
AND params='owner=1'
AND stored!='Shelly.Am'
ORDER BY display
LIMIT 5);
using sqlite, I'm trying to run a query with a limit clause, but instead of specifying a literal I am trying to use a column. Sadly I am getting an 'no such column' error. Is there a way of achieving what I mean without writing an external program?
Example
select * from ep where code=2 limit code
You have to use a subquery:
SELECT * FROM ep WHERE code = 2 LIMIT (SELECT code FROM ep WHERE ...)
Please note that the subquery must return a single value (if it returns multiple records, only the first one is used).
Following is the query that I use for getting a fixed number of records from a database with millions of records:-
select * from myTable LIMIT 100 OFFSET 0
What I observed is, if the offset is very high like say 90000, then it takes more time for the query to execute. Following is the time difference between 2 queries with different offsets:
select * from myTable LIMIT 100 OFFSET 0 //Execution Time is less than 1sec
select * from myTable LIMIT 100 OFFSET 95000 //Execution Time is almost 15secs
Can anyone suggest me how to optimize this query? I mean, the Query Execution Time should be same and fast for any number of records I wish to retrieve from any OFFSET.
Newly Added:-
The actual scenario is that I have got a database having > than 1 million records. But since it's an embedded device, I just can't do "select * from myTable" and then fetch all the records from the query. My device crashes. Instead what I do is I keep fetching records batch by batch (batch size = 100 or 1000 records) as per the query mentioned above. But as i mentioned, it becomes slow as the offset increases. So, my ultimate aim is that I want to read all the records from the database. But since I can't fetch all the records in a single execution, I need some other efficient way to achieve this.
As JvdBerg said, indexes are not used in LIMIT/OFFSET.
Simply adding 'ORDER BY indexed_field' will not help too.
To speed up pagination you should avoid LIMIT/OFFSET and use WHERE clause instead. For example, if your primary key field is named 'id' and has no gaps, than your code above can be rewritten like this:
SELECT * FROM myTable WHERE id>=0 AND id<100 //very fast!
SELECT * FROM myTable WHERE id>=95000 AND id<95100 //as fast as previous line!
By doing a query with a offset of 95000, all previous 95000 records are processed. You should make some index on the table, and use that for selecting records.
As #user318750 said, if you know you have a contiguous index, you can simply use
select * from Table where index >= %start and index < %(start+size)
However, those cases are rare. If you don't want to rely on that assumption, use a sub-query, for example using rowid, which is always indexed,
select * from Table where rowid in (
select rowid from Table limit %size offset %start)
This speeds things up especially if you have "fat" rows (e.g. that contain blobs).
If maintaining the record order is important (it usually isn't), you need to order the indices first:
select * from Table where rowid in (
select rowid from Table order by rowid limit %size offset %start)
select * from data where rowid = (select rowid from data limit 1 offset 999999);
With SQLite, you don't need to get all rows returned at once in a big fat array, you can get called back for every row. This way, you can process the results as they come in, which should address both your crashing and performance issues.
I guess you're not using C as you would already be using a callback, but this technique should be available in any other language.
Javascript example (from : https://www.npmjs.com/package/sqlite3 )
db.each("SELECT rowid AS id, info FROM lorem", function(err, row) {
console.log(row.id + ": " + row.info);
});