I am fetching data from MS app Insights . now I want to know that How paging can be implemented if there is lots of records fetching from query and all are not getting fetched in single query? I want paging in query.
I am using app insights rest api
https://dev.applicationinsights.io/quickstart
There is no direct way to do paging in the query.
Here is a workaround you can take a try:
For example, if you want to fetch 100 datas each time, you can use $top and $skip to mimic the paging function.
In your first query, fetch the first 100 data:
get /v1/apps/{app-id}/events/$all?$top=100
Then in the 2nd query, fetch the 101-200 data:
get /v1/apps/{app-id}/events/$all?$top=100&$skip=100
in the 3rd query, fetch the 201-300 data:
get /v1/apps/{app-id}/events/$all?$top=100&$skip=200
Please also let me know if you have more questions.
Related
At first sight, it's clear what the continuation token does in Cosmos DB: attaching it to the next query gives you the next set of results. But what does "next set of results" mean exactly?
Does it mean:
the next set of results as if the original query had been executed completely without paging at the time of the very first query (skipping the appropriate number of documents)?
the next set of results as if the original query had been executed now (skipping the appropriate number of documents)?
Something completely different?
Answer 1. would seem preferable but unlikely given that the server would need to store unlimited amounts of state. But Answer 2. is also problematic as it may result in inconsistencies, e.g. the same document may be served multiple times across pages, if the underlying data has changed between the page queries.
Cosmos DB query executions are stateless at the server side. The continuation token is used to recreate the state of the index and track progress of the execution.
"Next set of results" means, the query is executed again on from a "bookmark" from the previous execution. This bookmark is provided by the continuation token.
Documents created during continuations
They may or may not be returned depending on the position of insert and query being executed.
Example:
SELECT * FROM c ORDER BY c.someValue ASC
Let us assume the bookmark had someValue = 10, the query engine resumes processing using a continuation token where someValue = 10.
If you were to insert a new document with someValue = 5 in between query executions, it will not show up in the next set of results.
If the new document is inserted in a "page" that is > the bookmark, it will show up in next set of results
Documents updated during continuations
Same logic as above applies to updates as well
(See #4)
Documents deleted during continuations
They will not show up in the next set of results.
Chances of duplicates
In case of the below query,
SELECT * FROM c ORDER BY c.remainingInventory ASC
If the remainingInventory was updated after the first set of results and it now satisfies the ORDER BY criteria for the second page, the document will show up again.
Cosmos DB doesn’t provide snapshot isolation across query pages.
However, as per the product team this is an incredibly uncommon scenario because queries over continuations are very quick and in most cases all query results are returned on the first page.
Based on preliminary experiments, the answer seems to be option #2, or more precisely:
Documents created after serving the first page are observable on subsequent pages
Documents updated after serving the first page are observable on subsequent pages
Documents deleted after serving the first page are omitted on subsequent pages
Documents are never served twice
The first statement above contradicts information from MSFT (cf. Kalyan's answer). It would be great to get a more qualified answer from the Cosmos DB Team specifying precisely the semantics of retrieving pages. This may not be very important for displaying data in the UI, but may be essential for data processing in the backend, given that there doesn't seem to be any way of disabling paging when performing a query (cf. Are transactional queries possible in Cosmos DB?).
Experimental method
I used Sacha Bruttin's Cosmos DB Explorer to query a collection with 5 documents, because this tool allows playing around with the page size and other request options.
The page size was set to 1, and Cross Partition Queries were enabled. Different queries were tried, e.g. SELECT * FROM c or SELECT * FROM c ORDER BY c.name.
After retrieving page 1, new documents were inserted, and some existing documents (including documents that should appear on subsequent pages) were updated and deleted. Then all subsequent pages were retrieved in sequence.
(A quick look at the source code of the tool confirmed that ResponseContinuationTokenLimitInKb is not set.)
I have a search function and currently what I'm doing is, subscribe to my collection each time the search parameters change like this :
Meteor.subscribe('job_search', searchParams).ready();
My question is, do we really have to do this? When fetching the data I use the search params. Will subscribing to all the data at once load all the data and reduce the performance of the app?
The subscription itself doesn't fetch the data, but it can act as a filter. The helper does the fetching whenever the data changes or a reactive query parameter changes. As long as your helper query doesn't fetch all the data you should be ok.
We are creating an application MongoDB as database and we are using official C# driver for MongoDB. We have one collection which contains thousands of records and we want to create list with paging. I have gone through documentation but there is not efficient way of paging with MongoDB C# official driver.
My requirement is to exactly fetch only 50 records from database. I have seen many examples but that get all collection and perform skip and take via LINQ which is not going to work in our case as we don't want to fetch thousand of records in memory.
Please provide any example code or link for that. Any help will be appreciated.
Thanks in advance for help.
You can use SetLimit on the cursor that represents the query. That will limit the results from MongoDB, not only in memory:
var cursor = collection.FindAll(); // Or any other query.
cursor.SetLimit(50); // Will only return 50.
foreach (var item in cursor)
{
// Process item.
}
You can also use SetSkip to set a skip (surprisingly):
cursor.SetSkip(10);
Note: You must set those properties on the cursor before enumerating it. Setting those after will have no effect.
By the way, even if you do only use Linq's Skip and Take you won't be retrieving thousands of documents. MongoDB automatically batches the result by size (first batch is about 1mb, the rest are 4mb each) so you would only get the first batch and take the first 50 docs out of it. More on
Edit: I think there's some confusion about LINQ here:
that get all collection and perform skip and take via LINQ which is not going to work in our case as we don't want to fetch thousand of records in memory.
Skip and Take are extension methods on both IEnumerable and IQueryable. IEnumerable is meant for in memory collections, but IQueryable operations are translated by the specific provider (the C# driver in this case). So the above code is equivalent with:
foreach (var item in collection.AsQueryable().SetLimit(50))
{
// Process item.
}
I am curious to know how does paging work in asp.net?
If my query returns 500 records, and my gridview paging limits to 25 records per page, when the gridview loads, does the recordset return 25 records or 500 records?
If the recordset returns 25 records, how does ado communicate with SQL to return records for page two?
If the recordset returns 500 records, are they cached within the client side?
From MSDN:
the GridView control will perform
paging by getting all of the data
records from the source, displaying
only the records for the current page,
and discarding the rest.
So the answer is, it doesn't really do efficient "paging", like many aspects of Web Forms, its all abstracted away. It's not really doing a "SELECT TOP 10".
It's simply disregarding the records it doesnt need - but the DB call is still a "SELECT *".
That is why many people (myself included), prefer to write custom yet simple paging with LINQ, using the Skip and Take IEnumerable extension methods.
E.g
yourDbContext.Where(s => somePredicate).Skip((pageNum - 1) * pageSize).Take(pageSize);
You might want to depend on client side pagination approaches via javascript. jqGrid is an excellent jQuery based solution. Of course this would mean that you'd have to load the entire data on the page. Else you'd need to write your stored procedure to return data from a particular page. But in this case you'd have to implement your own paging techniques. Such stored procedures you create must always return two result sets. One that returns the actual data, and the other resultset which returns basic pagination info, like total no.of records, total pages, & current page.
I want to fetch data from two table buut i m realy confused.
just i want recent two users in my site but problem is there that two users releted information stored in other table .
I also used join query to fetch that data but proper result not get .
Are you getting the correct results when you run your query in SQL Server Management Studio (SSMS)? Make sure you can isolate where your problem is.
If the query returns the correct results when you run it in SSMS, then post up your code and we can help you find the problem.
If the query isn't returning your desired results post the table structure and the query and we can help you with that.