I am making multiple queries to freebase with the cursor option as described here. How long after making a query can I use the cursor returned? Can I change the limit field of the MQL query and still use the same cursor?
There is no time limit on the cursor, but it is specific to your query, so if you change the query, the cursor will no longer be valid.
Related
DynamoDB limits query response size to 1MB, so to maximize the number of items returned per query, I would like to reduce the size of an individual item. Since I query by GSI, one natural approach is to reduce the number of attributes projected. The problem is that a few of the attributes that I need is a nested attribute and not a top-level attribute. Including the top-level attribute will make the items entries in GSI too large.
I am aware there is no way to project a nested attribute in a GSI. Is there any work-around to reduce the item size while allowing me to access nested attribute within a single query?
I want to start by saying that your first assumption, that DynamoDB Query is limited to 1MB, is incorrect. Indeed, Query will return results after having read 1MB of data, but you can continue fetching the next results - a process known as paging. Please refer to the Query documentation where it is clearly explained how to do it.
If, despite learning this, you still want to retrieve only pieces of each item and not the entire items, that's easy to do - using a ProjectionExpression. If you pass a ProjectionExpression during a Query, it will return only parts of the retrieved items. It even allows you to return only a piece of a nested attribute, exactly like you wanted.
Be aware, though, that even if you only retrieve small pieces of large items, you will still pay for retrieving the entire item. That's a benefit of GSI you don't get by using ProjectionExpression - you don't get cheaper retrieval. Also, it won't help you with the single-page query you began with, because the 1MB page cutoff happens at the item-size level, not the returned data length level. So you'll still need to implement paging and not assume that a single page is always enough.
I have documents with a startDate and endDate. I would like to query between the range of the startDate and endDate. I can query between a range for one date like so:
whereField("startDate", isGreaterThan: start).whereField("startDate, isLessThan: end)
But I cannot query two fields like so:
whereField("startDate", isGreaterThan: start).whereField("endDate", isLessThan: end)
Firestore throughs an exception when using more than one field in a compound where statement.
No matter what is the platform that you are using for building your application, I'm sure that the error that you get is quite explicit. Cloud Firestore official documentation is also quite explicit regarding this topic. So there are some query limitations, when it comes to Firestore:
Cloud Firestore does not support the following types of queries:
Logical OR queries. In this case, you should create a separate query for each OR condition and merge the query results in your app.
So a Firestore query can perform a range filtering only on a single property. Since you are trying to filter ranges on two properties, you're geeting that error message and this is the expected behaviour since this is not possible in a single Firestore query.
To solve this, you can choose from one of the following solutions:
You can perform filtering on one (first) field in the query and on the other (second) field client-side, as also the official documentation indicates.
You can combine the values of the two range into a single field in some way that allows your use-case with a single field. A very successful example of such a combination would be the use of geohashes for filtering on latitude and longitude properties as Frank van Puffelen explained very well in this video, Querying Firebase and Firestore.
Another option is to change the way your are storing your data and model it differently. The most simple implementation would be to put all items within the startDate and endDate into a single collection. Since you didn't choose a tag for a platform, I will write the necessary query in Javascript but it can be simply written also for other programming languages. So you can query that collection with the following query:
db.collection("startDate-endDate").where('date','>=', start).where('date','<=', end);
Another even more general alternative would be to store all your items in a collection for each periode you need (year, monts or days) separately, and then perform the necessary number of queries to get the items you are looking for one of each collection, itemsFromFirstYear, itemsFromSecondYear and so on.
Please take also take a look at the official documentation regarding:
document on compound queries and their limitations
video on Cloud Firestore queries.
IMHO, I'd recommend picking up the first option.
When executing a Cosmos DB query, it is possible to specify a continuation token (x-ms-continuation). Cosmos DB doesn't seem to complain when a valid continuation token is specified with a query that differs from the original query.
What is the expected result of such a query? Or is it meaningless?
meaningless
Yes.
You've violated the API contract, so anything is possible.
Quite possibly what you'll get is the next page, as if the second query had run, but that isn't defined behaviour. When you violate the contract, anything that looks like it is working is just an accident.
Are collection group query prices the same as a single collection query? Or is there a minimum of one document read per collection even if the query for that collection returns no documents? Is there anything I should watch out for when it comes to billing?
Collections group queries are billed exactly like normal collection queries, as described in the documentation. It stands to reason that if a normal query that returns no documents incurs at least one read, then collection group queries would behave the same way.
There is a minimum of one document-read per query. So if you query across a group of collections and no documents are returned, that will be charged as a single document read.
The easiest way I find to remember this is to think of the read as a change for reading the index. Since a collection group query works from a single index, it is (at least) a single document read for reading that index.
For example, I want to get the number of freebase concepts which have a page in the english wikipedia.
Or I want to count the concepts which have the type - book/literature.
Yes, you can use "return":"count" to get the count of results for a query instead of the actual results of the query. For "hard" queries this may time out, but for simple queries like the number of books or instances of another type, it should work fine.