How to query complex firestore document (nested list of objects)? - firebase

I have this document:
{
"artifact": {
"id": 1,
"tags" : [
"art",
"art2"
],
"origins": [
{
"id": 2,
"tags" : [
"o1",
"o11"
],
"origins": [
{
"id": 3,
"tags" : [
"o2",
"o3"
]
}
]
}
]
},
"insertionDate": "15022023095425"
}
I need to write a query to check if an id matches the id of the deepest origins object of another document.
At first, I thought something like this would work:
common_elements = db.collection("tasks").where("artifact.origins.origins.id", "==", current_item_id)
Is it possible to query object properties in a nested list with Firestore?
If this is not possible, how should I remodel my data?
Could I store origins objects in another collection and reference them in this document?

I need to write a query to check if an id matches the id of the deepest origins object of another document.
Using your actual database schema, you cannot perform such a query. If you only need to filter documents from the tasks collection by the deepest ID, then you should consider creating a non-nested field inside the document called deepestId, which in your example should contain the value of 3 and perform the following query:
common_elements = db.collection("tasks").where("deepestId", "==", 3)
If such a query can return multiple results, then you should add another where() call, to filter the documents based on the deepestId and the id like this:
common_elements = db.collection("tasks").where("deepestId", "==", 3).where("id", "==", 1)
Another possible solution would be to duplicate the data by creating another collection that should contain only the tasks with the deepestId == 3.

Related

Firestore - Can you query fields in nested documents?

I currently have a data structure like this in Firebase Cloud Firestore
Database
+ ProductInventories (collection)
+ productId1 (document)
+ variantName (collection)
+ [auto-ID] (document)
+ location: "Paris"
+ count: 1334
How would I make a structuredQuery in POST to get the count for location `Paris'?
Intuitively it might have been a POST to https://firestore.googleapis.com/v1/projects/projectName/databases/(default)/documents/ProductInventories/productId1:runQuery with the following JSON
{
"structuredQuery": {
"from": [
{
"collectionId": "variantName",
"allDescendants": true
}
],
"where": {
"fieldFilter": {
"field": {
"fieldPath": "location"
},
"op": "EQUAL",
"value": {
"stringValue": "Paris"
}
}
}
}
}
With this I get error collection group queries are only allowed at the root parent, which means I need to make the POST to https://firestore.googleapis.com/v1/projects/projectName/databases/(default)/documents:runQuery instead. This however means I'll need to create a collection group index exemption for each variant (variantName) I have for each productId.
Seems like I would be better off to have below variantName collection level, the location as the name of the document, and I can then access the count directly without making a query. But seems to me the point of NoSQL was that I could be less careful about how I structure the data, so I'm wondering if there's a way for me to make the query as is with the current data structure.
Using collection names that are not known ahead of time is usually an anti-pattern in Firestore. And what you get is one of the reasons for that: you need to be able to create a collection group query across documents in multiple collections, you need to be able to define an index on the collection name - and that requires that you know those some time during development.
As usual, when using NoSQL databases, you can modify/augment your data structure to allow the use-case. For example, if you create a single subcollection for all variants, you can give that collection a fixed name, and search for paris and $variantName in there. This collection can either be a replacement of your current $variantName collections, or an addition to it.
have you tried something like this?
fb.firestore().collection('ProductInventories')
.doc('productId1')
.collection('variantName')
.where('location', '==', 'Paris')
.get()
.then(res=>{
res.data().docs.forEach((product, i)=>{
console.log('item ' + i + ': ' + product.count);
})
});

How is the order of the fields in firestore composite indexes decided?

This is my query:
db.requests
.where('userId', '==', uid)
.where('placeId', '==', placeId)
.where('endTime', '>=', Date.now())
.where('isFullfilled', '==', false);
So I manually wrote this index in firestore.indexes.json:
{
"collectionGroup": "requests",
"queryScope": "COLLECTION",
"fields": [
{
"fieldPath": "userId",
"order": "ASCENDING"
},
{
"fieldPath": "placeId",
"order": "ASCENDING"
},
{
"fieldPath": "endTime",
"order": "ASCENDING"
},
{
"fieldPath": "isFullfilled",
"order": "ASCENDING"
}
]
},
When run, I get an error "This query requires an index". And the automatically created index looks like this:
My manually created index on the other hand looks like this in GUI:
Why does it not accept my own index? Does the order of fields matter? I am not ordering query results. Is there any kind of pattern to index creation? This is really confusing and I can't find anything on this in the docs. It's really annoying to have to run every query against the cloud database to get the proper composite index field order.
Does the order of fields matter?
Yes, the order of the fields very much matters to the resulting index. I pretty much see such an index as:
Firestore creates a composite value for each document by concatenating the value of the fields in that index.
For a query it then can only query if the fields and order exactly match (with some exceptions for subsets of fields, and cases where it can do a zig-zag-merge-join).
For such a query Firestore finds the first entry in the index that matches the conditions. From there it then returns contiguous results (sometimes called a slice). It does not skip any documents, nor jump to another point in the index, nor reverse the index.
The field that you order on, or do a range query on (your >=) must be last in the index.
Note that this is probably not how it really works, but the model holds up pretty well - and is in fact how we recommend implementing multi-field filtering on Firebase's other database as explained here: Query based on multiple where clauses in Firebase

Flutter Firestore: update array of object by param

I have a document with these param:
"rent": "text",
"leases": [
{
"id": 1,
"title": "text",
},
{
"id": 2,
"title": "text",
}
]
I add new elements to the array like this:
.update({'leases': FieldValue.arrayUnion([newLease])});
But how can I update that id 2 only?
The array operations can either add an item that isn't in the array yet (FieldValue.arrayUnion) or remove an item that is in the array already (FieldValue.arrayRemove ). There is no way to atomically update an existing item, by its index or otherwise.
So that means the only option is to:
Read the document into your application code.
Get the array from it.
Update the array in your application.
Write back the entire array to the database.
Also see:
How can I update map data that's in a array in firebase? (Flutter)
Flutter: Update specific index in list (Firestore)
how to edit an array element present in firestore

How to update a nested object inside an array in DynamoDB

Consider the following document item / syntax in a DynamoDB table:
{
"id": "0f00b15e-83ee-4340-99ea-6cb890830d96",
"name": "region-1",
"controllers": [
{
"id": "93014cf0-bb05-4fbb-9466-d56ff51b1d22",
"routes": [
{
"direction": "N",
"cars": 0,
"sensors": [
{
"id": "e82c45a3-d356-41e4-977e-f7ec947aad46",
"light": true,
},
{
"id": "78a6883e-1ced-4727-9c94-2154e0eb6139",
}
]
}
]
}
]
}
My goal is to update a single attribute in this JSON representation, in this case cars.
My approach
I know all the sensors IDs. So, the easiest way to reach that attribute is to find, in the array, the route which has a sensor with any of the ids. Having found that sensor, Dynamo should know which object in the routes array he has to update. However, I cannot run this code without my condition being rejected.
In this case, update attribute cars, where the route has a sensor with id e82c45a3-d356-41e4-977e-f7ec947aad46 or 78a6883e-1ced-4727-9c94-2154e0eb6139.
var params = {
TableName: table,
Key:{
"id": "0f00b15e-83ee-4340-99ea-6cb890830d96",
"name": "region-1"
},
UpdateExpression: "set controllers.intersections.routes.cars = :c",
ConditionExpression: ""controllers.intersections.routes.sensors.id = :s",
ExpressionAttributeValues:{
":c": 1,
":s": "e82c45a3-d356-41e4-977e-f7ec947aad46"
},
ReturnValues:"UPDATED_NEW"
};
docClient.update(params, ...);
How can I achieve this?
Unfortunately, you can't achieve this in DynamoDB without knowing the array index. You have very complex nested structure. The DynamoDB API doesn't have a feature to handle this scenario.
I think you need the array index for controllers, routes and sensors to get the update to work.
Your approach may work in other databases like MongoDB. However, it wouldn't work on DynamoDB. Generally, it is not recommended to have this complex structure in DynamoDB especially if your use case has update scenario.
TableName : 'tablename',
Key : { id: id},
ReturnValues : 'ALL_NEW',
UpdateExpression : 'set someitem['+`index`+'].somevalue = :reply_content',
ExpressionAttributeValues : { ':reply_content' : updateddata }
For updating nested array element need to fing out array index . Then you can update nested array element in dynamo db.

Getting all root properties

Let's say I have a DocumentDB collection populated with documents that have this shape:
[{ "Name": "KT", "Dob": "5/25/1990", "Children": [], "IsMale": false },
{ "Name": "Chris", "Dob": "10/1/1980", "Children": [], "IsMale": true }]
Now let's say I don't the structure of the documents above.
Is there a query I can write that will return me a distinct list of those property names ("Name", "Dob", "Children", "IsMale")?
In other words, is there a way for be to sniff out the schema of those documents?
This might be a duplicate of this question. In any case, the answers there might give you some ideas.
tl;dr; The only way to do it is to read all of the docs. You can pull them all back to your machine or you can read them inside of a stored procedure and only send the calculated schema back to your machine.
You need a dynamic ORM or ODM for Azure DocumentDB like Slazure to do something like this. Example follows:
using SysSurge.Slazure.AzureDocumentDB.Linq;
using SysSurge.Slazure.Core;
using SysSurge.Slazure.Core.Linq.QueryParser;
public void EnumProperties()
{
// Get a reference to the collection
dynamic storage = new QueryableStorage<DynDocument>("URL=https://contoso.documents.azure.com:443/;DBID=DDBExample;TOKEN=VZ+qKPAkl9TtX==");
QueryableCollection<DynDocument> collection = storage.TestCustomers;
// Build collection query
var queryResult = collection.Where("SignedUpForNewsletter = true and Age < 22");
foreach (DynDocument document in queryResult)
{
foreach (KeyValuePair<string, IDynProperty> keyValuePair in document)
{
Console.WriteLine(keyValuePair.Key);
}
}
}

Resources