Firestore Pagination: how to define 'startAt'-cursor for REST? - firebase

I am trying to use a cursor and 'startAt', to paginate REST requests to Firestore. According to the Paginate-documentation, the cursor should equal to the last document of the previous query. As the REST-documentation is without an example, I tried to run it by inserting an entire document as cursor in the startAt-object; like this:
POST https://firestore.googleapis.com/v1/PROJECT-NAME/databases/(default)/documents/organizations/testManyInstructions:runQuery
{
"structuredQuery": {
"from": [
{
"collectionId": "instructions"
}
],
"where": {
"fieldFilter": {
"field": {
"fieldPath": "belongsToDepartementID"
},
"op": "EQUAL",
"value": {
"stringValue": "toplevel-document-id"
}
}
},
"orderBy": [
{
"field": {
"fieldPath": "instructionNumber"
},
"direction": "ASCENDING"
}
],
"startAt": {
"values": [{
"document": {
"name": "projects/PROJECT-NAME/databases/(default)/documents/organizations/testManyInstructions/instructions/i0",
"fields": {
"checkbox": {
"booleanValue": false
},
"retrainTimespanDays": {
"integerValue": "365000"
},
"approvedByName": {
"stringValue": ""
},
"instructionNumber": {
"stringValue": "instr. 0"
},
"instructionCurrentRevision": {
"stringValue": "A"
},
"instructionCurrentRevisionPublishingDate": {
"timestampValue": "1999-01-01T00:00:00Z"
},
"instructionFileURL": {
"stringValue": ""
},
"instructionTitle": {
"stringValue": "dummy Title0"
},
"instructionFileUploadDate": {
"timestampValue": "1999-01-01T00:00:00Z"
},
"belongsToDepartementID": {
"stringValue": "toplevel-document-id"
},
"approvedByEmailAdress": {
"stringValue": ""
}
},
"createTime": "2022-02-18T13:55:42.807103Z",
"updateTime": "2022-02-18T13:55:42.807103Z"
}
}
]
},
"limit": 5
}
}
without the "startAt"-Object, the following code works fine and returns 5 documents.
with the "startAt"-Object, this error is returned:
{
"error": {
"code": 400,
"message": "Invalid JSON payload received. Unknown name \"document\" at 'structured_query.start_at.values[0]': Cannot find field.",
"status": "INVALID_ARGUMENT",
"details": [
{
"#type": "type.googleapis.com/google.rpc.BadRequest",
"fieldViolations": [
{
"field": "structured_query.start_at.values[0]",
"description": "Invalid JSON payload received. Unknown name \"document\" at 'structured_query.start_at.values[0]': Cannot find field."
}
]
}
]
}
}
]
Please advise, how to set the cursor in the startAt-object correctly.

I've run a similar query using offset instead of startAt, so I tried modifying and got it to work. This is the rest api documentation I used.
startAt requires a Cursor object which is an array of Values.
https://firebase.google.com/docs/firestore/reference/rest/v1/StructuredQuery
https://firebase.google.com/docs/firestore/reference/rest/v1/Cursor
https://firebase.google.com/docs/firestore/reference/rest/Shared.Types/ArrayValue#Value
I would have preferred an example as well!
"startAt": {
"values": [{
"stringValue": "Cr"
}]
},
"orderBy": [{
"field": {
"fieldPath": "Summary"
}
}],
Good luck!

Related

BatchWrite REST API JSON Structure for FireStore

I know how to compose JSON when you want to create a single document on FireStore.
# Query String
"updateMask.fieldPaths": [
"`price`",
"`status`"
]
# Body
"fields": {
"price": {
"stringValue": "165.00"
},
"status": {
"stringValue": "active"
}
}
But when it comes to BatchWrite I cannot understand what the doc implies
https://cloud.google.com/firestore/docs/reference/rest/v1/projects.databases.documents/batchWrite
Currently I'm doing this with Python library I deployed on cloud computing resource(pipedream) but I want to do this within Make.com and its "Make API Call" module so I need to know the curl style or REST JSON syntax.
I'm assuming fields go to "writes" but not sure what goes to "labels" object.
Exactly what's the write structure for this call?
{
"writes": [
{
"update": {
"name": "projects/project_id/databases/(default)/documents/service_order/352003090342435.5f357270173e8c70878.-9486573",
"fields": {
"managed_by": {
"booleanValue": "false"
},
"location2": {
"booleanValue": "false"
},
"made_by": {
"stringValue": "352003090342435.5eda4f3c17284c58460.90921842"
},
"phone2": {
"stringValue": "0665439307"
},
"state": {
"stringValue": "confirm"
},
"write_date": {
"stringValue": "1597664391000"
},
"made_by_name": {
"stringValue": "FaridBenabdallah"
},
"description": {
"stringValue": "e1"
},
"accepted_offer": {
"booleanValue": "false"
},
"id": {
"stringValue": "352003090342435.5f357270173e8c70878.-9486573"
},
"location1": {
"stringValue": "352003090342435.5f3572a0173e8c7c336.-8281409"
},
"date_order": {
"booleanValue": "false"
},
"order_type": {
"stringValue": "Minuiserie"
},
"phone1": {
"stringValue": "0657331995"
}
}
},
"updateMask": {
"fieldPaths": [
"managed_by",
"location2",
"made_by",
"phone2",
"state",
"write_date",
"made_by_name",
"description",
"accepted_offer",
"id",
"location1",
"date_order",
"order_type",
"phone1"
]
}
}
]
}
This another stackoverflow article helped me
If there is null you need to change the intended value type to nullValue.

How to use ObjectSizeGreaterThan and ObjectSizeLessThan xml tags for AWS S3 command 'PutBucketLifeCycleConfiguration"?

Trying with a json input like this:
{
"Rules": [
{
"Expiration": {
"Date": "2023-01-01T00:00:00.000Z"
},
"ID": "id1",
"Filter": {
"And": {
"Tags": [
{
"Key": "k1",
"Value": "k1val1"
},
{
"Key": "k1u",
"Value": "k1vadl1"
}
],
"ObjectSizeGreaterThan": 100,
"ObjectSizeLessThan": 1000
}
},
"Status": "Enabled"
}
]
}
However, I am getting an error like this from the aws client:
Parameter validation failed:
Unknown parameter in LifecycleConfiguration.Rules[0].Filter.And: "ObjectSizeGreaterThan", must be one of: Prefix, Tags
Any idea why I get this error and how to correct it?
This seems to be working but only with the latest cli version aws-cli/2.7.30
I am getting the same error on aws-cli/1.19.39
{
"Rules": [
{
"Expiration": {
"Date": "2024-01-01T00:00:00.000Z"
},
"ID": "123",
"Filter": {
"And": {
"Prefix": "sricli",
"Tags": [
{
"Key": "ke",
"Value": "k1"
},
{
"Key": "je",
"Value": "k2"
}
],
"ObjectSizeGreaterThan": 102400,
"ObjectSizeLessThan": 204800
}
},
"Status": "Enabled"
}
]
}
Sri

Firestore Pagination: how to define **unique** 'startAt'-cursor for REST?

This is a follow up question to an already solved one.
For this previous question an answer was given, how to define a cursor for query-pagination with 'startAt' for REST, that relates to a range of documents. In the example below, the cursor relates to all documents with an 'instructionNumber.stringValue' equal to "instr. 101". According to my testing, this results in skipping of documents.
New question:
How has the cursor to be defined, to not only relate to the stringValue of a field, that the query is ordered by? But instead to a distinct document (usually defined by its document-id)?
"structuredQuery": {
"from": [{"collectionId": "instructions"}],
"where": {
"fieldFilter": {
"field": {
"fieldPath": "belongsToDepartementID"
},
"op": "EQUAL",
"value": {
"stringValue": "toplevel-document-id"
}
}
},
"orderBy": [
{
"field": {
"fieldPath": "instructionNumber"
},
"direction": "ASCENDING"
}
],
"startAt": {
"values": [{
"stringValue": "instr. 101"
}]
},
"limit": 5
}
}
For better understanding, here is the condensed schema of the documents.
{
"document": {
"name": "projects/PROJECT_NAME/databases/(default)/documents/organizations/testManyInstructions/instructions/i104",
"fields":
"belongsToDepartementID": {
"stringValue": "toplevel-document-id"
},
"instructionNumber": {
"stringValue": "instr. 104"
},
"instructionTitle": {
"stringValue": "dummy Title104"
},
"instructionCurrentRevision": {
"stringValue": "A"
}
},
"createTime": "2022-02-18T13:55:47.300271Z",
"updateTime": "2022-02-18T13:55:47.300271Z"
}
}
For a query with no ordering:
"orderBy": [{
"direction": "ASCENDING",
"field": {"fieldPath": "__name__"}
}],
"startAt": {
"before": false,
"values": [{"referenceValue": "last/doc/ref"}]
}
For a query with ordering:
"orderBy": [
{
"direction": "DESCENDING",
"field": {"fieldPath": "instructionNumber"}
},
{
"direction": "DESCENDING",
"field": {"fieldPath": "__name__"}
}
],
"startAt":
{
"before": false,
"values": [
{"stringValue": "instr. 101"},
{"referenceValue": "last/doc/ref"}
]
}
Be sure to use the same direction for __name__ as the previous "orderBy" or it will need a composite index.
To ensure you have identify unique document for starting at you'll always want to include the document ID in your call to startAt.
I'm not sure of the exact syntax for the REST API, but the Firebase SDKs automatically pass this document ID when you call startAt with a DocumentSnapshot.

Not able to filter out required property in Azure TSI Gen1 Get Events API response

I am using the below request body to fetch only the required property values.
"searchSpan": {
"from": {
"dateTime": "2021-11-20T00:00:00.000Z"
},
"to": {
"dateTime": "2021-11-20T23:00:00.000Z"
}
},
"predicateString": "[Params.Name] = 'power'",
"take": 100
}
}
The URL is like below:
https://12345678a-bcde-3e91-blah-2292933292aa.env.timeseries.azure.com/events?api-version=2016-12-12
Despite specifying the required property the response returns all properties as if it has not seen the predicate string. What might I be doing wrong?
{
"warnings": [],
"events": [
{
"schema": {
"rid": 0,
"$esn": "my-event-hub",
"properties": [
{
"name": "mytimestamp",
"type": "DateTime"
},
{
"name": "Params.Name",
"type": "String"
},
{
"name": "Params.Value",
"type": "Double"
}
]
},
"$ts": "2021-11-20T10:01:50Z",
"values": [
"2021-11-20T10:01:50Z",
"energy",
60
]
},
{
"schemaRid": 0,
"$ts": "2021-11-20T10:01:50Z",
"values": [
"2021-11-20T10:01:50Z",
"power",
10
]
},
{
"schemaRid": 0,
"$ts": "2021-11-20T10:01:50Z",
"values": [
"2021-11-20T10:01:50Z",
"strength",
200
]
},
]
}
Edit
I'm getting "Properties count error" in the TSI overview page. This might quite be the root cause but I don't know for sure
"For Time Series Insights environment ABC: You have used all 641/600 properties in your environment".

Actions-on-Google can not get UPDATES_USER_ID on Dialogflow SDK

I'm setting up an action which uses push notifications. Yet, on firebase I can't get "UPDATES_USER_ID" of user to save. It returns "undefined".
I followed the guide on this link and here is my code to get UPDATES_USER_ID.
app.intent('Setup', (conv, params) => {
conv.ask(new UpdatePermission({
intent: "notificationResponseIntent"
}));
});
app.intent("FinishNotificationSetup", (conv, params) => {
if (conv.arguments.get('PERMISSION')) {
conv.data.GoogleUserID = conv.arguments.get("UPDATES_USER_ID");
console.log(conv.data.GoogleUserID);
conv.ask("some response....");
}
});
And here is my webhook request when FinishNotificationSetup intent is invoked.
{
"responseId": "2f425fe5-db42-47dc-90a1-c9bc85f725d2",
"queryResult": {
"queryText": "actions_intent_PERMISSION",
"parameters": {},
"allRequiredParamsPresent": true,
"fulfillmentMessages": [
{
"text": {
"text": [
""
]
}
}
],
"outputContexts": [
{
"name": "projects/projectname/agent/sessions/ABwppHGD33Tyho41g9Mr2vzxePlskNmvOzCTxUiDGzENcl3C7RQs94aOQ8ae_DUlOApR0VBO9DuwAWdWr2frAA/contexts/actions_capability_screen_output"
},
{
"name": "projects/projectname-10c22/agent/sessions/ABwppHGD33Tyho41g9Mr2vzxePlskNmvOzCTxUiDGzENcl3C7RQs94aOQ8ae_DUlOApR0VBO9DuwAWdWr2frAA/contexts/actions_intent_permission",
"parameters": {
"PERMISSION": true,
"text": ""
}
},
{
"name": "projects/projectname-10c22/agent/sessions/ABwppHGD33Tyho41g9Mr2vzxePlskNmvOzCTxUiDGzENcl3C7RQs94aOQ8ae_DUlOApR0VBO9DuwAWdWr2frAA/contexts/_actions_on_google",
"lifespanCount": 98,
"parameters": {
"data": "{\"***":\"***",\"***":\"***"}"
}
},
{
"name": "projects/projectname-10c22/agent/sessions/ABwppHGD33Tyho41g9Mr2vzxePlskNmvOzCTxUiDGzENcl3C7RQs94aOQ8ae_DUlOApR0VBO9DuwAWdWr2frAA/contexts/actions_capability_account_linking"
},
{
"name": "projects/projectname-10c22/agent/sessions/ABwppHGD33Tyho41g9Mr2vzxePlskNmvOzCTxUiDGzENcl3C7RQs94aOQ8ae_DUlOApR0VBO9DuwAWdWr2frAA/contexts/actions_capability_audio_output"
},
{
"name": "projects/projectname-10c22/agent/sessions/ABwppHGD33Tyho41g9Mr2vzxePlskNmvOzCTxUiDGzENcl3C7RQs94aOQ8ae_DUlOApR0VBO9DuwAWdWr2frAA/contexts/google_assistant_input_type_keyboard"
},
{
"name": "projects/projectname-10c22/agent/sessions/ABwppHGD33Tyho41g9Mr2vzxePlskNmvOzCTxUiDGzENcl3C7RQs94aOQ8ae_DUlOApR0VBO9DuwAWdWr2frAA/contexts/actions_capability_web_browser"
},
{
"name": "projects/projectname-10c22/agent/sessions/ABwppHGD33Tyho41g9Mr2vzxePlskNmvOzCTxUiDGzENcl3C7RQs94aOQ8ae_DUlOApR0VBO9DuwAWdWr2frAA/contexts/actions_capability_media_response_audio"
}
],
"intent": {
"name": "projects/projectname-10c22/agent/intents/a12b6d3f-0f24-45e9-a1b2-5649083831b0",
"displayName": "FinishNotificationSetup"
},
"intentDetectionConfidence": 1,
"languageCode": "tr"
},
"originalDetectIntentRequest": {
"source": "google",
"version": "2",
"payload": {
"isInSandbox": true,
"surface": {
"capabilities": [
{
"name": "actions.capability.SCREEN_OUTPUT"
},
{
"name": "actions.capability.WEB_BROWSER"
},
{
"name": "actions.capability.ACCOUNT_LINKING"
},
{
"name": "actions.capability.MEDIA_RESPONSE_AUDIO"
},
{
"name": "actions.capability.AUDIO_OUTPUT"
}
]
},
"requestType": "SIMULATOR",
"inputs": [
{
"rawInputs": [
{
"inputType": "KEYBOARD"
}
],
"arguments": [
{
"textValue": "true",
"name": "PERMISSION",
"boolValue": true
},
{
"name": "text"
}
],
"intent": "actions.intent.PERMISSION"
}
],
"user": {
"lastSeen": "2019-04-30T07:23:23Z",
"permissions": [
"UPDATE"
],
"locale": "tr-TR",
"userId": "ABwppHHCEdtf23ZaNg0DaCv3fvshSUXUvYGXHe6kR7jbKacwIS6vDBBL7YXbN70jYa8KaXWZqbsyhFFSdsYLiw"
},
"conversation": {
"conversationId": "ABwppHGD33Tyho41g9Mr2vzxePlskNmvOzCTxUiDGzENcl3C7RQs94aOQ8ae_DUlOApR0VBO9DuwAWdWr2frAA",
"type": "ACTIVE",
"conversationToken": "[\"_actions_on_google\"]"
},
"availableSurfaces": [
{
"capabilities": [
{
"name": "actions.capability.AUDIO_OUTPUT"
},
{
"name": "actions.capability.SCREEN_OUTPUT"
},
{
"name": "actions.capability.WEB_BROWSER"
}
]
}
]
}
},
"session": "projects/projectname-10c22/agent/sessions/ABwppHGD33Tyho41g9Mr2vzxePlskNmvOzCTxUiDGzENcl3C7RQs94aOQ8ae_DUlOApR0VBO9DuwAWdWr2frAA"
}
To send notification, I've been using userID instead of UPDATES_USER_ID and it is working. Yet, it will be deprecated soon. So, I need to find a solution to get this ID and couldn't make it working. What do I need to do to get this ID?
I've found solution for this problem. While getting UPDATES_USER_ID conv.arguments.get() only works for first attempt. So, while building your action you must save it. If you didn't store or save, you can reset your profile and try again, you will be able to get.
app.intent("FinishNotificationSetup", (conv, params) => {
if (conv.arguments.get('PERMISSION')) {
if(!conv.user.storage.GoogleUserID)
{
conv.user.storage.GoogleUserID = conv.arguments.get("UPDATES_USER_ID");
//For best case
//store ID in your db.
}
console.log(conv.user.storage.GoogleUserID);
conv.ask("some response....");
}
});
For best case, try saving this to your database because conv.user.storage does not work sometimes.

Resources