Google Firestore - REST API without pagination - firebase

I've set up a database in Google Firebase - Firestore and already uploaded just about 500 documents into a single collection.
Now, I've also figured out the REST API to read "all" of the documents from that collection:
GET https://firestore.googleapis.com/v1/projects/my-project/databases/(default)/documents/my-collection
This request returns following response structure:
{
"documents": [
{
"name": "projects/my-project/databases/(default)/documents/my-collection/3D-Technologien7a69809a-dc97-4663-a661-2df3f6f034dd",
"fields": {
"start": {
"stringValue": "28.10.2021"
},
"location": {
"stringValue": "Hb"
},
"name": {
"stringValue": "3D-Technologien"
},
"end": {
"stringValue": "30.10.2021"
}
},
"createTime": "2020-10-09T16:22:53.865356Z",
"updateTime": "2020-10-09T16:22:53.865356Z"
}
],
"nextPageToken": "AFTOeJymPSjChtwIgEMwViyHTAnXjWr_wNAQ7t8h9ZioAI-tJRX7WxdS7TBCeHvREsNgmEGOSLtuDI_YCjIWU2BdlT1-QDoIPHtG4QCxxcjJaSB5NRA-o7smLhr6Pxil0tPUlLBdiLwyjOD-c7qm7QNTR9SqL7HNI3uNKbZi7MG3ng-WMLpoN6MWZ7uGjQGpgfysbwk"
}
It's pretty clear that the response is based on pagination, but I don't find any documentation on this and even worse, on how to adjust this. Currently, max. 20 documents are being returned for the request. I want all 500 to be returned.
Is there any configuration option for this?

According to the docs you can use query params pageSize and pageToken
?pageSize=20&pageToken=ABCDEF1234567890
https://cloud.google.com/firestore/docs/reference/rest/v1beta1/projects.databases.documents/list

Related

Sending a request to /adCreativesV2 entity doesn't return all the fields

We are in the process of designing a custom build report for our client and have recently stumbled upon a behavior which we could not puzzle. Basically, we are not getting returned the following fields
ad_creative_name
ad_creative_text
ad_creative_title
when sending a GET request as follows:
https://api.linkedin.com/v2/adCreativesV2/136236994?projection=(*,reference~(*))
We're getting a 200 OK response as per below:
"variables": {
"data": {
"com.linkedin.ads.SponsoredInMailCreativeVariables": {
"content": "urn:li:adInMailContent:5139314"
}
}
},
"test": false,
"servingStatuses": [
"RUNNABLE"
],
"type": "SPONSORED_INMAILS",
"version": {
"versionTag": "6"
},
"reference": "urn:li:adInMailContent:5139314",
"changeAuditStamps": {
"lastModified": {
"time": 1624098658000
},
"created": {
"time": 1624035613000
}
},
"review": {
"reviewStatus": "APPROVED"
},
"campaign": "urn:li:sponsoredCampaign:179783044",
"reference!": {
"message": "Unsupported URN domain: li:adInMailContent",
"status": 501
},
"id": 136236994,
"status": "ACTIVE"
}
From the response, you may also notice that we are also getting this warning
"message": "Unsupported URN domain: li:adInMailContent",
"status": 501
We've tried different API requests variations, including specifying different creatives that are active, but to no avail. Also, we were not able to find any info regarding the error above, so we are uncertain what might be causing it.
Thanks!

Q: Linkedin Vector Assest API, Registering Multi-Part Upload - No metadata value in response

I'm attempting to complete a multipart upload using Linkedin V2 Vector Assets and i've noticed that when registering the upload, the metadata field comes back blank in the response.
Here's the request to register the upload https://api.linkedin.com/v2/assets?action=registerUpload
{
"registerUploadRequest":{
"owner":"urn:li:organization:71736744",
"recipes":["urn:li:digitalmediaRecipe:feedshare-video"],
"serviceRelationships":[
{"identifier":"urn:li:userGeneratedContent","relationshipType":"OWNER"}
],
"supportedUploadMechanism":["MULTIPART_UPLOAD"],
"fileSize":50866165
}
and here is the response (removed all but one partUploadRequests object to make the response easier to read)
{
"value": {
"uploadMechanism": {
"com.linkedin.digitalmedia.uploading.MultipartUpload": {
"partUploadRequests": [
{
"headers": {
"Content-Type": "application/octet-stream"
},
"byteRange": { "lastByte": 4194303, "firstByte": 0 },
"url": "https://www.linkedin.com/dms-uploads/C5605AQFnYcNsf7K5Yw/uploadedVideo?sau=aHR0cHM6Ly93d3cubGlua2VkaW4uY29tL2FtYnJ5L2FtYnJ5LXZpZGVvLz94LWxpLWFtYnJ5LWVwPUFRRmp4SlBwRndkUUNnQUFBWGhjYkgwNzNuZmpacTRFM3AxUkpidTMxTlAxems3WmdoUHN1Ymp1TjVGTXVDRGFOaGczVnpHWXNpLW5OYWh2NlkxWkU5UDVIdktsQ2JxbGhhOUt1SElJSnBXTjI0TkEyWDIzM1ZiaS1aNW4yTmlXWWQyX1VlN1J2d0FEdkMwSG51ckNMY1ljMkdrczhKclFlZHRYNlZNN2dvTFJad2hpT0pxcFNZM29mXzdTTmRhZjZzQlNMS3FwaTJyWF9LYTRqTk9OcnBhTFpCam1KR3ZENEV3ZFd0VHEzNm44aEFQbDhCMmc3cVpOd3Q0Ny1TT1ZvcHRoT0dWcTB0cHAxU2VkbkY2aVNVQ3J6RVlwVHdzNDZTQlViME5HbGxITHo4a1ZNOGJxYkJjMDVCc054MDQ2ZDktZWdGb2RFa3ZKRmxsU0NRWFV2ZzdQb3VFaVNodkZtSHRYVHhLLWlkbnpKWjlQN1Q2b1ROQWJhWE5iR2g2QUdBQmQ3VVdjNkdGb01YaWFiazBhRlA4UzNpUXNlM05ISFNVdkFBdk1nSnV2a0xxWV9oYU1VTXdoeXFRT0w1a0NLM1NSd2RXTnpKSHZLMkhfY0ZqclYzcnB4VXJZckkxbUZ2QVl2Y3dUcVYyaDhWWHRpY1luNTA0am1jUjN3U0M5RVVQZEo2RFZEd1RJRmFvOTJYcmszbTFNS2pSRlFZMV94enVHZVk4SERKWTQwbDQySVRJNXlCLUp3b2V3Vjl3RGduOTZycmhoY1pLa0VOck5vdUZ4eDdodlBTNkZUVXcweXJWM3BaOUF5X2NtaHBXTVpwOVRFY0pJcGY3elFrQTF3ZFcyR3VIcEtGLUF4MldTZTdlR1pCOTdHaU5vLW9XamhkV2stR2MzNFh5UjFwckZsTF9sSFFUWU40S2pNLUI2Y2poZHFJV2xVTll6Z1lVaXdwNld2UEYwVFRObW5kQVgtTTBlX1VsaGFOQ190SDFEcG9Kem5IZExoN3IxMGk5VWE4MGpsZ3pDQzBsT3JKSkxuSEcwNm9jVzNJSzctT3NSZGhBSEFNdTBuWDNRdlhfSFoxWXpfNXgza3dyZzBPOVlmN2xIUFFiVW1lYllReUlJaGdxOEE1blpxVUxzeUpvZElLWk5WdUdlcjVuN21jTFV4TzRtNlBUczVtc3pHS0x4Z3o4b2FDdVNiWWliYzh2NkpxQktiN2RmejdnQWRFVVpfNnRLenVxUm11bGk5OGZVSVpOR3hfc3R1Mk9nVDYzYXRoQmZnRHQ0dGpnVi1BOWlaVXdtYWNLckxwVE5wNUdHZlBxZjhLaE1MZC1BVXotSVNoX19jRUlzM2laSXlpeC1pNlFpeGZyRmpxQnYwaDhwOGZpeXZ0Q2Job0xjU1g4aWNjX0lSX28xQXR0UmZB&pn=1&m=616825227&app=86804616&sync=0&v=beta&ut=1nBLJxAvwH5FI1",
"urlExpiresAt": 1616544717116
} ...
],
"metadata": ""
}
},
"asset": "urn:li:digitalmediaAsset:C5605AQFnYcNsf7K5Yw",
"mediaArtifact": "urn:li:digitalmediaMediaArtifact:(urn:li:digitalmediaAsset:C5605AQFnYcNsf7K5Yw,urn:li:digitalmediaMediaArtifactClass:uploadedVideo)"
}
}
The api documentation explains that I need to use the metadata value from the register upload response to complete the upload
I attempt making the request anyway without a value for the metadata field to https://api.linkedin.com/v2/assets?action=completeMultiPartUpload (removed all but one partUploadResponses object to make the response easier to read)
{
"completeMultipartUploadRequest": {
"mediaArtifact": "urn:li:digitalmediaMediaArtifact:(urn:li:digitalmediaAsset:C5605AQFnYcNsf7K5Yw,urn:li:digitalmediaMediaArtifactClass:uploadedVideo)",
"metadata": "",
"partUploadResponses": [
{
"headers": {
"ETag": "/ambry-video/signedId/AQGpSasdC3ezCgAAAXhcbTobXlB6_ujllr8BPUUAX4o9OgbHCxiRKoG9woM85dnGrxbUbd-5IXyMKO7syqyZXLgHFd8GQJkiMvvp1eHI9N32iJqMG-l_aFgSeXMbcqkFFcjuT22djM93dmKs_RBb58kppS3CadOPHSt5_tApCxAq0wgTfYQng3OkV_uVRdjIYftyfnHW3nF2vWz4hZBoX-YhmF6gDhXtN9dNTnU4QpHB7sGCecpjYgT8ypV-u5isvFrujHl3tl4.bin"
},
"httpStatusCode": 200
} ....
]
}
}
This comes back as a 200OK without any response body which is expected.
After that I check the upload status and which comes back with a code of SERVER_ERROR and I'm not able to create a post with the asset id of the uploaded asset.
Response from checking asset status https://api.linkedin.com/v2/assets/C5605AQFnYcNsf7K5Yw
{
"recipes": [
{
"recipe": "urn:li:digitalmediaRecipe:feedshare-video",
"status": "SERVER_ERROR"
}
],
"serviceRelationships": [
{
"relationshipType": "OWNER",
"identifier": "urn:li:userGeneratedContent"
}
],
"mediaTypeFamily": "VIDEO",
"created": 1616458317042,
"id": "C5605AQFnYcNsf7K5Yw",
"lastModified": 1616458454711,
"status": "ALLOWED"
}
Request to create a UGC post with the asset https://api.linkedin.com/v2/assets
{
"author": "urn:li:organization:71736744",
"lifecycleState": "PUBLISHED",
"specificContent": {
"com.linkedin.ugc.ShareContent": {
"media": [
{
"media": "urn:li:digitalmediaAsset:C5605AQFnYcNsf7K5Yw",
"status": "READY",
"title": {
"attributes": [],
"text": "Sample Video Create"
}
}
],
"shareCommentary": {
"attributes": [],
"text": "Some share text"
},
"shareMediaCategory": "VIDEO"
}
},
"visibility": {
"com.linkedin.ugc.MemberNetworkVisibility": "PUBLIC"
}
Response to the request
{
"serviceErrorCode": 0,
"message": "Resource assets does not exist",
"status": 404}
The only thing I can think of is since I don't have a metadata value when completing the upload I'm not able to use the video asset in UGC posts. Anyone encountered a problem similar to this?
Further Investigations
On further investigation the example from the API docs doesn't return a metadata field either. So this is what I think might be happening
I'm not uploading the file chunks correctly and the metadata field isn't required for completing the upload.
The API has a bug in it which prevents it from returning the metadata field when registering a multipart upload.
Figured it out, when using the slice method to break your file up into chunks it needs to be file.slice(firstByte, lastByte + 1)
It also seems that the metadata value from the register upload request isn't need to complete the upload.

Firebase Firestore REST Request - Query and Filter

I have a firestore database on a firebase project. I want to make rest request for filtering or querying data with postman. I'm using "https://firestore.googleapis.com/v1/projects//databases/(default)/documents/" to get the data in a known path in my database. Here is a sample of my database structure:
users > xxxxx > messages > yyyyy> "sent":"true"
where "users" and "messages" are collections, "xxxxx" and "yyyyy"are autogenerated document ids (xxxxx is autogenerated user id)
What I want to do is to find the "xxxxx"s (users) which have >"sent":"true"< data.
I get success if I know the "xxxxx" and "yyyyy" but I don't know them because they are autogenerated and different from each other in my database and don't know how to do it.
You need to run a Query, as explained here in the documentation of the REST API.
Since you want to query all the messages sub-collections of different user documents, you need to "simulate" a Collection Group Query in your StructuredQuery. The way to do that is to set the allDescendants element to true in the CollectionSelector.
So, issuing a POST HTTP Request on the following URL will do the trick:
var URL = "https://firestore.googleapis.com/v1/projects/<your-project-id>/databases/(default)/documents:runQuery";
The body of the POST Request shall contain:
"structuredQuery": {
"from": [{
"collectionId": "messages",
"allDescendants": true
}],
"where": {
"fieldFilter": {
"field": {
"fieldPath": "sent"
},
"op": "EQUAL",
"value": {
"stringValue": "true",
}
}
}
}
Note that you need to add a single field index to your Firestore DB, as follows:
Note also that, if your field sent is of type Boolean (and not String as shown in your question), you need to use a booleanValue element in your Value JSON element.
I am unable to get this to work for some reason.
I have a Collection called dzs which has some documents with auto generated id's.
I want to query and find a document with a specific email address.
When I try this in Postman, it returns (Error 400 Bad request)
"structuredQuery": {
"from": [{
"collectionId": "dzs",
"allDescendants": true
}],
"where": {
"fieldFilter": {
"field": {
"fieldPath": "email"
},
"op": "EQUAL",
"value": {
"stringValue": "123#123.com",
}
}
}
}
Add the parent collection/document path to the URL:
var URL = "https://firestore.googleapis.com/v1/projects/<your-project-id>/databases/(default)/documents/users/xxxxx:runQuery";
Then make the collectionId "messages" and allDescendents false:
"structuredQuery": {
"from": [{
"collectionId": "messages",
"allDescendants": false
}],
"where": {
"fieldFilter": {
"field": {
"fieldPath": "sent"
},
"op": "EQUAL",
"value": {
"stringValue": "true",
}
}
}
}
Source

How should a "write" request be structured for Firestore REST API (v1beta1)?

Based on the Google Discovery document, and RPC reference, it appears that the :write resource should be available for Firestore database interactions, but performing such a request to my (POST https://firestore.googleapis.com/v1beta1/projects/[my project]/databases/(default)/documents:write) results in:
[
{
"error": {
"code": 400,
"message": "Invalid value (Object), ",
"status": "INVALID_ARGUMENT",
"details": [
{
"#type": "type.googleapis.com/google.rpc.BadRequest",
"fieldViolations": [
{
"description": "Invalid value (Object), "
}
]
}
]
}
}
]
Is this possible? A related SO answer alludes to this being available as a means of field transforms, the same reason I require this, but I cannot construct valid a JSON body to succeed in the request. Currently, variations on the following don't work as expected when trying a minimum successful response:
{
"writes": [
{
"update": {
"name": "projects/{projectId}/databases/[my project]/documents/exampleId",
"fields": {
"example": {
"integerValue": 100
},
"timestamp": {
"nullValue": null
}
},
"transform": {
"document": "projects/[my project]]/databases/(default)/documents/examples/exampleId",
"fieldTransforms": [
{
"fieldPath": "timestamp",
"setToServerValue": "REQUEST_TIME"
}
]
}
}
}
]
}
First of all, note that you should use the v1 version of the REST API, not the betas.
To create a document, you would use the createDocument method, while to update a document you would use the patch one.
For the document creation you should therefore make a POST HTTP Request to the following URL
https://firestore.googleapis.com/v1/projects/<your-project-id>/databases/(default)/documents/<the-desired-collection>
with the following Request body:
{
fields: {
example: {
integerValue: 100
}
}
}
You need to use documents:commit instead of documents:write
also the name field should be in this format:
"name": "projects/projectID/databases/(default)/documents/collectionName/DocumentId"
See this post.

Google Analytics Reporting API V4 Lifetime value requests - invalid dimensions/metrics

I'm trying to make calls the Analytics Reporting API V4 and keep getting back unspecific error messages when trying to use certain dimensions and metrics. For example, I consistently get
{
"error": {
"code": 400,
"message": "Unknown dimension(s): ga:acquisitionTrafficChannel",
"status": "INVALID_ARGUMENT"
}
}
when passing ga:acquisitionTrafficChannel, despite it being documented as a valid dimension. Similarly, I get
{
"error": {
"code": 400,
"message": "Selected dimensions and metrics cannot be queried together.",
"status": "INVALID_ARGUMENT"
}
}
when passing ga:acquisitionSourceMedium (documented here), even when not passing any metrics whatsoever.
Are the docs out of date? Is there some documentation elsewhere about valid combinations of dimensions and metrics?
All the Lifetime Value reports and thus ga:acquisition... dimensions are only valid for App views not web views.
Secondly the cohort/LTV dimensions can only be queried in within a cohort requests for example:
POST https://analyticsreporting.googleapis.com/v4/reports:batchGet
{
"reportRequests": [
{
"viewId": "XXXX",
"dimensions": [
{
"name": "ga:cohort"
},
{
"name": "ga:acquisitionTrafficChannel"
}
],
"metrics": [
{
"expression": "ga:cohortSessionsPerUser"
}
],
"cohortGroup": {
"cohorts": [
{
"name": "cohort 1",
"type": "FIRST_VISIT_DATE",
"dateRange": {
"startDate": "2015-08-01",
"endDate": "2015-09-01"
}
},
{
"name": "cohort 2",
"type": "FIRST_VISIT_DATE",
"dateRange": {
"startDate": "2015-07-01",
"endDate": "2015-08-01"
}
}
],
"lifetimeValue": true
}
}
]
}
The error messages should probably be a bit clearer.
I ran into this problem as well. When I was in the Google Analytics Dashboard, I clicked on Acquisition->All Traffic->Channels and was fooled into thinking that I needed to combine the ga:acquisitionMedium dimension and ga:newUsers metric together.
When I clicked on ga:acquisitionMedium, it said that combining it with ga:newUsers was valid, despite the error that you mentioned in your question! In reality, I just needed to combine ga:medium and ga:newUsers together.
I know this is not the exact query that you were doing, but here is an example of how I queried New Users count where the dimension channel equaled "organic" (note that I am forming the JSON request with Javascript and then using JSON.stringify(req) to send it):
var req = {
reportRequests: [{
viewId: '<Your Google Analytics view ID>',
dimensions: [{ name: 'ga:medium' }],
dimensionFilterClauses: [{
filters: [{
dimensionName: 'ga:medium',
operator: 'EXACT',
expressions: ['organic']
}]
}],
dateRanges: [{ startDate: '2019-11-01', endDate: '2019-11-30' }],
metrics: [{ expression: "ga:newUsers" }]
}]
};
The above query returns 5,654, which is the same as seen in the "Acquisition" section of Google Analytics.
I definitely think the documentation and error message around this could be improved.

Resources