I have written a function that runs every x amount of time and changes a field in a document in cloud firestore if a specific condition applies. However, the function fails due to the function needing an index. I don't know what index I need.
This is the line that gives the error.
const snapshot = db.collectionGroup('people').where('dateEndFunctional', '<', today).where('state', '==', 'active').get()
.then(function(querySnapshot){
querySnapshot.forEach(function(doc){
This is the error message I'm getting
Error: 9 FAILED_PRECONDITION: The query requires an index. You can create it here: https://console.firebase.google.com/v1/r/project/fmis-online-dev/firestore/indexes?create_composite=Ck5wcm9qZWN0cy9mbWlzLW9ubGluZS1kZXYvZGF0YWJhc2VzLyhkZWZhdWx0KS9jb2xsZWN0aW9uR3JvdXBzL3Blb3BsZS9pbmRleGVzL18QAhoJCgVzdGF0ZRABGhUKEWRhdGVFbmRGdW5jdGlvbmFsEAEaDAoIX19uYW1lX18QAQ
If I follow the link, I always get an error, like the image below.
It states:
Loading failed.
An error occured while loading [link]. Try again.
However, eery time I try again, it gives the same error.
I have tried to create a combined index for both of the fields the where clauses are testing, as well as creating two seperate indexes for each of the fields. Both resulted in the same error. What index do I need for this function to properly work?
Thanks in advance for your help!
The link should work, just make sure you are logged into the right account on google cloud.....
For the people that want the answer:
A collection group index on people with state ASC and dateEndFunctional ASC
I don't know why it didn't work before, because I think I have tried this. Perhaps the order of fields matter. The link should take care of that
Related
I have a database of items and need to find the ones with matching tags and fetch them 10 at a time (or any given number).
I've tried the following:
(await admin
.firestore()
.collection('media')
.orderBy('id')
.where('descriptions.tagSet.' + 'Test', '==', true)
.where('descriptions.tagSet.' + 'Test2', '==', true)
.startAt(start)
.limit(10)
.get()).docs;
At first I got an error about no index existing for 'id' so I created one using the link Firestore gave me.
However the next error I got was that there's no composite index for descriptions.tagSet.Test.
Unfortunately tags can be any arbitrary number of things, it doesn't seem feasible to have indexes for each possible tag (unless there's a quick programmatic way to create one?).
I'm not sure how to solve this issue, it seems like something which should be simple enough.
I need to get the Marketo Leads who have changes on "progressionStatus" field (inside membership) with the API.
I can get all the leads related to a Program (with Get Leads by ProgramID API) without issues, but my need is to get those Leads with changes on "progressionStatus" column.
I was thinking to use the CreatedAt / UpdatedAt fields of the Program, so then, get all the leads related to those programs. But I didn't get the accurate results that I want.
Also, I tried to use the GET Lead changes API and use "fields" parameter to "progressionstatus" but that field don't exist.
It is possible to resolve this?
Thanks in advance.
You can get the list of Leads with progression status change by querying against the Get Lead Activities endpoint.
The Get Lead Changes endpoint could sound as a good candidate, but that endpoint only returns changes on the lead fields. Progression status change is not stored on the lead directly, so at the end that won't work. On the other hand the Get Leads by ProgramId endpoint returns –amongst others– the actual value of progressionStatus (program status of the lead in the parent program) but not the “change” itself, so you cannot process the resultset based on that.
The good news is that the progression status change is an activity type and luckily we have the above mentioned Get Lead Activities endpoint (which is also mentioned as the Query in the API docs) available to query just that. This endpoint also allows for filtering by activityTypeIds to narrow down the resultset to a single activity type.
Basically you have to call the GET /rest/v1/activities.json enpoint and pass the values of activityTypeIds and nextPageToken as query parameters (next to the access token obviously). So, first you need to get the internal Id of the activity type called “Change Status in Progression”. You can do that by querying the GET /rest/v1/activities/types.json endpoint and look for a record with that name. (I don't know if this Id changes from instance to instance, but in ours it is the #104). Also, to obtain a nextPageToken you have to make a call to GET /rest/v1/activities/pagingtoken.json as well, where you have to specify the earliest datetime to retrieve activities from. See more about Paging Tokens.
Once you have all of these bits at hand, you can make your request like that:
GET https://<INSTANCE_ID>.mktorest.com/rest/v1/activities.json?activityTypeIds=<TYPE_ID>&nextPageToken=<NEXTPAGE_TOKEN>&access_token=<ACCESS_TOKEN>
The result it gives is an array with items like below, which is easy to process further.
{
"id":712630,
"marketoGUID":"712630",
"leadId":824864,
"activityDate":"2017-12-01T08:51:13Z",
"activityTypeId":104,
"primaryAttributeValueId":1104,
"primaryAttributeValue":"PROGRAM_NAME",
"attributes":[
{"name":"Acquired By","value":true},
{"name":"New Status ID","value":33},
{"name":"Old Status ID","value":32},
{"name":"Reason","value":"Filled out form"},
{"name":"Success","value":false},
{"name":"New Status","value":"Filled-out Form"},
{"name":"Old Status","value":"Not in Program"}
]
}
Knowing the leadIds in question, you can make yet another request to fetch the actual lead records.
I'm struggling to make a (not so) complex query in firebase cloud firestore.
I simply need to get all docs where the id field == a given id.
Then order the results by their date and limit to 10 results.
So this is my current situation
db.firestore().collection('comments')
.where("postId",'==',idOfThePost)
.orderBy('date','asc').limit(10).get().then( snapshot => {
//Nothing happens and the request wasn't even been executed
})
I can get the result only if i don't use the orderBy query but i have to process this sorting for the needs of my application.
Someone has an idea to help me to fix this ?
thanks
You can do this by creating an Index in the firestore.
The first field of the index should be the equality field and the second field of the index should be the order by field.
Given your problem, you would need the following index:
first field: postId, 'asc'
second field: date, 'asc'
Please check the doc. It says
However, if you have a filter with a range comparison (<, <=, >, >=), your first ordering must be on the same field
you can try this code
db.firestore().collection('comments')
.where("postId",'==',idOfThePost)
.orderBy('postId')
.orderBy('date','asc').limit(10).get().then( snapshot => {
.....
})
My Workaround
If you're googling this you've probably realized it can't be done traditionally. Depending on your problem though there may be some viable workarounds, I just finished creating this one.
Scenario
We have an app that has posts that appear in a feed (kind of like Reddit), each post has an algorithmic score 'score' and we needed a way to get the 'top posts' from 12-24 hours ago. Trying to query sorted by 'score' where timestamp uses > and < to build the 12-24 hour ago range fails since Firebase doesn't allow multiple conditional querying or single conditional querying with an descending sort on another field.
Solution
What we ended up doing is using a second field that was an array since you can compound queries for array-contains and descending. At the time a post was made we knew the current hour, suppose it was hour 10000 since the server epoch (i.e. floor(serverTime/60.0/60.0)). We would create an array called arrayOfHoursWhenPostIsTwelveToTwentyFourHoursOld and in that array we would populate the following values:
int hourOffset = 12;
while (hourOffset <= 24) {
[arrayOfHoursWhenPostIsTwelveToTwentyFourHoursOld addObject:#(currentHour+hourOffset)];
hourOffset++;
}
Then, when making the post we would store that array under the field hoursWhenPostIsTwelveToTwentyFourHoursOld
THEN, if it had been, say, 13 hours since the post was made (the post was made at hour 10000) then the current hour would be 10013, so we could use the array-contains query to see if our array contained the value 10013 while also sorting by algorithm score at the same time
Like so:
FIRFirestore *firestore = [Server sharedFirestore];
FIRCollectionReference *collection = [firestore collectionWithPath:#"Posts"];
FIRQuery *query = [collection queryOrderedByField:#"postsAlgorithmScore" descending:YES];
query = [query queryWhereField:#"hoursWhenPostIsTwelveToTwentyFourHoursOld" arrayContains:#(currentHour)];
query = [query queryLimitedTo:numberToLoad];
Almost Done
The above code will not run properly at first since it is using a compound index query, so we had to create a compound index query in firebase, the easiest way to do this is just run the query then look at the error in the logs and firebase SDK will generate a link for you that you can navigate to and it will auto-generate the compound index for your database for you, otherwise you can navigate to firebase>database>index>compound>new and build it yourself using hoursWhenTwelveToTwentyFourHoursOld: Arrays, score: Descending
Enjoy!
same here, it is weird why can't. below is another sample. can't get the result. Hoping firebase can reply about this and update the document.
dbFireStore.collection('room').where('user_id.'+global.obj_user.user_id,'==',true).orderBy('last_update').get().then((qs)=>{
console.log(qs);
});
using other work-around solution is javascript array and array.sort()
I ran into the same issue yesterday on Android. The Callback was just not called. Today I suddenly got an error message. FAILED_PRECONDITION: The query requires an index. It even contains a URL in the error message to generate that index with one click.
It seems that if you want to use orderBy on your data, you need to create an index for that field. The index also needs to be in the correct order (DESC, ASC).
As per firestore document,
If you attempt a compound query with a range clause that doesn't map to an existing index, you receive an error. The error message includes a direct link to create the missing index in the Firebase console.
So just click that link you get in Logcat, it will be redirected to create index page, just create index. It will take some time. after enabling composite index, you will get the result as your requested query.
Stumbled across this looking for help when i found that using the orderBy function didnt work and the documentation still says it does not support it. A bit weird and unclear to be honest, because it does support it so long as you index your Firestore database. For example, this query now works fine for me having set up indexing:
const q = query(docRef, where("category", "==", 'Main'), orderBy('title', 'asc')
Indexing in Firestore
Console Log that even gives you the url to automatically create the index if you try and run with the above command.
Maybe I am missing something, or a later version of Firebase (I am using v9) simply does support it.
I am having an issue with my cube processing. It is failing in production after I made an update to the view used to build one of my dimensions. Here are the steps I took:
The update added a single field and did not touch anything else that was already present.
I updated the cube dimension in BIDS and did a full process of the cube successfully.
I have a scheduled job that reprocesses the cube every 15 mins and it
ran for 16 hours without issue.
Then the job started failing with the error "could not find attribute key".
The key it could not find was the ID column.
The ID column already existed, and is a numeric column.
I double checked to make sure there were no null ID fields, and there weren't.
The ID field was aliased in the data source view, so I tried updating the table definition to use a named query and aliasing the ID field directly in the query.
When I did that, I started getting errors that it could not find the [ID] field ( the
original field name).
Eventually I had to roll the changes back because I couldn't figure out the cause and I had to get production back up. Now, 17 hours later, the processing of the cube is failing again when no changes have been made. I'm getting the error now that the attribute key cannot be found when processing . When I look for the actual ID value that it gives me in the error, I find it in both of the views that make up my dimension and my fact table.
My underlying data source is Oracle 11g. Any help would be greatly appreciated.
Can you process the updated dimension(if you can restore the production db and BIDs project on local environment) ? check if it gives any error in dimension processing. Then try processing the related measure group alone and then try processing the complete cube and OLAP db. You will be able to find the error step more specifically and could be of help for analysis.
Also you can check this..
http://toddmcdermid.blogspot.in/2009/01/ssas-quick-reference-attribute-key.html
From the Transactions doc, second paragraph:
The intention here is for the client to increment the total number of
chat messages sent (ignore for a moment that there are better ways of
implementing this).
What are some standard "better ways" of implementing this?
Specifically, I'm looking at trying to do things like retrieve the most recent 50 records. This requires that I start from the end of the list, so I need a way to determine what the last record is.
The options as I see them:
use a transaction to update a counter each time a record is added, use the counter value with setPriority() for ordering
forEach() the parent and read all records, do my own sorting/filtering at client
write server code to analyze Firebase tables and create indexed lists like "mostRecent Messages" and "totalNumberOfMessages"
Am I missing obvious choices?
To view the last 50 records in a list, simply call "limit()" as shown:
var data = new Firebase(...);
data.limit(50).on(...);
Firebase elements are ordering first by priority, and if priorities match (or none is set), lexigraphically by name. The push() command automatically creates elements that are ordered chronologically, so if you're using push(), then no additional work is needed to use limit().
To count the elements in a list, I would suggest adding a "value" callback and then iterating through the snapshot (or doing the transaction approach we mention). The note in the documentation actually refers to some upcoming features we haven't released yet which will allow you to count elements without loading them first.