I am trying to kick off Google Cloud Function when two tables ga_sessions and events have successfully created in BigQuery (these tables can be created anytime in the gap of 3-4 hours).
I have written the following log stackdriver sink/log router to which Pub/Sub topic is subscribed (which in turn kick off google cloud function). However, it is not working. If I use sink/router individually for ga_sessions and events it works fine but when I combine them together then it doesn't work.
So my question is how do I take two different events from log stackdriver, combined them together & pass them to pub/sub topic
protoPayload.serviceData.jobCompletedEvent.job.jobConfiguration.load.destinationTable.datasetId="my_dataset"
protoPayload.serviceData.jobCompletedEvent.job.jobConfiguration.load.destinationTable.projectId="my-project"
protoPayload.authenticationInfo.principalEmail="firebase-measurement#system.gserviceaccount.com"
protoPayload.methodName="jobservice.jobcompleted"
protoPayload.serviceData.jobCompletedEvent.job.jobConfiguration.load.destinationTable.tableId:"events"
protoPayload.serviceData.jobCompletedEvent.job.jobConfiguration.load.writeDisposition:"WRITE_TRUNCATE"
protoPayload.serviceData.jobCompletedEvent.job.jobStatus.state:"DONE"
NOT protoPayload.serviceData.jobCompletedEvent.job.jobConfiguration.load.destinationTable.tableId:"events_intraday"
protoPayload.serviceData.jobCompletedEvent.job.jobConfiguration.load.destinationTable.datasetId="my_dataset"
protoPayload.serviceData.jobCompletedEvent.job.jobConfiguration.load.destinationTable.projectId="my-project"
protoPayload.authenticationInfo.principalEmail="analytics-processing-dev#system.gserviceaccount.com"
protoPayload.methodName="jobservice.jobcompleted"
protoPayload.serviceData.jobCompletedEvent.job.jobConfiguration.load.destinationTable.tableId:"ga_sessions"
NOT protoPayload.serviceData.jobCompletedEvent.job.jobConfiguration.load.destinationTable.tableId:"ga_sessions_intraday"
Thanks in advance for your help/guidance.
The trick here is to create a metric that would actually show 1 when both conditions are met.
Try creating a new logs based metric and swithc to "query editor". There you can create your own metric using MQL language.
To be able to create a single metric from two "metrics" you need to use something like this:
{ fetch gce_instance :: compute.googleapis.com/instance/cpu/utilization ;
fetch gce_instance :: compute.googleapis.com/instance/cpu/reserved_cores
} | join | div
And here's some useful info on how to create an alerting policy using MQL.
The code for the alerting policy can look like this:
{ fetch gce_instance :: compute.googleapis.com/instance/cpu/utilization ;
fetch gce_instance :: compute.googleapis.com/instance/cpu/reserved_cores
}
| join | div
| condition val() >1
This is just an example to demonstrate that it is very likely possible to create the metric to monitor the creation of BigQuery tables but you have to test it yourself.
Related
Below is a picture of the firebase database tree . I have items a, b , c. I want the value of totalresult = a + b + c
My requirement is : As the value of a or b or c gets updated , it should get automatically reflected in the totalresult item value.
Is there a way to set in firebase to do it automatically instead running a piece of code everytime to add these and update in firebase
Am able to run a piece of code to add these and update the value in totalresult. But I have to run it manually every time, which is not an ideal solution
There isn't an internal way to do this in firebase realtime database.
That said, while you still have to write code, you can write a firebase function to trigger on updates to those fields, and then apply the update to total result. This will be automatic instead of manual, as the trigger will happen for every event on the database.
Documentation is here for how to create such a trigger (probably using the "onWrite" event).
Of course, there are a few things to be aware of:
There will be a period of time while the function is running that the data is not updated. In other words, you should be tolerant of inconsistencies. (You will likely also want to do the actual writing to the total using a transaction)
You need to be careful to not run the function (or exit early) when "tot/total result" is being updated, or you could get into an infinite loop of functions (it'd be best to have the result object elsewhere in your tree)
I’m trying to create an app where I need to match two random users from my Firebase Database. The problem I have is that I‘m not sure how to connect every user of the database to random pairs:
As far as I thought the user might press a button to signal that he‘s ready, so a child of his UID like 'searching' turns from false to true. By receiving a DataSnapshot the user can see if another user is searching too. Then I thought of turning the 'searching' state to false for both users and create a new UniqueID to connect them.
But like that it‘s thought pretty short, I think and would cause some problems. Can anyone give me a hint on how to organize that searching process?
Actually using a special node like searching and setting it to true and false will be a good idea.
You can use orderByChild() to order all the users with searching node set to true and get the uid of any of those users.
To make the process random you could make a call to users, take the ids given back to you, and pick one at random with a basic random number.
If after making a call to users you have 10 user ids in an array you would want to get a random number between 0-9 and then make a call to firebase with the userId.
I don't believe Firebase has any built in code for this. So this would be the best thing you can do to achieve this feature in your app.
To give a basic idea, your database structure should look something like this:
--rootNode
|
|
-- uid1
| |
| - searching
| - other Fields
|
-- uid2
.
.
.
I'm trying to get all the documents that have my phone number as the fromNumber or the toNumber. My call right now is:
database.collection('documents').where('fromNumber','==',myPhoneNumber).get().then();
Instead of making 2 calls, one to check the fromNumber and the second one to check the toNumber, how can I check both at the same time and in the same .get()?
Btw: tested this code:
database.collection('documents')
.where('fromNumber','==',myPhoneNumber)
.where('toNumber','==',myPhoneNumber).get()
.then();
But it checks if both are true, it's an AND instead of the OR I'm looking for.
According to the official documentation, there a are some query limitations, when it comes to Firestore:
Cloud Firestore does not support the following types of queries:
Logical OR queries. In this case, you should create a separate query for each OR condition and merge the query results in your app.
As suggested in the documentation, you should create two separate queries and merge the result cliend side.
I tried to automate test to validate GA events.
My approach is :-
List item use google analytics real time reporting api.
Before the test ends i will hit this api and collect the last 30 mins data
This data will be a huge chunk of formattedJson string
and in this string i will search my GA events which was supposed to push.
This approach seems to be in-efficient.
My issue is to find the analytics data which corresponds to test user.
Each user has unique user id, hence, i am trying for making the request such that api returns me the filtered data based on some custom dimension "custom:user_id='user_unique_id'" .
Is it possible to get all data having condition e.g 'custom:user_id="XYZ"'.
Please advise, how to get all ga events data for a specific event label / custom dimension ? Also, does it support dimensionFilterClauses like reporting api v4 ?
We can do it by filtering.
e.g rt:eventCategory==ProductPage
earlier i was using quote, rt:eventCategory=='ProductPage' which wasn't supported.
My Firebase database is more than 800mb large and with more than 100.000 objects (news articles).
What I want to do is to fetch just the first 50 objects (most recent) and then to sort the objects got from the result of the first query according to child parameters.
So, for example, when the page is loaded, I need angularfire / firebase to load just first 50 objects and to stop loading the rest of objects in database. Then, I want to filter out just these 50 objects (articles) based on node category music.
So far, my first query seems to be fine (but if there is better way to ask firebase to load X objects and to stop, I would appreciate). But, the second part, I can’t figure it out because firebase throw an error.
The error is:
Query: Can't combine startAt(), endAt(), and limit(). Use limitToFirst() or limitToLast() instead
Here is my sample code:
var myArticlesRef = new Firebase(FIREBASE_URL + 'articles/');
var latestArticlesRef = myArticlesRef.limitToFirst(20); // is this the recommended way to ask firebase to stop
var latestArticlesOrder = latestArticlesRef.orderByChild('category').equalTo(‘Music’); // <- how to do something similar?
var latestArticlesInfo = $firebaseArray(latestArticlesOrder);
$scope.latestArticles = latestArticlesInfo;
console.log($scope.latestArticles);
This should work:
var query = myArticlesRef.orderByChild('category').equalTo(‘Music’).limitToFirst(20);
So you're asking Firebase to return the first 20 articles in the Music category.
While it is common to think of queries like this when coming from a relational/SQL mindset, I recommend that you consider this alternative data structure:
articles_by_category
music
article1: { }
article2: { }
article3: { }
...
technology
article4: { }
...
So instead of storing the articles in one big list, store them by category. That way to access the articles about music, you only have to do:
var query = ref.child('articles_by_category').child('music').limitToFirst(20);
With this approach the database doesn't have to execute any query and it can scale to a much higher number of users.
This is something you'll see regularly in a NoSQL world: you end up modeling your data for the way your application wants to query it. For a great introduction, see this article on NoSQL data modeling. Also read the Firebase documentation on data modeling and indexes.