I have a firebase connection with BigQuery and i want to extract the daily user engagement information that is present on the analytics dashboard.
Firebase Analytics Dashboard Daily User Engagement
I´ve already tried to find the numbers using EVENT_NAME filter as 'user_engagement', 'screen_view' and EVENT_PARAMS_KEY as 'engagement_time_msec', 'engaged_session_event'.
With the filter mentioned previously i couldn´t even get near from the firebase values.
Somebody knows how to reach them
According to the documentation, user engagement data collection is triggered periodically by Firebase SDK, while the app is running in the foreground. Saying this, for user_engagement we might be looking over the events, filtering for the entry where event_params.key = "engagement_time_msec" and fetching the event_params.value.int_value from there.
SELECT event_timestamp, user_pseudo_id,
(SELECT value.int_value FROM UNNEST(event_params) WHERE key =
"engagement_time_msec") AS engagement_time
FROM `firebase*.events_*`
WHERE event_name = "user_engagement"
Related
I'm looking to report on the outcome of a Firebase A/B test using Google Analytics / BigQuery tables.
To do this I need to identify the test group each user was in.
The Firebase documentation (https://firebase.google.com/docs/ab-testing/ab-concepts) suggests experiment and variant memberships are stored as user properties on every GA event in the normal GA event tables.
However, I can't find this information associated with GA events in the GA interface or BQ clickstream tables - I can only see it in the Firebase tables.
Where should I be looking? I've checked the Event Action, Event Category, Event Label etc. dimensions.
Use in your query to identify it
userProperty.value.string_value where userProperty.key = 'firebase_exp_000'
In Google Analytics (GA4) GUI, under the traffic acquisition report, it is possible to see app visits split by source.
However, I cannot see the same info in BigQuery.
According to [GA4] BigQuery Export schema documentation traffic_source is the "Name of the traffic source that first acquired the user". I have checked and in fact it seems that the value of traffic_source changes only when the user_pseudo_id changes, which means it persists until the app is reinstalled.
Scenario:
User A installed the app with a google-play campaign, then visits the app a second time following a Google cpc campaign and then a third time following a push notification.
Question:
In BigQuery, how can I see that the second visit was from cpc and the third from the push notification?
The traffic_source is indeed persisting over multiple sessions and it captures only the source of the first app install.
To get attribution at visit level, you need to use the parameters inside the firebase_campaign event, for example:
SELECT
user_id,
user_pseudo_id,
event_date,
event_timestamp,
event_name,
(SELECT value.string_value FROM UNNEST(event_params) WHERE key = 'source') AS source_,
(SELECT value.string_value FROM UNNEST(event_params) WHERE key = 'medium') AS medium_
FROM `project.table.events_*`
WHERE _TABLE_SUFFIX BETWEEN '20210316' AND '20210319'
AND event_name IN('firebase_campaign')
AND user_id = 'XXXXX'
I noticed there are thousands of events duplicated in the events tables of BigQuery (in an integration with Firebase).
My definition of duplicated is: 2 or more events that share the same data in all these fields:
event_timestamp, event_name, user_pseudo_id, app_info.id, device.advertising_id
It happens for automatically collected events, and also custom events. I found some of the parameters that could differ from one result to the other are (what make those events different):
event_server_timestamp_offset, geo.continent, geo.country
I guess there is no reason for a duplicated event at the same moment, same user, same app, same device, but one event is geo.continent=America and the other geo.continent=Asia.
Any thoughts why this is happening? Thanks in advance.
Google's explanation is that Firebase data duplication in BigQuery is mostly related to network issues on the client's side that cause events to be buffered and sent twice.
However there is a way to deduplicate these events by using event_server_timestamp_offset. This field is difference between the time the event was sent to Google's server and when it was received.
This means that given the same event_timestamp, event_name and user_pseudo_id you could take only the event with lower event_server_timestamp_offset to have a correct result.
You can also safely delete duplicates records from your event table.
Sorry I can't share sources for this because the answer came from Google Analytics support, as I was encountering the same issue.
We use the QUALIFY clause for deduplication Firebase events in BigQuery:
SELECT
*
FROM
`project.dataset.events_*`
QUALIFY
ROW_NUMBER() OVER (
PARTITION BY
user_pseudo_id,
event_name,
event_timestamp,
TO_JSON_STRING(event_params)
) = 1
Qualifying columns:
- name: user_pseudo_id
description: Autogenerated pseudonymous ID for the user -
Unique identifier for a specific installation of application on a client device,
e.g. "938642951.1666427135".
All events generated by that device will be tagged with this pseudonymous ID,
so that you can relate events from the same user together.
- name: event_name
description: Event name, e.g. "app_launch", "session_start", "login", "logout" etc.
- name: event_timestamp
description: The time (in microseconds, UTC) at which the event was logged on the client,
e.g. "1666529002225262".
- name: event_params
description: A repeated record (ARRAY) of the parameters associated with this event.
Firebase Analytics connected to BigQuery and the BQ table schema is described here:
https://support.google.com/firebase/answer/7029846
I would like to find out how each event record can be uniquely identified.
Originally I thought that a combination of a
user_pseudo_id and event_timestamp
is to be unique. But I found out that it is not unique...
I added: event_date, event_name, event_previous_timestamp, stream_id, etc. into the 'group by' clause, but nothing helps.
Can anybody advise me, what makes the event record unique, please?
We are using Google Advertising ID as unique device ID. A user may have logged on to your app with multiple devices but using the same account, so in this case user_id is not unique, user_pseudo_id for the same device will change if he/she re-installs the app. Only assumption out here is that the user has not intentionally reset his/her GAID.The GAID field can be found under event_params with Key as gaid in BigQuery. Hope this helps!
After a solid load of searching I'm still not able to find a way to automate the export of Google Firebase Analytics data.
I need to import a few KPIs in an existing Google Sheet every month.
Isn't there any option to query the Firebase API through the Google Sheet and GET the KPI I need into a cell somehow every month?
There are ways (and even plugins) to do that for Google Analytics data. So I was sure it should be possible for firebase aswell.
To import Firebase analytics data into a Google Spreadsheet, you'll need to go through BigQuery. There's a Google Spreadsheet add-on called OWOX but I couldn't get it to work, so here's how to do it via creating a Google Apps Script:
In Firebase under Project settings (under the gear icon) -> Account Linking : find the BigQuery tile and sign up if you haven't already. This DOES require moving to the Blaze pay-as-you-go plan, but there's a generous amount of free usage in this tier so you're likely to avoid costs, but you do need to provide the usual billing info.
After signing up for Blaze, you'll go back under this Account Linking section and you'll see that BigQuery is now linked to your apps. Click on "Manage Linking" and you'll likely see "Syncing dataset" for awhile... like several hours (note, this page doesn't seem to refresh on its own so you'll have to refresh manually to check status). Once this is complete, you'll be able to access your data via BigQuery on the Google Spreadsheets side.
In your Google Spreadsheet, select Tools -> Script Editor. There's a good starter script here: https://greenido.wordpress.com/2013/12/16/big-query-and-google-spreadsheet-intergration/
In Google Script editor, select Resources -> Advanced Google services and ensure you turn on BigQuery API. Now your script will have access to BigQuery data.
For my needs, I was just trying to get a count of some events in Firebase, so I wrote a function to invoke multiple queries like this:
function runQueries() {
var sql;
// fetch bigQuery data for Firebase app Create Wildlink (bottom row app)
sql = "#standardSQL\nSELECT count(event_dim) as event_count FROM `PROJECTID.TABLENAME.app_events_*`, UNNEST(event_dim) as event WHERE event.name = 'target_event_name'";
runQuery(sql, 1, 1, 'Label 1');
// fetch bigQuery data for Firebase app Share Extension (top row app)
sql = "#standardSQL\nSELECT count(event_dim) as event_count FROM `PROJECTID.TABLENAME.app_events_*`, UNNEST(event_dim) as event WHERE event.name = 'target_event_name_2'";
runQuery(sql, 3, 1, 'Label 2');
Browser.msgBox("Finished updating the results");
}
And then I changed the runQuery function to accept and use those params:
function runQuery(sql, insertRow, insertCol, label) {
...
sheet.getRange(insertRow, insertCol + 1, resultCount, tableRows[0].getF().length).setValues(resultValues);
sheet.getRange(insertRow, insertCol).setValue(label);
A few important notes about this method:
Events don't appear to be imported retroactively. From the moment you tie Firebase to BigQuery you start getting event data, but it doesn't import any old data.
Events are imported into day-parted tables. Note in the example above the wildcard in the table reference to span tables.
The example above uses StandardSQL (which was new to me). Note the #standardSQL\n in the above SQL strings. That changes from the default mode (LegacySQL). I ran into challenges using legacy SQL to get at the data.
Let me help you to deal with the OWOX BI BigQuery Add-on and simplify steps 3 to 6 from the answer above.
First of all, you really need to link BigQuery to your apps (Ian wrote good instruction how to do that in steps 1 and 2).
Then:
Add OWOX BI BigQuery Reports Add-on to your Chrome browser,
Open your Google Sheet, run add-on (Add-ons -> OWOX BI BigQuery Reports -> Add a new report),
Provide the add-on with the access to your BQ tables,
Select your Google BigQuery project in a drop-down list,
And create a new query (once again, Ian provide you with a good example of the query)
You can find some more details about OWOX BI BigQuery add-on in our Help Center. And feel free to write to us via email (bi#owox.com) or in chat - we’ll be happy to answer any of your questions.
Best regards, Eugene