Firebase Analytics connected to BigQuery and the BQ table schema is described here:
https://support.google.com/firebase/answer/7029846
I would like to find out how each event record can be uniquely identified.
Originally I thought that a combination of a
user_pseudo_id and event_timestamp
is to be unique. But I found out that it is not unique...
I added: event_date, event_name, event_previous_timestamp, stream_id, etc. into the 'group by' clause, but nothing helps.
Can anybody advise me, what makes the event record unique, please?
We are using Google Advertising ID as unique device ID. A user may have logged on to your app with multiple devices but using the same account, so in this case user_id is not unique, user_pseudo_id for the same device will change if he/she re-installs the app. Only assumption out here is that the user has not intentionally reset his/her GAID.The GAID field can be found under event_params with Key as gaid in BigQuery. Hope this helps!
Related
I have a firebase connection with BigQuery and i want to extract the daily user engagement information that is present on the analytics dashboard.
Firebase Analytics Dashboard Daily User Engagement
I´ve already tried to find the numbers using EVENT_NAME filter as 'user_engagement', 'screen_view' and EVENT_PARAMS_KEY as 'engagement_time_msec', 'engaged_session_event'.
With the filter mentioned previously i couldn´t even get near from the firebase values.
Somebody knows how to reach them
According to the documentation, user engagement data collection is triggered periodically by Firebase SDK, while the app is running in the foreground. Saying this, for user_engagement we might be looking over the events, filtering for the entry where event_params.key = "engagement_time_msec" and fetching the event_params.value.int_value from there.
SELECT event_timestamp, user_pseudo_id,
(SELECT value.int_value FROM UNNEST(event_params) WHERE key =
"engagement_time_msec") AS engagement_time
FROM `firebase*.events_*`
WHERE event_name = "user_engagement"
I noticed there are thousands of events duplicated in the events tables of BigQuery (in an integration with Firebase).
My definition of duplicated is: 2 or more events that share the same data in all these fields:
event_timestamp, event_name, user_pseudo_id, app_info.id, device.advertising_id
It happens for automatically collected events, and also custom events. I found some of the parameters that could differ from one result to the other are (what make those events different):
event_server_timestamp_offset, geo.continent, geo.country
I guess there is no reason for a duplicated event at the same moment, same user, same app, same device, but one event is geo.continent=America and the other geo.continent=Asia.
Any thoughts why this is happening? Thanks in advance.
Google's explanation is that Firebase data duplication in BigQuery is mostly related to network issues on the client's side that cause events to be buffered and sent twice.
However there is a way to deduplicate these events by using event_server_timestamp_offset. This field is difference between the time the event was sent to Google's server and when it was received.
This means that given the same event_timestamp, event_name and user_pseudo_id you could take only the event with lower event_server_timestamp_offset to have a correct result.
You can also safely delete duplicates records from your event table.
Sorry I can't share sources for this because the answer came from Google Analytics support, as I was encountering the same issue.
We use the QUALIFY clause for deduplication Firebase events in BigQuery:
SELECT
*
FROM
`project.dataset.events_*`
QUALIFY
ROW_NUMBER() OVER (
PARTITION BY
user_pseudo_id,
event_name,
event_timestamp,
TO_JSON_STRING(event_params)
) = 1
Qualifying columns:
- name: user_pseudo_id
description: Autogenerated pseudonymous ID for the user -
Unique identifier for a specific installation of application on a client device,
e.g. "938642951.1666427135".
All events generated by that device will be tagged with this pseudonymous ID,
so that you can relate events from the same user together.
- name: event_name
description: Event name, e.g. "app_launch", "session_start", "login", "logout" etc.
- name: event_timestamp
description: The time (in microseconds, UTC) at which the event was logged on the client,
e.g. "1666529002225262".
- name: event_params
description: A repeated record (ARRAY) of the parameters associated with this event.
I have been trying to understand the concept behind google client id, set as fullvisitor id in BigQuery Export Schema
I know that to define a session, a unique combination of fullvisitorid and visitid has to be found.
However,I couldn't find a good explanation regarding how google defines this id, and how permanent it is across sessions.
Thank you very much!
I'm not sure about what you didn't understand, but the client id is generated on the user's browser when the GA script is initialised and stored in the cookie.
After that, every hit sent by that instance of the script [that has the cookie] will have the cid value as the client id.
When implementing GA you can also set that value yourself if you want. Although there are better ways for doing that.
Here's some documentation on the cid and here is how the session is defined (thus how the session id is derived during data collection)
I have a question when using the User ID feature to track users across devices/browsers.
If I activate the transfer service from GA to Bigquery in which field will I get that information. Would it use the same fullVisitorId column or a new one will be created?
thanks,
mike
The field is called userId.
More info can be found here. Please note that for this data to appear in BigQuery, you must make sure to select the User ID view when setting up the GA/BigQuery link.
The GUID will be new field and can be found as a custom dimension.
I'm under the impression fullVisitorId being just a hash of clientId, there should be one-to-one mapping between the two. But here, I've a situation where few of the fullVisitorId are mapped to two different client Id (we're collecting GA Client ID into User scoped custom dimensions)
Is that possible ? under what circumstances?
Thanks for any clarification on this
Cheers!
[edit: ] attaching screenshot
You may be interested in reading about the Google Analytics schema for BigQuery. Some of the relevant parts are:
fullVisitorId: The unique visitor ID (also known as client ID).
visitId: An identifier for this session. This is part of the value usually stored as the _utmb cookie. This is only unique to the user. For a completely unique ID, you should use a combination of fullVisitorId and visitId.
So client ID and full visitor ID are synonymous, and if you want a unique ID for a particular visit, you should use a combination of fullVisitorId and visitId.