I recently connected my Google analytics (ga4) property to bigquery. I set up a daily export of the data. The data doesn't seem to get exported as I don't find any tables under the dataset in bigquery. The things I tried:
Delete the link and the dataset and start from scratch again. - Didn't work.
Check for service accounts - all were present and it still didn't work.
Issue: Dataset is getting created but the tables which should contain the raw data are not getting created.
Related
I need to access raw event data stored in Firebase. Thus, I have linked Firebase to Bigquery last month. Bigquery currently creates daily tables containing event data for a month. However, as the Bigquery documentation states, it is not possible to import data prior to linking to Bigquery. Does anyone know how these data can be exported?
The exact dataset from prior to linking cannot be exported in anyway.
The only workaround is that if you want to look up specific information, you can try using the GA4 Data API to fetch the information. Again, this will not give you the entire dataset export.
My team has linked our Firebase and BigQuery projects and set up the intraday table. However, the table is created unpredictably. I was able to use it yesterday (events_intraday_20200701), but it is already noon as of writing this, and there is still no intraday table (events_intraday_20200702) in the dataset. (The regular event tables are there, per usual). In the streaming area of the Firebase console, I can see hundreds of events being generated, but cannot query an intraday table to see them in realtime.
I also struggle to find resources clarifying when the table is created besides "raw event data is streamed into a separate intraday BigQuery table in real-time" from
https://support.google.com/firebase/answer/6318765?hl=en. Are there reasons why the table may not be created, or more details about what time during the day I can expect it to exist?
On a related note, is it true that Web events are not supported for the intraday table?
Thanks!
We have several websites (same website but different topleveldomains) and store the GA360 data in BigQuery.
The GA360 data for each website is stored in the same BigQuery project but in a different dataset (date partitioned).
Example:
GA360 GBQ Project
Website 1
Website 2
Website 3
We are now starting to use Looker and I want to be able to analyze the data. My goal is to be able to create dashboards containing the GA360 data for each individual website but also be able to create dashboards containing he data for all websites combined.
As far as my knowledge goes for both Looker and GBQ the joining can be done in both Looker and GBQ.
Since both GBQ and Looker are quite new to me I am wondering how and where these datasets should be combined in the most (cost)effective way.
So my question is:
What is the best way to combine multiple websites' datasets stored in GBQ to be able analyzed in Looker?
Thanks!
You should be able to join between tables in different datasets by explicitly specifying the fully qualified table names in queries while creating derived tables in Looker. The data processing will happen in BigQuery
After upgrading to Google Analytics in my Firebase project I linked up to a new GA property and had to re-setup my Bigquery integration (after accidentally linking/unlinking my GA account, my integration was turned off). I got it fixed by linking again but now all new data is fed into a new analytics_* dataset.
Since all my queries are referring to the old dataset it would be quite the effort renaming all of them to link to both the new dataset + the old dataset. Is it possible to either:
Change the destination table in the firebase Bigquery export (choosing the old dataset instead of the newly created one)
Somehow merge the two datasets (instead of copying them)
I understood it's impossible to rename datasets which would solve my issue if I could change the name of the new set to the old name and copy the contents of the old set to the new one.
UPDATE
I was able to fix it by:
unlinking the project again
using the firebase management api to link my firebase project again to the original GA property
https://firebase.google.com/docs/projects/api/reference/rest/v1beta1/projects/addGoogleAnalytics#request-body
This started feeding data back into my old property. I subsequently copied the partitioned tables from the newly created property/ table back into the old property (in Bigquery) using the same naming convention (eg. events_20190101) which correctly copied them in the partitioned dataset. I had to append some intraday events as well to the existing partitioned dataset but this solved my problem in the end.
According to the BigQuery export schema document for each single Firebase project linked to BigQuery, a single dataset named "analytics_" is added to your Bigquery project and such ID refers to your Analytics Property ID located in the Analytics Settings in Firebase (Settings -> Integrations -> Google Analytics).
It seems that this ID is generated automatically when you set up a property and it can be manually changed to a custom one.
Additionally, there's no way to merge datasets than copy the data between them. Thus, you could consider using scheduled queries to append the data in your old dataset.
Hope it helps
We have enabled continuous export of Google Analytics data to BigQuery which means we get ga_realtime_sessions_YYYYMMDD tables with data dumps throughout the day.
These tables are – usually! – left in place, so we accumulate a stack of the realtime tables for the previous n dates (n does not seem to be configurable).
However, every once in a while, one of the tables disappears, so there will be gaps in the sequence of dates and we might not have a table for e.g. yesterday.
Is this behaviour documented somewhere?
It would be nice to know which guarantees we have, as we might rely on e.g. realtime data from yesterday while we wait for the “finished” ga_sessions_YYYYMMDD table to show up. The support document linked above does not mention this.
As stated in this help article, these internal ga_realtime_sessions_YYYYMMDD tables should not be used for queries and the ga_realtime_sessions_view_YYYYMMDD view should be used instead for your queries, in order to obtain the fresh data and to avoid unexpected results.
In the case you want to use data from some day ago while you wait for the internal ga_realtime_sessions_YYYYMMDD tables to be created for today, you can choose to copy the data obtained from querying the ga_realtime_sessions_view_YYYYMMDD view, into a separate table at the end of a day for this purpose.