missing data firebase sync to bigquery sandbox - firebase

I use Bigquery Sandbox and export raw event data from Google Analytics to Bigquery. But The number of events 2 places are different.Bigquery much less Google Analytics.
My number of events per day usually ranges from 1 to 200 thousand events meanwhile GA usually ranges 7-10 million evets day. And The storage space I have left is 6.97/10 GB.

Related

Firebase real time database by time period

I would like to ask about firebase Google Analytics (GA)
Can real-time firebase see the data by last 3 hours or more ( Currently, it is limited to last 30 minutes).
How to see the reports filtered by hours? ( I would like to find which period has more active users, let says 6:00PM-7:00PM)

Does it cost to Backfill Data from linking BigQuery to Google Analytics?

I'm currently in the middle of the process of linking my Google analytics data to Big Query and the following note caught my attention when selecting the view to pick.
If this is the first time that you have linked this view, then data
will be backfilled for the smaller of 13 months or 10 billion hits.
Its a little unclear to me whether this 13 months of data will have costs in importing once I linked to BigQuery.
The process itself doesn't have a price component for back filling, but the storage it will occupy in BigQuery adds to your storage costs.
If you don't want old data, make sure you archive/remove/delete it.

Google Analytics Big Query export

What is the time taken by google analytics to start exporting historical data into the google cloud after the linking is complete between Big Query and Google Analytics.
According to Google's documentation:
Once the linkage is complete, data should
start flowing to your BigQuery project within 24 hours. 1 file will be
exported each day that contains the previous day’s data, and 3 files
will be exported each day that contain the current day's data. We will
provide a historical export of the smaller of 10 billion hits or 13
months of data within 4 weeks after the integration is complete.

Yesterday's data from BigQuery

We are having some issues pulling yesterday's Google Analytics data from BigQuery. Can anyone explain at what point a previous day's GA data is finalized?
There is some explanation here of the intraday tables, but it's not very clear:
https://support.google.com/analytics/answer/3437719?hl=en
To get previous day data do you need to need to use the intraday tables at all? Do you have access to the fully processed dataset at 8am local time? Or is it 8 hours after the current day UTC+14:00 (etc)?
I had a similar question and asked their support, this is the reply:
"According to this Google Analytics documentation , it states that '1 file will be exported each day that contains the previous day’s data, and 3 files will be exported each day that contain the current day's data'. In such, the minimum time that the data from Google Analytics to be exported to BigQuery was 8 hours. Although Google Analytics can be linked to BigQuery, the availability of data depends on how it was served by Google Analytics 360."
But based on experience, it's really a minimum time. Sometimes there are delays of 4-5 hours.
My team has been pressing Google's support for providing SLA's for BigQuery dump, so they updated the documentation:
This feature is not governed by a service-level agreement (SLA).
In practice we are experiencing regular delays anywhere between 2 to 12 hours.

Google Analytics show real time goal hits but not on the conversions report

I'm trying to report conversions to Google Analytics from the server side of an app after a payment is successfully processed. I'm using the Measurement Protocol from the devguides. https://developers.google.com/analytics/devguides/collection/protocol/v1/
The problem is that it successfully shows the goal hits on the real time conversions report, but this are not showed in the normal conversions report as goal completions.
Is there any difference between 'goal hit' and 'goal completion' I'm missing? Or is there any delay on the data that makes into the regular conversions report?
There is a delay. Per documentation it's 24-48 hours (4 hours on a 360 account), although usually the data shows up somewhat faster.
Documentation:
Processing latency is 24-48 hours. Standard accounts that send more
than 200,000 sessions per day to Analytics will result in the reports
being refreshed only once a day. This can delay updates to reports and
metrics for up to two days. To restore intra-day processing, reduce
the number of sessions your account sends to < 200,000 per day. For
Analytics 360 accounts, this limit is extended to 2 billion hits per
month.
I used to think there was long delays in data showing up in GA reports as well, until I discovered a small bug in the GA system in regards to time zones. The system automatically selects the date for you on the reports, but if you live in a time zone like Australia or The Philippines, these can be out of sync, and therefore, the most recent data doesn't show up.
I now always set the date to "Today" or to the last few days, and I find all data comes through within minutes, not hours.

Resources