I've read at the link "Set up BigQuery Export" that the import process from GA 360 happens 3 times every day:
"3 files will be exported each day that contain the current day's
data"
Is there a way to make this load happen every hour or a in a shorten period of time?
Related
I use Bigquery Sandbox and export raw event data from Google Analytics to Bigquery. But The number of events 2 places are different.Bigquery much less Google Analytics.
My number of events per day usually ranges from 1 to 200 thousand events meanwhile GA usually ranges 7-10 million evets day. And The storage space I have left is 6.97/10 GB.
I have enabled the BigQuery integration in Firebase Console for more than a week now and I noticed that:
The daily export data is only up to D-2 when it should be D-1
The Intraday dataset usually contains up to 2 days of data
So I want to know if there is any config or where I can find at what time do the daily export of Firebase data takes place? And if possible how do I change it?
What is the time taken by google analytics to start exporting historical data into the google cloud after the linking is complete between Big Query and Google Analytics.
According to Google's documentation:
Once the linkage is complete, data should
start flowing to your BigQuery project within 24 hours. 1 file will be
exported each day that contains the previous day’s data, and 3 files
will be exported each day that contain the current day's data. We will
provide a historical export of the smaller of 10 billion hits or 13
months of data within 4 weeks after the integration is complete.
I'm trying to report conversions to Google Analytics from the server side of an app after a payment is successfully processed. I'm using the Measurement Protocol from the devguides. https://developers.google.com/analytics/devguides/collection/protocol/v1/
The problem is that it successfully shows the goal hits on the real time conversions report, but this are not showed in the normal conversions report as goal completions.
Is there any difference between 'goal hit' and 'goal completion' I'm missing? Or is there any delay on the data that makes into the regular conversions report?
There is a delay. Per documentation it's 24-48 hours (4 hours on a 360 account), although usually the data shows up somewhat faster.
Documentation:
Processing latency is 24-48 hours. Standard accounts that send more
than 200,000 sessions per day to Analytics will result in the reports
being refreshed only once a day. This can delay updates to reports and
metrics for up to two days. To restore intra-day processing, reduce
the number of sessions your account sends to < 200,000 per day. For
Analytics 360 accounts, this limit is extended to 2 billion hits per
month.
I used to think there was long delays in data showing up in GA reports as well, until I discovered a small bug in the GA system in regards to time zones. The system automatically selects the date for you on the reports, but if you live in a time zone like Australia or The Philippines, these can be out of sync, and therefore, the most recent data doesn't show up.
I now always set the date to "Today" or to the last few days, and I find all data comes through within minutes, not hours.
Before Tuesday, March 14th, we saw the data lag in Google Analytics at approximately 1-2 hours. (It was never immediate.) You can see this effect on the Conversions > Ecommerce > Overview page if you search by date and select "today" to "today" (1 day's worth of data)
As of Tuesday, March 14th, we started seeing the lag for this overview report anywhere from 8-12 hours, with an inconsistent aggregation time. For example, it is now 4 PM here on the east coast (EDT), and here is a screenshot of our GA overview tab (I have obscured the revenue number for our privacy). As you can see, there are no numbers after 6:00 AM.
We saw this same effect yesterday (about 8-10 hour lag), and the following day the overview report seemed to fix itself (catch up with all of the aggregated data).
Now, what's more interesting, is that if we either A) Add a "Secondary Dimension" or B) use a "Custom Report", we can see all our data near real-time. For example, if I switch into the Ecommerce > Sales Performance report, then add a Secondary Dimension of "Hour of Day", I can see all my data through 2 PM today (about a 2 hour lag as it is now 4:22 pm as I am writing this)
[
Note that to replicate this I sorted the "Hour by day" column by descending order (showing most recent first.)
Our questions are:
(1) Does anyone know why searching by Secondary Dimension or Custom Report shows us the data in more real-time than just looking at the overview report?
(2) Can anyone else confirm that what used to be a 2-3 hour delay now appears as if it is a 8-12 hour delay, starting on or around March 14th (possibly a few days earlier, this is the first day we can remember seeing this effect)
We are using Universal Analytics (with Enhanced E-commerce) implemented via the newer analytics.js. We are NOT using the older ga.js (we moved away from that about a year ago.)
We are not a GA 360 customer, just a regular free account.
From Google Analytics Help Center article.
Processing latency is 24-48 hours. Standard accounts that send more than 200,000 sessions per day to Analytics will result in the reports being refreshed only once a day. This can delay updates to reports and metrics for up to two days. To restore intra-day processing, reduce the number of sessions your account sends to < 200,000 per day. For Analytics 360 accounts, this limit is extended to 2 billion hits per month.
What it means is that for Standard accounts up to 48h delay is normal, if you have more data it can take more if you have less data it can be faster.
Regarding your observation that certain reports load faster than others this is linked to the design of Google Analytics Backends. Google will generate pre-aggregated tables with common reports to speed up consult and that sometimes can takes longer to process. Other non-common reports can't be answered by aggregated reports so it can be responded by a different backend that already has fresher data. So it is considered normal to see different levels of freshness in different reports.
Google Analytics 360 has fresher data of course.
This other table from the HC article highlights some of the differences and has more info.