I would like to obtain data via the Google analytics api (Java). Every half an hour once I would be running a script which needs to get me the required variables over the previous half an hour, that is details of the person who has used my site over the last half hour. I do not know if this is possible since the start date and end date parameters can be used only in date format and not in datetime format. How should I proceed with this? Thanks in advance
Google Analytics core reporting API provides 2 dimensions which are the hour and the minute. This should allow you to accomplish what you want, just make sure to wait till the hour and minute you are querying are updated.
Check
https://developers.google.com/analytics/devguides/reporting/core/dimsmets#view=detail&group=time&jump=ga_hour
Related
Background:
I'm having the Firebase analytics data exported to BigQuery. And I'm using cron jobs to crunch data in BigQuery for getting insight.
Problem:
To be able to only crunch delta data i.e. the data that has arrived since last time I ran my cron job I need a way to figure out the time when the data arrived at server, since the event_timestamp is generated at client and can be cached at client before sent.
Insights:
I have laborated with event_server_timestamp_offset (offset) which I thought I could use together with event_timestamp. But I was expecting the offset to only be positive but it can also be negative. And when I look at the MAX and MIN for the offset in the entire exported Firebase analytics dataset and re-calculate it to years instead of microseconds I can get more than 18 years offset.
Query:
SELECT
MAX(event_server_timestamp_offset)/(1000000*60*60*24) max_days,
MIN(event_server_timestamp_offset)/(1000000*60*60*24) min_days
FROM
`analytics_<project_id>.events_*`
Result: max_days=6784.485790436655,
min_days=-106.95833052104166
Question:
How can I figure out the server arrival time for my Firebase exported BigQuery data so I can run cron jobs crunching only delta data?
Can I use event_server_timestamp_offset together with event_timestamp? If so, how?
Best regards,
Daniel
Surprisingly enough, this question not having a clear answer for almost 2 years, I am leaving here the answers I got from the Firebase support team. The format is - question asked followed by the answer of the support staff.
Q1. event_date - The date on which the event was logged (YYYYMMDD format in the registered timezone of your app). Does it mean that the event occurred on that date, or that it was actually collected on that date?
A1. Per documentation, event_date refers to the date on which the event is logged/occurred. Note that event_date is based on the Analytics timezone setting of your Firebase Project.
Q2. event_timestamp - The time (in microseconds, UTC) at which the event was logged on the client. Is it safe to assume that this is the exact timestamp the event occurred on client side (in the app timezone of course)?
A2. Yes, this is based on the device timezone setting. However, event_timestamp may be skewed if the device time is incorrect.
Q3. event_server_timestamp_offset - Timestamp offset between collection time and upload time in micros. This is the main field that causes all the misunderstandings - in our BigQuery table for the year 2020 this field takes values in a range between 5 days and -2 days. I mean how can the colleciton time be 2 days ahead?
A3. The event_server_timestamp_offset field in the export schema is the time difference between when the event took place and the app uploaded it to our server. In other words, this is the estimated difference between the client's local time and the actual time, according to our servers. The values of this field are usually positive, but can be negative as well if the device time setting is incorrect.
Q4. One last question is very important - can we ignore the
event_server_timestamp_offset field and just rely on event_timestamp -
as the exact date and time the event occurred on the clientside (not
collected, not uplaoded, etc). If not- please explain how we can get
the exact datetime of the event occuring on the clientside. But if yes
please let me know why do we need the event_server_timestamp_offset field?
A4. Yes, you may actually ignore it and use event_timestamp alone. However, as mentioned earlier, event_timestamp could be off if the device time setting incorrect, but it shouldn't really affect the bigger picture of your analytics data as cases like this are usually one-off.
We use the event_date as the indicator and load the data once a day.
Can someone give little clarification how to interpret following parameter:
deletionRequestTime: datetime`:
This marks the point in time up to which all user data for the specified end user and Google Analytics property or Firebase project should be deleted.`
If I set it to 1st Jan2018 (GTM), does it delete all user data:
from that date till today (which is how I interpret).. meaning all 2018 data will be gone?
or, (from epoch time) till that date ... meaning all 2016/2017 etc. data is gone and all that remains is 2018 data?
When trying the API > refreshed User Explorer report in GA interface > I notice all-time data seems is gone (giving me impression that this filed is not respected?). But let me wait 72hrs since API request to draw any conclusion..
Thanks for any clarification.
Cheers!
First off i dont think your miss-interperting it I dont think the documentation is clear.
The following is from userDeletionRequest
deletionRequestTime datetime
This marks the point in time up to which all user data for the specified end user and Google Analytics property or Firebase project should be deleted.
Now to me that means that its a point in time that the data should be deleted. as in one day? one minute a time stamp? would this then mean you will need to loop though every hour minute in a day to delete everything.
My current answer is this is confusing. I am going to contact the team for clarification they are in West coast USA we wont get an answer back for several hours. I will updated this when i know more.
Clarification from Google
As per documentation, deletionRequestTime represents a timestamp up to
which all use data will be deleted. In other words, all data from the
beginning of time until the point returned in deletionRequestTime will
be deleted.
I don't believe you can set the deletionRequestTime field. It is set to the time you make the call.
I believe that is why you are seeing this behavior.
I am using the following URL to generate an item for the transaction.
http://www.google-analytics.com/collect?v=1&tid=someID&cid=someID&ti=126540&ta=Another%20type&tr=400.00&tt=0&ea=Purchase&pa=purchase&ec=Enhanced%20Ecommerce&ts=0&cu=UAH&t=event&pr1id=cha803478-WTC%2FNBK&pr1nm=foo%202PP%20crew%20socks&pr1ca=foo&pr1pr=199.00&pr1br=foo&pr1qt=1
After a few days I will need to change it, because any data in it will be not correct. Is there a way to specify date on this URL so that the modification of the transaction was the day it was created?
No. While there is the queue time parameter that allows for a few hours offset there is not way to specify a date in a hit (that would force Google to reprocess a lot of data to integrate hits for previous dates in the reports).
I build application which pull sales data from Amazon and push it to Google Analytics.
I'm "hitting" transaction to Analytics and it's all works fine, the problem is that in Amazon the sales data updated after 48 hours (maximum) so I need to pull the report two days later.
Does anyone know if there's a way to send the date of the transaction? Can't find it in the Measurement Protocol API.
Thank you all, Peace and love :)
Not really. You have the queue time parameter (offset between current time and time the data was tracked), but as it says in the documentation:
Values greater than four hours may lead to hits not being processed.
"May" indicates that this is not a completely fixed limit but I'm pretty sure that 48 hours are not covered (Philipp Walton, a Google engineer who frequents SO once mentioned that the queue time limit is somehow related to timezones and rollover between days, so I'd guess that two days are way to long).
Apart from that there is as of yet no way to set a date for a hit.
You can try posting a Custom Dimension for each event you send, where the dimension is formatted to the date-time:-
e.g.
&t=event&cd1=20170423081321&cm1=24 etc
Only problem is: when you generate a GA custom report, the values are accumulated per day. If anyone knows how to get round that, would love to hear it!
I'm using "Reporting google Analitics API" and I can’t find information about what the last “end date” with data in Analytics is.
For example, let's suppose you want to retrive the last month’s data.
When do you have to perform the query?
The first day of the current month?
...or the second one?
...or maybe the third one?
And only another question: are the returned data for days in pacific time?
Google Analytics API is supposed to have access to the same data you have in the interface.
Google says that data can take up to 24h to process. The time it takes to really update the data depends on the type and size of the account. Small accounts are updated multiple times a day and can have data available in just a few hours. Once you reach 1M hits a month you are moved to a different mode where the data on your account is updated only once a day. Google Analytics Premium customers have updates more often even for large ammounts of traffic.
There's no way to tell through the API what is exactly the time of the last hit processed. You can query the data for today by the hour and see for yourself though.
Usually you don't care and just want to make sure that the data you're querying has been fully processed for that day.
So if you query data for yesterday there's a chance it has not being completely updated, for example if it's midnight the data for yesterday is just a couple minutes ago and probably haven't been completely processed yet. The safest bet in this case is to query data for 2 days ago.
So if today is 2012-06-15 and you want to get 1 month of data a safe approach is to query data with start-date=2012-05-13 and end-date=2012-06-13. This will most of the time give you data for days that have been fully processed, but it's not 100% safe as well. Google Analytics have had outages in the past where data took longer than that to process, these are not usual though. When you get the data out it's really hard to tell just for the API if the data for those days have been fully processed or not, using the 2 days ago isea you just make it more likely that it is.
The days are aggregate following your timezone settings configured on the Google Analytics profile.