Context: I wish to collect real-time, hit-level clickstream data from a website that will ideally pushed into an AWS kinesis data stream (or elsewhere). This is to build a machine learning software for a client that already has Adobe analytics and Google analytics implemented on their website.
Question: Instead of building a tracking code that tracks the clickstream data and pushes it to our own AWS Kinesis data stream or some form of storage under our control, the goal is to piggy-back the tracking code (already implemented by Adobe and Google analytics) that sends a duplicate of the tracked data directly into an AWS kinesis stream. I understand that there are ways to export certain a certain granularity of data from google and adobe analytics (once it's already arrived in these platforms) but these export options don't really satisfy the requirement of raw, unprocessed, hit-level, real-time clickstream data.
Is it possible to modify the tracking code in a way that a duplicate of the tracked data can be redirected to a custom route, ideally AWS Kinesis?
As I understand it, the analytics tracking code is essentially Javascript code (tag) embedded in the website that imports a method from a URL that does the event tracking and the uploading. If I could redirect the data at this point in the already implemented analytics stage then I could get the real-time raw data that I need.
I haven't figured out a way to create and redirect a duplicate of the tracked data by modification of this tag. I doubt the imported method is customisable if it is imported from an adobe or google automatically created URL?
Any detailed answers or even links to information would be helpful,
Thanks.
Related
For some reason PowerBI does not accept GA4 as a connection, so my data is stuck in GA4.
PowerBI does support Universal Analytics (UA), but I cannot find a way to convert my GA property.
Is there a way to do this or should I start over again and create a new property as UA?
PowerBI currently doesn't support GA4 as a destination because the underlying api is not product ready yet. See the response in the microsoft power BI community here: https://community.powerbi.com/t5/Desktop/Power-BI-Google-Analytics/td-p/1679530
For information about the api's and their current state:
This is a link to the api documentation the plugin is using (for universal analytics): https://developers.google.com/analytics/devguides/reporting/core/v4
This is a link to the api documentation of GA4 (which is still in beta): https://developers.google.com/analytics/devguides/reporting/data/v1
Regarding a solution for your issue:
In GA4 is it possible to export data to big query, which can be processed there and imported to PowerBI: https://learn.microsoft.com/en-us/power-bi/connect-data/desktop-connect-bigquery
The export only works from the moment you enable it, so it is not possible to access the data this way before the enablement.
Second option is using data studio to connect to your GA4 data to be able to access the data you have measured before.
First off as everyone says you cant change the type of your Google analytics account if it was created as UA then its UA if it was created as GA4 then its GA4. There is also no way to exxport data.
Now lets look at the underlying cause of your problem Google analytics connector in Power BI Desktop rely on the Google Analytics Core Reporting API this API is used to extract data from Google Analytics UA accounts. There is a very big difference between the underlying data structure of a UA account and a GA4 account they are not completable in any way shape or form.
Thats why the Google analytics team is currently developing the Google analytics data api This api has the ability to connect to Google analytics GA4 accounts and extract data. At the time this post was created Power BI Google Analytics The Google analytics data api had just been released and was still in alpha test phase as you can see from the change log this API is still under development. I haven't checked but it looks like it may even be out of beta now. (TBH I am not really sure of the exact date it was released publicly as I have been in the alpha test group for a long time.)
However that being said a lot of the client libraries used by this api are still under development as well Google .Net Client library for Google Analytics Data API for example is still in beta. At the time that post was written i am almost 100% sure the client library had not been released.
This means that any power bi connector is probably not going to be released until both the API and the client library are considered stable. I know the team is working hard on the API and I Haven't seen any breaking changes in a very long time.
So to the user who added a bounty to this looking for an updated answer.
IMO I don't think you have long to wait before everything is stable enough for applications like the Power BI to start appearing that support GA4. That is assuming that someone creates one.
You can't covert GA4 to UA (or UA to GA4).
In your case you can create a new UA Property and start to track in it:
Any ideas how I get historical data out of Google Analytics?
My problem here is I didn't realise the Firebase / Analytics export to BigQuery wasn't retrospective for history, so I only have data from when I turned on the integration and I need everything from when the Firebase App was launched. The main data I need is events with their parameters.
I have tried a few things:
Connect PowerBI to Google Analytics. I can authenticate and it only
shows "Default Account for Firebase" with no children - it doesn't
show the apps underneath that account.
https://ga-dev-tools.appspot.com/query-explorer/ has the same
problem (not showing child properties).
I can load the
properties and therefore analytics dataset using Google Data Studio,
but this does not have the functionality I require that is in Power
BI. I also couldn't locate the event parameters which I need either.
Any ideas on how to export the data?
You can try to use Google Analytics Data API to export report programmatically: https://developers.google.com/analytics/devguides/reporting/data/v1
However the data are not raw like with BigQuery Export. So if you want raw data there are no solution for the historical data.
I trying to find a better solution than google analytics to create dashboards for visually representing custom data.
My use case is that I need something to track and record custom data that is not passed in through custom web events into google analytics.
I was able to find a hacky solution to the problem initially by running a periodic function over my data on firebase and sending it as custom events to google-analytics. (I mention the approach in this answer. The linked blog post gives further details.)
The problem that I faced was that it wasn't always reliable as although the function did run successfully every day, the data was not shown in the google analytics dashboard.
The data I to visualize is gathered as further insights by writing basic code on existing data
I am aware that it is possible to integrate google analytics for various use cases like even the smallest button click, but I am trying to either send custom events( raised outside of the application by running cron jobs or scheduled functions ) through some API call or maybe directly save and update the data in some other place where there is direct access to visualization
Does anyone know a better solution?
The Google Analytics Measurement Protocol allows developers to make HTTP requests to send raw user interaction data directly to Google Analytics servers, so you can use it to send your data generate outside your application:
https://developers.google.com/analytics/devguides/collection/protocol/v1
My client send me spreadsheet that contains all the transactions. Now I have to push those sheet data in Google Analytics, So How can I push those data in Google Analytics? Should I have to create a PHP Script that will read from spreadsheet and push to Google Analytics?
Depending upon how this data looks. You should look into the Measurement protocol which will allow you to send additional hits to Google analytics make sure that you check the qt parameter which will allow you to set the time that the hit came in. There is no library for this you will have to code these calls yourself.
In the even that this is custom data then you could also upload it directly. You can use the Google api php client library for this to help you with your calls.
In either case yes your going to have to read the speedsheet yourself and then create the code in which ever language you choose to upload it.
I have a question to the following case. We want to track a content platform using google tag manager. However, not every time the platform is online but GTM would send data to our internal server. Therefore our concern is if data collected during this offline period will be kept or if we loose them.
Do you know if there is some period during which data collected offline through Google Tag Manager is kept and once it gets online then it is sent to Google Analytics?
Thank you,
Lukas
No, that is not how Google Tag Manager works. GTM for web is basically a javascript injection engine. It bundles your configured tags,triggers and variables with a selector engine and injects that into your page. There is no serverside component that stores data.
I'm sure one could come up with a solution to your problem - e.g. store your data with localstorage in the browser, poll you server to see if it is available, and when it's online send the data with a queue time parameter to Google Analytics. However that has nothing to do with GTM.
Having said this, it is hard to understand your use case - if your server is offline, then where does the data come from ?
If you have an offline PWA app (with a Service Worker), you can use the Workbox Google Analytics module to handle the collection of data, and to report it upstream when your site comes back online.
This module has a service worker fetch handler that intercepts the calls that you would make with analytics.js or gtag.js, and stores your data locally in IndexDB in the event that the call fails because it is offline.