Some GTM wisdom needed.
We use measure protocol to send some transactions to Google Analytics & using GTM too in "thank you page". We are not happy with that.
Is there a way we can send Data Layer Variables from backend using some GTM call? I want to know how you solve that using GTM & dataLayer
My first idea is to try emulate measure protocol using something similar to https://developers.google.com/tag-manager/devguide#adding-data-layer-variables-for-devices-without-javascript-support
Context: Some payment methods notify transactions to my backend system via a background HTTP post method call from their backend system, not an interactive browser (where server-side code do the measure protocol request)
I would like to add more clearance into terminology you are using.
Measurement protocol is HTTP request to google-analytics.com containing all key=value pairs in its query string or in POST data.
Use documentation on Goodle Developers. This is direct input into Google Analytics.
DataLayer is usualy JavaScript object containing key : value pairs Check documentation here. This keys are then accessible in GTM interface to be paired with pre-prepared TAGs in their particular fields. In case of non-javascript solution are these parameters translated into Image Request and sended in different way to Google Analytics.
Adding variables for devices without JavaScript is about specifying JS dataLayer object in flatten mode like GET parameters. This mean you do not emulate Measurement protocol, you "simulate" dataLayer.
My suggestion:
If you have your GTM setted in standard manners (by Google Documentation), then use "Adding Data Layer Variables for Devices without JavaScript Support" and it will save you a lot of time.
If you are using custom JavaScript implementations, do it with Measurement Protocol by Image Request.
Related
Is it possible to use GA4 Measurement Protocol to send events to Google Analytics and view and analyze them in the GA dashboard without using gtag.js or any other front-end script? The use case would be that some events are being sent to my server and I will just push these events to GA through the API.
One thing that makes me think is that the official Measurement Protocol API say:
In order for an event to be valid, it must have a client_id that has already been used to send an event from gtag.js. You will need to capture this ID client-side and include it in your call to the measurement protocol. In send an event to your property, we use "client_id" as the client_id. You will need to replace this with a real client_id that comes from gtag.js.
(https://developers.google.com/analytics/devguides/collection/protocol/ga4/verify-implementation?client_type=gtag)
That suggests that only events that have a valid client_id that originate from gtag.js will be counted.
I did some experimenting with randomly generated client_ids and what I discovered was that I was able to see my events in the Realtime section of the GA4 console (the Event count by Event name section), but all the other sections would be empty and the Users in last 30 min section would always show 0.
Can someone please explain to me why it's zero and if such a use case is valid at all? Thanks
tl;dr
You can use any value in client_id, as long as it uniquely identifies the user (we use a GUID/UUID), but it seems like you also need to send a value in user_id. We use the same value for both.
Also, you need to add the 'engagement_time_msec' parameter to get any user metrics to register.
Longer answer:
We're trying to do the same, i.e. send all events to the GA4 Measurement Protocol from the server, so that it is not dependent on the current user's GDPR cookie settings.
We currently do this for a Universal Analytics property with no issues, but it seems that Google is trying to prevent this in future, by restricting the scope of the Measurement Protocol in GA4, whilst forcing everyone to move to it by July 1st 2023. See the documentation at https://developers.google.com/analytics/devguides/collection/protocol/ga4#full_server-to-server, where it states:
While it is possible to send events to Google Analytics solely with
measurement protocol, only partial reporting may be available. The
purpose of measurement protocol is to augment existing events
collected via gtag, GTM, or Firebase.
We have something working with GA4, in that the events are being registered on the GA4 property correctly, using a client id that is just a GUID/UUID that we define in our own site cookies. So, any value can be used in the client id, as long as it uniquely identifies the user. The same value is used to populate the user_id parameter.
When sending events, the realtime event details were showing on the GA4 dashboard, but user metrics were not until we also populated the 'engagement_time_msec' parameter, as described in https://stackoverflow.com/a/71482548/7205473
We still have issues with things like getting the user location and the platform details, which previously were automatically populated by passing the IP address and the User Agent, but which seem to no longer work in GA4.
We were also passing page load timing events through the Measurement Protocol, but again, these features seem to have been removed in GA4.
It is possible to use GA4 directly without gtag.ja or the Firebase SDK. Its not supported, so it takes some work. We have this working in a desktop app reasonably well. There a couple things that need to be done.
As stated elsewhere the "engagement_time_msec" param must be set using the "_et" parameter. This is the number of milliseconds between now and the previous event.
The client id "cid" has a specific format; it should be:
"randomNumbers(10).unixTimeStamp()"
The session id "sid" format is:
"randomNumbers(10)"
The "_z" parameter needs to be set. I think this is a cache buster. Looking deep into the gtag.js code it is a url safe base64 encoding of "CCD", which always results in the value "ccd.v9b"
The page hash parameter "_p" can be set to this; not totally sure its correct but it works.
"randomString(3).randomString(3)"
Set the "User-Agent" HTTP request header in whatever framework/lib you are using. GA4 uses this to determine many things including Operating System. You will need to create a fake user agent based on the local device information. This is what we use for a Windows 11 x64:
"myco.testapp/4.0.0 (Windows NT 10.0; Win64; x64)"
The IP will be taken fromn the web request which is where the geolocation data comes from.
Since a full working example is worth 1,000 words of documentation; here is a "test" event with a parameter "animal=dog":
https://www.google-analytics.com/g/collect?cid=0078745494.1659679529&_et=364&_p=pfJ.Aev&seg=1&sid=2678664821&tid=G-???&ul=en&v=2&_z=ccd.v9b&en=test&ep.animal=dog
It's possible to extract outgoing GA4 request from a GTM container debug/preview view and map any GA4 (automatically collected and custom) event.
Example page_view request URL:
https://www.google-analytics.com/g/collect?v=2&tid=G-XXXXXXXXXX>m=3oes1i1&_p=1545013558&_dbg=1&cid=P%2FdJWyULMwcT21TMrzn7pZdlNt%2FxtttGVqGUmqNYbhc%3D.1669722847&ul=nl-nl&sr=2560x1440&uaa=x86&uab=64&uafvl=Not_A%2520Brand%3B99.0.0.0%7CGoogle%2520Chrome%3B109.0.5414.75%7CChromium%3B109.0.5414.75&uamb=0&uam=&uap=Windows&uapv=10.0.0&uaw=0&_s=1&_uip=XXX.XXX.XXX.X&sid=1674235261&sct=1&dl=https%3A%2F%2FXXXXXXXXXX.com%2F%3Fgtm_debug%3D1674235654105&dr=https%3A%2F%2Ftagassistant.google.com%2F&dt=OM%20test&jscid=XXXXXXXXXX.1669722847&seg=1&en=page_view
Tip: use Postman to analyse and experiment with parameters
regardless of the platform used to make a call the Measurement Protocol, you should use a client id generated by gtag.js, or the app ID if using Firebase.
I have settled my google tag manager as from the guide.
I have got in place a page that redirect just after the tag gtm-load is collected into the dataLayer. It's actually a "transition" page that should redirect instantaneously.
In this way I make sure google-analytics.com/r/collect is called, as I can see from the network ta of my browser, seems the only way to do so for an "instant redirect page".
However I need to test it from the back-end side.
Is there any way to have a feedback from Google Analytics about the data is actually sent? For example something like google-analytics.com/get/data/lastEntry, so that I can use a restAPI to check it out?
As far as I know, google analytics provide only metrics through a web page, and no actual data sent to.
Moreover there are some Rest API here but they are only for configuration purpose.
you might pass a JavaScript function as a hitCallback parameter of analytics send command and it will be called right after hit data were sent:
https://developers.google.com/analytics/devguides/collection/analyticsjs/field-reference#hitCallback
Here's an example of how to use it with GTM alongside another useful feature of eventcallback
https://www.simoahava.com/gtm-tips/hitcallback-eventcallback/
Hope this helps.
There is a realtime API in Google Analytics, so what I have done for testing is to call my test URL with utm campaign parameters attached. Then I made a call to the realtime API and filtered by my custom campaign.
The realtime API is fairly limited (no session based values, obviously, you cannot test custom dimensions etc), but at least this tells you if your hit has registered in GA.
I am having some pages where I need to track page views and get data for reports. So I need to use some tracking events without using gtag.js.
I have tried solutions like GIF Request Parameters
But this solution was there in ga.js and its a legacy
The format used in your linked documentation is deprecated since Google introduced Universal Analytics in 2012.
However the current solution to send requests to Googke Analytics without a library, the Measurement Protocol, works broadly the same way. You have an endpoint (google-analytics.com/collect) and add the necessary parameters as described in the reference, depending on what kind of interaction you want to track. The endpoint returns a 200 http status and a gif (no matter if the request has actually been logged in analytics, this just confirms that you actually hit the server).
Is it possible to get the info about website traffic without Google Analytics, for example?
For tracking without Javascript you can use the Measurement Protocol, i.e. you send a request to the Google Analytics Server with query parameter that specify the type of interaction (pageview, event etc) and the data you want to track.
General info is found in the protocol reference, a list of valid parameters is here, and if you want to test your tracking requests you can use the hit builder, which allows you to assemble, validate and send a hit to Google Analytics.
As for tracking without cookies that probably won't work very well. You need a persistent id to assemble hits into sessions, and sessions into visits. Such an ID is usually stored in a cookie (your "no javascript" requirement means that things like local storage are out of the question). You can either decorate all your links with a client id and use that to persist the parameter from page to page, or you use some sort of server-side browser fingerprinting.
All in all this might be somewhat less trivial than you assume, especially if you do not only want to track pageviews but also events that do not load a new page.
I'm working on a web application that uses node.js + express. Our team has decided that all related pages must use a single URL and be distinguished only by its method.
So, for example: we have a group edition page. GET /groups gets the html, which includes the edition form, and POST /groups saves the changes.
The issue is that I only desire to track the POST /groups request. However, on the Google Analytics configurations, I only found URL-related tracking, nothing about specific HTTP methods.
Can this be done?
If you have implemented GA via the standard client-side tracking code then no, GA will not be aware of the http method (JavaScript has no access to http headers). It should however be trivial to render a Javascript if condition into the code depending on the request method.