How can I measure session/users with the GA4 Measurement Protocol? - google-analytics

The documentation for the GA4 measurement protocol only describes sending events, and notes that the key events for measuring session duration etc. are reserved (automatically provided by gtag/firebase).
I'm attempting to use the measurement protocol in a .NET desktop environment (not firebase/web/android/ios). Sending events works as expected (they show up on the dashboard), but the user count always shows as zero. Sending a session_started event to the debug endpoint does not pass validation (states that it is a reserved event name).
Is it possible to manually send the events required to track sessions or is there another way to get an accurate user count without using firebase/gtag?
Protocol reference: https://developers.google.com/analytics/devguides/collection/protocol/ga4

You need to include the 'engagement_time_msec' parameter with a value of '1', e.g.:
{
"client_id": "XXXXXXXXXXXX",
"timestamp_micros": 1655281222643640,
"events": [
{
"name": "page_view",
"params": {
"page_location": "https://test.com",
"page_title": "Home Page",
"engagement_time_msec": 1
}
}
]
}
This was answered here: https://stackoverflow.com/a/71482548/7205473

Related

GA4 dimensions like language is (not set) for events sent by the measurment protocol

I am using GA4 measurement protocol to record offline conversions to Google Analytics. I have two conversion events: sale and purchase.
Events are being recorded and in the Monetisation overview I can even see the revenue.
But it seems that GA is not linking these events with other dimensions of the user. For example in any a report breaking down by some dimension of the user, like user's language or OS or other dimensions, I get something lie this:
Note the language code for all sale and purchase events is "(not set)".
But in the user explorer I can see that the sale and purchase events are correctly associated with the client_ids.
How can I make GA4 to associate events through the measurement protocol with the events recorded on the website from gtag for the same client_id?
Example of the payload for a sale event:
{
"client_id": "1799600509.1659478566",
"timestamp_micros": "1659855760249000",
"events": [
{ "name": "sale", "params": { "first_billing": 1 } }
]
}
Example of the payload for a purchase event:
{
"client_id": "1799600509.1659478566",
"timestamp_micros": "1659855760249000",
"non_personalized_ads":false,
"events": [
{
"name": "purchase",
"params": {
"items": [],
"currency": "USD",
"value": 1.2
}
}
]
}
The measurement protocol for ga4 is locked down a lot more then the one for UA was.
Things like language code can not be set via the measurement protocol so will appear as not set.
Until the team decides to open up more for the measurement protocol there's not much that can be done about it other then to use the sdks they supply rather than the measurement protocol directly.
You may want to fill out this and add your voice to the request that they add open up for things.
[Feedback Survey] Measurement Protocol in Google Analytics
It seems like the GA4 MP documentation is off. In addition to the client_id under the root object, you should also add the latest client session_id and session_number params (under the params object of each event).
This should fix the linking of the client device and other dimensions.
Checkout this bug report which shows an example of exactly what you're trying to get.
https://issuetracker.google.com/issues/243678686
I think that Kevin from the google team didn't get the point right and rejected that report.

Always get “Cannot parse non Measurement Protocol hits”

I have a little Python program that other people use and I would like to offer opt-in telemetry such that can get an idea of the usage patterns. Google Analytics 4 with the Measurement Protocol seems to be the thing that I want to use. I have created a new property and a new data stream.
I have tried to validate the request and set it to www.google-analytics.com/debug/mp/collect?measurement_id=G-LQDLGRLGZS&api_secret=JXGZ_CyvTt29ucNi9y0DkA via post and send this JSON payload:
{
"app_instance_id": "MyAppId",
"client_id": "TestClient.xx",
"events": [
{
"name": "login",
"params": {}
}
]
}
The response that I get is this:
{
"validationMessages": [
{
"description": "Cannot parse non Measurement Protocol hits.",
"validationCode": "INTERNAL_ERROR"
}
]
}
I seem to be doing exactly what they do in the documentation or tutorials. I must be doing something wrong, but I don't know, what is missing. What do I have to do in order to successfully validate the request?
Try to remove /debug part in the URL. In the example you followed it is not present so it is not quite exactly the same.
we just came across the same issue and the solution for us was to put https:// in front of the URL. Hope this helps.

Mobile data reported in GA Measurement Protocol appear in realtime but not in daily summary

I've been attempting to log activity on a mobile-like device using the Google Analytics Measurement Protocol. All of these attempts have validated using the validation URL, and I can see activity when I look at the real-time reports on the Analytics website. But when I look at the Home or Overview reports for the day - no activity is shown.
The view is set for "All Mobile App Data".
The POST body looks something like this:
v=1&tid=UA-000000000-1&ds=app&qt=1601&uid=uid-zzzzz&t=screenview&cd=Foo&an=Foo%20App%20Name&aid=com.example.foo&aiid=com.example.foo&av=0.0.1&ua=Mozilla%2F5.0%20(Linux%3B%20Android%207.0%3B%20SM-G930V%20Build%2FNRD90M)%20AppleWebKit%2F537.36%20(KHTML%2C%20like%20Gecko)%20Chrome%2F59.0.3071.125%20Mobile%20Safari%2F537.36
The ua field is just a pre-defined string. I found that if I omitted it, the Real Time monitoring listed the hits as desktop hits, although I was in a Mobile report and the ds field was "app".
Am I missing a field that is required? Is there some reason why it is showing up in the real-time report, but not in a daily report? Is there some other way to diagnose why the data is vanishing, or confirm the data is actually being captured?
When i check the debug endpoint the hit is valid
Request:
https://www.google-analytics.com/debug/collect?v=1&tid=UA-XXX-1&ds=app&qt=1601&uid=uid-zzzzz&t=screenview&cd=Foo&an=Foo%20App%20Name&aid=com.example.foo&aiid=com.example.foo&av=0.0.1&ua=Mozilla%2F5.0%20(Linux%3B%20Android%207.0%3B%20SM-G930V%20Build%2FNRD90M)%20AppleWebKit%2F537.36%20(KHTML%2C%20like%20Gecko)%20Chrome%2F59.0.3071.125%20Mobile%20Safari%2F537.36
Response
{
"hitParsingResult": [ {
"valid": true,
"parserMessage": [ ],
"hit": "/debug/collect?v=1\u0026tid=UA-53766825-1\u0026ds=app\u0026qt=1601\u0026uid=uid-zzzzz\u0026t=screenview\u0026cd=Foo\u0026an=Foo%20App%20Name\u0026aid=com.example.foo\u0026aiid=com.example.foo\u0026av=0.0.1\u0026ua=Mozilla%2F5.0%20(Linux%3B%20Android%207.0%3B%20SM-G930V%20Build%2FNRD90M)%20AppleWebKit%2F537.36%20(KHTML%2C%20like%20Gecko)%20Chrome%2F59.0.3071.125%20Mobile%20Safari%2F537.36"
} ],
"parserMessage": [ {
"messageType": "INFO",
"description": "Found 1 hit in the request."
} ]
}
I cannot use one of the mobile libraries from Firebase - this is not one of the platforms they support. I do not wish to pretend this is a web page - there is no associated hostname or path. I do not wish to use Events since I can't do event Behavior Flow, which is one of the things I'm interested in seeing.
I'm aware that it can sometimes take "a day or so" for results to first appear. The site was setup over five days ago at this point, and has received data during that time.
Good thought about the anti-spam setting, however the setting appears to be correct:
I've also tried using GET instead of POST - no change, it still shows the hit in real-time, but then it vanishes.
However, I know that it can record hits permanently. There were two hits from a spammer in Russia that have shown up in the daily report (I wasn't there to see it show up in real-time). I don't know what they did, but would love to find out since it might help figure out how I can add a record.
In the real-time reports, it correctly points out the data center all the hits are coming from. Perhaps that is filtering it out somewhere out of my control?
Try adding Cid I know it says this is an optional parameter but for mobile accounts I belive it may be required.
Client ID
Optional.
This field is required if User ID (uid) is not specified in the request. This anonymously identifies a particular user, device, or browser instance. For the web, this is generally stored as a first-party cookie with a two-year expiration. For mobile apps, this is randomly generated for each particular instance of an application install. The value of this field should be a random UUID (version 4) as described in http://www.ietf.org/rfc/rfc4122.txt.
Example value: 35009a79-1a05-49d7-b876-2b884d0f825b
Although this says it needs to be a UUIDv4, it does work with other UUIDs (I've tested it with a v5, which is a hash against the value used for the uid parameter).

Google Analytics Measurement Protocol and Enhanced e-commerce not capturing revenue

I'm trying to use the measurement protocol to record an offline purchase on our backend system. The problem is, after I send the Measurement Protocol request, the goal conversion is captured correctly in GA's reports, but the revenue earned from the purchase is always displayed as 0.00 in the reports, even though I am sending a value of 500.00 in the revenue parameter.
Here is the request I am sending to the measurement protocol:
https://www.google-analytics.com/collect?
v=1
&tid=UA-XXXXXXXXXX
&cid=XXXXXXX.XXXXXX
&uid=123
&t=pageview
&dh=http://example.com
&dp=order/thank_you
&dt=thank+you
&ti=127210
&ta=offline+conversion
&tr=500.00
&pa=purchase
&pr1id=1864
&pr1nm=Abc+Program
&pr1ca=Software
&pr1pr=500.00
&cu=CNY
&sc=end
&aip=1
The next day, when I check GA's Conversions reports, the goal conversion is tracked successfully, but the e-commerce report always displays the revenue earned for this purchase as 0.00, even though I've specified the price to be 500.00 in the request. I've already tried the following tasks to resolve the problem, but to no avail:
Double-check that Enhanced E-commerce is enabled in GA (it is).
Change &t=pageview to &t=transaction, but this modification results in neither the goal nor the revenue to be captured in GA.
Re-constructed the request in Google's Hit Builder and validated the hit (it is valid)
Sent the request to /debug/collect to validate the hit (it is valid). Below is the response:
"hitParsingResult": [ {
"valid": true,
"parserMessage": [ {
"messageType": "INFO",
"description": "IP Address from this hit was anonymized to 106.111.63.0.",
"messageCode": "VALUE_MODIFIED"
} ],
"hit": "/debug/collect?v=1\u0026tid=UA-XXXXXXXX-X\u0026cid=XXXXXXXXX.XXXXXXXXXX\u0026uid=123\u0026t=pageview\u0026dh=http%253A%252F%252Fmydomain.com\u0026ti=127210\u0026ta=offline%20conversion\u0026tr=500.00\u0026pa=purchase\u0026pr1id=1864\u0026pr1nm=Abc%20Program\u0026pr1ca=Software\u0026pr1pr=500.00\u0026cu=CNY\u0026sc=end\u0026aip=1"
} ],
"parserMessage": [ {
"messageType": "INFO",
"description": "Found 1 hit in the request."
} ]
Here is the screenshot of the above purchase in GA's conversion reports:
As shown above, the revenue from the purchase is 0.00, even though the revenue I sent to GA is 500.00.
Any idea why Google is not tracking the revenue?
Would also appreciate any thoughts on how to better troubleshoot this issue... because Google's /debug/collect script is not providing much insight into the problem. Any idea what other approaches to take for troubleshooting this?
Thanks!
Okay, found the problem. I didn't include the quantity of product purchased in the request data. In order for revenue to be tracked, you not only have to provide the product price (pr1pr), you also have to provide the product quantity (pr1qt).
So, I added the following parameter into the Measurement Protocol request:
&pr1qt=1
And now revenue is being tracked successfully.

Using webhooks with Google Analytics

I'm trying to integrate my CRM with Google Analytics to monitor lead changes (from lead to sell) and so on. As I understood, I need to use Google Measurement Protocol, to receive webhooks from CRM and translate it to Analytics Conversions.
But in fact, I don't really understand how to do it. I need to make some script, to translate webhook code to analytics, but where I need to place that script? Are there some templates? And so on.
So, If you know some tutorials/courses/freelancers to help me with intergrating webhooks with Analytics - I need your advice.
Example of webhook from CRM:
{
"leads": {
"status": {
"id": "25399013",
"name": "Lead title",
"old_status_id": "7039101",
"status_id": "142",
"price": "0",
"responsible_user_id": "102525",
"last_modified": "1413554372",
"modified_user_id": "102525",
"created_user_id": "102525",
"date_create": "1413554349",
"account_id": "7039099",
"custom_fields": [
{
"id": "427183",
"name": "Checkbox custom field",
"values": ["1"]
},
{
"id": "427271",
"name": "Date custom field",
"values": ["1412380800"]
},
{
"id": "1069602",
"name": "Checkbox custom field",
"values": ["0"]
},
{
"id": "427661",
"name": "Text custom field",
"values": ["Валера"]
},
{
"id": "1075272",
"name": "Date custom field",
"values": ["1413331200"]
}
]
}
}
}
"Webhook" is a fancy way of saying that your CRM can call a web based service whenever something interesting happens (i.e. the CRM can "hook" into a web based application). E.g. if a new lead is created you can call an url with the lead details as parameters.
Specifics depend on your CRM, but when you set up a webhook there should be a field to set a url; the script that evaluates the CRM data is located at the URL.
You have that big JSON thing as your example - No real way to tell without knowing your system, but I assume that is sent as request body. So in your script you evaluate the request body, extract the parameters you want to send to analytics (be mindful that you are not allowed to store personally identifiable information) and sent it via the measurement protocol as described in the documentation linked in the other answer.
Depending on the system you might even be able to call the measurement protocol without having a custom script in between (after all the measurement protocol is an url with a few parameters).
This is an awfully generic answer, but then the question is really broad.
I've done just this in my line of work.
You need to first decide your data model on how you would like the CRM data to look within Google Analytics. This could be just mapping Google Analytics' event category, event label, event action to your data, or perhpas using custom dimensions and metrics.
Then to make it most useful, you would like to be able to link the CRM activity of a customer to their online activity. You can do this if they login online. In that case, you can set the cid and/or uid of the user to your CRM id.
Then, if you send in a GA hit with the same cid/uid in your Measurement Protocol hit, you will link the online sessions with your offline CRM activity.
To make the actual record hit Google Analytics, you will need to program something that takes the CRM data and turns it into a Measurement Protocol hit, which is essentially just a URL with the correct parameters. Look here for reference: https://developers.google.com/analytics/devguides/collection/protocol/v1/reference
An example could be: http://www.google-analytics.com/collect?v=1&tid=UA-123456-1&cid=5555&t=pageview&dp=%2FpageA
We usually have this as a seperate process, that fires when the CRM data is written to its database (the webhook in your example). If its a lot of data, you should probably implement checks to see if the hit was sucessful, and caching in case the service is not online - you have an optional parameter that gives you 4 hours leeway in sending data.
Hope this gets you at least started.

Resources