I am trying to generate a report using Google Analytics Explore tab using Free Form technique. Few weeks ago I could use Message name, stream name and time to see all the notification name, platform and total no of click. I exported the same to excel file.
but today when I tried to generate the same I couldn't find "Message Name" dimension. Is this field removed from pre defined/custom dimensions from GA? or am I doing something wrong?
My main purpose is to get all list of notifications sent via Firebase.
Any help will be deeply appreciated.
Given that you excluded the obvious issues like using the too-fresh data, the proper way to debug it is to export the data into a sample BQ table, then conduct exactly the same analysis that you're trying to conduct in GA4's explorer. From there, if your issue is with explorer's filters, you will quickly see it.
If, however, you're able to see your event properties in BQ, but not able to get the explorer to display them... Well, Google likely saved quite a lot of money on GA4. UA was pretty expensive. GA4 now introduces all these amazing features like data retention limits, properties' values cardinality bugs, odd inconsistencies between explore's reports and default reports and so on.
For now, the best way to really access your data minus all the artificial limitations of GA4 is to ETL your data from there either through the reporting API or exporting it to BQ.
Related
I'm pretty sure the answer to this question is "no", but I would like to get a definitive answer from an official source, and also understand what my alternative options might be.
Long story short, my app has old data in it that used to include user email addresses as a GET parameter. Those URLs are showing up as unique page view URLs in Google analytics, like this:
I don't want to be recording email addresses in my Google Analytics account (for privacy reasons), and I have fixed the code that was causing this in the first place, but I also want to delete or scrub the old data that currently exists in Google Analytics.
From everything I've read, it doesn't sound like this is possible without completely deleting the property, maybe even the account?
To be clear, I am NOT interested in creating new views that don't include URLs with email parameters in them, or otherwise change the view and not the data. The data needs to be gone and be completely inaccessible to anyone with access to this Google Analytics account.
Here are the options I've come up with:
Delete the property and start over. I'm pretty sure this will
actually delete the collected data, but it's not clear to me if I
would have to actually delete the account itself to achieve that.
Set the data retention time to the lowest possible value (looks like 14 months right now) and wait 14 months for it to go away https://support.google.com/analytics/answer/7667196?hl=en
Perform some kind of magic to get in contact with an actual human at Google who could help me scrub or remove this data.
Does this sound right? Are there options I'm missing? If there's a way to do this through a Google API that would not be a problem.
If this is still a relevant issue. GoogleAnalytics provides a way to delete some data. Universal Analytics https://support.google.com/analytics/answer/9450800?hl=en and GA4 https://support.google.com/analytics/answer/9940393?hl=en&ref_topic=2919631
You are right: changing the data, that you have collected, and Google Analytics have already processed, is not possible. You have the option to make changes during processing with various filters, e.g. Search-and-replace filters, but as it is written in this official support article:
Like all filters, search-and-replace filters only apply to hits
collected after you've applied the filter to the view (filters cannot
change historical data).
Regarding you suggested options:
Deleting a view or property will result in a permanent loss of data after a 35 days period of waiting time. (While this could be undone.) So unless the requirement of scrubbing the collected PII is more important than having your historical data, this should not be a way to go. The same applies to deleting the whole account, so it would be enough to delete affected properties or views.
From the article you have linked as well, you can see, that data retention is about removing user and event level data, and it will not affect the data in aggregated reports. My understanding is, that an already created, page level report will keep showing the page with an email address:
Keep in mind that standard aggregated Google Analytics reporting is
not affected.
I hope these references help you to evaluate your options. Sorry for not being able to come up with a solution, but the basic concept is, as highlighted in this Google article:
Once Analytics processes the data, it’s stored in a database where it can’t be changed
I am looking for options to ingest Google Analytics data(historical data as well) into Redshift. Any suggestions regarding tools, API's are welcomed. I searched online and found out Stitch as one of the ETL tools, help me know better about this option and other options if you have.
Google Analytics has an API (Core Reporting API). This is good for getting the occasional KPIs, but due to API limits it's not great for exporting great amounts of historical data.
For big data dumps it's better to use the Link to BigQuery ("Link" because I want to avoid the word "integration" which implies a larger level of control than you actually have).
Setting up the link to BigQuery is fairly easy - you create a project in the Google Cloud Console, enable billing (BigQuery comes with a fee, it's not part of the GA360 contract), add your email address as BigQuery Owner in the "IAM&Admin" section, go to your GA account and enter the BigQuery Project ID in the GA Admin section, "Property Settings/Product Linking/All Products/BigQuery Link". The process is described here: https://support.google.com/analytics/answer/3416092
You can select between standard updates and streaming updated - the latter comes with an extra fee, but gives you near realtime data. The former updates data in BigQuery three times a day every eight hours.
The exported data is not raw data, this is already sessionized (i.e. while you will get one row per hit things like the traffic attribution for that hit will be session based).
You will pay three different kinds of fees - one for the export to BigQuery, one for storage, and one for the actual querying. Pricing is documented here: https://cloud.google.com/bigquery/pricing.
Pricing depends on region, among other things. The region where the data is stored might also important be important when it comes to legal matters - e.g. if you have to comply with the GDPR your data should be stored in the EU. Make sure you get the region right, because moving data between regions is cumbersome (you need to export the tables to Google Cloud storage and re-import them in the proper region) and kind of expensive.
You cannot just delete data and do a new export - on your first export BigQuery will backfill the data for the last 13 months, however it will do this only once per view. So if you need historical data better get this right, because if you delete data in BQ you won't get it back.
I don't actually know much about Redshift, but as per your comment you want to display data in Tableau, and Tableau directly connects to BigQuery.
We use custom SQL queries to get the data into Tableau (Google Analytics data is stored in daily tables, and custom SQL seems the easiest way to query data over many tables). BigQuery has a user-based cache that lasts 24 hours as long as the query does not change, so you won't pay for the query every time the report is opened. It still is a good idea to keep an eye on the cost - cost is not based on the result size, but on the amount of data that has to be searched to produce the wanted result, so if you query over a long timeframe and maybe do a few joins a single query can run into the dozens of euros (multiplied by the number of users who use the query).
scitylana.com has a service that can deliver Google Analytics Free data to S3.
You can get 3 years or more.
The extraction is done through the API. The schema is hit level and has 100+ dimensions/metrics.
Depending on the amount of data in your view, I think this could be done with GA360 too.
Another option is to use Stitch's own specfication singer.io and related open source packages:
https://github.com/singer-io/tap-google-analytics
https://github.com/transferwise/pipelinewise-target-redshift
The way you'd use them is piping data from into the other:
tap-google-analytics -c ga.json | target-redshift -c redshift.json
I like Skyvia tool: https://skyvia.com/data-integration/integrate-google-analytics-redshift. It doesn't require coding. With Skyvia, I can create a copy of Google Analytics report data in Amazon Redshift and keep it up-to-date with little to no configuration efforts. I don't even need to prepare the schema — Skyvia can automatically create a table for report data. You can load 10000 records per month for free — this is enough for me.
I have two view/profiles linked to my google analytics account. I want to fetch the hourly data for the current day, ie
start date:today
end date: today
with a few filters and dimensions.
Now I am getting the response for one view that means it is possible in google analytics, however for the other view its showing all the values as 0- this applies both to the gui and the api.
Can anyone suggest me how to enable it for the other view as well?
You cannot. Google Analytics needs some processing time. It might be that some data appears immediately, especially on small accounts, but it's not guaranteed and not a thing you can "enable" or count on.
Updated: Okay, that was a dumb answer. Still, there is a processing latency event in GA Premium. It is possible to get realtime data, but that's a different API with limited data (the core reporting API might return data, but no guarantees for that).
But I admit, since your problem is that you do not get data for the whole day yor have a different problem. But with a premium account you should be able to contact your account manager/technical support.
This may be a possible duplicate of this question, but according to all the Google Analytics documentation I really should be able to pull my list of custom segments.
Since I have a very large list of them, it would be suboptimal for me to manually copy the segment ids over one at a time.
I'm following this walk through. Steps to reproduce:
Create a custom segment using date of first session in your Google Analytics account.
Authorize the Google Analytics guide to access your Google Analytics account.
Try their on-page query tester, and inspect whether your custom segment is there.
One thing I've already ruled out was the user that created the segment. I've manually created a segment with the same user that I'm querying the API with and it still does not show. Is there a flag I need to set somewhere to include custom segments?
Edit:
It turns out that it will list some custom segments, but not ones created with date of first session, so this is a duplicate of this question, which means that there is a bug in the Google Analytics API.
There was a bug which is now fixed. So it is now possible to list the Date of Session Segments in the Google Analytics Management API by calling the segments.list() method.
So after days of trying to solve this one I've come to the conclusion that it cannot be done as asked.
There is, however, another way to do it. For every segment set up a daily (or weekly, etc) email report to a email as a TSV. In each email body specify the name of the segment so when you're consuming the emails you can know which segment the attached TSV is for. It doesn't look like the daily reports were designed with segments in mind, since non of the metadata included in the TSV mentions which segment it is for.
From there it's trivial. Connect to the email address using an IMAP client once a day and update the numbers.
Note that the daily email only contains the numbers for that day (not a specified range), so you'll need to first generate the report one time with the historical data to load in.
While hacky, one nice thing about this approach is that it keeps your reports in sync with your (faked through email) api code (provided you match the column headings in the TSV). So, if for example, a new filter is included into a report, the new daily fields will continue to update.
Unfortunately though, the past data won't be reflected in the change.
Obviously this isn't great, but if you are monitoring daily cohorts it's the best you've got if you need to stay with Google Analytics. I have raised this as a bug to the Google Analytics developers, but I haven't heard back as to whether or not they plan to fix it.
I'm doing some complex reports for google analytics and would like to ask you if the following is possible. The client wants to have just organic data for a bunch of metrics. Like pageviews, visitBounceRoutes, etc. The query I ended up with is the following:
https://www.googleapis.com/analytics/v3/data/ga?dimensions=ga:source,ga:medium,ga:keyword,ga:day,ga:month,ga:year&end-date=2013-11-20&fields=columnHeaders/name,rows,totalResults,totalsForAllResults&filters=ga:medium==organic&ids=ga:79067749&metrics=ga:pageviews,ga:pageviewsPerVisit,ga:visitors,ga:avgTimeOnSite,ga:newVisits,ga:visitBounceRate&start-date=2013-10-20
However the response is as follows:
'{"totalResults":0,"columnHeaders":[{"name":"ga:source"},{"name":"ga:medium"},{"name":"ga:keyword"},{"name":"ga:day"},{"name":"ga:month"},{"name":"ga:year"},{"name":"ga:pageviews"},{"name":"ga:pageviewsPerVisit"},{"name":"ga:visitors"},{"name":"ga:avgTimeOnSite"},{"name":"ga:newVisits"},{"name":"ga:visitBounceRate"}],"totalsForAllResults":{"ga:pageviews":"0","ga:pageviewsPerVisit":"0.0","ga:visitors":"0","ga:avgTimeOnSite":"0.0","ga:newVisits":"0","ga:visitBounceRate":"0.0"}}'
Can the dimensions ga:source,ga:medium,ga:keyword be mixed with the above metrics? It seems they can't since if I omit them the API returns an array of values 1 per each day within the specified range.
Where can I find more information about this and what categories are mixable? https://developers.google.com/analytics/devguides/reporting/core/dimsmets just shows all the available metrics but do not explains how they are combined and which one would be valid requests. I'm new at the analytics API and would be great any kind of help or guidance
Thanks a lot
Google Analytics Query Explorer is your friend for playing around with analytics dimensions/metrics/filters ;-)
Try http://ga-dev-tools.appspot.com/explorer/?dimensions=ga:source,ga:medium,ga:keyword,ga:day,ga:month,ga:year&metrics=ga:pageviews,ga:pageviewsPerVisit,ga:visitors,ga:avgTimeOnSite,ga:newVisits,ga:visitBounceRate&filters=ga:medium%253D%253Dorganic&start-date=2013-10-20&end-date=2013-11-20&max-results=100
Some thoughts:
Those dimensions & metrics should work -- maybe there was no organic data recorded during that time range?
Try removing the ga:medium==organic filter and see what your data looks like.
Does the profile you're using (ga:79067749) have any filters on it? If so, maybe try a different profile that has unfiltered data. (Analytics best practices -- make sure you have a profile with no filters applied that captures all data.)
As Mike said, there is no problem with the combination of metrics and dimensions you are using.
If you are entering the URL query directly in the browser problem might be the lack of URL encoding in your query string. For example, you need to convert == to %253D%253D
For example, instead of ga:medium==organic, you need ga:medium%253D%253Dorganic
If you build your query in the Google Analytics Query Explorer as Mike suggests, you can grab the direct link to your report by clicking the link symbol in the upper left: