I am working on a project to remove gtag based tracking from a website. We have concerns about not being able to implement tracking due to poor network connections. The ideal solution would be to handle conversion tracking on the backend when we receive a purchase request (so as not to send too many requests.) It seems that Google Analytics Measurement Protocol would be the correct tool for the job. However, this (somewhat frustrating) note mentions that "only partial reporting may be available."
We have explored server side GTM, but that still adds unwanted network requests and is not the approach we want to go for. It seems that there are workarounds such as in this question, but they seem pretty precarious. Is there any other api available, or approach that might fit the use case we are looking for?
Related
My website uses Google Ads Conversion and Google Analystics. From time to time, I will see Chrome reports errors in accessing Google related URLs, such as
GET https://www.googleadservices.com/pagead/conversion_async.js net::ERR_TIMED_OUT
A screenshot is put below:
THe error is generated from Google Tag Manager with the following URL:
https://www.googletagmanager.com/gtag/js?id=AW-1071615551&l=dataLayer&cx=c
I am curious why this will occur so frequently since Google has the best server and network connection in the world?
I have seen some report indicating Google Server is the best, for example:
Google CDN is the 2nd fast, see https://www.cdnperf.com/
Another report is not in English so I do not put it here.
I've been working with various Google services for a while now. Have never seen a timed out or a different kind of errorous hit except in cases:
Sudden client connection loss. Especially with mobile traffic.
Second Most popular issue - adblockers. They tend to block tracking too. Often by interfering with the network requests, resulting in network errors that then surface in datadog/dynatrace and whatnot.
When it's impossible for the client to play its encryption part, which is likely either due to there being a middle man sniffing the traffic, or, which is actually more common, the client having a completely wrong date. This should be quite a rare occasion
When there's a firewall-like filtering in place blocking requests to Google servers. Even more rare in my experience.
I've been searching for many hours, but I havent been able to come up with a solution to the following problem:
I want to be able to test the implementation of Google Analytics in an app, in real real time. Using google real-time analytics is insufficient, because you sometimes have to wait several minutes for your new tag to show up.
A solution should be packet sniffing through Wireshark. I've attempted to do this, But even when I do find some analytics related calls, I cant seem to extract any useful information from them.(I have to admit, I dont know enough about networks to understand wireshark's features....)
Another solution should be using a proxy like Fiddler or Charles. I tried both, set up SSL, rooted an android phone to make sure all the traffic was using the proxy, but even then I'm not able to find any google analytics calls from my app.It seems like I'm looking in the wrong place or something, but for websites I do see these GA calls come by.
What am I doing wrong? It's so easy to find this stuff on web, and so far it has seemed impossible to get if from an app. Any help is appreciated.
By default Google Analytics uploads the hits data over SSL connection. It might be difficult for Wireshark to see inside the encrypted connection. The HTTP stack will likely check the certificates and refuse to talk to the wrong server as well. You can change the upload to non-SSL (plain HTTP) by calling tracker.setUserSecure(false):
https://developer.android.com/reference/com/google/android/gms/analytics/Tracker.html#setUseSecure(boolean)
That should allow you to see the uploads with Wireshark or any other packet analyzer. Make sure you don't ship your application with this setting.
The hits will also have some delay before they are uploaded over the network. If you are running on Google Play device (or emulator with Google Play Services) your hits will be dispatched by Google Play within few minutes. When running on non-Google Play devices (or emulator without Google Play Services) you can set the dispatch interval with googleAnalytics.setLocalDispatchPeriod(int seconds) call or explicitly trigger a dispatch with googleAnalytics.dispatchLocalHits()
Say, I'm developing a Windows (if OS is important) application that will be available to download for free and I would like then to collect some usage statistics. In the easiest case - count of application launches. It looks superfluous to maintain a server (e.g. VDS) just for this.
I've been thinking to use Google Analytics for this (manually send requests to GA server). This will probably work, but it is not GA designed for - the idea looks like a hack.
What are the options here?
I don't think this is a hack. It's all just data about user interaction. There is little logical difference between opening a desktop app and clicking a button vs opening a web page and following a link. Both are measurable user actions you can track, aggregate and put on graphs.
In fact, Google provides a lower level HTTP based "Measurement Protocol" that is intended for exactly that.
https://developers.google.com/analytics/devguides/collection/protocol/v1/
From the overview:
The Google Analytics Measurement Protocol allows developers to make
HTTP requests to send raw user interaction data directly to Google
Analytics servers. This allows developers to measure how users
interact with their business from almost any environment
Just put an HTTP request with the correct parameters in your application launch or button click code and it will collect the data. Any data you want to collect.
In other answers to this question there are suggestions like making web services or storing the data locally but why reinvent the wheel? Google Analytics already provides the collecting and reporting tools and it seems like a good solution.
I'm using this link https://www.google.com/reader/api/0/stream/contents/feed/FEEDHERE?output=json&n=20
to fetch feeds using Google's algorithm. As you can see I'm not adding any other parameters, just fetching the returned data in JSON format. My app will be heavily used hopefully and if I send a lot of requests to this link, will Google block my access or something?
Is there anything I can include, like userip, url for my app (so if they have problem to just contact me) or something else?
The most basic answer to your question is that Google will change its Terms of Service whenever it likes, and you've got no say in the matter. So if it's allowed today, it might not be allowed tomorrow, at Google's whim.
On this issue, though, you seem fairly safe. From the Terms of Service (these is the general document, since Reader doesn't seem to have a specific one):
Don’t misuse our Services. For example, don’t interfere with our Services or try to access them using a method other than the interface and the instructions that we provide.
Google provides RSS and Atom. They provide these feeds, so I assume they expect that they'll be used. They don't say that it's a misuse to point someone else at those feeds, so it looks OK for now, but they could add such a clause at any time.
All online services are subject to the terms and conditions of the providers of those services. So, as others have said, they may be ok with your use today, but they can change their mind any time down the line. I doubt including a URL or email or contact info will help anything, because when these services change, they don't notify every user of the service, they just announce the change publicly, and usually they give several month's notice in order to give users a chance to adapt their applications, but this is not standardized or enforced so there is no guarantee. One example would be the fairly recent discontinuance of the Google Finance API (for which no replacement has been announced).
The safest approach would be to design your app such that this feature that uses google's functionality is decoupled as much as possible from the rest of your app, so that, when or if the availability of the service changes (ie it's no longer available at all) you can adapt your app to use some other source for the feeds with minimal impact to the rest of the app. Design for change and plan for the worst.
What i'm wondering is, what kind of behaviour does google analytics show when a ddos attack occurs? Any theories?
My theory would be that an effective DDoS platform/script would not include anything as heavyweight as a JavaScript engine, and that therefore the DDoS activity would not show up in Google Analytics at all.
The point of a DDoS attack is to overwhelm the server with a flood of requests. Any CPU cycles that are spent evaluating JavaScript in the response that the server sends back are cycles that could better be used churning out more requests to the server. I would fully expect a properly executed DDoS attack to not waste time parsing the response from the server, or even reading it off of the underlying socket, let alone interpreting and executing and JavaScript that may be embedded in the markup or fetching scripts and other resources from domains other than the target server.
Of course, this does not preclude the possibility of an exceptionally naive DDoS attack implemented using web frameworks and libraries that do evaluate embedded JavaScript. Such an attack would not (or rather, should not if you've implemented your server code correctly) be very effective, but it would likely generate a spike in Google Analytics traffic.
It depends on the way that the DDOS is implemented. If it's simply an executable distributed to multiple machines, making simple HTTP queries using native TCP sockets, then Google Analytics wouldn't notice anything at all: because the JavaScript that gets returned would never be executed.
However, other sorts of DDOS attacks could leverage actual browsers distributed across many machines. For instance, if you could hack the Yahoo home page and insert an <iframe src='takemedown.com'> into it, you could easily DDOS "takemedown.com". In this particular scenario, GA would certainly detect the impressions, and because (depending on the scenario) there might be an HTTP referrer tag, you could possibly run a report in GA that could pull out the suspicious impressions.
But there are other similar scenarios that wouldn't leave any particular footprints. For instance, if you could hack Lady Gaga's twitter account, you could send out a link to her 16MM followers, and a significant number would immediately click on it: and since most of those clicking on it would probably be doing so from within a separate app, there wouldn't be any referrer tag, and no particular way of identifying the requests.
In other words, it all depends, but it's probably not a terribly useful avenue to investigate. In many (most?) scenarios, GA wouldn't even recognize the impression; and in many others, wouldn't have any reasonable way of picking out the good impressions from the bad.
It will show up 100% some significant peaks in google analytics , simply because there are huge number of requests from multiple sources having huge bounce rate !
When a HTTP DDoS attack occurs the attacker is either using several (thousands) of computers to do so. Sometimes, it's also done with servers. When they make the request, they don't render the javascript or anything - they simply in most cases just make a GET request to the webpage.
So no, it shouldn't really have an impact on GoogleAnalytics
Well, I'm also searching this kind of information, but I have some considerations about the answer:
You will probably not see the attack itself with Google analytics, but you should see the results, I mean, a DDoS is a "distributed deny of service", so, if the service is effectively denied, then you should see a flat line on the graph on Google analytics.
It depends how the bot works, but here's what happened to my website:
Google Analytics real time report for the monk
As well as the increase in traffic you will likely see your bounce rate go sky high and average time on page significantly drop - which I'm sure can have a negative impact on SERPS.
For me it coincided with a Google update so first I put it down to that, but I started getting a lot of traffic to the root page, terms, and privacy, with many prefixed with /?m=0 which is in itself odd (and I'd love for someone to shed light).
The attack caused a great deal of timeouts and was painful to fix:
In short, I hooked up CloudFlare, then created Security -> WAF rules to challenge countries where I was receiving most of the bot traffic. I also switched on the basic bot attack mode (there's a more effective super bot attack mode with the paid subscriptions).
The other interesting point of note was why was my site subject to a DDOS attack. I wish I knew, but at a similar time to when the attack started I was approached by someone who enquired about buying the website. Possibly a tactic to get me to sell it/sell it cheap.