I have a desktop application that uses CEF for displaying a built in web page.
I have customized the User-Agent (Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) DesktopApp MyAppName/1.0 (MyApp release 1.0 stamp 99999) Safari/537.36) but Google Analytics only shows as Safari 537.36.
Are browsers outside the known universe of real browsers supported by GA when looking up browsers used? I would like this to instead be MyApp instead of Safari or Chrome.
I just looked at my browser reports and unless "aaa", "ddd" and "this is a test ua" are actually existing browsers it would seem that GA also tracks unknown user agents.
More seriously, the measurement protocol (on top of which Google Analytics is built) allows for a user agent override parameter (&ua), which probably would make very little sense if you could only pass in known browser names (after all this is meant so support e.g IoT devices which might not even have a real user agent name).
Related
I am exploring the Google PageSpeed insights api and there in the response I see a tag called:
{
...
lighthouse.userAgent:'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/84.0.4147.140 Safari/537.36'
...
}
Docs mention userAgent="The user agent that was used to run this LHR."
https://developers.google.com/speed/docs/insights/rest/v5/pagespeedapi/runpagespeed#LighthouseResultV5 What does that mean? How is this performance aggregated by running on all browsers?
PS: This is for Desktop version.
What does that mean?
This lets you know what browser was used to run the test.
It is useful for if you believe that there is an issue with Lighthouse (a bug in your report) so you can test it directly on the same browser that Lighthouse uses.
There is also the "environment" object which contains how Lighthouse presented itself (it sent a header saying "treat me like this browser") to the website that was being tested. (lighthouseResult.environment.networkUserAgent)
This is useful so you can check your server isn't blocking requests for that user agent etc.
It is also useful for checking your server logs to see what requests Lighthouse made etc. etc.
See the Wikipedia page for user agent for more info on user agents
How is this performance aggregated by running on all browsers?
As for your second question it doesn't quite make sense, but it has no impact on performance unless your server does something different for that user agent string if that is what you mean.
I've been working on open tracking by attaching tracking pixel to an email which is sent via PHP web application that I've been developing. The first problem I encountered was marking email as open just it was sent. I found help here: False open trackings using SES and gmail where I found link to the article describing how to prevent triggering opens by Google bot by checking user agent. I created the following function to check if I deal with the bot:
function isGoogleBot(string $userAgent): bool
{
$googleBotUserAgent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246 Mozilla/5.0';
return $userAgent === $googleBotUserAgent;
}
I also read some articles about caching images by Gmail but unfortunately I'm still dealing with some problems.
I did the following test in Chrome, Firefox, Edge and Safari:
I sent an email via application that I've been working on and then opened it in Gmail web client.
Then I went to the application to see open tracking statistics.
I repeated above test twice to make sure every time I get the same result.
With the last email I checked what is happening with subsequent opens of the same email.
Here are my observations:
In every browser except Firefox when I open email for the first time
an extra open is being made so I see 2 opens in the statistics. In
Firefox only 1 open is numbered.
On subsequent opens only 1 open is being added to the total opens
statistics every time I open the email. The exception is also Firefox
where total opens counter is not changing. No matter how many times I
open the email the total opens count is equal 1.
Is there any solution how to make reliable open tracking in every browser when email is opened in Gmail?
Recently I put some hidden links in a web site in order to trap web crawlers. (Used CSS visibility hidden style in order to avoid human users accessing it).
Any way, I found that there were plenty of HTTP requests with a reference of browsers which have accessed the hidden links.
E.g : "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"
So now my problems are:
(1) Are these web crawlers? or else what can be?
(2) Are they malicious?
(3) Is there a way to profile their behaviour?
I searched on the web but couldn't find any valuable information. Can you please provide me some resources, or any help would be appreciated.
This is a HTTP user agent. They are not malicious at all. It's following the pattern, for example Mozilla/<version> and so on. A browser is a user-agent for example. However, they can be used by attackers and this can be identified by looking at anomalies. You can read this paper.
The Hypertext Transfer Protocol (HTTP) identifies the client software
originating the request, using a "User-Agent" header, even when the
client is not operated by a user.
The answer to your questions are, in order:
They are not web crawlers. They are user agents. Common term for a web developer.
Generally they aren't malicious but they can be, as I suggest, look at the paper.
I don't understand what you mean by profiling behaviour, they aren't malware!
Can someone please explain what "Safari (in-app)" means in Google Analytics under Audience | Technology | Browser & OS?
For the Google Analytics for our website, we suddenly started to see significant traffic from this source.
It sounds like it just means that visitors are coming to us through browsers embedded within apps (e.g. like a web viewing control) except that there doesn't seem to be reason why we should be getting such traffic and so suddenly.
We went from zero traffic from this source to almost 40% of our traffic in only two days for no apparent reason! We haven't done anything that can explain this sudden new source of traffic (e.g. we haven't released any apps ourselves) that point back at our website. We're hoping that, if we can find out what "in-app" actually means, we'll be able to understand this traffic.
Thank you
I did some tests on a separate page available only for me using IOS 8.2 and updated 1Password and Facebook App:
General Safari browser is reported as Safari 8.0:
Mozilla/5.0 (iPhone; CPU iPhone OS 8_2 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12D508 Safari/600.1.4
1Password is reported as Safari 6.0:
Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_3 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10B329 Safari/8536.25
Facebook App is reported as Safari (in-app)
Mozilla/5.0 (iPhone; CPU iPhone OS 8_2 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Mobile/12D508 [FBAN/FBIOS;FBAV/27.0.0.10.12;FBBV/8291884;FBDV/iPhone7,2;FBMD/iPhone;FBSN/iPhone OS;FBSV/8.2;FBSS/2; FBCR/Play;FBID/phone;FBLC/en_US;FBOP/5]
Tested on 2015.03.31 (yyyy.mm.dd)
Visitors using built-in search box within Safari are now having their searches sent though Google SSL Search, if they use iOS 6. Searches done with Google and through Safari's search box will continue to grow, as people continue to upgrade to iOS 6 (currently for iPhone 4/4S highest version is 6.1.3, for iPhone 5 is 6.1.4). Also Google updated their user-agent parsing code and UA like this:
1Password
Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) 1Password/4.1.2 (like Version/10B143 Mobile/6.1 Safari/8536.25)
will be categorised like Safari (in-app).
Read more in the articles below and do not trust Google Analytics fully.
Hope this helps!
iOS 6 change Google traffic from Safari
Safari (in-app) as reported by Google Analytics
As you concluded, many people think those represent traffic from iOS apps opening links not on the browser but inside the app (WebView). However, I haven't being able to find a concrete Google Documentation on that.
As for the sudden increase, it is not unusual. Just yesterday I saw how HTC Hero (#1 Device in visits) went from 25.000 visits/day to 0 in two days, and Elocity A7 A7 Internet Tablet from 0 to 30.000 visits/day at the same time. Seems, Google keeps updating its User-Agent Parsing code.
I noticed it being reported in email tracking. Could it be Email Opens in Safari if you have set up email tracking in GA.
Looks like people accessing your website via a home screen bookmark. "Add to Home Screen" function, rather than via the Safari browser are listed as "Safari (in-app)".
I'm looking at a GA account that only watches iPads displaying a "WebApp". It looks like those devices running 10.3.2 and higher display as "Safari (in-app)" and those below show as "Safari" only.
This would also explain the sudden jump in metrics. As users update their iOS will suddenly be listed as "Safari (in-app)".
This question already has answers here:
Why do all browsers' user agents start with "Mozilla/"?
(6 answers)
Closed 4 years ago.
When I myself send many requests to the server I found it amazing that in IE if I choose opera user string that the value of user string was
User-Agent Opera/9.80 (Windows NT 6.1; U; en) Presto/2.2.15 Version/10.00
But if I choose another browser in Internet Explorer that it puts Mozilla 5.0 in the user string first.
When I send the ajax request from Chrome that I found same thing that they put user string
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.20 (KHTML, like Gecko) Chrome/11.0.672.2 Safari/534.20
I found that Mozilla is an organization that doesn't have anything to do with Google and Microsoft. Perhaps it was a competitor for both. Why do MSFT and Google both put Mozilla in their user agent? Is there any reason for putting Mozilla in connection string?
Why do chrome and IE both put Mozilla in the userstring when they send the request? I do not know why but is there any specific reason for that?
See: user-agent-string-history
It all goes back to browser sniffing and making sure that the browsers are not blocked from getting content they can support. From the above article:
And Internet Explorer supported frames, and yet was not Mozilla, and so was not given frames. And Microsoft grew impatient, and did not wish to wait for webmasters to learn of IE and begin to send it frames, and so Internet Explorer declared that it was “Mozilla compatible” and began to impersonate Netscape, and called itself Mozilla/1.22 (compatible; MSIE 2.0; Windows 95), and Internet Explorer received frames, and all of Microsoft was happy, but webmasters were confused.