When a user is visiting your site, is there a way of know what type of device they are using: computer, tablet, or mobile device? For example, when I get emails sometimes I see that it says sent via iphone; how do they know that? Any help or point me in the right direction would be very helpful.
Short answer: You can't.
Long answer:
All you have is the information in the HTTP User-Agent header, which usually contains the OS name and version.
Usually, browsers running on Mac OS and Linux send enough information to identify the exact OS. For example, here's my User-Agent header:
Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.7) Gecko/2009030423 Ubuntu/8.10 (intrepid) Firefox/3.0.7
You can see that I'm running Ubuntu 8.10 Intrepid Ibex.
And here's what Firefox and Safari 4 Beta report on my MacBook Pro:
Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.7) Gecko/2009021906 Firefox/3.0.7
Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16
Windows browsers, on the other hand, usually only report the OS version and not the specific package (Pro, Business, etc.):
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:x.x.x) Gecko/20041107 Firefox/x.x
I did not write this awnser i copied it from: Detect exact OS version from browser Thanks to: Can Berk Güder
Related
Please find below XPATH am using to scrape price from Myntra site. I can able to scrape from all other sites except Myntra and same below XPATH is working in my local windows system with Selenium,Python3 version and using chrome driver.
Driver path : driver = webdriver.Chrome("/usr/lib/chromium-browser/chromedriver", options=chrome_options);
variable_name = driver.find_element_by_xpath('//*[#id="mountRoot"]/div/div/div/main/div[2]/div[2]/div[1]/p[1]/span/strong').text
link for reference: https://www.myntra.com/beauty-gift-set/kama-ayurveda/kama-ayurveda-round-the-clock-skincare-gift-set/12800176/buy
When hosted to EC2 ubuntu machine getting below error:
Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="mountRoot"]/div/div/div/main/div[2]/div[2]/div[1]/p[1]/span/strong"}
Tried changing XPATH like driver.find_element_by_xpath('//*[#class="pdp-price"]//*').text but no luck.
Use the below XPath
driver.find_element_by_xpath('//span[#class="pdp-price"]//strong').text
Or by Using the below CSS selector
driver.find_element_by_css_selector('.pdp-price strong').text
The above works only if the site is in GUI mode whereas for headless displays Access Denied attached below screenshot for your reference. Since application blocks headless mode
Add the below user agent argument and load the web driver to your chrome driver options
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument("--headless")
chrome_options.add_argument(f'user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36')
Just add this user-agent option while launching headless chrome :
--user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
chromeLauncher.launch({
chromeFlags: ["--headless", '--disable-gpu', `--user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"`],
chromePath: '/usr/bin/google-chrome'
})
Here are some spider user agent strings I've seen recently. They all seem to include a URL prefixed with +:
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; meanpathbot/1.0; +http://www.meanpath.com/meanpathbot.html)
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Is that just a convention that most spiders follow? Or is it specified somewhere. I couldn't find it.
It's just a convention that some spiders follow. There is no constraint on what people can put in a user agent header.
Take a look at this list of user agents that contain "GoogleBot". You'll notice that many of these don't contain "+http".
I use qml WebView QT 5.2.
WebView {
anchors.fill: parent
url: "http://google.com"
}
Loaded page content is scaled depending on WebView width. How can I get default scale like in browser?
left picture my WebView, right - Google Chrome
To achieve the same scaling as in Picture 2 from you question , you have to set the user-agent property of WebView to the chrome one as follow :
import QtWebKit.experimental 1.0
...
WebView {
anchors.fill: parent
url: "http://google.com"
experimental.userAgent:"Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36"
}
...
[Update]
notice the difference between the 3 images from 3 different user-agent strings
------------------------------------------User-agent Android Browser------------------------------------------------
"Mozilla/5.0 (Linux; U; Android 4.0.3; ko-kr; LG-L160L Build/IML74K) AppleWebkit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30"
------------------------------------------user-agent Chrome Browser------------------------------------------------
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.146 Safari/537.36"
------------------------------------------user-agent Ipad-Macosx/Safari------------------------------------------------
"Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5355d Safari/8536.25"
Although this topic is pretty old, I had just encountered the same issue with QT 5.11 in a Ubuntu Touch app. Turned out that I just had to set the following:
QGuiApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
(or QApplication, respectively)
Note that this must be set before the application instance is created:
int main(int argc, char *argv[])
{
QGuiApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
QGuiApplication *app = new QGuiApplication(argc, (char**)argv);
This lead to the webview scaling the same way the native browser on ubuntu touch does (Morph browser). I did not need to set any user agent information.
For those also working with Ubuntu Touch, the MainView size does have a direct impact on content scaling in the webview. I left the defaults set by the creation process (clickable create), which is:
width: units.gu(45)
height: units.gu(75)
This leads to correct scalingon the device (in this case, a Volla Phone).
When I run this command:
wget --user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:21.0) Gecko/20100101 Firefox/21.0" http://yahoo.com
...I get this result (with nothing else in the file):
<!-- hw147.fp.gq1.yahoo.com uncompressed/chunked Wed Jun 19 03:42:44 UTC 2013 -->
But when I run wget http://yahoo.com with no --user-agent option, I get the full page.
The user agent is the same header that my current browser sends. Why does this happen? Is there a way to make sure the user agent doesn't get blocked when using wget?
It seems Yahoo server does some heuristic based on User-Agent in a case Accept header is set to */*.
Accept: text/html
did the trick for me.
e.g.
wget --header="Accept: text/html" --user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:21.0) Gecko/20100101 Firefox/21.0" http://yahoo.com
Note: if you don't declare Accept header then wget automatically adds Accept:*/* which means give me anything you have.
I created a ~/.wgetrc file with the following content (obtained from askapache.com but with a newer user agent, because otherwise it didn’t work always):
header = Accept-Language: en-us,en;q=0.5
header = Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
header = Connection: keep-alive
user_agent = Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0
referer = /
robots = off
Now I’m able to download from most (all?) file-sharing (streaming video) sites.
You need to set both the user-agent and the referer:
wget --header="Accept: text/html" --user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:21.0) Gecko/20100101 Firefox/21.0" --referer connect.wso2.com http://dist.wso2.org/products/carbon/4.2.0/wso2carbon-4.2.0.zip
I have confirmed that my Control Adapters are not triggering in Chrome and Safari. I've debugged, and the breakpoints inside the adapters just don't get hit in Chrome/Safari, when they work perfectly find in Firefox/IE. So, for Chrome/Safari, IIS is just ignoring the mapping.
My AdapterMappings.browser file looks like this:
<browsers>
<browser refID="Default">
<controlAdapters>
[...adapters here....]
</controlAdapters>
</browser>
</browsers>
This should provide mappings for all browsers, correct?
I used the Charles proxy to check what user agents were being sent. They are:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1064 Safari/532.5
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7
Any idea why this would be? Everything I've read tells me that my browser mappings are correct? And, as I said this works for IE/Firefox, so I know my configuration is technically correct.
We found the problem. It was errant code that specifically disabling the adapters in Chrome/Safari. They work fine, normally.