Illegal characters in path depending on User-Agent? - http

I have two identical calls to ASP.NET, the only difference is the User-Agent. I used Fiddler to reproduce the issue.
The HTTP request line is:
PUT http://localhost/API/es/us/havana/club/tickets/JiWOUUMxukGVWwVXQnjgfw%7C%7C214 HTTP/1.1
Works with:
User-Agent: Mozilla/5.0 (Linux; Android 4.3; Nexus 10 Build/JSS15Q) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2307.2 Safari/537.36
Fails with:
User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 8_0 like Mac OS X) AppleWebKit/600.1.3 (KHTML, like Gecko) Version/8.0 Mobile/12A4345d Safari/600.1.4
Everything else is 100% the same.

In my case, the root cause was MVC's MultipleViews and DisplayMode providers. This allows MVC apps to magically pick up device-specific views; e.g.
custom.cshtml
customer.mobile.cshtml
This article has a good explanation of the functionality as well as details how to turn it off:
https://learn.microsoft.com/en-us/archive/msdn-magazine/2013/august/cutting-edge-creating-mobile-optimized-views-in-asp-net-mvc-4-part-2-using-wurfl
I fixed this by adding Microsoft.AspNet.WebPages package to my project and adding a call to this code in my startup (application_start in global.asax or if using OWIN, the method decordated w/ OwinStartup attribute):
public static void RegisterDisplayModes()
{
// MVC has handy helper to find device-specfic views. Ain't no body got time for that.
dynamic modeDesktop = new DefaultDisplayMode("") { ContextCondition = (c => { return true; }) };
dynamic displayModes = DisplayModeProvider.Instance.Modes;
displayModes.Clear();
displayModes.Add(modeDesktop);
}

Related

CSS is sometimes loading URLs relative to page

I see on MDN that relative URLs are relative to the css file, not to the current page.
I want to understand if there any exceptions or gotchas with using relative urls in CSS, as I can see in my access logs that some requests are using the wrong relative path.
Previous threads on here suggest there might be issues with old IE versions, but that doesn't seem to be what is causing this as the user agents here are modern.
The particularly annoying thing is that this doesn't affect all requests and I'm unable to reproduce the problem myself. I could just replace all the relative urls in my css with absolute and continue with my life but I want to understand what is causing this and discover if there is a better solution.
CSS file is loaded like <link rel="stylesheet" type="text/css" href="/static/css/common.min.css?v6" /> and in this
I have relative urls to images defined , e.g.
#go, #search_go {background: #EEE url(../img/silk/magnifier.png) no-repeat .4em .25em; padding-left: 1.8em;}
The CSS is minified using UglifyCSS but using Inspect element in my browser makes me think this is not the issue (also, of course it does work in the majority of requests).
An example of a bad request looks like
ip.ip.ip.ip - - [09/Jan/2022:20:22:20 +0000] "GET /foo/img/silk/magnifier.png HTTP/2.0" 404 1575 "https://example.com/foo/search/" "Mozilla/5.0 (Android 11; Mobile; rv:95.0) Gecko/95.0 Firefox/95.0"
An example of a 'good' request looks like
ip.ip.ip.ip - - [09/Jan/2022:21:07:32 +0000] "GET /static/img/silk/magnifier.png HTTP/2.0" 200 615 "https://example.com/foo/search/" "Mozilla/5.0 (iPhone; CPU iPhone OS 15_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Mobile/15E148 Safari/604.1"
To note this is not just affecting a single browser, some other user agents which show up in the access logs with "bad" requests are
"Mozilla/5.0 (Android 11; Mobile; rv:95.0) Gecko/95.0 Firefox/95.0"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:95.0) Gecko/20100101 Firefox/95.0"
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36"```

Simulate file download with Gatling

Good morning,
I would like to simulate a file download with Gatling. I'm not sure that a simple get request on a file ressource really simulate it:
val stuffDownload: ScenarioBuilder = scenario("Download stuff")
.exec(http("Download stuff").get("https://stuff.pdf")
.header("Content-Type", "application/pdf")
.header("Content-Type", "application/force-download"))
I want to challenge my server with multiple downloads within the same moment and I need to be sure I have the right method to do it.
Thanks in advance for your help.
EDIT: Other headers I send:
"User-Agent" -> "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36",
"Accept" -> "application/json, text/plain, */*; q=0.01",
"Accept-Encoding" -> "gzip, deflate, br",
"Accept-Language" -> "fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7",
"DNT" -> "1",
"Connection" -> "keep-alive"
It looks technically globally fine except that:
you have 2 Content-Type ?
Is there a mistake in second one ?
Also aren't you missing other browser headers like User-Agent ?
Aren't you missing an important one related to Compression like Accept-Encoding ?
But regarding functional part, aren't you missing some steps before it ?
I mean do your user access immediately the link or do they hit a login screen , then do a search and finally click on a link ?
Also, is it always the same file ? Shouldn't you introduce a kind of variability using Gatling CSV Feeders for example with a set of files ?

How can I set WebView content scale (qml QT 5.2)

I use qml WebView QT 5.2.
WebView {
anchors.fill: parent
url: "http://google.com"
}
Loaded page content is scaled depending on WebView width. How can I get default scale like in browser?
left picture my WebView, right - Google Chrome
To achieve the same scaling as in Picture 2 from you question , you have to set the user-agent property of WebView to the chrome one as follow :
import QtWebKit.experimental 1.0
...
WebView {
anchors.fill: parent
url: "http://google.com"
experimental.userAgent:"Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36"
}
...
[Update]
notice the difference between the 3 images from 3 different user-agent strings
------------------------------------------User-agent Android Browser------------------------------------------------
"Mozilla/5.0 (Linux; U; Android 4.0.3; ko-kr; LG-L160L Build/IML74K) AppleWebkit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30"
------------------------------------------user-agent Chrome Browser------------------------------------------------
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.146 Safari/537.36"
------------------------------------------user-agent Ipad-Macosx/Safari------------------------------------------------
"Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5355d Safari/8536.25"
Although this topic is pretty old, I had just encountered the same issue with QT 5.11 in a Ubuntu Touch app. Turned out that I just had to set the following:
QGuiApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
(or QApplication, respectively)
Note that this must be set before the application instance is created:
int main(int argc, char *argv[])
{
QGuiApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
QGuiApplication *app = new QGuiApplication(argc, (char**)argv);
This lead to the webview scaling the same way the native browser on ubuntu touch does (Morph browser). I did not need to set any user agent information.
For those also working with Ubuntu Touch, the MainView size does have a direct impact on content scaling in the webview. I left the defaults set by the creation process (clickable create), which is:
width: units.gu(45)
height: units.gu(75)
This leads to correct scalingon the device (in this case, a Volla Phone).

WebClient.DownloadString seems to change some of the html from an external site

I have an ASP.NET website (.aspx) that I call from within an ASP.NET MVC 4 mobile website (.cshtml) to get its html response string. Both sites are hosted on a Windows Server 2008 R2 system. They are created and published with VS2010 Professional.
-If I go directly to the external site and view source then it is correct.
-If I use any of the below ways of getting the external html:
using (WebClient client = new WebClient())
{
html = client.DownloadString(strUrl);
}
or
using (WebClient client = new WebClient())
{
byte[] DataBuffer = client.DownloadData(strUrl);
html = Encoding.ASCII.GetString(DataBuffer);
}
or
WebResponse objResponse;
WebRequest objRequest = System.Net.HttpWebRequest.Create(strUrl);
objResponse = objRequest.GetResponse();
using (StreamReader sr = new StreamReader(objResponse.GetResponseStream()))
{
html = sr.ReadToEnd();
sr.Close();
}
then the html is changed from this ( where the font-family is set on a parent table ):
<td align="right" style="color:Red;background-color:White;width:4.375em;border-bottom:1px solid black;border-right:1px solid black;">-27.0%</td>
to this:
<td align="right" bgcolor="White" style="border-bottom:1px solid black;border-right:1px solid black;"><font face="Arial,sans-serif" color="Red">-27.0%</font></td>
I doesn't look like anything else has changed other than the font style is changed to a tag, the background color moved from a style to a tag attribute, and the width style being completely removed. This happens on the entire page.
If I put a break point on the html variable and view it then the html has already been changed by the time DownloadString is called.
Anyone know why this is happening?
Thanks in advance.
edit:
this link: WebClient.DownloadString() Not Producing Exact HTML
is not quite the same thing as I am not using Ajax or JavaScript on the external page.
edit:
here are the request headers from fiddler and the site that calls the other site (I used Chrome):
GET / HTTP/1.1
Connection: keep-alive
Accept: */*
User-Agent: Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: .ASPXBrowserOverride=Mozilla%2f4.0+(compatible%3b+MSIE+6.0%3b+Windows+CE%3b+IEMobile+8.12%3b+MSIEMobile+6.0);
going to the site directly I get this request header:
Connection: keep-alive
Cache-Control: max-age=0
User-Agent: Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: .ASPXBrowserOverride=Mozilla%2f4.0+(compatible%3b+MSIE+6.0%3b+Windows+CE%3b+IEMobile+8.12%3b+MSIEMobile+6.0);
edit:
If I look at the client object in debug mode client.Headers is empty before and after DownloadString is called.
Also, after DownloadString is called here are the client.ResponseHeaders:
{Content-Length: 267123
Cache-Control: private
Content-Type: text/html; charset=utf-8
Date: Tue, 27 Nov 2012 18:37:27 GMT
Set-Cookie: ASP.NET_SessionId=******; path=/; HttpOnly
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
}
Solution:
Unfortunately I cannot accept two answers. Both Icarus and James Lawruk's answers helped me to solve the problem. I am picking an answer based on what most recently lead me to the final solution. So thanks to you both!
So here is the solution in a nutshell:
Use fiddler to view the request headers and find the user-agent.
Modify the code as follows:
using (WebClient client = new WebClient())
{
client.Headers.Add("user-agent", "Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11");
html = client.DownloadString(strUrl);
}
Try setting the user-agent value and experiment with different browsers. This may prove the Web site is switching the HTML response based on the user-agent header.
webClient.Headers.Add("user-agent", "Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_2 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8H7 Safari/6533.18.5");
var iphoneHtml = webClient.DownloadString("http://www.yoursite.com");
webClient.Headers.Add("user-agent", "Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11");
var safariHtml = webClient.DownloadString("http://www.yoursite.com");
Most likely Browser Sniffing as Dour pointed out on his comment because WebClient does not change the resulting HTML at all.
You can probably verify this if you use Fiddler and set up the request headers in exactly the same way WebClient does. I bet you you get the same HTML output.

Why wouldn't the default Control Adapter mappings work on Chrome or Safari?

I have confirmed that my Control Adapters are not triggering in Chrome and Safari. I've debugged, and the breakpoints inside the adapters just don't get hit in Chrome/Safari, when they work perfectly find in Firefox/IE. So, for Chrome/Safari, IIS is just ignoring the mapping.
My AdapterMappings.browser file looks like this:
<browsers>
<browser refID="Default">
<controlAdapters>
[...adapters here....]
</controlAdapters>
</browser>
</browsers>
This should provide mappings for all browsers, correct?
I used the Charles proxy to check what user agents were being sent. They are:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1064 Safari/532.5
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7
Any idea why this would be? Everything I've read tells me that my browser mappings are correct? And, as I said this works for IE/Firefox, so I know my configuration is technically correct.
We found the problem. It was errant code that specifically disabling the adapters in Chrome/Safari. They work fine, normally.

Resources