I am accessing one website (I am hiding origin website name as it is against the policy) using browser and jetty/apache httpclient.
The website works fine with web browser.
Using api I am able to login into website,gets the session cookie JSESSIONID and home page html content. But after that when I submit any form or call the links from html I receive the HTTP error code 412(Pre condition failed).
I understand this error is due problem in client header. I set all the headers from browser(checked using inspect element in chrome browser). Still I have the same error.
I am not able to track down which header is causing the problem.
Here is the Header from browser
Request Headers
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8
Accept-Encoding:gzip, deflate
Accept-Language:en-GB,en-US;q=0.8,en;q=0.6
Cache-Control:max-age=0
Connection:keep-alive
Content-Length:318
Content-Type:application/x-www-form-urlencoded
Cookie:language=en_IN; __gads=ID=14c64d8f9fd7de54:T=1424658276:S=ALNI_Mba1kvJO4mLo7R-T2jUJE9zCYck5A; SLB_Cookie=ffffffff09461c2d45525d5f4f58455e445a4a422971; JSESSIONID=36m4Oo6dCML_Wvx-Wgmm9rtLh9mbURxnZhWIVwg-zHaNzFQeUt9C!-1989013783; _ga=GA1.3.379900459.1428120216
Host:www.irctc.co.in
Origin:https://www.example.com
Referer:https://www.example.com/context/home
Upgrade-Insecure-Requests:1
User-Agent:Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36
Same header I am setting from jetty client.
Request request = httpClient.newRequest(url);
request.method(HttpMethod.POST);
request.agent(USER_AGENT);
request.accept("text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
request.header(HttpHeader.REFERER,"https://www.example.com/context/home");
request.header(HttpHeader.ACCEPT_ENCODING, "gzip, deflate");
request.header(HttpHeader.ACCEPT_LANGUAGE, "en-GB,en-US;q=0.8,en;q=0.6");
request.header(HttpHeader.CACHE_CONTROL, "max-age=0");
request.header(HttpHeader.CONNECTION, "keep-alive");
request.header(HttpHeader.CONTENT_TYPE, "application/x-www-form-urlencoded");
request.header(HttpHeader.HOST, "www.example.com);
for (Map.Entry<String, String> entry : params.entrySet()){
request.param(URLEncoder.encode(entry.getKey(), "UTF-8"), URLEncoder.encode(entry.getValue(), "UTF-8"));
if(!StringUtils.isEmpty(content)){
content+="&";
}
content+=URLEncoder.encode(entry.getKey(), "UTF-8")+"="+URLEncoder.encode(entry.getValue(), "UTF-8");
}
request.header(HttpHeader.CONTENT_LENGTH, ""+content.length());
I see JSESSIONID and SLB_Cookie are present in the request. Since the website is out of our control I really can not track what is the issue.
Please help me to resolve this issue. Any pointers to resolve the issue on client side is appreciated. is there any way we can make sure which header causing this issue.
I solved the problem.
Issue with the form parameters value. I was sending encoded values where were encoded by jetty client again
Related
I am getting the following error message on my angular/asp.net web api project.
XMLHttpRequest cannot load http://localhost:7291/api/products. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:7305' is therefore not allowed access. The response had HTTP status code 500.
I know it has something to do with CORS not being implemented correctly, but I'm not sure what I'm doing wrong. I'm following a tutorial and as far as I can tell I've got everything right?
Here's the info from the network tab in chrome debug.
Remote Address:[::1]:7291
Request URL:http://localhost:7291/api/products
Request Method:GET
Status Code:500 Internal Server Error
Response Headers
(8)
Request Headers
view source
Accept:application/json, text/plain, /
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Host:localhost:7291
Origin:http://localhost:7305
Referer:http://localhost:7305/
User-Agent:Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36
Here is where I map the API url.
(function () {
"use strict";
angular
.module("common.services", ["ngResource"])
.constant("appSettings",
{
serverPath: "http://localhost:7291/"
})
}());
And this is how I'm setting up the EnableCOrsAttribute:
[EnableCorsAttribute("http://localhost:7305", "*", "*")]
Can anyone see what I'm doing wrong? If more code is needed please let me know. Thanks.
First Step, use a tool like Postman or Fiddler to verify your service endpoint first to ensure functionality.
It looks as if this is not a CORS issue at all, due to the 500 response:
Status Code:500 Internal Server Error
A CORS issue on a normal success response is usually manifested as a status 0 or -1 in the client, but what you are experiencing is an error on the server side. I have seen this in my own code and suspect that your implementation on the server side is only injecting the CORS headers at the end of processing the request and as the processing abnormally aborted the CORS headers didn't make it in there.
Once you have confirmed functionality do an OPTIONS request on your endpoint to verify CORS:
OPTIONS / HTTP/1.1
Host: localhost:7291
Cache-Control: no-cache
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
Then inspect the headers of the response, if your CORS is enabled correctly on the server you should see Access-Control-Allow headers similar to this:
Access-Control-Allow-Methods: GET, POST, PUT, PATCH, DELETE, OPTIONS, ETAG
Access-Control-Allow-Origin: http://localhost:7305
Looking at the network panel in the developer tools of google chrome I can read the HTTP request and response messages of each file in a web page and, in particular, I can read the start line and the headers with all their fields.
I know (and I hope that is right) that the start line of each HTTP message has a specific and rigorous structure (different for request and response message, of course) and any element inside a start line cannot be missed.
Unlike the start line, the header of an HTTP message contains additional informations, so, I guess, the header fields are facultative or, at least, not so strictly requested like the fields in the start line.
Considering all this, I'm wondering: who sets the header fields in an HTTP message? Or, in other words, how are determined the header fields of an HTTP message?
For example, i can actually see that the HTTP request message for a web page is this:
GET / HTTP/1.1
Host: www.corriere.it
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.130 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: it-IT,it;q=0.8,en-US;q=0.6,en;q=0.4,de;q=0.2
Cookie: rccsLocalPref=milano%7CMilano%7C015146; rcsLocalPref=milano%7CMilano; _chartbeat2=DVgclLD1BW8iBl8sAi.1422913713367.1430683372200.1111111111111111; rlId=8725ab22-cbfc-45f7-a737-7c788ad27371; __ric=5334%3ASat%20Jun%2006%202015%2014%3A13%3A31%20GMT+0200%20%28ora%20legale%20Europa%20occidentale%29%7C; optimizelyEndUserId=oeu1433680191192r0.8780217287130654; optimizelySegments=%7B%222207780387%22%3A%22gc%22%2C%222230660652%22%3A%22false%22%2C%222231370123%22%3A%22referral%22%7D; optimizelyBuckets=%7B%7D; __gads=ID=bbe86fc4200ddae2:T=1434976116:S=ALNI_MZnWxlEim1DkFzJn-vDIvTxMXSJ0g; fbm_203568503078644=base_domain=.corriere.it; apw_browser=3671792671815076067.; channel=Direct; apw_cache=1438466400.TgwTeVxF.1437740670.0.0.0...EgjHfb6VZ2K4uRK4LT619Zau06UsXnMdig-EXKOVhvw; ReadSpeakerSettings=enlarge=enlargeoff; _ga=GA1.2.1780902850.1422986273; __utma=226919106.1780902850.1422986273.1439110897.1439114180.19; __utmc=226919106; __utmz=226919106.1439114180.19.18.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided); s_cm_COR=Googlewww.google.it; gvsC=New; rcsddfglr=1441375682.3.2.m0i10Mw-|z1h7I0wH.3671792671815076067..J3ouwyCkNXBCyau35GWCru0I1mfcA3hRLNURnDWREPs; cpmt_xa=5334,5364; utag_main=v_id:014ed4175b8e000f4d2bb480bdd10606d001706500bd0$_sn:74$_ss:1$_st:1439133960323$_pn:1%3Bexp-session$ses_id:1439132160323%3Bexp-session; testcookie=true; s_cc=true; s_nr=1439132160762-Repeat; SC_LNK_CR=%5B%5BB%5D%5D; s_sq=%5B%5BB%5D%5D; dtLatC=116p80.5p169.5p91.5p76.5p130.5p74p246.5p100p74.5p122.5; dtCookie=E4365758C13B82EE9C1C69A59B6F077E|Corriere|1|_default|1; dtPC=-; NSC_Wjq_Dpssjfsf_Dbdif=ffffffff091a1f8d45525d5f4f58455e445a4a423660; hz_amChecked=1
how these header fields are chosen? Who/what chose them? (The browser? Not me, of course...)
p.s.:
hope my question is clear, please, forgive my bad english
All internet websites are hosted on HTTP servers, these headers are set by the http server who is hosting the webpage. They are used to control how pages are shown, cached, and encoded.
Web browsers set the headers when requesting the pages from the servers. This mutual communication protocol is the HTTP protocol linked above.
here is a list of all the possible header fields for a request message: the question is, why the broser chooses only some of them?
The browser doesn't include all possible request headers in every request because either:
They aren't applicable to the current request or
The default value is the desired value
For instance:
Accept tells the server that only certain data formats are acceptable in the response. If any kind of data is acceptable, then it can be omitted as the default is "everything".
Content-Length describes the length of the body of the request. A GET request doesn't have a body, so there is nothing to describe the length of.
Cookie contains a cookie set by the server (or JavaScript) on a previous request. If a cookie hasn't been set, then there isn't one to send back to the server.
and so on.
I am using chrome to load a web page twice. On the second load, I still see all the js files being requested with 304 of course returned. I see the css files NOT requested and just load from cache. why is my js not loading from cache?
From chrome, here is the request/response on the second time(and on the third time it is re-requested instead of listenting to cache-control:max-age=3600
Request URL:http://myextremestore.com/public/js/bootstrap/theme.js
Request Method:GET
Status Code:304 Not Modified
Request Headersview source
Accept:*/*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8,zh-CN;q=0.6,zh;q=0.4
Cache-Control:max-age=0
Connection:keep-alive
Cookie:_ga=GA1.2.1887017343.1389240672; _usrp_dbba204e-e224-4c2e-9715- 645207f1698c_0=eyJMYXN0VmlzaXRUaW1lIjoxMzg5ODEwNzk2LCJTZXNzaW9uTnVtYmVyIjoxNCwiSXNTZXNzaW9uVmlzaXRMb2dnZWQiOmZhbHNlLCJUcmllZFRvSW52aXRlIjp0cnVlLCJTZXNzaW9uUGFnZVZpZXciOjE1LCJUb3RhbFBhZ2VWaWV3Ijo1MywiTG9jYWxRdWFyYW50aW5lIjpudWxsLCJXYXNJbml0QW1pbmF0aW9uIjp0cnVlfQ==; PLAY_SESSION="4c82c3ec3facde8b5cdd167313527d0b9b7b91df-s=500&___AT=df52f59436377f3500fb5700808f35bd5631e539&key=dean%40buffalosw.com"
Host:myextremestore.com
If-Modified-Since:Wed, 08 Jan 2014 18:48:42 GMT
If-None-Match:"1389206922000-1544568346"
Referer:http://myextremestore.com/admin/products
User-Agent:Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36
Here is the response
Response Headersview source
Cache-Control:max-age=3600
ETag:"1389206922000-1544568346"
Server:Play! Framework;master-5afad9b;prod
I am testing this on http://myextremestore.com and for some reason firefox is working just fine and I only see a single request server side when using firefox but chrome keeps asking for the same js files.
IE11 is working just fine as well. This is very weird.
Any ideas appreciated.
thanks,
Dean
oh, so it turns out in chrome hitting enter in the url window results in same behavior as pushing the refresh button!!! In firefox, hitting enter in the url does NOT result in hitting the refresh button.
I do not however know why chrome on refresh is only asking for js files and not asking for css files...that seems weird. Anyways, it works as I expect now.
So I set up an ejabberd XMPP server and use nginx as a proxy on an EC2 instance. The Strophe.js echobot example can connect from my Chrome browser. Here are the Request Headers:
Request URL:http://foo.eu-west-1.compute.amazonaws.com/http-bind
Request Method:POST
Status Code:200 OK
Request Headers
Accept:*/*
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
Connection:keep-alive
Content-Length:115
Content-Type:application/xml
Host:foo.compute.amazonaws.com
Origin:foo.eu-west-1.compute.amazonaws.com
Referer:foo.eu-west-1.compute.amazonaws.com/examples/echobot.html
User-Agent:Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11
Request Payload
<body rid='2692151172' xmlns='http://jabber.org/protocol/httpbind' sid='15f6dc6cbc7a7d69b5db531c2ebf7828e094f043'/>
Response Headers
Access-Control-Allow-Headers:Content-Type
Access-Control-Allow-Origin:*
Connection:keep-alive
Content-Length:286
Content-Type:text/xml; charset=utf-8
Date:Fri, 04 Jan 2013 11:14:14 GMT
Server:nginx/1.2.4
I ported the echobot example to work in Spotify as an app. However, Strophe cannot connect. Here is the Web Inspector network log:
Request URL:http://foo.eu-west-1.compute.amazonaws.com/http-bind
Request Headers
POST http://foo.eu-west-1.compute.amazonaws.com/http-bind HTTP/1.1
Origin: sp://chatify
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.15 Safari/535.11
Content-Type: application/xml
Request Payload
<body rid='790399813' xmlns='http://jabber.org/protocol/httpbind' to='ec2-54-246-45-111.eu-west-1.compute.amazonaws.com' xml:lang='en' wait='60' hold='1' content='text/xml; charset=utf-8' ver='1.6' xmpp:version='1.0' xmlns:xmpp='urn:xmpp:xbosh'/>
Now. The RequiredPermissons in the manifest.json look as follows:
"RequiredPermissons": [
"http://foo.eu-west-1.compute.amazonaws.com",
"foo.eu-west-1.compute.amazonaws.com",
"http://foo.eu-west-1.compute.amazonaws.com/http-bind",
"foo.eu-west-1.compute.amazonaws.com/http-bind"
]
I can load sources from foo.eu-west-1.compute.amazonaws.com so I think the permissons work.
The Spotify app uses a different origin, "sp://chatify". I read that this can cause problems. And Access-Control-Allow-Origin:* should be added to the header. I did so in the nginx config just to find out that it had been sent all along. You can see it in Response Headers of the first request.
Strophe.js itself says in the logs of the example if turned on:
error 0 happened
So any suggestions?
The manifest seems to be right
Access-Control-Allow-Origin;* seem right
It works in a browser but not Spotify.
Thx for your help!
Update:
If I open the echobot.html locally it still works. The Origin is then null.
Okay, I got it to work. And the solution is a classic. However, my debugging might help others.
At first I looked at the Web Inspector. It showed that all request were canceled. No the inspector is not always right as mentioned in many places. This time it was, though.
So Strophe.js could not establish a connection. To see if any custom Spotify App could establish a connection I tried the Google Maps example from the tutorial app. It did not work with the native Linux version and the Windows under wine. It did however work under windows.
My next step was to see if my app could load any resources from the net. So I simply added an iframe: . It did not load my page.
I took a close look at the manifest of the tutorial app and my app. I could not find any difference. So I just copied "RequiredPermissions" to see that I had a typo. Once that was fixed the iframe and eventually strophe.js worked.
It is funny. I looked for typos a few days ago I could not find any.
To sum up:
Make sure your server sets Access-Control-Allow-Origin;*
Set the Permissions. E.g.
"RequiredPermissions": [
"http://ec2-foo.eu-west-1.compute.amazonaws.com",
"ec2-foo.eu-west-1.compute.amazonaws.com",
"http://ec2-foo.eu-west-1.compute.amazonaws.com/http-bind",
"ec2-foo.eu-west-1.compute.amazonaws.com/http-bind"
]
I have a problem regarding ajax PUT method. When I send a file whose size over approximate 1M, there will be no response for missing content-length header field. The
request header just like this:
Accept:*/*
Content-Type:application/x-www-form-urlencoded; charset=UTF-8
Origin:http://xxxxxxxxxx
Referer:http://xxxxxx
User-Agent:Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.11 (KHTML, like Gecko)
Chrome/20.0.1132.57 Safari/536.11
X-Requested-With:XMLHttpRequest
My codes about ajax request as follows:
var file = ev.dataTransfer.files[0];
$.ajax({
url: url,
processData:false,
data: file,
type: 'PUT',
success : function(){},
error : function(){}
});
The problem happened both in Chrome and Firefox, I don't know how to deal with it. Is it related with Browsers or PUT method?
I can't fully parse "When I send a file which over approximate 1M, the request will have no response for missing content-length in my request header.".
Is the problem that the request doesn't have a Content-Length header field, and the server rejects the request because of that? That would be a server bug (because the browsers then probably used chunked encoding instead).