When using Fiddler, it pops up an alert dialog.
Fiddler has detected a protocol violation in session #14.
The Request's Host header did not match the URL's host component.
URL Host: proxy.music.pp.com
Header Host: 119.147.22.41
And it shows that Fiddler changed HTTP Header's host to "proxy.music.pp.com", is there any way to disable Fiddler changing it?
From my book:
Swap the Host Header
When Fiddler gets a request whose URL doesn’t match its Host header, the original Host value is stored in the session flag X-Original-Host and then the Host value is replaced with the host parsed from the URL. The following script, placed inside your FiddlerScript's BeforeRequest function, reverses behavior by routing the request to the host specified by the original Host header.
if (oSession.BitFlags & SessionFlags.ProtocolViolationInRequest)
{
var sOverride = oSession["X-Original-Host"];
if (!String.IsNullOrEmpty(sOverride))
{
oSession["X-overrideHost"] = sOverride;
oSession["ui-backcolor"] = "yellow";
// Be sure to bypass the gateway, otherwise overrideHost doesn't work
oSession.bypassGateway = true;
}
}
You can do this with rules.
Go into Customize rules, and find the function: OnBeforeRequest(oSession: Session)
Then add the following as the last line of that function:
if (oSession.HostnameIs("proxy.music.pp.com")) { oSession.host = "119.147.22.41"; }
I didn't understand what the "url host" was supposed to be - I mean, normally, there is only the Host header in HTTP.
Looking closer, it appears that this violation occurs for an initial proxy "setup" request, which looks like this:
CONNECT targaryen:45633 HTTP/1.1
Host: targaryen
This is where the error makes sense to me.
I couldn't get what I needed for the existing answers, but Eric Law's answer gave me what I needed to fix my issue. I was having a problem where I was calling a web server by IP address, and I had to add a Host header to get the right endpoint. Fiddler was changing the Host header to the IP address I was calling and removing my Host header value, which caused the call to fail. I added a line to Eric's script to solve this. I put the rule into OnBeforeRequest in the Fiddler Rules Script.
if (oSession.BitFlags & SessionFlags.ProtocolViolationInRequest)
{
var sOverride = oSession["X-Original-Host"];
if (!String.IsNullOrEmpty(sOverride))
{
oSession["X-overrideHost"] = sOverride;
oSession["ui-backcolor"] = "yellow";
oSession.oRequest["Host"] = sOverride;
// Be sure to bypass the gateway, otherwise overrideHost doesn't work
oSession.bypassGateway = true;
}
}
Took this Error:
And changed it to this:
Related
We have a downstream application which is setting some custom headers to the requests from browser before hitting nginx. nginx serves only static contents.
ie browser >> application A >> nginx
The requirement is that the nginx should be able to return all the headers which it receives as is to the downstream server which would give it back to the browser. by default its returning only the generic headers ( cookies etc, expiry etc ) and not retuning the custom ones sent by the downstream server.
For instance, there is a header with name appnumber which nginx receives with value app01. I tried to explicitly set it with the following rule to set it manually if it exist, but did not help as it throws error that variables are not allowed.
if ($appnumber) {
add_header appnumber $appnumber;
}
Can someone please guide me here?
The request headers are stored under $http_ variable. You could try something like
if ($appnumber) {
add_header appnumber $http_appnumber;
}
Refer http://nginx.org/en/docs/http/ngx_http_core_module.html and nginx - read custom header from upstream server
Is there any way I can get nginx to not forward a specific request header to uwsgi?
I want to enable nginx basic auth, but if the Authorization header gets forwarded to my app it breaks things (for reasons, I won't go into). If it was just a simple proxy_pass I would be able to do proxy_set_header Authorization ""; but I don't think this works with uwsgi_pass and there's no equivalent uwsgi_set_header as far as I can see.
Thanks.
Try hide header and ignore header directives:
uwsgi_hide_header
Syntax: uwsgi_hide_header field; Default: — Context: http, server,
location
By default, nginx does not pass the header fields “Status” and
“X-Accel-...” from the response of a uwsgi server to a client. The
uwsgi_hide_header directive sets additional fields that will not be
passed. If, on the contrary, the passing of fields needs to be
permitted, the uwsgi_pass_header directive can be used.
uwsgi_ignore_headers
Syntax: uwsgi_ignore_headers field ...; Default: —
Context: http, server, location Disables processing of certain
response header fields from the uwsgi server. The following fields can
be ignored: “X-Accel-Redirect”, “X-Accel-Expires”,
“X-Accel-Limit-Rate” (1.1.6), “X-Accel-Buffering” (1.1.6),
“X-Accel-Charset” (1.1.6), “Expires”, “Cache-Control”, “Set-Cookie”
(0.8.44), and “Vary” (1.7.7).
If not disabled, processing of these header fields has the following
effect:
“X-Accel-Expires”, “Expires”, “Cache-Control”, “Set-Cookie”, and
“Vary” set the parameters of response caching; “X-Accel-Redirect”
performs an internal redirect to the specified URI;
“X-Accel-Limit-Rate” sets the rate limit for transmission of a
response to a client; “X-Accel-Buffering” enables or disables
buffering of a response; “X-Accel-Charset” sets the desired charset of
a response.
It's probably too late for you but for anyone who would have the same problem, this answer provides a valid solution.
In this case the Authorization header could be passed by using the following directive:
uwsgi_param HTTP_Authorization "";
https://groups.google.com/forum/#!topic/golang-nuts/OwGvopYXpwE
As seen in this thread, when http.Client sends requests to redirects, the header gets reset.
There is a workaround like:
client.CheckRedirect = func(req *http.Request, via []*http.Request) error {
if len(via) >= 10 {
return fmt.Errorf("too many redirects")
}
if len(via) == 0 {
return nil
}
for attr, val := range via[0].Header {
if _, ok := req.Header[attr]; !ok {
req.Header[attr] = val
}
}
return nil
}
But my question is how do I do this if I want to http Request through my proxy server.
When http request goes through proxy server, does header get all reset? Do I have to set up another http.Client in proxy?
I set up my proxy server using https://github.com/elazarl/goproxy
Thanks,
I looked briefly at the code, at it looks like goproxy doesn't handle 3XX responses in any special way – they're simply routed back to your client, so it can react accordingly. In your case, you'll issue another request with all the headers set.
If it wasn't your concern and you were just wondering if simply existence of proxy requires any hack on the client side, then no, http.Client won't remove any headers when proxy is in use, it won't even know there's a proxy involved.
You can easily test these assumption by setting up you own a simple server (in Go for example) which is returning a redirect response (to itself or some nc listener) and printing all the headers. When routing the client to the server through your proxy, you can make sure everything looks good on the server side.
I have seen previous posts about Jetty BadMessage: 400 No Host for HttpChannelOverHttp and I can confirm that I am able to repeat the problem.
I have a Jetty route in Camel Blueprint, which creates another request and forwards on to a Dropwizard service via Camel HTTP.
.process(new Processor() {
//Creates Object for request
}
.marshal(jsonFormat)
.convertBodyTo(String.class)
.setHeader(Exchange.HTTP_URI, simple(serviceEndpoint))
.setHeader(Exchange.HTTP_METHOD, constant(HttpMethod.POST))
.to(userviceEndpoint)
When this request executes, I see the following error on Dropwizard
WARN [2014-11-12 23:15:35,333] org.eclipse.jetty.http.HttpParser: BadMessage: 400 No Host for HttpChannelOverHttp#3aa99dd2{r=0,a=IDLE,uri=-}
This happens constantly, and this problem does not occur when I send a request to the DW service using SOAP-UI (using the serviceEndpoint URL).
Please if anyone has solved this problem, I would like to know how. Thank you.
Capture your network traffic, and post the HTTP request headers you are sending to Jetty.
Odds are that your HTTP client is not sending the Host: header (which is required on HTTP/1.1)
In my case, I was setting header with null value. After removing header having null value from request solved the issue.
Jetty Version: 9.3.8
I got this error when I was making a request with incorrectly formatted headers. So instead of having the header as "X_S_ID: ABC" I had "X_S_ID: ["X_S_ID":BLAH]". So the error sometimes may not literally mean you need to pass a Host header.
Fixing the headers fixed this. Print the exact request you are making and make sure all headers are correctly formatted.
I am doing a proof of concept to demonstrate how we might implement 3scale in our stack. In one example I want to do some POST request body manipulation to create an API façade that maps what might be a legacy API format to a new internal one. Eg. change something like
{ "foo" : "bar" , "deprecated" : true }
into
{ "FOO" : "bar" }
The Lua module docs for content_by_lua, which seems like the appropriate method say
Do not use this directive and other content handler directives in the same location. For example, this directive and the proxy_pass directive should not be used in the same location.
My understanding is that the content_by_lua is a content handler like proxy_pass, only one of which can be used per location.
I don't think there's any way to remove proxy_pass as that's the basis of how the proxying works, so is it possible capture the request in a separate location, use content_by_lua, then pass to the location implementing proxy_pass or is there a different method like rewrite_by_lua which is more appropriate?
If it helps anyone else, I added the following function (my first bit of Lua) which removes the user_key parameter which 3scale requires for authorization but is invalid for our API if forwarded on:
function remove_user_key()
ngx.req.read_body()
-- log the original body so we can compare to the new one later
local oldbody = ngx.req.get_body_data()
log(oldbody)
-- grab the POST parameters as a table
local params = ngx.req.get_post_args()
-- build up the new JSON string
local newbody = "{"
for k,v in pairs(params) do
-- add all the params we want to keep
if k ~= "user_key" then
log("adding"..k.." as "..v)
newbody = newbody..'"'..k..'":"'..v..'",'
else
log("not adding user_key")
end
end
--remove the last trailing comma before closing this off
newbody = string.sub(newbody, 0, #newbody-1)
newbody = newbody.."}"
ngx.req.set_body_data(newbody)
log(newbody)
end
if ngx.req.get_method() == "POST" then
remove_user_key()
end
I will suggest you to use access_by_lua
in nginx.conf
location / {
#host and port to fastcgi server
default_type text/html;
set $URL "http://$http_host$request_uri";
access_by_lua_file /home/lua/cache.lua;
proxy_pass http://$target;
-------
---------
in cache.lua file you can do something like :
if ngx.req.get_method() == "POST" then
-- check if request method is POST
-- implement your logic
return
end
Adding to what Prashant already mentioned: when you download your Nginx configuration files from 3scale you will notice that there is a Lua file included. This file is already being called from access_by_lua_file.
That file would be in my opinion the best place to add your body manipulation code. It will be executed for every request before the proxy_pass to your API server is sent.
Also, this is a really good in-depth blog article about how to accomplish different transformation to your requests using Lua inside Nginx:
Recently I had to manipulate upstream based on JSON value in post request and I found this useful: NGINX LUA and JSON
It is basic configuration but gives an idea how to do it.