I'm doing some researching on switching from Apache to Nginx as a reverse proxy in front of a Grails application on the backend. I'm playing around with some URL rewriting and have run into an issue with the response being sent back from my back end. I can handle the location header rewrite but I'm wondering what the best way to process the actually content is for link and such.
Is nginx_substitutions_filter the preferred method or is there another module that folks use to do content replacement in the response body?
I've thought about creating a Grails plugin to handle rendering the correct content based on additional request headers but now I'm thinking that would be best handled outside the application to allow for the most flexibility and loose coupling.
Are there any articles about best practices for doing URL rewriting/response post processing for reverse proxy scenarios?
You can use the Lua module to capture the response and manipulate it like a Lua string. Silly example to upper case the output:
res = ngx.location.capture('/some/path')
ngx.print(string.upper(res.body))
see http://wiki.nginx.org/HttpLuaModule#ngx.location.capture
If you want to replace only the headers, HeadersMore 3rd party module is great for that.
Other than that, susbstiution module seems to be the only option.
But I would suggest you make the backend return the correct page. Modifying every response uses resources and takes time.
Related
I am trying to migrate a legacy system to use artifactory. However I have two blockers:
the old scripts require PyPixmlrpc, which artifactory doesn't support
they also make use of upload_docs, not supported by artifactory's pypi implementation either
a smaller issue, the old scripts call register and they expect 200 instead of 204 http status code.
Would it be possible for me to write a plugin to implement this?
Looking at https://www.jfrog.com/confluence/display/RTF/User+Plugins I couldn't find a callback for when POST /api/pypi/<index-name> is requested.
If I can make
work for the methods we actually use, to just pretend it deployed docs and to respond with the correct status code I will be happy enough.
As you say, there is no plugin hook for the Pypi API endpoints. It would be possible to use the altResponse endpoint to customize artifact downloads, but then you would be restricted to GET requests with no request body, which is also not a good option for you.
I think the most viable approach would be to define a custom executions endpoint. With this, you can specify the acceptable method, read the body, and set your own response code and body. The main shortcoming with this is that you can't customize the path (it's always /api/plugins/execute/[execution_name]), but this can be worked around.
Execution endpoints can take params in the following form:
/api/plugins/execute/[execution_name]?params=[param_name]=[param_val]
Say your plugin takes a param path, which represents the API path your old scripts are going to call. Then you can set your base URL to /api/plugins/execute/[execution_name]?params=path=/, so that the API path is appended to the param. Alternatively, you can use nginx or another reverse proxy to rewrite the original API path to this form.
(Since you'll be using XML-RPC, I don't suppose you'll need to worry about any of this path stuff, but I'm including it anyway for completeness.)
Some issues with this approach:
Execution endpoints only allow String responses, so sending binary data in the response body might be finnicky. However, no such limitation exists with the request body.
If you need more than one request method, you'll need more than one execution endpoint. This means you'll need to use a reverse proxy to rewrite each method to a separate endpoint. Again, since XML-RPC just uses POST, this probably won't be an issue for you.
Execution endpoints can't customize response headers. Therefore, if your scripts expect a particular Content-Type or other header, you'll need to use a reverse proxy to insert it into the response.
I am creating a REST API in Go, and I want to build URLs to other resources in my replies.
Based on the http.Response I can get the Host and URL.
However, how would I go about getting the transport scheme used by the server? http or https?
I attemped to check if server.TLSConfig is nil and then assuming it is using http since it says this in the documentation for http.Server:
TLSConfig *tls.Config // optional TLS config, used by ListenAndServeTLS
But it turns out this exists even when I do not run the server with ListenAndServeTLS.
Or is this way of building my URLs the wrong way of doing things? Is there some other normal way of doing this?
My preferred solution when running http and https is just to run a simple listener on :80 that redirects all traffic to https. Then any real traffic can be assumed to be https.
Alternately I believe you can access a request's URL at req.URL.Scheme to see the protocol.
Or do you mean for the entire application? If you accept configuration to switch between http and https, then can't you look at that and see which they chose? I guess I'm missing some context maybe.
It is also common practice for apps to take a baseURL via flag or config to generate external urls with.
My web app makes request to third party servers, and we sometimes route them trough proxies. I'd like to be able to "see what they see" -- see what the request looks like once its been routed through the proxy.
Specifically, I'm interested in how much identifying information about the source (my web app) is left in the request once it reaches the destination, having been routed through the proxy.
Does anyone know an easy way to do this? Maybe a web service that will just echo back all the information about the incoming request in the outgoing response?
Not a full answer, but maybe you can try:
http://www.cantoni.org/2012/01/08/simple-webservice-echo-test
And the other 2 webs mentioned there:
http://respondto.it/
http://requestb.in/
To setup a URL to send your requests and see if the info provided helps you.
I'm just stating this as an idea that came to me. You could try sending requests to your own URL, which you control (i.e. a resource in your own web application). That way, you can use your debugging infrastructure or other facilities (basically anything you want) to inspect the request that's coming into your application. It seems to me this might be the most powerful / easiest way to do this. It won't let you test the URL you were trying to test, but in terms of proxy visibility, it might be what you need.
Good luck!
If the proxy supports the TRACE method and the Max-Forwards header you can use that. Not all do, however.
http://myfileserver/images/car/chevrolet.gif
I have this file server holding the files such as images, doc files etc. Now i want to intercept the http request and based on the file extension i want perform some action such as redirection to some other webpage.
What is the best and the easier way to accomplish this thing? I am using asp.net framework for my applications.
Pls suggest the approach.
Thx
If you are looking to intercept the request for specific file types, then go with an Http Handler. Here is the MSDN link explaining their usage - Http Handlers
In the WCF world if you're looking to handle an HTTP request with a different option based on the requested filetype, you may want to look into adding an IDispatchOperationSelector, which allows the service to route the code through a different operation. The default HTTP implementation is the WebHTTPDispatchOperationSelector, which is explained pretty well here and here.
If you want to remain in the Asp.Net world, I'd recomend going with custom message handlers. Here's an article by Mike Wasson explaining how these work & where they fall in the Asp stack.
Use-case: Upload a simple image file to a server, which clients could later retrieve
Designate a FTP Server for the job.
HTTP Put: It can directly upload files to a server without the need of a server side
component to handle the bytestream.
HTTP Post: Handle the bytestream by the server side component.
I think to safely use PUT on a public website requires even more effort than using POST (and is less commonly done) due to potential security issues. See http://bitworking.org/news/PUT_SaferOrDangerous.
OTOH, I think there are plenty of resources for safely uploading files with POST and checking them in the server side script, and that this is the more common practice.
PUT is only appropriate when you know the URL you are putting to.
You could also do:
4) POST to obtain a URL to which you then PUT the file.
edit: how are you going to get the HTTP server to decide whether it is OK to accept a particular PUT request?
What I usually do (via PHP) is HTTP POST.
And employ PHP's move_uploaded_file() to get it to whatever destination I want.