Why is an OPTIONS request sent and can I disable it? - http

I am building a web API. I found whenever I use Chrome to POST, GET to my API, there is always an OPTIONS request sent before the real request, which is quite annoying. Currently, I get the server to ignore any OPTIONS requests. Now my question is what's good to send an OPTIONS request to double the server's load? Is there any way to completely stop the browser from sending OPTIONS requests?

edit 2018-09-13: added some precisions about this pre-flight request and how to avoid it at the end of this reponse.
OPTIONS requests are what we call pre-flight requests in Cross-origin resource sharing (CORS).
They are necessary when you're making requests across different origins in specific situations.
This pre-flight request is made by some browsers as a safety measure to ensure that the request being done is trusted by the server.
Meaning the server understands that the method, origin and headers being sent on the request are safe to act upon.
Your server should not ignore but handle these requests whenever you're attempting to do cross origin requests.
A good resource can be found here http://enable-cors.org/
A way to handle these to get comfortable is to ensure that for any path with OPTIONS method the server sends a response with this header
Access-Control-Allow-Origin: *
This will tell the browser that the server is willing to answer requests from any origin.
For more information on how to add CORS support to your server see the following flowchart
http://www.html5rocks.com/static/images/cors_server_flowchart.png
edit 2018-09-13
CORS OPTIONS request is triggered only in somes cases, as explained in MDN docs:
Some requests don’t trigger a CORS preflight. Those are called “simple requests” in this article, though the Fetch spec (which defines CORS) doesn’t use that term. A request that doesn’t trigger a CORS preflight—a so-called “simple request”—is one that meets all the following conditions:
The only allowed methods are:
GET
HEAD
POST
Apart from the headers set automatically by the user agent (for example, Connection, User-Agent, or any of the other headers with names defined in the Fetch spec as a “forbidden header name”), the only headers which are allowed to be manually set are those which the Fetch spec defines as being a “CORS-safelisted request-header”, which are:
Accept
Accept-Language
Content-Language
Content-Type (but note the additional requirements below)
DPR
Downlink
Save-Data
Viewport-Width
Width
The only allowed values for the Content-Type header are:
application/x-www-form-urlencoded
multipart/form-data
text/plain
No event listeners are registered on any XMLHttpRequestUpload object used in the request; these are accessed using the XMLHttpRequest.upload property.
No ReadableStream object is used in the request.

Have gone through this issue, below is my conclusion to this issue and my solution.
According to the CORS strategy (highly recommend you read about it) You can't just force the browser to stop sending OPTIONS request if it thinks it needs to.
There are two ways you can work around it:
Make sure your request is a "simple request"
Set Access-Control-Max-Age for the OPTIONS request
Simple request
A simple cross-site request is one that meets all the following conditions:
The only allowed methods are:
GET
HEAD
POST
Apart from the headers set automatically by the user agent (e.g. Connection, User-Agent, etc.), the only headers which are allowed to be manually set are:
Accept
Accept-Language
Content-Language
Content-Type
The only allowed values for the Content-Type header are:
application/x-www-form-urlencoded
multipart/form-data
text/plain
A simple request will not cause a pre-flight OPTIONS request.
Set a cache for the OPTIONS check
You can set a Access-Control-Max-Age for the OPTIONS request, so that it will not check the permission again until it is expired.
Access-Control-Max-Age gives the value in seconds for how long the response to the preflight request can be cached for without sending another preflight request.
Limitation Noted
For Chrome, the maximum seconds for Access-Control-Max-Age is 600 which is 10 minutes, according to chrome source code
Access-Control-Max-Age only works for one resource every time, for example, GET requests with same URL path but different queries will be treated as different resources. So the request to the second resource will still trigger a preflight request.

Please refer this answer on the actual need for pre-flighted OPTIONS request: CORS - What is the motivation behind introducing preflight requests?
To disable the OPTIONS request, below conditions must be satisfied for ajax request:
Request does not set custom HTTP headers like 'application/xml' or 'application/json' etc
The request method has to be one of GET, HEAD or POST. If POST, content type should be one of application/x-www-form-urlencoded, multipart/form-data, or text/plain
Reference:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS

When you have the debug console open and the Disable Cache option turned on, preflight requests will always be sent (i.e. before each and every request). if you don't disable the cache, a pre-flight request will be sent only once (per server)

Yes it's possible to avoid options request. Options request is a preflight request when you send (post) any data to another domain. It's a browser security issue. But we can use another technology: iframe transport layer. I strongly recommend you forget about any CORS configuration and use readymade solution and it will work anywhere.
Take a look here:
https://github.com/jpillora/xdomain
And working example:
http://jpillora.com/xdomain/

For a developer who understands the reason it exists but needs to access an API that doesn't handle OPTIONS calls without auth, I need a temporary answer so I can develop locally until the API owner adds proper SPA CORS support or I get a proxy API up and running.
I found you can disable CORS in Safari and Chrome on a Mac.
Disable same origin policy in Chrome
Chrome: Quit Chrome, open an terminal and paste this command: open /Applications/Google\ Chrome.app --args --disable-web-security --user-data-dir
Safari: Disabling same-origin policy in Safari
If you want to disable the same-origin policy on Safari (I have 9.1.1), then you only need to enable the developer menu, and select "Disable Cross-Origin Restrictions" from the develop menu.

As mentioned in previous posts already, OPTIONS requests are there for a reason. If you have an issue with large response times from your server (e.g. overseas connection) you can also have your browser cache the preflight requests.
Have your server reply with the Access-Control-Max-Age header and for requests that go to the same endpoint the preflight request will have been cached and not occur anymore.

I have solved this problem like.
if($_SERVER['REQUEST_METHOD'] == 'OPTIONS' && ENV == 'devel') {
header('Access-Control-Allow-Origin: *');
header('Access-Control-Allow-Headers: X-Requested-With');
header("HTTP/1.1 200 OK");
die();
}
It is only for development. With this I am waiting 9ms and 500ms and not 8s and 500ms. I can do that because production JS app will be on the same machine as production so there will be no OPTIONS but development is my local.

You can't but you could avoid CORS using JSONP.

you can also use a API Manager (like Open Sources Gravitee.io) to prevent CORS issues between frontend app and backend services by manipulating headers in preflight.
Header used in response to a preflight request to indicate which HTTP headers can be used when making the actual request :
content-type
access-control-allow-header
authorization
x-requested-with
and specify the "allow-origin" = localhost:4200 for example

After spending a whole day and a half trying to work through a similar problem I found it had to do with IIS.
My Web API project was set up as follows:
// WebApiConfig.cs
public static void Register(HttpConfiguration config)
{
var cors = new EnableCorsAttribute("*", "*", "*");
config.EnableCors(cors);
//...
}
I did not have CORS specific config options in the web.config > system.webServer node like I have seen in so many posts
No CORS specific code in the global.asax or in the controller as a decorator
The problem was the app pool settings.
The managed pipeline mode was set to classic (changed it to integrated) and the Identity was set to Network Service (changed it to ApplicationPoolIdentity)
Changing those settings (and refreshing the app pool) fixed it for me.

OPTIONS request is a feature of web browsers, so it's not easy to disable it. But I found a way to redirect it away with proxy. It's useful in case that the service endpoint just cannot handle CORS/OPTIONS yet, maybe still under development, or mal-configured.
Steps:
Setup a reverse proxy for such requests with tools of choice (nginx, YARP, ...)
Create an endpoint just to handle the OPTIONS request. It might be easier to create a normal empty endpoint, and make sure it handles CORS well.
Configure two sets of rules for the proxy. One is to route all OPTIONS requests to the dummy endpoint above. Another to route all other requests to actual endpoint in question.
Update the web site to use proxy instead.
Basically this approach is to cheat browser that OPTIONS request works. Considering CORS is not to enhance security, but to relax the same-origin policy, I hope this trick could work for a while. :)

One solution I have used in the past - lets say your site is on mydomain.com, and you need to make an ajax request to foreigndomain.com
Configure an IIS rewrite from your domain to the foreign domain - e.g.
<rewrite>
<rules>
<rule name="ForeignRewrite" stopProcessing="true">
<match url="^api/v1/(.*)$" />
<action type="Rewrite" url="https://foreigndomain.com/{R:1}" />
</rule>
</rules>
</rewrite>
on your mydomain.com site - you can then make a same origin request, and there's no need for any options request :)

It can be solved in case of use of a proxy that intercept the request and write the appropriate headers.
In the particular case of Varnish these would be the rules:
if (req.http.host == "CUSTOM_URL" ) {
set resp.http.Access-Control-Allow-Origin = "*";
if (req.method == "OPTIONS") {
set resp.http.Access-Control-Max-Age = "1728000";
set resp.http.Access-Control-Allow-Methods = "GET, POST, PUT, DELETE, PATCH, OPTIONS";
set resp.http.Access-Control-Allow-Headers = "Authorization,Content-Type,Accept,Origin,User-Agent,DNT,Cache-Control,X-Mx-ReqToken,Keep-Alive,X-Requested-With,If-Modified-Since";
set resp.http.Content-Length = "0";
set resp.http.Content-Type = "text/plain charset=UTF-8";
set resp.status = 204;
}
}

What worked for me was to import "github.com/gorilla/handlers" and then use it this way:
router := mux.NewRouter()
router.HandleFunc("/config", getConfig).Methods("GET")
router.HandleFunc("/config/emcServer", createEmcServers).Methods("POST")
headersOk := handlers.AllowedHeaders([]string{"X-Requested-With", "Content-Type"})
originsOk := handlers.AllowedOrigins([]string{"*"})
methodsOk := handlers.AllowedMethods([]string{"GET", "HEAD", "POST", "PUT", "OPTIONS"})
log.Fatal(http.ListenAndServe(":" + webServicePort, handlers.CORS(originsOk, headersOk, methodsOk)(router)))
As soon as I executed an Ajax POST request and attaching JSON data to it, Chrome would always add the Content-Type header which was not in my previous AllowedHeaders config.

Related

Sec-Fetch-Mode instead of Preflight

I created login FE and finished it.
And as per usual my goto for ajax was Axios. My code is as follows.
const baseUrl = http://localhost:5000/project/us-central1/api
Axios.post(
`${baseUrl}/v1/user/login`,
{ ...data },
{
headers: {
Authorization: 'Basic auth...'
}
},
).then(r => console.log(r).catch(e =>console.log(e));
Now when i try to send request to my local firebase cloud function.
I get a 400 bad request.
after checking the request, I was wondering why it wasn't sending any preflight request, which it should do(to the best of my knowledge) but instead I saw a header named Sec-Fetch-Mode. I searched anywhere it's a bit abstract. And I can't seem to figure anything why my request still fails.
Is there anything Im missing in my config of axios?
My FE is running on a VSCode Plugin named live server(http://127.0.0.1:5500)
Also, my firebase cloud function has enabled cors
// cloud function expres app
cors({
origin: true
})
Any insights would be very helpful.
The OPTIONS request is actually being sent, because you are sending a cross-origin request with an Authorization header which is considered as non-simple. It doesn't show in developer tools because of a feature/bug in Chrome 76 & 77. See Chrome not showing OPTIONS requests in Network tab for more information.
The preflight request is a mechanism that allows to deny cross-origin requests on browser side if the server is not CORS aware (e.g: old and not maintained), or if it explicitly wants to deny cross-origin requests (in both cases, the server won't set the Access-Control-Allow-Origin header). What CORS does could be done on server side by checking the Origin header, but CORS actually protects the user at browser level. It blocks the disallowed cross-origin requests even before they are sent, thus reducing the network traffic, the server load, and preventing the old servers from receiving any cross-origin request by default.
On the other hand, Sec-Fetch-Mode is one of the Fetch metadata headers (Sec-Fetch-Dest, Sec-Fetch-Mode, Sec-Fetch-Site and Sec-Fetch-User). These headers are meant to inform the server about the context in which the request has been sent. Based on this extra information, the server is then able to determine if the request looks legitimate, or simply deny it. They exist to help HTTP servers mitigate certain types of attacks, and are not related to CORS.
For example the good old <img src="https://mybank.com/giveMoney?amount=9999999&to=evil#attacker.com"> attack could be detected on server side because the Sec-Fetch-Dest would be set to "image" (this is just a simple example, implying that the server exposes endpoints with the GET method with unsafe cookies for money operations which is obviously not the case in real life).
As a conclusion, fetch metadata headers are not designed to replace preflight requests, but rather to coexist with them since they fulfill different needs. And the 400 error has likely nothing to do with these, but rather with the request that does not comply with the endpoint specification.
You are missing a dot on your spread operator, this is the correct syntax:
{ ...data }
Note the three dots before “data”.
Please see the use of spread operators with objects here:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_syntax

How is it possible to enable CORS from server-side?

From my little experience in web API, same origin policy is a policy of browsers i.e, browser doesn't allow to make requests to other hosts rather than the origin. I wonder how it is possible to enable CORS from server side(talking about ASP.net Web API)?
This is how i enable CORS in webAPI
namespace WebService.Controllers
{
[EnableCors(origins: "*", headers: "*", methods: "*")]
public class TestController : ApiController
{
// Controller methods ...
}
}
If CORS is a browser thing, isn't it more logical to enable it from client side. Can anybody clear this out
Here’s an attempt at a short summary of how it works: The browser is where the same-origin policy and cross-origin restrictions are enforced. Specifically, browsers block frontend JavaScript code from being able to access responses from cross-origin requests—unless the servers the requests are made to send the response header Access-Control-Allow-Origin in responses.
In other words, the way for getting browsers to relax the same-origin policy is for servers to use the Access-Control-Allow-Origin header to indicate they’re opting in to cross-origin requests.
So, browsers are the place where any cross-origin restrictions are either being applied or relaxed.
One case that helps to illustrate how it works is a simple cross-origin POST. As long as a cross-origin POST doesn’t have any custom request headers that will trigger browsers to do a CORS preflight OPTIONS request, a browser will go ahead and make the request, even cross-origin. And the server that POST is sent to will go ahead and accept it and then send a response.
What happens then is where the cross-origin restrictions from browsers kick in—because if that POST request was sent from frontend JavaScript code using XHR or the Fetch API or an Ajax method from some JavaScript library, then unless the response includes the Access-Control-Allow-Origin header, browsers won’t allow the frontend code to access the response (even though the server accepted the POST and it succeeded).
Anyway, I hope the above helps to clarify what enabling CORS support in servers actually means, and what effects it has, and that the actual policy enforcement is performed by browsers.
Of course all of the above just describes the simplest case, where there are no characteristics of the request that will trigger browsers to do a CORS preflight OPTIONS request.
But still in that case, the policy enforcement is all performed by the browser—in fact even more so, in that, for example, browsers won’t allow a POST with custom headers to even be sent to a server to begin with unless the server explicitly indicates (in its response to the preflight OPTIONS) that the server has opted in to receiving cross-origin requests which include that custom header.

Fetch API, custom request headers, CORS, and cross-origin redirects

I need to make an HTTP GET request with custom request headers in-browser and process the result as it streams in. The Fetch API is ideal for this:
fetch('https://example.com/resource', {
method: 'GET',
headers: {
'X-Brad-Test': 'true'
},
cache: 'no-store',
mode: 'cors'
}).then((res) => {
const reader = res.body.getReader();
// etc.
});
This works quite well. Since there are custom headers, the browser pre-flights the request with an OPTIONS request to /resource. I have configured my server to respond with a 204 No Content and the following headers:
Access-Control-Allow-Headers: X-Requested-With, Range, If-Range, X-Brad-Test
Access-Control-Allow-Origin: *
The browser is happy with this, then makes a GET request, the server returns a 200 OK with the data, and the browser allows me to access the response headers and body.
The problem comes in when there is a redirect. The OPTIONS request succeeds with the 204 No Content and the same headers as before. The browser makes the correct GET request, and on the server I send a 302 with a Location: header. Chrome throws the following error:
Fetch API cannot load https://example.com/resource. Redirect from 'https://example.com/resource' to 'http://some-other-origin/resource' has been blocked by CORS policy: Request requires preflight, which is disallowed to follow cross-origin redirect.
This was unexpected, and seems nonsensical to me. I expected the browser to follow the redirect, and do another pre-flight request for this new location, but it didn't do that.
Stranger still is that I can sort of hack around this client-side. I can make an HTTP request without my custom header, figure out where I ended up after redirects by looking at the Response object, then make a second request at the new target with my custom headers. This doesn't work in all cases of course, and I'd rather not rely on this hack. I'd rather find a proper way.
Two Questions:
What is the proper way to allow the client to follow redirects? Is there some sort of Access-Control-* header I can use?
Why does this restriction exist? What security issue is prevented by not following and running pre-flight on the followed URL?
Supporting redirects to requests that require a preflight is very recent change to Fetch (which defines CORS).
https://github.com/whatwg/fetch/commit/0d9a4db8bc02251cc9e391543bb3c1322fb882f2
I believe some implementations have started adjusting their implementations, but this will take some time to reach everyone.

Https page doing Http request using CORS

Reading the CORS spec it states [abridged]:
... user agents are allowed to terminate the algorithm and not
make a request. This could be done because e.g.:
...
https to http is not allowed.
...
This appears to be the behavior in Chrome, however I have not found any reference to this behavior on the various browser vendors documentation.
For performance reasons we wish to make an http request from a https page for certain content. It is a request to a different domain entirely with the correct CORS headers to do http requests (which work ok http->http).
Are there any headers that can be set to allow https -> http? It seems sensible to disallow mixed-mode on a single domain / sub-domains, but cross-domain seems over protective, especially if the other domain allows http normally.
I'm imagining the 'preflight check' or a 'simple request' would be allowed if it didn't have HSTS headers or alike.

Is caching in browser automatic?

I have a JavaScript app that sends requests to REST API, the responses from server have cache headers (like ETag, cache-control, expires). Is caching of responses in browser automatic, or the app must implement some sort of mechanism to save the data?
An AJAX request is no different from a normal request - it's a GET/POST/HEAD/whatever request being sent by the browser, and it is handled as such. This is confirmed here:
The HTTP and Cache sub-systems of modern browsers are at a much lower level than Ajax’s XMLHttpRequest object. At this level, the browser doesn’t know or care about Ajax requests. It simply obeys the normal HTTP caching rules based on the response headers returned from the server.
As per the jQuery documentation, caches can also be invalidated in at least one usual way (appending a query string):
cache (default: true, false for dataType 'script' and 'jsonp')
Type: Boolean
If set to false, it will force requested pages not to be cached by the browser. Note: Setting cache to false will only work correctly with HEAD and GET requests. It works by appending "_={timestamp}" to the GET parameters. The parameter is not needed for other types of requests, except in IE8 when a POST is made to a URL that has already been requested by a GET.
So in short, given the same headers, AJAX responses are cached the same way as other requests.
Browser handles automatically cache of resources. What you seem to be asking about is the actuall response from the server.
You will need to set that up yourself in your application. You can do so both on front-end and backend.
Most of JS frameworks have cache control implemented, for example:
jQuery
$.ajaxSetup({
// Disable caching of AJAX responses
cache: false
});
AngularJS
$http.defaults.cache = false;
etc.
On backend it really depends on what language you using, what server engine etc.
Checkout Memcached for example
http://memcached.org/
As with anything with web development there are odd things here and there, for example some IE versions are automatically chaching requests and you have to add unique id to the url to prevent that.
From https://developer.mozilla.org/en-US/docs/AJAX/Getting_Started :
Note 2: If you do not set header Cache-Control: no-cache the browser will cache the response and never re-submit the request, making debugging "challenging." You can also append an always-diferent aditional GET parameter, like the timestamp or a random number (see bypassing the cache)
Browser should handle the cache automatically.
Check this article, there's only clear cache method for javascript.
https://developer.chrome.com/extensions/browsingData
If the server sends the response with any of cache headers browsers should respect it. there is no difference between resources or ajax requests.
also you can specify cache headers in your ajax calls to make it not use caches and fetch the whole response from the server.
Most modern browsers support browser caching because of the cache and expire headers
http://www.arlocarreon.com/blog/http/http-requests-and-your-browsers-cache/
There are 2 apects of an HTTP request that can qualify it for being cached:
- Specifc HTTP cache and expire headers
- A unique URL
Interesting read:
http://www.mobify.com/blog/beginners-guide-to-http-cache-headers/

Resources