I built a Next-app on the frontend and deployed it on Vercel (and a Django-app on the backend deployed on Heroku). Authentication is implemented with JWTs and and a Next-API route connecting the Next-Frontend with the Django-Backend.
The authentication workflow relevant for my problem is the following:
Whenever an authenticated user visits a protected page, a request (with the current refresh-token) is sent to the backend, asking for a fresh pair of access & refresh tokens. This works perfectly fine when testing locally the production-optimized version of the Nextjs-app (via next build & next start): I receive a 200 OK response with the new refresh- and access-tokens.
However - when I do the same thing in my Vercel-deployed app, after the first request/response-cycle for new tokens (which works as expected), I always get back a 304 Not Modified-response, which does not have any cookies in its Headers. I can see that my Django-server sends off the new tokens but somehow they get lost on the Next-server.
I did some research and if I understand correctly, Vercel has some caching rules, that lead to this behavior. However, I could not find any solution on how to avoid this from happening.
Does anyone know, what to do in this case?
I figured it out by myself, sharing the solution for anyone who might run into this problem in the future. When it comes to caching - Vercel's production servers behave differently than the test server you can run with npm run start. While the former caches certain responses by default for performance reasons, the latter does not do that.
To deactivate the caching for the response giving me the unwanted 304 Not Modified status, i simply added a Cache-Control-header with no-cache, no-store, max-age=0 and must-revalidate directives to it:
export default async function refresh(req, res) {
...
if (apiRes.status === 200) {
// "Cache-Control"-header needed for production to avoid Vercel's default
// response caching mechanism
res.setHeader(
"Cache-Control",
"no-cache, no-store, max-age=0, must-revalidate"
);
...
I have an DotNet Core API which we are running in Kestrel Server
In the response headers it adds the header "Content-Length" which is causing an issue.
Is there any way to remove the Content-Length Header for all the responses ?
I tried below code But it only removes the "Server: Kestrel" type header.
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
.UseKestrel(options => options.AddServerHeader = false);
});
If I remember correctly, AddServerHeader only corresponds to the type of WebServer, meaning "Kestrel" keyword.
I don't think there is a built-in mechanism to remove "Content-Length", thus you should create a middleware and after the result you should remove this header.
I would also indicate that when removing this header the client will continue reading the response (or wait), so you will need to force close the connection each time, which isn't recommended.
You can further read here: https://stackoverflow.com/a/15995101/303254
This question already has an answer here:
How to send an HTTP 4xx-5xx response with CORS headers in an ASPNET.Core web app?
(1 answer)
Closed 3 years ago.
I am migrating and existing angular js app to angular while sticking to an existing api.
I tried to implement a logon (ASP NET / OWIN) and encountered following situation after making a http request to logon endpoint:
All possible server responses (2xx,4xx,5xx) could not be fetched in javascript code.
After researching I found the cause: CORS
This is because new angular app runs on different port than the locally hosted api while the old angular js ab is being hosted on the same port.
So I added following to ValidateClientAuthentication in API:
context.Response.Headers.Append("Access-Control-Allow-Origin", "http://localhost:4200");
context.Response.Headers.Append("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS");
Now I am finally able to fetch 2xx responsed but still do not get 4xx and 5xx results.
Why is that?
And how is it possible that API calls to other endpoints of same API do not require setting those headers?
UPDATE:
I receive this error message for all 4xx responses even if I have added those headers in API:
:63323/api/logon:1 Failed to load resource: the server responded with a status of 400 (Bad Request)
vendor.js:68504 ERROR TypeError: You provided an invalid object where a stream was expected. You can provide an Observable, Promise, Array, or Iterable.
at subscribeTo (vendor.js:158625)
at subscribeToResult (vendor.js:158782)
at CatchSubscriber.error (vendor.js:151269)
at MapSubscriber.Subscriber._error (vendor.js:147999)
at MapSubscriber.Subscriber.error (vendor.js:147979)
at XMLHttpRequest.onLoad (vendor.js:24621)
at ZoneDelegate.push../node_modules/zone.js/dist/zone.js.ZoneDelegate.invokeTask (polyfills.js:2785)
at Object.onInvokeTask (vendor.js:70070)
at ZoneDelegate.push../node_modules/zone.js/dist/zone.js.ZoneDelegate.invokeTask (polyfills.js:2784)
at Zone.push../node_modules/zone.js/dist/zone.js.Zone.runTask (polyfills.js:2557)
UPDATE 2:
this is the code that handles http request:
return this.httpClient
.post(this.appConfigService.buildAuthorizationUrl('/logon'), data, {
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
})
.toPromise()
.then(
(response: any) => {
// hit when 2xx and cors headers present
},
(error: any) => {
// never hit
}
)
.catch(error => {
// never hit
});
UPDATE 3:
As I understood CORS middleware might(?) drop all heades in case 4xx or 5xx responses which will prevent browsers from fetching responses. But like you can see my 400 response from a different endpoint on my api seems to be valid:
HTTP/1.1 400 Bad Request
Cache-Control: private,no-cache, no-store, must-revalidate
Pragma: no-cache
Content-Length: 112
Content-Type: application/json; charset=utf-8
Expires: 0
Server: Microsoft-IIS/10.0
Access-Control-Allow-Origin: http://localhost:4200
X-AspNet-Version: 4.0.30319
X-SourceFiles: =?UTF-8?B?QzpcVXNlcnNcS2F5IFphbmRlclxEb2N1bWVudHNcZXZlXFNvdXJjZVxJbmZvcnNIVC5HZW5lc2lzQXBpXGF1ZGl0dHJhaWw=?=
X-Powered-By: ASP.NET
X-UA-Compatible: IE=edge
Date: Wed, 06 Nov 2019 16:01:08 GMT
[{"ErrorMessage":"Der Objektverweis wurde nicht auf eine Objektinstanz festgelegt.","PropertyName":"Exception"}]
still I encounter the mentioned error message.
Most likely your ASP.NET backend doesn't have CORS enabled, according to the fact that if you set it manually you're able to make a request. If you set a header (e.g. Content-Type) to your request, whether it was manually set or automatically, Angular sends a preflight request for the backend to determine if the resource can be accessed with the specific request headers or not.
There's a built-in support for preflight request handling, which you can enable using attributes to your controllers, and a little configuration in the initial startup class.
For ASP.NET (.NET Framework based) check this guide: https://learn.microsoft.com/en-us/aspnet/web-api/overview/security/enabling-cross-origin-requests-in-web-api
For ASP.NET Core check this one: https://learn.microsoft.com/en-us/aspnet/core/security/cors?view=aspnetcore-3.0
Why does all responses from ASP.NET contain Cache-Control: private? Even a 404 response? Is there something in IIS that sets this default value, and is there a way to configure it? Or is there something in ASP.NET that sets this?
For dynamic content (that is, all MVC results) I would not like it to be cached by the browser, since it is dynamic and can change at any time. Static content is hosted on a CDN, so is not served by IIS.
Edit:
To clarify, I understand very well what Cache-Control: private is, the difference between private, public, no-store, etc and how/when to use them. The question I have is why Cache-Control: private is added by default by IIS/ASP.NET and how to prevent it from being added by default. I understand that it can be useful to cache dynamic pages, but in my application I don't want to cache dynamic pages/responses. For example, I don't want XHR JSON responses to be cached, since they contain dynamic content. Unfortunately the server adds Cache-Control: private to all responses automatically, so I have to manually override it everywhere.
How to reproduce: Open visual studio and create a new ASP.NET Framework (yes, framework, no not Core. We are not able to migrate our system to core yet) solution with an MVC project. Now start the project in IIS Express (just press the play button), and use F12 devtools in the browser to look at the http response. You will see that it contains Cache-Control: private. My question is, what adds this header, and how can I prevent it from being added by default?
Adding my bit to the great answers, given by community;
1. http caching header attrubute Cache-Control: private is added by default by IIS/ASP.NET ?
Cache request directives
Standard Cache-Control directives that can be used by the client in an HTTP request.
Cache-Control: max-age=<seconds>
Cache-Control: max-stale[=<seconds>]
Cache-Control: min-fresh=<seconds>
Cache-Control: no-cache
Cache-Control: no-store
Cache-Control: no-transform
Cache-Control: only-if-cached
Cache response directives
Standard Cache-Control directives that can be used by the server in an HTTP response.
Cache-Control: must-revalidate
Cache-Control: no-cache
Cache-Control: no-store
Cache-Control: no-transform
Cache-Control: public
Cache-Control: private
Cache-Control: proxy-revalidate
Cache-Control: max-age=<seconds>
Cache-Control: s-maxage=<seconds>
IIS uses the secure and more obvious/useful one for default, which happens to be private
2. how to prevent it from being added by default?
IIS/asp.net allows this to be configured from the day it was introduced like this, ref1, ref2, ref3, ref4 and
System.Web Namespace
The System.Web namespace supplies classes and interfaces that enable browser-server communication. This namespace includes the System.Web.HttpRequest class, which provides extensive information about the current HTTP request; the System.Web.HttpResponse class, which manages HTTP output to the client; and the System.Web.HttpServerUtility class, which provides access to server-side utilities and processes. System.Web also includes classes for cookie manipulation, file transfer, exception information, and output cache control.
protected void Application_BeginRequest()
{
Context.Response.Cache.SetCacheability(HttpCacheability.NoCache);
}
TL; DR
Caching is not used by default for dynamic ASP.NET pages. You need to make an efforts to enable caching in ASP.NET.
Presence of 'Cache-Control: private' header does not mean at all that cached version of the page will be used on repeated requests.
--
There is a very simple test to validate above statements. Create an action that returns current time:
public ActionResult Index()
{
ViewBag.CurrTime = DateTime.Now.ToString("T");
return View();
}
View:
#{
ViewBag.Title = "Home Page";
}
<h1>#ViewBag.CurrTime</h1>
If you refresh such page in abrowser, you'll see fresh time on every request:
There is a possibility to use caching with ASP.NET MVC but you should make some efforts to enable it. See this article for details.
If in spite of this you still for some reason want to exclude any possibility of the caching, you could do it by setting specific HTTP headers. There is a great SO answer that lists which headers should be set:
Cache-Control: no-cache, no-store, must-revalidate
Pragma: no-cache
Expires: 0
You could use action filter to set those headers in every ASP.NET response:
public class CachingHeadersFilterAttribute : ActionFilterAttribute
{
public override void OnResultExecuted(ResultExecutedContext filterContext)
{
var response = filterContext.HttpContext.Response;
response.Cache.SetCacheability(HttpCacheability.NoCache);
response.Cache.AppendCacheExtension("no-store, must-revalidate");
response.AppendHeader("Pragma", "no-cache");
response.AppendHeader("Expires", "0");
base.OnResultExecuted(filterContext);
}
}
In FilterConfig.cs (crated automatically in ASP.NET MVC template):
public class FilterConfig
{
public static void RegisterGlobalFilters(GlobalFilterCollection filters)
{
filters.Add(new HandleErrorAttribute());
filters.Add(new CachingHeadersFilterAttribute());
}
}
Here are result headers from the response:
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store, must-revalidate
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Expires: -1
Vary: Accept-Encoding
Server: Microsoft-IIS/10.0
X-AspNetMvc-Version: 5.2
X-AspNet-Version: 4.0.30319
X-SourceFiles: =?UTF-8?B?RDpcRHJvcGJveFxwcm9nXFN0YWNrT3ZlcmZsb3dcUTQ3MjI0NTYxQ2FjaGVcUTQ3MjI0NTYxQ2FjaGU=?=
X-Powered-By: ASP.NET
Date: Mon, 13 Nov 2017 17:44:33 GMT
Content-Length: 837
As you see there is no 'Cache-Control: private' header.
But again, I don't see a reason why you should add such filter to your application.
what adds this header?
The IIS 7 and later (included in the default installation) sends it to Web clients as one of the cache-related HTTP headers.
and how can I prevent it from being added by default?
I describe two ways of disabling cache mechanism (server-side approach and client-side approach) with a focus on server-side based on your question:
Server-Side Approach
Follow this steps Inside the Internet Information Services (IIS) Manager to change Cache-Control value (while I think this works for static contents):
In the Connections pane, go to the site, application, or
directory for which you want to disable caching.
In the Home pane, double-click HTTP Response Headers.
Click Set Common Headers ... in the Actions pane.
Check the box to expire Web content, select the option to expire after a specific interval or at a specific time,
and then click OK.
One way is to annotate your controller as follows which is quite powerful and the response headers contain a Cache-Control: public, max-age=0 header.:
[OutputCache(Duration = 0)]
public class SomeController : Controller {
}
You can also Define a cache profile in your application's Web.config file and in the profile, include duration and varyByParam settings:
<caching>
<outputCacheSettings>
<outputCacheProfiles>
<add name="nocache" duration="0"
varyByParam="none" />
</outputCacheProfiles>
</outputCacheSettings>
</caching>
Then use it by the [OutputCache CacheProfile="nocache"] before the action/controller.
Note that there is a configuration in web.config file (following) which won’t prevent caching. Instead it just indicates that no kind of caching mechanism should be applied. By just disabling the output cache we get the default cache headers used by ASP.net MVC which falls back to Cache-Control: private, thus again opening the browser the possibility to cache requests.
<caching>
<outputCache enableOutputCache="false" />
</caching>
Client-Side Approach
Using cache: false inside your js request like:
$.ajax({
type: 'GET',
cache: false,
url: '/nation',
...
});
For additional information, visit:
Client Cache <clientCache>
How to: Set the Cacheability of an ASP.NET Page Declaratively
output-caching-in-aspnet-mvc
An answer from RickNZ, copied from https://forums.asp.net
Cache-Control private says that it's OK for the client to cache the page, subject to its expiration date. The expiration can either be provided with Cache-Control: max-age, or with an Expires HTTP header. In the default case, the page is set to expire immediately, which means that it won't be cached.
One of the purposes of Cache-Control: private is really to tell intermediate proxies that they should not cache the page.
BTW, just because a page is dynamic doesn't mean that it should never be cached. There are many cases where caching a dynamic page is appropriate. You can cache not only at the client, but also in proxies and in the server's output cache.
More info:
IIS 7.0 - IIS adding "private" to cache-control, where is that coming from
Private vs Public in Cache-Control
https://msdn.microsoft.com/en-us/library/ms524721(v=vs.90).aspx
https://msdn.microsoft.com/en-us/library/system.web.httpcacheability(VS.71).aspx
https://forums.asp.net/t/1443346.aspx?Cache+control+private+
https://forums.asp.net/t/2052325.aspx?Remove+the+private+value+from+the+Cache+Control+in+the+Response+Header
The Cache-Control: private header is added to the response by the framework by default.
https://learn.microsoft.com/en-us/dotnet/api/system.web.configuration.httpruntimesection.sendcachecontrolheader?view=netframework-4.8
Definition
Gets or sets a value indicating whether the cache-control:private header is sent by the output cache module by default.
...
Remarks
The HttpResponse class checks both the HttpRuntimeSection.SendCacheControlHeader property and the OutputCacheSection.SendCacheControlHeader property to determine whether to send the cache-control:private header in the HTTP response. If either property is set to false, the header will not be sent. When the cache-control header is set to private, then the client will not cache the response in a shared cache.
Support for the SendCacheControlHeader property in the HttpRuntimeSection class is provided for compatibility with legacy applications; this property is obsolete in the .NET Framework version 2.0. For more information, see the OutputCacheSection class.
To prevent the Cache-Control: private header from being added, simply disable the OutputCacheSection.SendCacheControlHeader property, which defaults to true.
https://learn.microsoft.com/en-us/dotnet/api/system.web.configuration.outputcachesection.sendcachecontrolheader?view=netframework-4.8#system-web-configuration-outputcachesection-sendcachecontrolheader
Property Value
true if the sending of cache-control:private header is enabled; otherwise, false. The default is true.
Example web.config
<configuration>
<system.web>
<caching>
<outputCache sendCacheControlHeader="false" />
</caching>
</system.web>
</configuration>
The default is specified in System.Web.HttpResponse.CacheControl:
/// <devdoc>
/// <para>
/// Provided for ASP compatiblility. Use the <see cref='System.Web.HttpResponse.Cache'/>
/// property instead.
/// </para>
/// </devdoc>
public string CacheControl {
get {
if (_cacheControl == null) {
// the default
return "private";
}
return _cacheControl;
}
While you can override the header through (global) filters, this doesn't work for error pages caused by authentication/authorization. Luckily there's a nice entry point for each request, allowing you to override this default:
// In Global.asax.cs:
protected void Application_BeginRequest()
{
Context.Response.CacheControl = "no-cache";
}
Furthermore when there's an error and the YSOD (yellow error page) is rendered through ReportRuntimeError, the framework will call ClearHeaders and your custom cache-control setting will be overridden. I haven't found a solution for this.
Addressing the question:
You will see that it contains Cache-Control: private. My question is,
what adds this header, and how can I prevent it from being added by
default?
The Short answer is: as others have noted, this is a default setting by IIS(7+).
To answer more fully: If you are looking to control or modify the default setting Cache-Control: private for caching HTTP requests, server side.
This will depend on your the which specific version of IIS you are running, and how specific you want to be with your modification. Generally speaking however you can do this using your HTTP Response Headers interface (in IIS Manager => Features view - IIS)
If you want to simply disable caching for a Web site or Application:
go to IIS Manager
Select Site/Application desired
in Features View select HTTP Response Headers:
In actions pane click: Set Common Headers
check Expire Web Content
Set to: Immediately
OK
alternatively, from your command line:
appcmd.exe set config "Default Web Site" -section:system.webServer/staticContent /clientCache.cacheControlMode:"DisableCache"
For Detailed Information and specifics please check:
https://learn.microsoft.com/en-us/iis/configuration/system.webserver/staticcontent/clientcache
Hope this is what your were looking for, cheers.
I created a new solution from WebApi .Net Core 2.0 template in Visual Studio.
I added the following in startup.
public void ConfigureServices(IServiceCollection services)
{
services.AddMvc();
services.AddHttpCacheHeaders(opt => opt.MaxAge = 600);
services.AddResponseCaching();
}
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
app.UseResponseCaching();
app.UseHttpCacheHeaders();
app.UseMvc();
}
Then, with postman, I hit http://localhost:xxxx/api/values which is a endpoint created by the template and that returns ["value1","value2"]
Note that I made sure that Postman doesn't send no-cache header (in Postman settings).
The HttCacheHeaders service comes from that repo. It adds HTTP Cache Headers. So my endpoint response header is:
Cache-Control: public,max-age=600
Content-Type: application/json; charset=utf-8
Date: Fri, 29 Sep 2017 14:02:29 GMT
ETag: C5DFA8974BB722D27E71EE50D3D14625
Expires: Fri, 29 Sep 2017 14:03:29 GMT
Last-Modified: Fri, 29 Sep 2017 14:02:29 GMT
Server: Kestrel
Transfer-Encoding: chunked
Vary: Accept, Accept-Language, Accept-Encoding
X-Powered-By: ASP.NET
X-SourceFiles: =?UTF-8?B?................
The problem is that nothing gets cached. the Ouput windows only shows The response could not be cached for this request.
So I'm a bit lost on how to use ASP.NET Core ResponseCaching Middleware.
Update
If I don't use the HttCacheHeaders service but add [ResponseCache(Duration = 600)] to the action of my controller, the cache works.
Note that reason that I want to use HttCacheHeaders is for ETag and Last-Modified to later do Validation Caching as well as Expiration Caching.
This is kind of a non-answer, with some troubleshooting tips.
I tried your exact code & it worked fine for me. Maybe a bug has been fixed in the HttpCacheHeaders or ResponseCaching repo?
Unfortunately, debugging the server-side ResponseCaching is tricky because it has weird rules & there's not adequate logging. When I've had similar issues with it in the past I've had to pull down Microsoft's source code to step through it & find the issue with my code.
The note you found in the output window "The response could not be cached for this request" is a clue.
There's 2 parts to the server-side caching of a request. The server has to prime the cache the first time the url is requested. It will serve the cached version the 2nd time it's requested. Pay attention to when the error message shows up, if it's on the 1st or 2nd request. That'll tell you if it couldn't be stored in the cache or if it couldn't be retrieved from the cache.
The rules for both storage & retrieval are in this source code file: https://github.com/aspnet/ResponseCaching/blob/3bf5f6a1ce69b65c998d6f5c739822a9bed4a67e/src/Microsoft.AspNetCore.ResponseCaching/Internal/ResponseCachingPolicyProvider.cs
Your "Cache-Control:public,max-age=600" header should match these rules just fine.
You already found the no-cache/no-store "gotcha". There are also a few other others with ResponseCaching to watch out for:
Authenticated requests & responses with set-cookie won't be cached. Only requests using GET or HEAD method will be cached. If the QueryString is different, it'll make a new cache entry. Also, usually you'll want a "Vary" header to prevent caching if certain conditions of a request differ from the previously-cached request (example: user-agent, accept-encoding, etc).
On a side note, honoring the no-cache/no-store request headers was probably a poor design choice since ASP.Net Core's ResponseCache will most likely be used by a server who owns the response, rather than an intermediary cache like a CDN/ISP. I've extended the base ResponseCache with an option to disable honoring these headers (as well as serialize the cache to disk, rather than in-memory only). It's an easy drop-in replacement for the default cache.
You can find my extension here: https://github.com/speige/AspNetCore.ResponseCaching.Extensions https://www.nuget.org/packages/AspNetCore.ResponseCaching.Extensions
HttpCacheHeader is not a cache store.
This middleware handles the "backend"-part: it generates the correct cache-related headers, and ensures a cache can check for expiration (304 Not Modified) & preconditions (412 Precondition Failed) (often used for concurrency checks).
Source