Firebase Hosting is caching my dynamic `robots.txt` on the CDN - firebase

From Firebase docs, we get that it's supposed to set Cache-Control: private as default.
Firebase Hosting- Manage cache behavior
I generate my robots.txt on the fly, hence it might change from one request to the next. Because of that, I don't want it cached by the CDN. But to my surprise, I'm getting CDN hits for robots.txt.
See:
I do the same with ads.txt and sitemap.xml. The same is happening with ads.txt, but not with sitemap.xml.
Maybe it's something related to .txt files. Could this be a bug?
Do I have to explicitly set the Cache-Control: private?

You can configure that specific file's cache-control header through the firebase.json file like this:
{
"hosting": [
{
...,
"headers": [
{
"source": "robots.txt",
"headers": [
{
"key": "Cache-Control",
"value": "no-cache"
}
]
}
]
}
]
}
More info in the docs here.

Related

CORS error when using Firebase Hosting and Cloud Run

I have a SPA with the following setup:
Frontend: React deployed using Firebase Hosting
Backend: GraphQL API written in Django deployed on Cloud Run
Everything works locally but when I deploy the app and try to make requests, I get a CORS error due to preflight missing allow origin header:
Access to fetch at 'https://cloud-run-api-hash.a.run.app/graphql/' from origin 'https://project-id.web.app' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
Using django-cors-headers, I have already added the Firebase domain to the CORS_ORIGIN_WHITELIST in my Django settings.
My firebase.json looks like this:
{
"hosting": {
"public": "build",
"ignore": [
"firebase.json",
"**/.*",
"**/node_modules/**"
],
"rewrites": [
{
"source": "/authenticated/**",
"run": {
"serviceId": "cloud-run-serviceID",
"region": "us-central1"
}
},
{
"source": "**",
"destination": "/index.html"
}
],
"headers": [ {
"source": "**/*.#(eot|otf|ttf|ttc|woff|font.css)",
"headers": [ {
"key": "Access-Control-Allow-Origin",
"value": "*"
} ]
} ]
}
}
My intention is to trigger the cloud run container after the user is authenticated and is routed to https://project-id.web.app/authenticated, but I'm not seeing any Cloud Run logs after I login to the app. Furthermore, when I try to send any HTTP requests I get the CORS error shown above. Not sure where went wrong here because I strictly followed the Cloud Run/Firebase Hosting documentation. Any advice is greatly appreciated.
It turns out that the problem was with django-cors-headers. I put the allowed domains in CORS_ALLOWED_ORIGINS instead of CORS_ORIGIN_WHITELIST and it worked. According to the documentation, this setting was renamed. I still unsure why CORS_ORIGIN_WHITELIST doesn't work on Cloud Run though... it should work as an alias.

Using Firebase Hosting, how can I restrict PURGE request to specific IPs

I have a website hosted on Firebase, using static html, no server-side function is used to deliver the result.
When running curl -X PURGE https://mywebsite.com -v -L the result is:
{ "status": "ok", "id": "20755-1619059392-3560756" }
I need a way to restrict this action to specific IPs so that not anybody can reset my cache which might result in extra costs.
Also it seems that Firebase uses Varnish to manage cache (which is something am null at).
My client's security consultant sent us this recommendation on how to handle this issue, I'm not sure exactly if this is .htaccess syntax or what:
# Varnish recommends to using PURGE method only by valid user,
# for example by ip limiting and for other return 405 Not allowed:
acl purge {
"localhost";
"192.168.55.0"/24;
}
sub vcl_recv {
# allow PURGE from localhost and 192.168.55...
if (req.method == "PURGE") {
if (!client.ip ~ purge) {
return(synth(405,"Not allowed."));
}
return (purge);
}
}
I don't know how to apply this in Firebase Hosting, again am not using Server Functions, just the regular firebase.json with the following headers:
"headers": [
{
"source": "*.[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f].+(css|js)",
"headers": [
{
"key": "Cache-Control",
"value": "public,max-age=31536000,immutable"
}
]
},
{
"source": "**/*.#(json|eot|otf|ttf|ttc|woff|font.css)",
"headers": [
{
"key": "Access-Control-Allow-Origin",
"value": "*"
}
]
}
]
The following code is VCL code:
acl purge {
"localhost";
"192.168.55.0"/24;
}
sub vcl_recv {
# allow PURGE from localhost and 192.168.55...
if (req.method == "PURGE") {
if (!client.ip ~ purge) {
return(synth(405,"Not allowed."));
}
return (purge);
}
}
This code allows you to extend the behavior of Varnish. This code has to be added to your /etc/varnish/default.vcl file.
After adding this code to your VCL file, you have to reload your varnishd process to activate this VCL configuration.
If reloading varnishd is not an option, you can also activate the new VCL file using the following commands:
sudo varnishadm vcl.load purge_acl /etc/varnish/default.vcl
sudo varnishadm vcl.use purge_acl
For more information about Varnish and the VCL programming language, please have look at http://varnish-cache.org/docs/6.0/reference/index.html
There is no solution for the moment. This is the answer I received from the firebase support :
Hi Damien,
My name is Sergei, thanks for reaching out. I'll be assisting you.
The first thing to be addressed here is the fact that Varnish services
fall outside of our scope, so our information about its
implementations with our hosting services are not the most abundant.
Unfortunately, right now we can only control the caching behavior with
our existing tools.
I am sorry we cannot provide the functionality you need at the time,
if you would like to, we can submit the feature request so this is
visible to our engineering team.

Firebase Hosting rewrite glob behaviour issue

I have set up a firebase hosting project with three pages and my custom domain setup . These pages get hit based on different paths in the url. The firebase.json file is as follows
{
"hosting": {
"public": "public",
"ignore": [
"firebase.json",
"**/.*",
"**/node_modules/**"
],
"rewrites": [ {
"source": "/main",
"destination": "/index.html"
},{
"source": "/waitlistview/**",
"destination": "/waitlistview.html"
},{
"source": "/mwaitlistview/**",
"destination": "/waitlistview-minimal.html"
}
],
"cleanUrls": true
}
}
Now when i try a url :
foodini.co.in/waitlistview
Instead of getting an error with 404 page, i still hit my page
Whereas if i hit the url:
foodini.co.in/mwaitlistview
I get the 404 page as i expect.
How to i make the first url to also go to 404 page
According to firebase hosting docs, Hosting first tries to map a file inside the public directory to the path in the url. Only if there is no such file will a rewrite take effect.
In my case, I had a file /waitlistview.html directly inside the public folder. Thus firebase hosting just mapped the url to that file even without a trailing slash. The rewrite never occurred and thus there was no 404 page
Also for anyone interested:
your default domain name directly maps to index.html without the need for any specific path
Also /waitlistview** does not match /waitlistview/Ahmedabad/xyz path via GLOB matching so please have a slash before **
add
"trailingSlash": false,
before the line
"cleanUrls": true
Reason is that "foodini.co.in/waitlistview" does not match "/waitlistview/", it would match "/waitlistview".

Unexpected trailing slash behaviour with Firebase hosting

I'm seeing unexpected behaviour (at least as per me) with trailing slashes on Firebase Hosting. I expect Firebase to serve URLs with a trailing slash (except for actual files present in the hosting directory). For this I set trailingSlash to true in firebase.json.
This resulted in:
example.com redirect to example.com/index.html/
example.com/js/script.js redirect to example.com/js/script.js/
example.com/images/image.png redirect to example.com/images/image.png/
The expected behaviour would be:
example.com redirect to example.com/
example.com/js/script.js served as it is without any redirect
example.com/images/image.png served as it is without any redirect
The addition of trailing slash to actual file URLs makes the browser/server think script.js is a directory instead of file and results in a 404. The addition of index.html to the root simply isn't elegant.
Expecting cleanUrls to do the trick I set cleanUrls to true along with trailingSlash and immediately the site goes into an endless redirect loop and fails to load. How can I stop Firebase from adding "index.html" at the root and the trailing slash for the actual js or image assets?
Edit:
As requested by Frank here's the firebase.json I'm using:
{
"firebase" : "example",
"public" : "public",
"cleanUrls" : false,
"trailingSlash" : true,
"ignore" : [ "firebase.json", "**/.*", "**/node_modules/**" ],
"rewrites": [{
"source" : "**",
"destination" : "/index.html"
}],
"headers": [{
"source" : "**/*.#(eot|otf|ttf|ttc|woff|font.css)",
"headers" : [{
"key" : "Access-Control-Allow-Origin",
"value" : "*"
}]
}, {
"source" : "**/*.#(jpg|jpeg|gif|png)",
"headers" : [{
"key" : "Cache-Control",
"value" : "max-age=600"
}]
}, {
"source" : "**/*.#(html|js)",
"headers" : [{
"key" : "Cache-Control",
"value" : "max-age=300"
}]
}]
}
P.S.: I have tried setting both trailingSlash and cleanUrls to true and also tried setting these true/false individually.
https://firebase.google.com/docs/hosting/full-config#trailingslash
When true, it will redirect URLs to add a trailing slash. When false
it will redirect URLs to remove a trailing slash. When unspecified,
trailing slashes are used only for directory index files (e.g.
about/index.html).
Just do not add that option and it will only affect urls
As for your question, If you want to invoke the command "rewrites", Server can execute "rewirtes" command when he cannot find any index.html in your root directory.
The solution one:
you just put index.html in your root directory under your "public" folder. I switch to your public folder and deploy your server.
The solution two:
you put index.html under some folder which you can named whatever. Then use JSON format: "rewrites": [{ "source" : "**", "destination" : "/folderName/index.html" }], but you cannot put any index.html under your root folder.
The solution three:
you create root folder, put your JS/CSS/IMG/ folders and index.html under the root folder. Then you deploy server JSON file like this.
{
"firebase": "you app name",
"public": ".",
"ignore": [
"firebase.json",
"**/.*",
"**/node_modules/**"
],
I'm sure this is too late for Vinny, but I just ran into this same issue. Adding <base href='/'>to my html files prevented the extra unexpected trailing slash behavior for me.
"hosting": {
"public": "...",
"ignore": [...],
"trailingSlash": false
}
Simply add trailingSlash property and set it to false will remove the trailing slash when firebase redirects. Check out their details documentation here

Firebase Hosting - Wildcard Redirections

I want to use Firebase Hosting to host an angular application and I need to create a redirection to some old files in another URL.
According with the Firebase Documentation you can do basic redirections
"redirects": [ {
"source" : "/foo",
"destination" : "/bar",
"type" : 301
}, {
"source" : "/firebase/*",
"destination" : "https://www.firebase.com",
"type" : 302
} ]
But I need a wildcard redirection
"redirects": [ {
"source" : "/config/*",
"destination" : "//oldsiteurl/config/[match-request]",
"type" : 302
}]
So, basically I need that myapp.firebase.com/config/some.json redirects to //oldsiteurl/config/some.json. I have a lot of json files so I do not want to match file by file.
Did you know if this is possible?
Thanks!
For people landing on this page, it is now possible to have wildcards in the URLs:
Sometimes it is desirable to capture parts of the source URL of a redirect and re-use them in the destination. You can do this using a : prefix to identify the segment and an optional * after the name to indicate that it should capture the rest of the URL
(source)
"redirects": [
{
"source": "/subdomain/:random*",
"destination": "https://subdomain.myapp.com/:random*",
"type": 301
}
]
From the Firebase documentation on URL redirects (emphasis mine):
If a match is found, an HTTP redirect response is set with the "Location" header set to the static destination string, which can be a relative path or an absolute URL.
So it looks like the wild-carded part of the match is not carried over into the redirect.

Resources