We are using Amazon cloudfront to serve static files from our website. We are copying these image references to another database on a mobile app, so that the images in the app are served from Cloudfront as well, so we need the URL to be permanent.
A URL for an image on our site looks something like this: http://d3q35tken14acg.cloudfront.net/cdn/farfuture/M17vstJzweaXVBR4penpg6CEv_v8DwxSKZIqZKlR6rY/mtime:1493753067/sites/unfestival/files/Screen%20Shot%202017-05-02%20at%2020.18.53.png
The mtime:1493753067 in the URL makes me wonder if the URL will expire.
My question: do URLs like this expire, or are they permanent?
The way CloudFront works can be a bit confusing, but the cache applies over the file content, not the URL. It's intended to be used for CDNs and such, where URLs need to be static, and it just ensures you can retrieve the files from a nearby region, reducing latency.
So basically Cloudfront URLs shouldn't expire for most of the cases.
Looks like you are using signed urls. It totally depends upon the time you specified when you created the URL.
If you specify time of one day while creating URL, one day after cloudfront will not answer for this URL marking this as unauthorised.
Related
This question was migrated from Stack Overflow because it can be answered on Webmasters Stack Exchange.
Migrated 2 days ago.
Context
I'm having a Cloud Run service, that generates images, hosted via Firebase hosting. I benefits from the included Firebase Hosting CDN. Because, with Firebase hosting:
Any requested static content is automatically cached on the CDN.
🎉
Creating CDN cache works well
The first time when requesting an image it will be generated on the fly, subsequent requests it will get the CDN-version.
This works amazingly well.
Purging one explicit URL works too
Sometimes I need to remove all live versions of an image. Therefore, I can do a PURGE request, and remove the specific path from the CDN.
But, how to work with wildcards or query strings?
It seems that the Firebase CDN cache has different keys for all variations of a URL. If I would e.g. issue a request to a specific URL with a PURGE method:
curl -X PURGE https://my.firebasehosting.site.com/path/to/content
This will purge the CDN cache for that URL, but not for https://my.firebasehosting.site.com/path/to/content?someAdditionalStuffHere
What would the way to PURGE urls at locations like:
https://my.firebasehosting.site.com/path/to/content
https://my.firebasehosting.site.com/path/to/content?someAdditionalStuffHere
https://my.firebasehosting.site.com/path/to/content?width=200&height=100
https://my.firebasehosting.site.com/path/to/**
etc?
...In one go, without knowing all variants that have been generated? Is this even possible?
I really need to be able to access images in my firebase storage dynamically by creating a URL. Something that would look like this:
https://firebasestorage.googleapis.com/v0/b/<bucket>.appspot.com/o/userImages%2F<userUID>2%2F<imageUID>?alt=media
I know it it would be possible since I managed to load the image in my browser. However, my concern is security.
I would need to set the rule allow read: if true; for this to work
If someone with bad intentions wanted to see users' images, would they be able to see all the images in my bucket or would they need to guess the userUID and the imageUID?
What you're asking isn't possible without custom code. Direct download URLs are not affected by Firebase security rules at all.
If you want to limit access to direct download URLs of any kind, you will need some sort of custom backend service that checks the end user's permission before delivering the content. This means you will have to create your own endpoint that serves the content of the file in Storage.
The rules don't work on the URLs. BUT the download URL has a token in it which can be generated by you or is generated by the bucket by default a UUID which is always unique.
https://firebasestorage.googleapis.com/v0/b/[bucket].appspot.com/o/userImages%2F[userUID]2%2F[imageUID]?alt=media&token=[accessToken]
I am looking for a way to limit access to AWS S3 hosted data in a controlled and at least semi-secure way. I have various resources in a number of S3 buckets, with CloudFront as CDN. I then have a WordPress based website using a theme that allows me to sell "courses". Finally I manage my domains so I can create a sub domain for the content download link, i.e. content.domainname.com.
Ideally I want to limit access to content to a specific set of courses, so only people who have bought the course, and are linking to the content from a web page in that course, can (easily) get at the data.
I know I can use an AWS Web ACL rule to check the referer, to limit downloads to links on my domain. And I think I can expand on that to test more of the URL, so in www.domainname.com/paid/coursename/page.html I could have a rule that tests for the bold portion of the path and refuses otherwise.
However, I also know that referer can be easily spoofed, and more importantly some browsers and internet security software will replace the referer, and I don't want my site security to force customers to change their security settings. So, is there another option, to include some sort of data in the HTTP request, that limits access in a way that is both somewhat secure, but not dependent on a client side settings? Perhaps something like a hash that I could include in the link itself? Or, maybe the WordPress API and AWS Web ACL Rules can communicate is some way so as to validate the logged on user has membership in the course? Grasping at straws here I suspect.
Additionally, there will be a PowerShell script that can be downloaded and run, which will access downloadable content as well. Again, I want to limit access, but in this case I need to be able to maintain the criteria on AWS as I have subscription and non subscription versions of the courses, and the PS script should only download for customers on subscription. So, I could provide the PS script with something like a customer ID, then maintain a list of customer IDs that are currently on subscription so the Web ACL rule could filter. But again, I suspect that HTTP header won't get the job done, because it could be changed by internet security at the customer location. But now I am limited by what PowerShell can do with regards to HTTP requests.
I know, rather an open ended question, but hopefully someone can at least point me in the right direction. It sure seems like both needs are something that AWS should be able to do, I am just so out of my depth here I don't know where to start, and AWS documentation requires that you have some clue to get you going.
I have a situation where in Firebase Storage users store their avatar to /users/{uid}.jpg
If I then use the Storage API to get the download URL and display the image, it ends up being very slow to make the first request because the download URL is not cached anywhere.
So my solution is to get the DownloadURL when the user uploads the image and store that in Firebase allowing the client image provider to automatically cache the image which speeds up loads considerably.
However, there is one problem with this solution.
If a user replaces their avatar, the old link becomes broken instead of updated. I believe this is because a new token is generated each time something is uploaded for security reasons but these are of no benefit to me.
So my question is twofold:
1) How can I allow a user to upload an avatar to a path that is dedicated to them such as /users/{uid}.jpg, get a bare download URL that can be cached by the client, and have that URL remain the same even when the file changes at /users/{uid}.jpg
2) If this is not possible, what is the normal way to solve this issue?
Download URLs are opaque. The contents of the actual URL itself is an implementation detail of the system, and it's not supported to dig around in its contents. The URLs can't be dissected or composed.
You can use a storage trigger with Cloud Functions to automatically generate a signed URL whenever something changes in your storage bucket.
So instead of serving from a hard-coded URL, simply retrieve the URL from an updated value in the datastore (or any data storage system). Every time the user updates the avatar, simply store the new URL in the datastore and you can query for it when you need it.
Is it possible to serve two different pages based on the user agent.
I want to serve pagename-iphone.html if the user agent matches iPhone and pagename-other.html for all other user agents. I want all pages on the site to follow this pattern. Is it possible to do this at the CDN level (cloud front, akamai etc).
thanks for your help!
I think what you are after is User Agent based caching, like vary: User-Agent.
In theory, a server provide Cache service can definitely do so, however, as far as I can tell CloudFront and most of other major CDN providers don' support so.
The basic reason is very straightforward that the currently there are too many User-Agent header, and it's almost unique on every single browser, not mention the different versions of the same browser. If you purely do things based on the whole User-Agent, you will lost the benefit of CDN cache most of the time.
Some of the more advanced servers allow you to add condition based on headers, for example, in Varnish, you can even add if,else logic for returning different values. But this is not available for majority of CDNs.
In the other hand, you should not rely on CDN to return different html pages. CDN is more commonly used to accelerate artifacts (js/css/imgs) instead of the whole page.
EDIT: Actually, I just recieved an email from AWS mentioned now CloudFront starts to support this:
Mobile Device Detection: You can now cache and deliver customized
content to your viewers on different devices (e.g. mobile vs. desktop)
based on the value of the User Agent header.
Please refers to: http://aws.amazon.com/about-aws/whats-new/2014/06/26/amazon-cloudfront-device-detection-geo-targeting-host-header-cors/ for more details.