I'm using Amazon cloud drive to save many video recordings. I want to run my own program in the cloud to process and/or edit these files.
Is there a way to access the data in my cloud drive from a program running on an EC2 instance?
Yes and no.
You can use Amazon Cloud Drive REST API to access files. To authenticate on Cloud Drive through your personal device you also need small web server to get authentication tokens.
Main problem you need to get security tokens by registering your security profile and "whitelist" it. Since September "whitelist" became unavailable without special review which takes about a month and almost 100% you get rejected. Even documentation for Cloud Drive is not accessible on dev console page anymore.
I would recommend to reconsider using Amazon Cloud Drive.
Related
I have a Node.js API (built with Express.js) hosted on Firebase Cloud Functions and an accompanying web application hosted on Firebase Hosting which is supposed to interact with the aforementioned API.
I would like to restrict calls to the API so that only the web application would be able to call the cloud functions. How can I achieve that?
What I have tried:
using the App Check or more precisely Google's reCAPTCHA v3 for web apps. I have whitelisted the domain of the web application and have activated App Check token validation server side as well. The problem with App Check, however, is that I am able to obtain the reCAPTCHA attestation token from the browser (after authenticating through the web app) and use that with requests made from anywhere. This enables bombarding the API from outside the web application and defeats the purpose of using App Check in the first place.
Something to note here, the documentation for activating App Check in Cloud Functions instructs the usage of functions.https.onCall(). However, since my API is built using Express.js, I had to use a workaround to be able to use functions.https.onRequest() as instructed here.
restricting the Cloud Function from Google Cloud console to allow only clients of the same project to call the function as instructed here. Unfortunately, my web application hosted on Firebase Hosting does not seem to belong under the same Google Cloud project. Apps hosted on Firebase Hosting do not appear in Google Cloud console. Furthermore, after adjusting the Ingress settings of the functions to "allow internal traffic only", I am receiving CORS errors when attempting to access the API through the web application. I am unable to access the API from anywhere else though, which is partly the desired outcome.
Before anyone proposes limiting the domains in CORS options from within the API, while this might serve the purpose of restricting access to the API endpoints, it still would allow calling the function rapidly and thus, potentially, racking up the bill.
I am grateful for any suggestions!
Firebaser here.
This is a great question! Doug has already made some great points above, and I'll just add to that by saying that the TTL on App Check tokens reduce the replay window that you observed in your first bullet point, and this TTL is configurable.
The default TTL for reCAPTCHA v3 is 1 day to protect against running out of quota, but you can set a shorter TTL to increase the cost for an attacker trying to set up a replay attack. But please do be mindful of the associated trade-offs as described in the documentation. You can read about the associated quotas and limits here.
Unfortunately, web applications redirected from Firebase Hosting can't hook up to the GCP internal network. App Check is actually exactly what you are looking for in this situation.
Finally, because we are continuously working on improving the App Check platform, the comments you leave here are valuable for us as we decide on what anti-abuse features we want to work on next.
I have implemented Firebase Authentication in my app, using the Google Provider and the "https://www.googleapis.com/auth/drive.appdata" scope so my app can access its application folder within Google Drive
This part works fine and I retrieve the FireBaseUser once the authentication completes
What I now want to do is to access my app storage on Google Drive, but I don't know how to do this using the result of the current authentication
I tried to follow this: https://github.com/gsuitedevs/java-samples/blob/master/drive/quickstart/src/main/java/DriveQuickstart.java
But this doesn't work on Android.
The main issue is how to create the Credentials object
Do you have any idea on how to initialize a Drive.Builder instance so I can write/read to the app Google Drive folder?
Thanks
This cannot be done directly, because the Firebase & Google login are not the same, even when having logged in to Firebase with a Google account. On Android one meanwhile only has the "last logged in Google account" available for the user's Drive (the one on the device, which has nothing to do with Firebase Authentication, where Google may only be used as an "Authentication Provider").
So there are generally two options available to you:
Grant a GCP service-account access to Drive API and use Cloud Functions to access it.
Instead use Cloud Storage, which is within the Firebase eco-system and is similar to Drive.
I could also think of a combined solution approach, where users would use Cloud Storage (together with Firebase Authentication) and having a Cloud Function which uses a service-account, which eg. copies or moves uploaded files into a folder your Drive.
Concerning that (obviously server-side) Java example, the credentials.json clearly hints for a service account... and this is exactly where you can obtain this file from. However, for Android this is pretty useless - because it has major security implications, to package service-account credentials in an easy to de-compile package and distribute it on the WWW (to everybody). The Google Play Store likely would not permit you to publish or even upload that, because there are security checks in place. You could in best case only deploy that code as App Engine module, but not as an Android module.
Sorry for having destroyed your delusions and for not being able to provide a ready-made solution for 500 imaginary internet points, which pay nothing - but at least I can tell what is technically possible and what isn't - which effectively might save you lots of time, trying to accomplish the impossible.
My Current Plan:
I'm currently creating an IOS App that will access/change java/python files that are stored in the Google Cloud Storage. Once confirmed the App will talk with App Engine that will have a Compute Engine VM receive files and compile them. Once compiled have the result returned back to the IOS App
Is there any better or easier method to achieve this task? Should I use firebase or Google Cloud Functions? Would it be any help
Currently, I'm lost how to design and have requests sent between many platforms.
It would also depend on what type of data processing you are doing to the files in Cloud Storage. Ideally you would want to avoid as many "hops" between services as possible. You could do everything via Cloud Functions and listen on GCS Triggers. You can think of Cloud Functions as a sudo App Engine Backend to use for quick request handling.
Use Cloud Functions to respond to events from Cloud Storage or Firebase Storage to process files immediately after upload
If you are already using Firebase, it would be better to stay within their ecosystem as much as possible. If you are doing bigger or more intensive data processing you might want to look at different options.
With more information and current pain points, we may be able to offer more insight.
I am aware of how the permission system works in AWS:
By giving an EC2 instance a specific IAM role, it is possible to give all programs running on that specific EC2 instance some set of permissions for accessing other AWS services (e.g. permission to delete an EBS volume).
Is there something similar for Openstack? If you would like a program that is running on an Openstack server to be able to programmatically make changes through the Openstack API:s, how do you solve that?
The scenario I am thinking of is this:
You create a new Rackspace OnMetal cloud server together with an extra Rackspace Cloud Block Storage volume, and copy a big input data file to it with scp. You log in to the server with ssh and start a long running compute job. It would be great if the compute job by itself would be able to copy the result files to Rackspace Cloud Files and then unmount and delete the
Rackspace Cloud Block Storage volume that was used as temporary storage during the computation.
Rackspace's Role Based Access Control (RBAC) system is similar to AWS IAM roles. It lets you create users that restricted to specific APIs and capabilities. For example, a readonly cloud files user, or a cloud block storage administrator.
You could create a new user that only has access to the areas required for this compute job, e.g. cloud block storage and cloud files. Then your job would use that user's apikey to request a token and call the cloud block storage and cloud files api.
You did not mention a specific language but I recommend using an SDK, as it will handle the API specifics and quirks and get you up and running more quickly.
I'm not sure if this was answered somewhere in the documentation, but looking at the pricing tells me that Firebase has to work with client connected to the Internet at all times.
I am currently developing an application with Java-backed server and with client side done in GWT (Javascript). The application is meant to be deployed for "corporate" clients - that means in intranets.
Will Firebase be operational in these circumstances?
Well, I guess I should have read into FAQ more thoroughly:
Can I run Firebase on my own servers?
Firebase is a hosted cloud service, so the general answer is “no.” Some customers have very specific needs for where their sensitive data is physically stored and managed, so we may make exceptions. Contact us for more info.