I understand that much of Firebase's services are wrappers around the Google Cloud platform (functions, storage etc..) I would like to obtain analytics on Google Cloud Storage downloads on an object basis (downloads, time). In the firebase console, it shows the amount of requests as well as the amount of data downloaded, but I don't know which objects were downloaded and how often.
Is there a logging method or API in Google Cloud I can utilize to obtain this data?
You have the Access Logs & Storage Logs that allows you to get the information for all the requests made to a bucket.
You will have to set up a bucket dedicated to store Access and Storage logs and then set logging for each bucket pointing to the dedicated one.
Related
I want to create a Udemy like video platform where a user can see all videos but can watch videos only that he has purchased.
I am making a rest call to get the videos from the storage bucket from an angular application, using Firebase authentication here. In my GET request to storage bucket I am passing the access token that I got from Firebase authn.
Does this access token can be used to determine scope of the user to access video in a bucket?
Assume if I have given read access for a video in a bucket for a specific user, using the access token can I get the video? But every time I tried it shows unauthorized. Is there any other way to verify users access to storage bucket object.
Google recommend to not use ACL because it's hard to manage and to have a global view on the authorization.
In most cases, Cloud Identity and Access Management (Cloud IAM) is the recommended method for controlling access to your resources.
Caution: Permissions can be granted either by ACLs or Cloud IAM policies. In general, permissions granted by Cloud IAM policies do not appear in ACLs, and permissions granted by ACLs do not appear in Cloud IAM policies. The only exception is for ACLs applied directly on a bucket and certain bucket-level Cloud IAM policies, as described in Cloud IAM relation to ACLs.
IMO, the best pattern is to have a database on your side with the file on GCS allowed per user. You can store these in Firestore: affordable, pay as you use, generous free tier. For downloading the video, you can generate a temporarily access to the user by generating a signedUrl.
As per mention Guillaume Cloud Storage use the ACL pattern in order to have a control to the access of the resources stored in their buckets.
Nevertheless, when you need to storage wide objects per user in this case a video, you can store these in Firestore: affordable, pay as you use, generous free tier. This is a very suitable option since Firestore can use as another resource
Is recommended for this scenario generate a signed URL
I have an app which uses firebase storage and I am observing an unusual traffic to the storage. The bandwidth usage is unusually high, which shouldn't happen with the usual behavior of the app.
I need to isolate a user or a stored file which causes this unusual bandwidth usage. Is there any way I can get more information about the storage usage, other than the information shown at the Usage tab of firebase console?
Firebase Storage only tracks and shows aggregate data for the usage in your project. It does not tie this usage to specific Firebase Authentication users.
If you need that, consider adding your own tracking for this, for example by logging uploads/downloads to Google Analytics for Firebase.
You can also log to Stack Driver, although it might only be in aggregate there too. For the exact metrics, see the Google Cloud Platform documentation on its storage metrics.
Alternatively you could reach out to Firebase support for personalized help in troubleshooting. But I'm not sure if they have access to the type of data you're looking for.
TL;DR:
Does my firebase project incur charges/use up quota by running firebase functions locally using $firebase serve --only functions?
Details:
I have a collection in my firestore database called locations with thousands of documents that include location data such as address and postCode. I want to add some geolocation data to each document using an external api service (Open Postcode Geo API). I've written an HTTP-triggered firebase function, triggered by hitting the endpoint /api/geocodeLocations. This cloud function will run through each document in the locations collection and use the postCode data to make an api call to Open Postcode Geo which will respond with the geolocation data (along with other data). It will then get the data it needs from the response and write it to the document in firestore.
I am aware that if I deploy this function and call it by hitting https://my-project/api/geocodeLocations, I will start racking up my quotas and maybe incur charges for the amount of external requests that it's making (and more than likely time out...). However, if I run $firebase serve --only functions locally on my dev machine to emulate functions and then hit the url http://localhost:5000/my-project/us-central1/app/api/geocodeLocations, will I still rack up quotas for external requests?
I don't think I will, as the requests are between my local machine and the external api, but I can't find any firebase documentation on quotas when running locally.
Thanks in advance!
Running functions locally in any way doesn't cost anything in terms of billing or quota for the Cloud Functions product.
If you read data out of Firestore or Realtime Database in a function running locally, you will still be billed for that access, but it won't be in relation to Cloud Functions billing or quotas.
I am using Firebase for almost 2 months now for my app in production and right now I am on Spark plan for sometime until I get user base. I see that the Spark pricing allows 1GB of downloads per day and 5GB of total storage but nowhere on the console I am able to see how much storage quota I have used up now. This is really important to a lot of users, if you show usage information for Database you should show it for storage as well.
As long as Storage is concerned it's actually a Google Cloud Storage Bucket, you can visit the Google Cloud Console and see the details. Firebase currently does not show the bucket usage details.
You can send them a feature request from this link :
https://firebase.google.com/support/contact/bugs-features/
Like Raghvendra said, you can view it on the Google Cloud Console at the following link: https://console.cloud.google.com/storage/browser/my-app.appspot.com
my-app will be your project name, and is the same as the one you use on Firebase: https://console.firebase.google.com/project/my-app/storage/my-app.appspot.com/files
How to automate the process of uploading .CSV files from Google Cloud onto Big Query.
Google Cloud Storage provides access and storage log files in CSV formats which can be directly imported into BigQuery for analysis. In order to access these logs, you must set up log delivery and enable logging. The schemas are available online, in JSON format, for both the storage access logs and storage bucket data. More information is available in the Cloud Storage access logs and storage data documentation.
In order to load storage and access logs into BigQuery from the command line, use a command such as:
bq load --schema=cloud_storage_usage_schema.json my_dataset.usage_2012_06_18_v0 gs://my_logs/bucket_usage_2012_06_18_14_v0