I want to delete an image. All I have is the download url.
In flutter i am able to get file path from download url and use that path to delete the file in cloud storage.
Is that possible to get the file path from download url and use that path to delete the image from cloud functions?
or is there any better/ faster way / more efficient way to delete an image from cloud storage only with the download url
Google Cloud Storage object URL has the following parts:
https://storage.cloud.google.com/[bucket_name]/[path/and/the/object/name*]?[autentication_if_needed]
*Path in Cloud Storage is "virtual", in fact it is an integral part of the object name/identification. Cloud Console and gsutil simulates folders for the user interface output.
There are several methods to delete object:
From the Cloud Console
Using Cloud SDK command: gsutil rm gs://[BUCKET_NAME]/[OBJECT_NAME]
Using Client Libraries, for example with python:
def delete_blob(bucket_name, blob_name):
"""Deletes a blob from the bucket."""
storage_client = storage.Client()
bucket = storage_client.get_bucket(bucket_name)
blob = bucket.blob(blob_name)
blob.delete()
print('Blob {} deleted.'.format(blob_name))`
Please keep in mind, that you need proper permissions to delete the object for the user/service account used to perform the operation.
Related
I followed the official docs for scheduling firestore export via cloud function and cloud scheduler.
It works perfectly only for the first time creating the necessary exports at the right location.
When I run it again I get following error in cloud function.
Error: 3 INVALID_ARGUMENT: Path already exists: /red115.appspot.com/daily_backup/users.overall_export_metadata
Why doesn't it overwrite on existing data?
I followed official docs, gave the necessary roles & permissions to principal account.
I have Recreated the setup from the docs you shared. I am assuming you also did the Configure access permissions part. I have scheduled the firebase function for firestore export every 5 minutes on all collections. And even after running 4 times with the schedule I have not got the Error you have mentioned.
The Firebase Storage Rules you provided do not affect the Firestore export functionality, they are only for the Firebase Storage.
But if you are experiencing this error I will recommend you to first check whether the export is already created at the given specified location on that bucket.
If Yes then To overwrite existing data on Firestore export, you could add a step in your Cloud Function to delete the existing export if there exists before running the new export. Something like this : admin.storage().bucket().delete() (be specific about this thing) method to delete the existing export before running the new export.
OR
You could change the export path to include a timestamp or a version number, so that each export is saved to a unique location. Which is a default behavior.
If No then there must be a typo that happened in the following line while providing the bucket link const bucket = 'gs://BUCKET_NAME';
Make sure you provide the same gs://BUCKET_NAME in the functions/index.js and gsutil iam ch serviceAccount:PROJECT_ID#appspot.gserviceaccount.com:admin \ gs://BUCKET_NAME
This thread also talks about the wrong gsutil path you have a look at once as well..
I am attempting to copy the contents of a folder in one Firebase project's Storage bucket to the storage bucket of another Firebase project.
I have been following the firestore docs and this SO question.
Both projects have the necessary permissions to other's service accounts.
Here is what I have done:
When attempting to transfer files from a folder in the default bucket of Project-A to the default bucket of Project-B using the cloud shell terminal, I first set the project to 'Project-A'. I then ran gcloud beta firestore export gs://[PROJECT_A_ID] --collection-ids=[FOLDER_TO_TRANSFER] --async. This succeeds and creates a folder called "2019-08-26T21:23:23_26014/" in Project-A. This folder contains some metadata.
Next, I tried beginning the import by setting the project to Project-B and running gcloud beta firestore import gs://[PROJECT_A_ID]/2019-08-26T21:23:23_26014
This completes and the logs display this message:
done: true
metadata:
'#type': type.googleapis.com/google.firestore.admin.v1beta1.ImportDocumentsMetadata
collectionIds:
- [FOLDER_TO_TRANSFER]
endTime: '2019-08-26T21:25:56.794588Z'
inputUriPrefix: gs://[PROJECT_A_ID]/2019-08-26T21:23:23_26014
operationState: SUCCESSFUL
startTime: '2019-08-26T21:25:19.689430Z'
name: projects/[PROJECT_B]/databases/(default)/operations/[SOME_ID_STRING]
response:
'#type': type.googleapis.com/google.protobuf.Empty
However, the Project-B storage bucket doesn't have any new files or folders. It looks like the import did nothing. Am I missing something?
You can create a transfer job in the GCP Console. You can specify source/destination buckets from different projects as long as you have access permissions. You can specify the folder by setting "Specify file filters":
https://console.cloud.google.com/storage/transfer
You can also use the gsutil tool, which is part of gcloud, to move or copy your objects to another bucket.
So your default buckets would be gs://[PROJECT_A_ID].appspot.com and gs://[PROJECT_B_ID].appspot.com Let's say you wanted to copy over the contents of my_directory:
gsutil cp -r gs://[PROJECT_A_ID].appspot.com/my_directory gs://[PROJECT_B_ID].appspot.com
I am using the REST API of Firebase Realtime Database from an AppEngine Standard project with Java. I am able to successfully put data under different locations, however I don't know how I could ensure atomic updates to different paths.
To put some data separately at a specific location I am doing:
requestFactory.buildPutRequest("dbUrl/path1/17/", new ByteArrayContent("application/json", json1.getBytes())).execute();
requestFactory.buildPutRequest("dbUrl/path2/1733455/", new ByteArrayContent("application/json", json2.getBytes())).execute();
Now to ensure that when saving a /path1/17/ a /path2/1733455/ is also saved, I've been looking into multi path updates and batched updates (https://firebase.google.com/docs/firestore/manage-data/transactions#batched-writes, only available in Cloud Firestore?) However, I did not find whether this feature is available for the REST API of the Firebase Realtime Database as well or only through the Firebase Admin SDK.
The example here shows how to do a multi path update at two locations under the "users" node.
curl -X PATCH -d '{
"alanisawesome/nickname": "Alan The Machine",
"gracehopper/nickname": "Amazing Grace"
}' \
'https://docs-examples.firebaseio.com/rest/saving-data/users.json'
But I don't have a common upper node for path1 and path2.
Tried setting as the url as the database url without any nodes (https://db.firebaseio.com.json) and adding the nodes in the json object sent, but I get an error: nodename nor servname provided, or not known.
This would be possible with the Admin SDK I think, according to this blog post: https://firebase.googleblog.com/2015/09/introducing-multi-location-updates-and_86.html
Any ideas if these atomic writes can be achieved with the REST API?
Thank you!
If the updates are going to a single database, there is always a common path.
In your case you'll run the PATCH command against the root of the database:
curl -X PATCH -d '{
"path1/17": json1,
"path2/1733455": json2
}' 'https://yourdatabase.firebaseio.com/.json'
The key difference with your URL seems to be the / before .json. Without that you're trying to connect to a domain on the json TLD, which doesn't exist (yet) afaik.
Note that the documentation link you provide for Batched Updates is for Cloud Firestore, which is a completely separate database from the Firebase Realtime Database.
I am developing with Firebase and have data stored in the Realtime Database. I need to share my database structure for a question here on Stack Overflow, or just take a backup before making breaking changes. How can I do this using the Firebase Console?
Data can be exported from the Firebase Realtime Database as JSON:
Login to the Database section of the Firebase Console.
Navigate to the node you wish to export by clicking on it in the list (skip this to export all data).
Click the 3-dot overflow menu icon, at the top-right of the data panel.
Click Export JSON from the menu.
Likewise, you can import a structure in the same fashion, using Import JSON.
There is an Node.js tool called firebase-export, similar to firebase-import but not from Firebase itself, that will export JSON from the command line.
Firebase export helper utility for exporting excluded JSON from Firebase.
To install
npm install -g firebase-export
Usage example
$ firebase-export --database_url https://test.firebaseio-demo.com --firebase_secret '1234' --exclude 'settings/*, users/*/settings'
Github Repo
Note: Firebase has a REST API, so you can use any language to retrieve (export) data:
curl 'https://[PROJECT_ID].firebaseio.com/users/jack/name.json'
Here's an example curl request with filters
curl 'https://dinosaur-facts.firebaseio.com/dinosaurs.json?orderBy="height"&startAt=3&print=pretty'
If you have a large JSON file then it is safe to download it using Postman's Import feature because downloading a large JSON file sometimes faces failure in the middle of the way. You just need to click save the response after the response is reached.
I noticed that to use Firebase Storage (Google Cloud Storage) I need to come up with a unique file name to upload a file.
I then plan to keep a copy of that Storage file location (https URL or gs URL) in the Firebase Realtime database, where the clients will be able to read and download it separately
However I am unable to come up with unique filenames for the files located on Firebase Storage. Using a UUID generator might cause collisions in my case since several clients are uploading images to a single Firebase root
Here's my plan. I'd like to know if it will work
Lets call my firebase root : Chatrooms, which consists of keys : chatroom_1, chatroom_2 ...chatroom_n
under chatroom_k I have a root called "Content", which stores Push keys that are uniquely generated by Firebase to store content. Each push key represents a content, but the actual content is stored in Firebase Storage and a key called URL references the URL of the actual content. Can the filename for this content on Firebase storage have the same randomized Push key as long as the bucket hierarchy represents chatroom_k?
I am not sure if storage provides push() function but a suggestion would be the following:
Request a push() to a random location to your firebase database and use this key for a name.
At any case you will probably need to store this name to the database too.
In my application I have a node called "photos" and there I store the information about the images I upload. I first do a push() to get a new key and I use this key to rename the uploaded image to.
Is this what you need or I misunderstood something?
So I had the same problem and I reached this solution:
I named the files with time and date and the user uid, so it is almost impossible to have two files with the same name and they will be different every single time.
DateFormat dtForm = new SimpleDateFormat("yyyy.MM.dd.HH.mm.ss.");
String date = dtForm.format(Calendar.getInstance().getTime());
String fileName = date + FirebaseAuth.getInstance().getCurrentUser().getUid();
FirebaseStorage
.getInstance()
.getReference("Folder/SubFolder/" + fileName)
.putFile(yourData)
With this the name of the files are going to be like this "2022.09.12.11.50.59.WEFfwds2234SA11" for example