I have created a document with pdf attachment using below code and it's working (able to retrieve attached file).
var myDoc = new { id = "42", Name = "Max", City="Aberdeen" }; // this is the
document you are trying to save
var attachmentStream = File.OpenRead("c:/Path/To/File.pdf"); // this is the
document stream you are attaching
var client = await GetClientAsync();
var createUrl = UriFactory.CreateDocumentCollectionUri(DatabaseName,
CollectionName);
Document document = await client.CreateDocumentAsync(createUrl, myDoc);
await client.CreateAttachmentAsync(document.SelfLink, attachmentStream, new
MediaOptions()
{
ContentType = "application/pdf", // your application type
Slug = "78", // this is actually attachment ID
});
I can upload a document directly in blob storage and put that blob URL in the document.
Can anyone help me to understand the value of inbuild attachment feature? how this is better than blob and other option? where cosmos DB keep attachment?
I want to understand which scenario we should consider this option (I know 2GB per account limitation)
Can anyone help me to understand the value of inbuild attachment
feature?how this is better than blob and other option?
Based on this official doc, you could get answer for your question.
You could store two types data:
1.binary blobs/media
2.metadata (for example, location, author etc.) of a media stored in a remote media storage
In addition,attachments has garbage disposal mechanism which is different with Azure Blob Storage I think.
Azure Cosmos DB will ensure to garbage collect the media when all of
the outstanding references are dropped. Azure Cosmos DB automatically
generates the attachment when you upload the new media and populates
the _media to point to the newly added media. If you choose to store
the media in a remote blob store managed by you (for example,
OneDrive, Azure Storage, DropBox, etc.), you can still use attachments
to reference the media. In this case, you will create the attachment
yourself and populate its _media property.
So,per my understanding,if your resource data will be frequently added or deleted, I think you could consider using attachment. You just need to store remote URL into _media property.
where cosmos DB keep attachment?
Attachment is stored in the collection as JSON format document,it can be created, replaced, deleted, read, or enumerated easily using either REST APIs or any of the client SDKs. As I know, it can't display on the portal so far.
BTW, Azure cosmos db is more expensive than blob storage usually.I think cost is an important factor to consider. More details, you could refer to the price doc.
Hope I'm clear on this.
Related
Duplicate of: Firebase storage URL keeps changing with new token
When a user uploads a profile pic I store this in firebase storage with the file name as the uid.
Lets say the user then goes and makes say 100 posts + 500 comments and then updates their profile image.
Currently I have a trigger which goes and updates the profile image url in all of the post and comment documents. The reason I have to do this is that when the image is changed in storage the access token is changed and this is part of the url so the old url no longer works.
What I want to do is not have the access token change. If I can do this I can avoid the mass updates that will massively increase my firestore writes.
Is there any way to do this? or an alternative?
Edit:
Another solution if you don't mind making the file public.
Add this storage rule and you won't have to use a token to access the file.
This will allow read access to "mydir" globally in any subfolder.
match /{path=**}/mydir/{doc} {
allow read: if true;
}
There are only two options here:
You store the profile image URL only once, probably in the user's profile document, and look it up every time it is needed. In return you only have to write it once.
You store the profile image URL for every post, in which case you only have to load the post documents and not the profile URL for each. In return you'll have to write the profile URL in each post document, and update it though.
For smaller networks the former is more common, since you're more likely to see multiple posts from the same user, so you amortizing the cost of the extra lookup over multiple posts.
The bigger the network of users, the more interesting the second approach becomes, as you'll care about read performance and simplicity more than the writes you're focusing on right now.
In the end, there's no singular right answer here though. You'll have to decide for yourself what performance and cost profile you want your app to have.
Answer provided by #Prodigy here: https://stackoverflow.com/a/64129850/10222449
I tried this and it works well.
This will save millions of writes.
var storage = firebase.storage();
var pathReference = storage.ref('users/' + userId + '/avatar.jpg');
pathReference.getDownloadURL().then(function (url) {
$("#large-avatar").attr('src', url);
}).catch(function (error) {
// Handle any errors
});
In my Flutter app, I have a userData collection on Cloud Firestore where I store user's data including name, image url, etc.. The user can create posts, add comments to post, etc. similar to any other social apps out there and so I have multiple other collections where the user's info is stored including the link to their profile image.
Let's say if the user adds a comment to a post, I save their name, profile image url and comment text as a document inside "postComment" collection and then I display his/her profile image, name and the comment text on the screen by reading this collection and document.
Now, if the user updates their profile image or even their name which will be reflected in the userData collection, I need to make sure that their name and image url are updated in all other collections as well.
What's the easiest and least costly way to do that? Do I need to loop through all my collections and their documents and update the field values, or is there like a simple cloud function that can handle this?
Thanks!
I also store user profile images in Firestore Storage, BUT I use a very consistent schema to make the images easy to "guess":
When I have a document such as "/People/{userID}", and within the document is a field "image" which stores the URL to the image...
...then I store it in Firestore at the path "People/{userID/image/image.jpg" (eg). This way it is trivial to generate a StorageRef to it, and a downloadURL.
All the other uses of it always are to the now-standardized URL. Change the image in Storage; all references update.
For most "user" applications, the only use of the image is to feed it to a web-page, so just the URL is needed, and let the browser do the rest of the work.
As Fattie somewhat more aggressively stated, generally all you need is the URL. But following by itself that means you still would have to find all the references and update them if the user changes the URL. Saving a copy in Firestore Storage, and using that consistent URL, means all references will be "updated" just by changing what is stored at that location. Disadvantage is it will count as a storage read when fetched.
I'm finding duplicating data in NoSQL is great when it's fairly static - created once, and not dynamically changed (which is a LOT of cases). If your application doesn't fit that, it's better to store a reference to the source-of-truth, and incur the cost of the "lookup".
Here's a couple utilities I use to make this easier:
export const makeStorageRefFromRecord = (
record,
key = null,
filename = null
) => {
return FirebaseStorage.ref(
record.ref.path + (key ? "/" + key : "") + (filename ? "/" + filename : "")
);
};
export const makeFileURLFromRecord = (record, key = null, filename = null) => {
return FirebaseStorage.ref(
record.ref.path + (key ? "/" + key : "") + (filename ? "/" + filename : "")
).getDownloadURL();
};
("key" is essentially the fieldname)
remember the refpath is a string of the "/" separated collection/document path to the record, and is completely knowable in a simple situation, such as "People/{userID}". If you keep this internal, you can use "filename" as simple as "image.jpg" so it's always the same - it's unique, because of the path.
Do I need to loop through all my collections and their documents and update the field values
Minimally, yes, that's what you have to do.
or is there like a simple cloud function that can handle this?
You can certainly write your own Cloud Function to do this as well. There is not an existing function that will just do what you want - you have to code it.
Alternatively, you can just store the URL is one document, store the ID of that document in the other documents that need to refer to it, and have the client make an query for the single document with the URL you need.
There are multiple ways to do that.
The best way to do that is instead of storing the profile picture image again and again, you can store document references. If you are storing the images as base64, this would also save a lost of space and is cost efficient.
Another way of doing it is less efficient but you can store the image in firestore and refer it from there.
Both of these are from refereces
The last way of doing it and probably the most inefficient is by querying. You can go to that collection of post (Or if you store each post as a collection, loop through all of them) and then add a where filter and search for the imageURL or more safely a unique ID and then you can change them all one by one
These are the ways that I know
I've always used the Cloud Firestore Database (and the old real-time one) to store text, and then use the Storage for images.
While using SurveyJS and AngularFirestore, I discovered I can push binary files into and out of the Firestore Database with the attached code. My question is: Is this OK?? I mean it works great, but I don't want to incur a cost or network slowdown...Thanks
var resultAsString = JSON.stringify(this.survey.data);
this.qs.saveSupplierQuestionnaire(this.companyid, this.id,this.survey.data)
...
saveSupplierQuestionnaire(userid:string, questionnaireid:string, questionnaireData:any) {
var resultAsString = JSON.stringify(questionnaireData);
var numCompleted = 0; /////test grading
const dbRef = this.afs.collection<questionnaire>('companies/' + userid + '/questionnaires/').doc(questionnaireid).update({results:resultAsString})
If it meets the needs of your application, then it's OK.
You should be aware than any time a document is read, the entire document is transferred to the client. So, even if you don't use the field with the binary data, you are going to make the user wait for the entire contents to be downloaded. This is true for all fields of a document, regardless of their types. There is really nothing special about binary fields, other than how the data is typed.
I have a need to create a copy of a Google Doc with a specific ID - not the "friendly" name like MyDocument, but the name that makes it unique in the GoogleSphere - the one like 1x_tfTiA9-b5UwAf3k2fg6y6hyZSYQIvhSNn-saaDs4c.
Here's the scenario why I would like to do this:
I have a newsletter which is in the form of a Google Doc. The newsletter is published on a website by embedding the document in a web page inside an <iframe> element. Also published in the same way is a "large print" version of the newsletter that is the same, apart from the fact that the default font size is 24pt, rather than 11pt.
I am trying to automate the production of the large print version, but in such a way that the unique ID of the large print document doesn't change, so that the embedded <iframe> for it still works.
I have experimented in the past with Google Apps Scripts routines for creating a deep copy of a document but the deep copy functions don't play nicely with images and tables, so I could never get a complete copy. If I could implement a "Save As" function, where the operand was an existing unique ID, I think this would do what I want.
Anyone know how I might do this?
I delved into this, attempting to set the id of the "large print" version of the file in a variety of ways:
via copy(): var copiedFile = Drive.Files.copy(lpFile, spFile.id, options);
which yields the error:
Generated IDs are not currently supported for copy requests
via insert(): var newFile = Drive.Files.insert(lpFile, doc.getBlob(), options);
which yields the error:
Generated IDs are not supported for Google Docs formats
via update(): Drive.Files.update(lpFile, lpFile.id, doc.getBlob(), options);
This method successfully updates the "large print" file from the small print file. This particular line, however, uses the Document#getBlob() method, which has issues with formatting and rich content from the Document. In particular, as you mention, images and tables in are not preserved (among other things, like changes to the font, etc.). Compare pre with post
It seems that - if the appropriate method of exporting formatted byte content from the document can be found - the update() method has the most promise. Note that the update() method in the Apps Script client library requires a Blob input (i.e. doc.getBlob().getBytes() will not work), so the fundamental limitation may be the (lack of) support for rich format information in the produced Blob data. With this in mind, I tried a couple methods for obtaining "formatted" Blob data from the "small print" file:
via Document#getAs(mimetype): Drive.Files.export(lpFile, lpFile.id, doc.getAs(<type>), options);
which fails for seemingly sensible types with the errors:
MimeType.GOOGLE_DOCS: We're sorry, a server error occurred. Please wait a bit and try again.
MimeType.MICROSOFT_WORD: Converting from application/vnd.google-apps.document to application/vnd.openxmlformats-officedocument.wordprocessingml.document is not supported.
These errors do make sense, since the internal Google Docs MimeType is not exportable (you can't "download as" this filetype since the data is kept however Google wants to keep it), and the documentation for Document#getAs(mimeType) indicates that only PDF export is supported by the Document Service. Indeed, attempting to coerce the Blob from doc.getBlob() with getAs(mimeType) fails, with the error:
Converting from application/pdf to application/vnd.openxmlformats-officedocument.wordprocessingml.document is not supported.
using DriveApp to get the Blob, rather than the Document Service:
Drive.Files.update(lpFile, lpFile.id, DriveApp.getFileById(smallPrintId).getBlob(), options);
This has the same issues as doc.getBlob(), and likely uses the same internal methods.
using DriveApp#getAs has the same errors as Document#getAs
Considering the limitation of the native Apps Script implementations, I then used the advanced service to obtain the Blob data. This is a bit trickier, since the File resource returned is not actually the file, but metadata about the file. Obtaining the Blob with the REST API requires exporting the file to a desired MimeType. We know from above that the PDF-formatted Blob fails to be properly imported, since that is the format used by the above attempts. We also know that the Google Docs format is not exportable, so the only one left is MS Word's .docx.
var blob = getBlobViaURL_(smallPrintId, MimeType.MICROSOFT_WORD);
Drive.Files.update(lpFile, lpFile.id, blob, options);
where getBlobViaURL_ implements the workaround from this SO question for the (still-broken) Drive.Files.export() Apps Script method.
This method successfully updates the existing "large print" file with the exact content from the "small print" file - at least for my test document. Given that it involves downloading content instead of using the internal, already-present data available to the export methods, it will likely fail for larger files.
Testing Script:
function copyContentFromAtoB() {
var smallPrintId = "some id";
var largePrintId = "some other id";
// You must first enable the Drive "Advanced Service" before this will work.
// Get the file metadata of the to-be-updated file.
var lpFile = Drive.Files.get(largePrintId);
// View available options on the relevant Drive REST API pages.
var options = {
updateViewedDate: false,
};
// Ideally this would use Drive.Files.export, but there is a bug in the Apps Script
// client library's implementation: https://issuetracker.google.com/issues/36765129
var blob = getBlobViaURL_(smallPrintId, MimeType.MICROSOFT_WORD);
// Replace the contents of the large print version with that of the small print version.
Drive.Files.update(lpFile, lpFile.id, blob, options);
}
// Below function derived from https://stackoverflow.com/a/42925916/9337071
function getBlobViaURL_(id, mimeType) {
var url = "https://www.googleapis.com/drive/v2/files/"+id+"/export?mimeType="+ mimeType;
var resp = UrlFetchApp.fetch(url, {
headers: { Authorization: 'Bearer ' + ScriptApp.getOAuthToken()}
});
return resp.getBlob();
}
With the new Firebase API you can upload files into cloud storage from client code. The examples assume the file name is known or static during upload:
// Create a root reference
var storageRef = firebase.storage().ref();
// Create a reference to 'mountains.jpg'
var mountainsRef = storageRef.child('mountains.jpg');
// Create a reference to 'images/mountains.jpg'
var mountainImagesRef = storageRef.child('images/mountains.jpg');
or
// File or Blob, assume the file is called rivers.jpg
var file = ...
// Upload the file to the path 'images/rivers.jpg'
// We can use the 'name' property on the File API to get our file name
var uploadTask = storageRef.child('images/' + file.name).put(file);
With users uploading their own files, name conflicts are going to be an issue. How can you have Firebase create a filename instead of defining it yourself? Is there something like the push() feature in the database for creating unique storage references?
Firebase Storage Product Manager here:
TL;DR: Use a UUID generator (in Android (UUID) and iOS (NSUUID) they are built in, in JS you can use something like this: Create GUID / UUID in JavaScript?), then append the file extension if you want to preserve it (split the file.name on '.' and get the last segment)
We didn't know which version of unique files developers would want (see below), since there are many, many use cases for this, so we decided to leave the choice up to developers.
images/uuid/image.png // option 1: clean name, under a UUID "folder"
image/uuid.png // option 2: unique name, same extension
images/uuid // option 3: no extension
It seems to me like this would be a reasonable thing to explain in our documentation though, so I'll file a bug internally to document it :)
This is the solution for people using dart
Generate the current date and time stamp using:-
var time = DateTime.now().millisecondsSinceEpoch.toString();
Now upload the file to the firebase storage using:-
await FirebaseStorage.instance.ref('images/$time.png').putFile(yourfile);
You can even get the downloadable url using:-
var url = await FirebaseStorage.instance.ref('images/$time.png').getDownloadURL();
First install uuid - npm i uuid
Then define the file reference like this
import { v4 as uuidv4 } from "uuid";
const fileRef = storageRef.child(
`${uuidv4()}-${Put your file or image name here}`
);
After that, upload with the file with the fileRef
fileRef.put(Your file)
In Android (Kotlin) I solved by combining the user UID with the milliseconds since 1970:
val ref = storage.reference.child("images/${auth.currentUser!!.uid}-${System.currentTimeMillis()}")
code below is combination of file structure in answer from #Mike McDonald , current date time stamp in answer from # Aman Kumar Singh , user uid in answer from #Damien : i think it provides unique id, while making the firebase storage screen more readable.
Reference ref = firebaseStorage
.ref()
.child('videos')
.child(authController.user.uid)
.child(DateTime.now().millisecondsSinceEpoch.toString());