How to store keywords in firebase firestore - firebase

My application use keywords extensively, everything is tagged with keywords, so whenever use wants to search data or add data I have to show keywords in auto complete box.
As of now I am storing keywords in another collection as below
export interface IKeyword {
Id:string;
Name:string;
CreatedBy:IUserMin;
CreatedOn:firestore.Timestamp;
}
export interface IUserMin {
UserId:string;
DisplayName:string;
}
export interface IKeywordMin {
Id:string;
Name:string;
}
My main document holds array of Keywords
export interface MainDocument{
Field1:string;
Field2:string;
........
other fields
........
Keywords:IKeywordMin[];
}
But problem is auto complete reads data frequently and my document reads quota increases very fast.
Is there a way to implement this without increasing reads for keyword ? Because keyword is not the real data we need to get.
Below is my query to get main documents
query = query.where("Keywords", "array-contains-any", keywords)
I use below query to get keywords in auto complete text box
query = query.orderBy("Name").startAt(searchTerm).endAt(searchTerm+ '\uf8ff').limit(20)
this query run many times when user types auto complete search which is causing more document reads

Does this answer your question
https://fireship.io/lessons/typeahead-autocomplete-with-firestore/
Though the receommended solution is to use 3rd party tool
https://firebase.google.com/docs/firestore/solutions/search
To reduce documents read:
A solution that come to my mind however I'm not sure if it's suitable for your use case is using Firestore caching feature. By default, firestore client will always try to reach the server to get the new changes on your documents and if it cannot reach the server, it will reach to the cached data on the client device. you can take advantage of this feature by using the cache first and reach the server only when you want. For web application, this feature is disabled by default and you can enable it like in
https://firebase.google.com/docs/firestore/manage-data/enable-offline
to help you understand this feature more check this article:
https://firebase.google.com/docs/firestore/manage-data/enable-offline

I found a solution, thought I would share here
Create a new collection named typeaheads in below format
export interface ITypeAHead {
Prefix:string;
CollectionName:string;
FieldName:string;
MatchingValues:ILookupItem[]
}
export interface ILookupItem {
Key:string;
Value:string;
}
depending on the minimum letters add either 2 or 3 letters to Prefix, and search based on the prefix, collection and field. so most probably you will end up with 2 or 3 document reads for on search.
Hope this helps someone else.

Related

Updating all the references to an image url in Cloud Firestore when the image is updated using Flutter

In my Flutter app, I have a userData collection on Cloud Firestore where I store user's data including name, image url, etc.. The user can create posts, add comments to post, etc. similar to any other social apps out there and so I have multiple other collections where the user's info is stored including the link to their profile image.
Let's say if the user adds a comment to a post, I save their name, profile image url and comment text as a document inside "postComment" collection and then I display his/her profile image, name and the comment text on the screen by reading this collection and document.
Now, if the user updates their profile image or even their name which will be reflected in the userData collection, I need to make sure that their name and image url are updated in all other collections as well.
What's the easiest and least costly way to do that? Do I need to loop through all my collections and their documents and update the field values, or is there like a simple cloud function that can handle this?
Thanks!
I also store user profile images in Firestore Storage, BUT I use a very consistent schema to make the images easy to "guess":
When I have a document such as "/People/{userID}", and within the document is a field "image" which stores the URL to the image...
...then I store it in Firestore at the path "People/{userID/image/image.jpg" (eg). This way it is trivial to generate a StorageRef to it, and a downloadURL.
All the other uses of it always are to the now-standardized URL. Change the image in Storage; all references update.
For most "user" applications, the only use of the image is to feed it to a web-page, so just the URL is needed, and let the browser do the rest of the work.
As Fattie somewhat more aggressively stated, generally all you need is the URL. But following by itself that means you still would have to find all the references and update them if the user changes the URL. Saving a copy in Firestore Storage, and using that consistent URL, means all references will be "updated" just by changing what is stored at that location. Disadvantage is it will count as a storage read when fetched.
I'm finding duplicating data in NoSQL is great when it's fairly static - created once, and not dynamically changed (which is a LOT of cases). If your application doesn't fit that, it's better to store a reference to the source-of-truth, and incur the cost of the "lookup".
Here's a couple utilities I use to make this easier:
export const makeStorageRefFromRecord = (
record,
key = null,
filename = null
) => {
return FirebaseStorage.ref(
record.ref.path + (key ? "/" + key : "") + (filename ? "/" + filename : "")
);
};
export const makeFileURLFromRecord = (record, key = null, filename = null) => {
return FirebaseStorage.ref(
record.ref.path + (key ? "/" + key : "") + (filename ? "/" + filename : "")
).getDownloadURL();
};
("key" is essentially the fieldname)
remember the refpath is a string of the "/" separated collection/document path to the record, and is completely knowable in a simple situation, such as "People/{userID}". If you keep this internal, you can use "filename" as simple as "image.jpg" so it's always the same - it's unique, because of the path.
Do I need to loop through all my collections and their documents and update the field values
Minimally, yes, that's what you have to do.
or is there like a simple cloud function that can handle this?
You can certainly write your own Cloud Function to do this as well. There is not an existing function that will just do what you want - you have to code it.
Alternatively, you can just store the URL is one document, store the ID of that document in the other documents that need to refer to it, and have the client make an query for the single document with the URL you need.
There are multiple ways to do that.
The best way to do that is instead of storing the profile picture image again and again, you can store document references. If you are storing the images as base64, this would also save a lost of space and is cost efficient.
Another way of doing it is less efficient but you can store the image in firestore and refer it from there.
Both of these are from refereces
The last way of doing it and probably the most inefficient is by querying. You can go to that collection of post (Or if you store each post as a collection, loop through all of them) and then add a where filter and search for the imageURL or more safely a unique ID and then you can change them all one by one
These are the ways that I know

Request.auth.metadata in security rules?

I have a Firebase project where I'd like for users to be able to see when other users created their profiles. My initial hope was that I could use "user.metadata.creationTime" on the frontend to pass the date into the user's extra info document and verify that it is correct by having "request.resource.data.datecreated == request.auth.metadata.creationTime" as a Database Rule, but it looks like it is not possible according to the documentation.
Is there any way I can verify that the creation date is correct on the backend?
More info edit: Below is the code that is being triggered when a user creates a new account on my profile. The three values are displayed publicly. I'm creating a niche gear for sale page so being able to see when a user first created their account could be helpful when deciding if a seller is sketchy. I don't want someone to be able to make it seem like they have been around for longer than they have been.
db.collection('users').doc(user.uid).set({
username: "Username-156135",
bio: "Add a bio",
created: user.metadata.creationTime
});
Firestore rules:
match /users/{id} {
allow get;
allow create, update: if request.resource.data.username is string &&
request.resource.data.bio is string &&
request.resource.data.created == request.auth.metadata.creationTime;
}
user.metadata.creationTime, according to the API documentation is a string with no documented format. I suggest not using it. In fact, what you're trying to do seems impossible since that value isn't available in the API documentation for request.auth.
What I suggest you do instead is use a Firebase Auth onCreate trigger with Cloud Functions to automatically create that document with the current time as a proper timestamp. Then, in security rules, I wouldn't even give the user the ability to change that field, so you can be sure it was only ever set accurately by the trigger. You might be interested in this solution overall.

How to do pattern searching in fire base real time DB [duplicate]

I am using firebase for data storage. The data structure is like this:
products:{
product1:{
name:"chocolate",
}
product2:{
name:"chochocho",
}
}
I want to perform an auto complete operation for this data, and normally i write the query like this:
"select name from PRODUCTS where productname LIKE '%" + keyword + "%'";
So, for my situation, for example, if user types "cho", i need to bring both "chocolate" and "chochocho" as result. I thought about bringing all data under "products" block, and then do the query at the client, but this may need a lot of memory for a big database. So, how can i perform sql LIKE operation?
Thanks
Update: With the release of Cloud Functions for Firebase, there's another elegant way to do this as well by linking Firebase to Algolia via Functions. The tradeoff here is that the Functions/Algolia is pretty much zero maintenance, but probably at increased cost over roll-your-own in Node.
There are no content searches in Firebase at present. Many of the more common search scenarios, such as searching by attribute will be baked into Firebase as the API continues to expand.
In the meantime, it's certainly possible to grow your own. However, searching is a vast topic (think creating a real-time data store vast), greatly underestimated, and a critical feature of your application--not one you want to ad hoc or even depend on someone like Firebase to provide on your behalf. So it's typically simpler to employ a scalable third party tool to handle indexing, searching, tag/pattern matching, fuzzy logic, weighted rankings, et al.
The Firebase blog features a blog post on indexing with ElasticSearch which outlines a straightforward approach to integrating a quick, but extremely powerful, search engine into your Firebase backend.
Essentially, it's done in two steps. Monitor the data and index it:
var Firebase = require('firebase');
var ElasticClient = require('elasticsearchclient')
// initialize our ElasticSearch API
var client = new ElasticClient({ host: 'localhost', port: 9200 });
// listen for changes to Firebase data
var fb = new Firebase('<INSTANCE>.firebaseio.com/widgets');
fb.on('child_added', createOrUpdateIndex);
fb.on('child_changed', createOrUpdateIndex);
fb.on('child_removed', removeIndex);
function createOrUpdateIndex(snap) {
client.index(this.index, this.type, snap.val(), snap.name())
.on('data', function(data) { console.log('indexed ', snap.name()); })
.on('error', function(err) { /* handle errors */ });
}
function removeIndex(snap) {
client.deleteDocument(this.index, this.type, snap.name(), function(error, data) {
if( error ) console.error('failed to delete', snap.name(), error);
else console.log('deleted', snap.name());
});
}
Query the index when you want to do a search:
<script src="elastic.min.js"></script>
<script src="elastic-jquery-client.min.js"></script>
<script>
ejs.client = ejs.jQueryClient('http://localhost:9200');
client.search({
index: 'firebase',
type: 'widget',
body: ejs.Request().query(ejs.MatchQuery('title', 'foo'))
}, function (error, response) {
// handle response
});
</script>
There's an example, and a third party lib to simplify integration, here.
I believe you can do :
admin
.database()
.ref('/vals')
.orderByChild('name')
.startAt('cho')
.endAt("cho\uf8ff")
.once('value')
.then(c => res.send(c.val()));
this will find vals whose name are starting with cho.
source
The elastic search solution basically binds to add set del and offers a get by wich you can accomplish text searches.
It then saves the contents in mongodb.
While I love and reccomand elastic search for the maturity of the project, the same can be done without another server, using only the firebase database.
That's what I mean:
(https://github.com/metaschema/oxyzen)
for the indexing part basically the function:
JSON stringifies a document.
removes all the property names and JSON to leave only the data
(regex).
removes all xml tags (therefore also html) and attributes (remember
old guidance, "data should not be in xml attributes") to leave only
the pure text if xml or html was present.
removes all special chars and substitute with space (regex)
substitutes all instances of multiple spaces with one space (regex)
splits to spaces and cycles:
for each word adds refs to the document in some index structure in
your db tha basically contains childs named with words with childs
named with an escaped version of "ref/inthedatabase/dockey"
then inserts the document as a normal firebase application would do
in the oxyzen implementation, subsequent updates of the document ACTUALLY reads the index and updates it, removing the words that don't match anymore, and adding the new ones.
subsequent searches of words can directly find documents in the words child. multiple words searches are implemented using hits
SQL"LIKE" operation on firebase is possible
let node = await db.ref('yourPath').orderByChild('yourKey').startAt('!').endAt('SUBSTRING\uf8ff').once('value');
This query work for me, it look like the below statement in MySQL
select * from StoreAds where University Like %ps%;
query = database.getReference().child("StoreAds").orderByChild("University").startAt("ps").endAt("\uf8ff");

SetPermission With sensenet API - version 6.3

I am working on sensenet API. I faced an issue with setPermission on sensenetAPI security.
As per concern, when I create a document I would like to give See, open, Save and RunApplication permission as a default for newly created document to the user(User is taken from the function parameter).
To achieve this I use below code
public static void SetCollabUserSecurity(string myUserEmailId, Node myNodetToSetSecurity)
{
var domainName = "Builtin";
var strUsername = GetNameFromEmail(myUserEmailId);
User user;
using (new SystemAccount())
{
user = User.Load(domainName, strUsername);
if (user != null && user.Enabled)
{
var myUser = user;
myNodetToSetSecurity.Security.SetPermission(myUser, true, PermissionType.See,
PermissionValue.Allow);
myNodetToSetSecurity.Security.SetPermission(myUser, true, PermissionType.Open,
PermissionValue.Allow);
myNodetToSetSecurity.Security.SetPermission(myUser, true, PermissionType.Save,
PermissionValue.Allow);
myNodetToSetSecurity.Security.SetPermission(myUser, true, PermissionType.RunApplication,
PermissionValue.Allow);
}
}
}
While I am using this function, my process for creating document becomes time consuming. It takes around 40 second time for execution.
So in case of, if I would like to share the same newly created document with multiple users, lets say there are 3 user and I want to give the above permission to all of them then my single function call takes 120 second (2 minute) time to simply assign permission.
Is there any Odata REST API call available or any sensenet library call available through which I can assign...
1) multiple permission to multiple user for single document or
2) multiple permission to single user for single document
Can anyone help to come out from this issue?
Thanks!
C# api
On the server there is a c# api for managing permissions, please check this article for details. You may use the AclEditor class for setting multiple permissions in one round. Please note that you have to call the Apply method at the end to actually perform the operation.
// set permissions on folder1, folder2 and file1 for two users and a group
SecurityHandler.CreateAclEditor()
.Allow(folder1.Id, user1.Id, false, PermissionType.Open, PermissionType.Custom01)
.Allow(folder2.Id, user2.Id, false, PermissionType.Open)
.Allow(file1.Id, editorsGroup.Id, false, PermissionType.Save)
.Apply();
As a side note: in most cases it is better to work with groups than users when assigning permissions. So it is advisable to give permissions to a group and put users into the group as members instead of assigning permissions to users directly.
Also: it is easier to maintain a simpler security structure, for example if you assign a permission on the parent container (e.g. a folder) instead of on individual files. Of course if you have to set permission per file, then it is fine.
OData api
The same api is available from the client through the REST api. Please take a look at the SetPermissions action in this article or the similar api in the JavaScript client library of sensenet.

Firebase "Where" like search

Tryng to get a simple result using "Where" style in firebase but get null althe time, anyone can help with that?
http://jsfiddle.net/vQEmt/68/
new Firebase("https://examples-sql-queries.firebaseio.com/messages")
.startAt('Inigo Montoya')
.endAt('Inigo Montoya')
.once('value', show);
function show(snap) {
$('pre').text(JSON.stringify(snap.val(), null, 2));
}
Looking at the applicable records, I see that the .priority is set to the timestamp, not the username.
Thus, you can't startAt/endAt the user's name as you've attempted here. Those are only applicable to the .priority field. These capabilities will be expanding significantly over the next year, as enhancements to the Firebase API continue to roll out.
For now, your best option for arbitrary search of fields is use a search engine. It's wicked-easy to spin one up and have the full power of a search engine at your fingertips, rather than mucking with glacial SQL-esque queries. It looks like you've already stumbled on the appropriate blog posts for that topic.
You can, of course, use an index which lists users by name and stores the keys of all their post ids. And, considering this is a very small data set--less than 100k--could even just grab the whole thing and search it on the client (larger data sets could use endAt/startAt/limit to grab a recent subset of messages):
new Firebase("https://examples-sql-queries.firebaseio.com/messages").once('value', function(snapshot) {
var messages = [];
snapshot.forEach(function(ss) {
if( ss.val().name === "Inigo Montoya" ) {
messages.push(ss.val());
}
});
console.log(messages);
});
Also see: Database-style queries with Firebase

Resources