I am using firebase for data storage. The data structure is like this:
products:{
product1:{
name:"chocolate",
}
product2:{
name:"chochocho",
}
}
I want to perform an auto complete operation for this data, and normally i write the query like this:
"select name from PRODUCTS where productname LIKE '%" + keyword + "%'";
So, for my situation, for example, if user types "cho", i need to bring both "chocolate" and "chochocho" as result. I thought about bringing all data under "products" block, and then do the query at the client, but this may need a lot of memory for a big database. So, how can i perform sql LIKE operation?
Thanks
Update: With the release of Cloud Functions for Firebase, there's another elegant way to do this as well by linking Firebase to Algolia via Functions. The tradeoff here is that the Functions/Algolia is pretty much zero maintenance, but probably at increased cost over roll-your-own in Node.
There are no content searches in Firebase at present. Many of the more common search scenarios, such as searching by attribute will be baked into Firebase as the API continues to expand.
In the meantime, it's certainly possible to grow your own. However, searching is a vast topic (think creating a real-time data store vast), greatly underestimated, and a critical feature of your application--not one you want to ad hoc or even depend on someone like Firebase to provide on your behalf. So it's typically simpler to employ a scalable third party tool to handle indexing, searching, tag/pattern matching, fuzzy logic, weighted rankings, et al.
The Firebase blog features a blog post on indexing with ElasticSearch which outlines a straightforward approach to integrating a quick, but extremely powerful, search engine into your Firebase backend.
Essentially, it's done in two steps. Monitor the data and index it:
var Firebase = require('firebase');
var ElasticClient = require('elasticsearchclient')
// initialize our ElasticSearch API
var client = new ElasticClient({ host: 'localhost', port: 9200 });
// listen for changes to Firebase data
var fb = new Firebase('<INSTANCE>.firebaseio.com/widgets');
fb.on('child_added', createOrUpdateIndex);
fb.on('child_changed', createOrUpdateIndex);
fb.on('child_removed', removeIndex);
function createOrUpdateIndex(snap) {
client.index(this.index, this.type, snap.val(), snap.name())
.on('data', function(data) { console.log('indexed ', snap.name()); })
.on('error', function(err) { /* handle errors */ });
}
function removeIndex(snap) {
client.deleteDocument(this.index, this.type, snap.name(), function(error, data) {
if( error ) console.error('failed to delete', snap.name(), error);
else console.log('deleted', snap.name());
});
}
Query the index when you want to do a search:
<script src="elastic.min.js"></script>
<script src="elastic-jquery-client.min.js"></script>
<script>
ejs.client = ejs.jQueryClient('http://localhost:9200');
client.search({
index: 'firebase',
type: 'widget',
body: ejs.Request().query(ejs.MatchQuery('title', 'foo'))
}, function (error, response) {
// handle response
});
</script>
There's an example, and a third party lib to simplify integration, here.
I believe you can do :
admin
.database()
.ref('/vals')
.orderByChild('name')
.startAt('cho')
.endAt("cho\uf8ff")
.once('value')
.then(c => res.send(c.val()));
this will find vals whose name are starting with cho.
source
The elastic search solution basically binds to add set del and offers a get by wich you can accomplish text searches.
It then saves the contents in mongodb.
While I love and reccomand elastic search for the maturity of the project, the same can be done without another server, using only the firebase database.
That's what I mean:
(https://github.com/metaschema/oxyzen)
for the indexing part basically the function:
JSON stringifies a document.
removes all the property names and JSON to leave only the data
(regex).
removes all xml tags (therefore also html) and attributes (remember
old guidance, "data should not be in xml attributes") to leave only
the pure text if xml or html was present.
removes all special chars and substitute with space (regex)
substitutes all instances of multiple spaces with one space (regex)
splits to spaces and cycles:
for each word adds refs to the document in some index structure in
your db tha basically contains childs named with words with childs
named with an escaped version of "ref/inthedatabase/dockey"
then inserts the document as a normal firebase application would do
in the oxyzen implementation, subsequent updates of the document ACTUALLY reads the index and updates it, removing the words that don't match anymore, and adding the new ones.
subsequent searches of words can directly find documents in the words child. multiple words searches are implemented using hits
SQL"LIKE" operation on firebase is possible
let node = await db.ref('yourPath').orderByChild('yourKey').startAt('!').endAt('SUBSTRING\uf8ff').once('value');
This query work for me, it look like the below statement in MySQL
select * from StoreAds where University Like %ps%;
query = database.getReference().child("StoreAds").orderByChild("University").startAt("ps").endAt("\uf8ff");
Related
My application use keywords extensively, everything is tagged with keywords, so whenever use wants to search data or add data I have to show keywords in auto complete box.
As of now I am storing keywords in another collection as below
export interface IKeyword {
Id:string;
Name:string;
CreatedBy:IUserMin;
CreatedOn:firestore.Timestamp;
}
export interface IUserMin {
UserId:string;
DisplayName:string;
}
export interface IKeywordMin {
Id:string;
Name:string;
}
My main document holds array of Keywords
export interface MainDocument{
Field1:string;
Field2:string;
........
other fields
........
Keywords:IKeywordMin[];
}
But problem is auto complete reads data frequently and my document reads quota increases very fast.
Is there a way to implement this without increasing reads for keyword ? Because keyword is not the real data we need to get.
Below is my query to get main documents
query = query.where("Keywords", "array-contains-any", keywords)
I use below query to get keywords in auto complete text box
query = query.orderBy("Name").startAt(searchTerm).endAt(searchTerm+ '\uf8ff').limit(20)
this query run many times when user types auto complete search which is causing more document reads
Does this answer your question
https://fireship.io/lessons/typeahead-autocomplete-with-firestore/
Though the receommended solution is to use 3rd party tool
https://firebase.google.com/docs/firestore/solutions/search
To reduce documents read:
A solution that come to my mind however I'm not sure if it's suitable for your use case is using Firestore caching feature. By default, firestore client will always try to reach the server to get the new changes on your documents and if it cannot reach the server, it will reach to the cached data on the client device. you can take advantage of this feature by using the cache first and reach the server only when you want. For web application, this feature is disabled by default and you can enable it like in
https://firebase.google.com/docs/firestore/manage-data/enable-offline
to help you understand this feature more check this article:
https://firebase.google.com/docs/firestore/manage-data/enable-offline
I found a solution, thought I would share here
Create a new collection named typeaheads in below format
export interface ITypeAHead {
Prefix:string;
CollectionName:string;
FieldName:string;
MatchingValues:ILookupItem[]
}
export interface ILookupItem {
Key:string;
Value:string;
}
depending on the minimum letters add either 2 or 3 letters to Prefix, and search based on the prefix, collection and field. so most probably you will end up with 2 or 3 document reads for on search.
Hope this helps someone else.
I am trying to pass a object { key:value} and send it to meteor publish so i can query to database.
My Mongo db database has (relevant datas only) for products:
products : {
categs:['Ladies Top','Gents'],
name : Apple
}
In meteor Publish i have the following:
Meteor.publish('product', (query) =>{
return Clothings.find(query);
})
In client i use the following to subscribe:
let query = {categs:'/ladies top/i'}; // please notice the case is lower
let subscribe = Meteor.subscribe('product',query);
if (subscribe.ready()){
clothings = Products.find(query).fetch().reverse();
let count = Products.find(query).fetch().reverse().length; // just for test
}
The issue is, when i send the query from client to server, it is automatically encoded eg:
{categs:'/ladies%top/i'}
This query doesnot seem to work at all. There are like total of more than 20,000 products and fetching all is not an option. So i am trying to fetch based on the category (roughly around 100 products each).
I am new to ,meteor and mongo db and was trying to follow existing code, however this doesnot seem to be correct. Is there a better way to improve the code and achieve the same ?
Any suggestion or idea is highly appreciated.
I did go through meteor docs but they dont seem to have examples for my scenario so i hope someone out there can help me :) Cheers !
Firstly, you are trying to send a regex as a parameter. That's why it's being encoded. Meteor doesn't know how to pass functions or regexes as parameters afaict.
For this specific publication, I recommend sending over the string you want to search for and building the regex on the server:
client:
let categorySearch = 'ladies top';
let obj = { categorySearch }; // and any other things you want to query on.
Meteor.subscribe('productCategory',obj);
server:
Meteor.publish('productCategory',function(obj){
check(obj,Object);
let query = {};
if (obj.categorySearch) query.category = { $regex: `/${obj.categorySearch}/i` };
// add any other search parameters to the query object here
return Products.find(query);
});
Secondly, sending an entire query objet to a publication (or Method) is not at all secure since an attacker can then send any query. Perhaps it doesn't matter with your Products collection.
I am new to firebase and I am having a bit of a nightmare trying to adapt old code to what is now deprecated and what is not. I am trying to write a function which updates one "single" record in my datasource using the now approved $save()promise but it is doing some really strange stuff to my data source.
My function (should) enables you to modify a single record then update the posts json array. However, instead of doing this, it deletes the whole datasource on the firebase server and it is lucky that I am only working with testdata at this point because everything would be gone.
$scope.update = function() {
var fb = new Firebase("https://mysource.firebaseio.com/Articles/" + $scope.postToUpdate.$id);
var article = $firebaseObject(ref);
article.$save({
Title: $scope.postToUpdate.Title,
Body: $scope.postToUpdate.Body
}).then(function(ref) {
$('#editModal').modal('hide');
console.log($scope.postToUpdate);
}, function(error) {
console.log("Error:", error);
});
}
Funnily enough I then get a warning in the console "after" I click the button:
Storing data using array indices in Firebase can result in unexpected behavior. See https://www.firebase.com/docs/web/guide/understanding-data.html#section-arrays-in-firebase for more information. Also note that you probably wanted $firebaseArray and not $firebaseObject.
(No shit?) I am assuming here that $save() is not the right call, so what is the equivalent of $routeParams/$firebase $update()to do a simple binding of the modified data and my source? I have been spending hours on this and really don't know what is the right solution.
Unless there's additional code that you've left out, your article $firebaseObject should most likely use the fb variable you created just before it.
var article = $firebaseObject(fb);
Additionally, the way in which you're using $save() is incorrect. You need to modify the properties on the $firebaseObject directly and then call $save() with no arguments. See the docs for more.
article.Title = $scope.postToUpdate.Title;
article.Body = $scope.postToUpdate.Body;
article.$save().then(...
I want to build a realtime quiz game which randomly matches two players (according to their winning rate if they are logged in). I've read through the book Discover Meteor and have a basic understanding of the framework, but I just have no idea of how to implement the matching part. Anyone know how to do that?
if you want to match users who have scores close to each other, you can do something like this : mongodb - Find document with closest integer value
The Meteor code for those Mongo queries is very similar, but there are some subtle differences that are kind of tricky. In Meteor, it would look something like this :
SP // "selected player" = the User you want to match someone up with
var score = SP.score; // selected player's score
var queryLow = {score: {$lte:score},_id:{$ne:SP._id}};
var queryHigh = {score:{$gte:score},_id:{$ne:SP._id}};
// "L" is the player with the closest lower score
var L=Players.findOne(queryLow,{sort:{score:-1},limit:1});
// "H" is the player with the closest higher score
var H=Players.findOne(queryHigh,{sort:{score:1},limit:1});
so, now you have references to the players with scores right above and right below the 'selected player'. In terms of making it random, perhaps start with a simple algorithm like "match me with the next available player who's score is closest" , then if it's too predictable and boring you can throw some randomness into the algorithm.
you can view the above Meteor code working live here http://meteorpad.com/pad/4umMP4iY8AkB9ct2d/ClosestScore
and you can Fork it and mess about with the queries to see how it works.
good luck! Meteor is great, I really like it.
If you add the package peppelg:random-opponent-matcher to your application, you can match together opponents like this:
On the server, you need to have an instance of RandomOpponentMatcher like this:
new RandomOpponentMatcher('my-matcher', {name: 'fifo'}, function(user1, user2){
// Create the match/game they should play.
})
The function you pass to RandomOpponentMatcher will get called when two users been matched to play against each other. In it, you'll probably want to create the match the users should play against each other (this package does only match opponents together, it does not contain any functionality for playing games/matches).
On the client, you need to create an instance of RandomOpponentMatcher as well, but you only pass the name to it (the same name as you used on the server):
myMatcher = new RandomOpponentMatcher('my-matcher')
Then when the users is logged in and which to be matched with a random opponent, all you need to do is to call the add method. For example:
<template name="myTemplate">
<button class="clickMatchesWithOpponent">Match me with someone!</button>
</template>
Template.myTemplate.events({
'click .clickMatchesWithOpponent': function(event, template){
myMatcher.add()
}
})
When two different logged in users has clicked on the button, the function you passed to RandomOpponentMatcher on the server will get called.
One implementation might be as follows:
A user somehow triggers a 'looking for game' event that sets an attribute on user.profile.lookingForGame to true. The event then makes a call to a server side Meteor method which queries for all other online users looking for games.
From there you it really depends on how you want to handle users once they 'match'.
To determine all online users, try using the User Status package:
https://github.com/mizzao/meteor-user-status
Once added, any online user will have an attribute in the profile object of 'online'. You can use this to query for all online users.
Tryng to get a simple result using "Where" style in firebase but get null althe time, anyone can help with that?
http://jsfiddle.net/vQEmt/68/
new Firebase("https://examples-sql-queries.firebaseio.com/messages")
.startAt('Inigo Montoya')
.endAt('Inigo Montoya')
.once('value', show);
function show(snap) {
$('pre').text(JSON.stringify(snap.val(), null, 2));
}
Looking at the applicable records, I see that the .priority is set to the timestamp, not the username.
Thus, you can't startAt/endAt the user's name as you've attempted here. Those are only applicable to the .priority field. These capabilities will be expanding significantly over the next year, as enhancements to the Firebase API continue to roll out.
For now, your best option for arbitrary search of fields is use a search engine. It's wicked-easy to spin one up and have the full power of a search engine at your fingertips, rather than mucking with glacial SQL-esque queries. It looks like you've already stumbled on the appropriate blog posts for that topic.
You can, of course, use an index which lists users by name and stores the keys of all their post ids. And, considering this is a very small data set--less than 100k--could even just grab the whole thing and search it on the client (larger data sets could use endAt/startAt/limit to grab a recent subset of messages):
new Firebase("https://examples-sql-queries.firebaseio.com/messages").once('value', function(snapshot) {
var messages = [];
snapshot.forEach(function(ss) {
if( ss.val().name === "Inigo Montoya" ) {
messages.push(ss.val());
}
});
console.log(messages);
});
Also see: Database-style queries with Firebase