Text search with Meteor and Mongo 2.6 - meteor

I'd like to perform the equivalent of this Mongo shell command in meteor(server-side, of course):
db.articles.find(
{ $text: { $search: "apple pie" } },
{ score: { $meta: "textScore" } }
).sort( { score: { $meta: "textScore" } } ).limit(10)
I have been able to do:
return Articles.find( { $text: { $search: "apple" } },
{ sort: {"name":1}, limit:20});
However, searching for "pie apple" doesn't work - it only does exact matching. Neither does trying to sort by score.
I am using mongo 2.6.3 with a text index on the name field in articles. Searching from within mongo shell works perfectly.
Also, has anyone successfully implemented a text search with a different approach? My database has 10k entries and I only need to search within a single field, and return 20 best matches.

3 steps (meteor 1.0.4+) now using MongoDB 3.0. Assuming you already have the YourCollection collection i.e.
YourCollection = new Meteor.Collection("yourCollection");
A. Index your collection (server side) here below is how to index all the fields, more info here
Meteor.startup(function (){
YourCollection._ensureIndex(
{"$**": "text"},
{"name": "searchIndex"}
); }
B. Create the publication (server side)
Meteor.publish("search-yourCollection", function(searchField)
{
return YourCollection.find({"$text": {"$search": searchField}},
{
fields: {
score: {$meta: "textScore"}
},
sort: {
score: {$meta: "textScore"}
}
});
});
C. Subscribe to the publication and find (client side)
var whatToSearch = "abc"; // can be taken out of the session
Meteor.subscribe("search-yourCollection", whatToSearch);
var results = YourCollection.find({score:{"$exists":true}});
Remark: The publication will add a score property to all the returned items. Ensuring this property exists within the find function {"$exists":true} will make sure you are finding the elements returned by the search-yourCollection publication. This is mandatory if you are subscribing to another publication adding items into YourCollection published set.

I've been able to get this to work using something similar to:
Implementing MongoDB 2.4's full text search in a Meteor app
The differences are:
MongoInternals.defaultRemoteCollectionDriver().mongo.db.executeDbCommand
and the searchDinosaurs function in the link above looks like:
if (query && query !== '') {
var searchResults = _searchArticles(query);
var results = [];
for (var i = 0; i < searchResults.length; i++) {
results.push({
id: searchResults[i].obj._id,
score: searchResults[i].score});
}
var ids = [];
results.sort(function(a,b) { return a.score < b.score } );
for (var i = 0; i < 20; i++) {
if (results[i]!=null){
ids.push(results[i].id);
}
}
return ids;
Here I'm sorting the results on score and returning the top 20 id's. The only issue is, once the user is subscribed to these 20 articles, I have to find and sort them once again client-side using regex search in minimongo. If anyone has suggestions or improvements, I'd love to hear them.

Related

Dynamically toggle resource column visibility

I have a FullCalendar scheduler on a webapp which has 2 way databinding for resources and events, all working great. I want to be able to present the user with a dropdown that enables them to toggle the visibility of a column, ideally completely client side.
I have tried a combination of addResource / removeResource however my issue here is that a rerender of the calendar (e.g. when a new event is added) then displays the previously removed resource. I can work around this however would prefer a really simple approach using JS / CSS. I currently cannot find a way to set a resource to not be visible, or to have zero width - is this possible?
There is an easy way to do this:
Store resources in an array variable resourceData.
Create another array called visibleResourceIds to store the ids of any resources you want to show.
In the resources callback function, filter resourceData to only contain the resources where the resource id exists in visibleResourceIds. Return the filtered array and fullcalendar will only add the desired resources for you.
To remove a resource from view, simply remove the resource id from visibleResourceIds and refetchResources. To add the resource back in, add the id to visibleResourceIds and refetchResources. DONE.
JSFiddle
var resourceData = [
{id: "1", title: "R1"},
{id: "2", title: "R2"},
{id: "3", title: "R3"}
];
var visibleResourceIds = ["1", "2", "3"];
// Your button/dropdown will trigger this function. Feed it resourceId.
function toggleResource(resourceId) {
var index = visibleResourceIds.indexOf(resourceId);
if (index !== -1) {
visibleResourceIds.splice(index, 1);
} else {
visibleResourceIds.push(resourceId);
}
$('#calendar').fullCalendar('refetchResources');
}
$('#calendar').fullCalendar({
defaultView: 'agendaDay',
resources: function(callback) {
// Filter resources by whether their id is in visibleResourceIds.
var filteredResources = [];
filteredResources = resourceData.filter(function(x) {
return visibleResourceIds.indexOf(x.id) !== -1;
});
callback(filteredResources);
}
});
I had the same challenge. Instead of a dropdown, I use checkboxes, but the workings will be the same.
My resources are stored in a variable, when I uncheck a box, the resource is removed and the resource's object is added to another array with the resourceId as key, and the index added to the object to restore the object in the same column as it originally was. When re-checking the box, the object is added to the resources array and the resources refetched.
/* retrieve the resources from the server */
var planningResources;
var removedResource = [];
$.ajax({
url: '/planning/resources/',
method: 'get',
success: function (response) {
planningResources = response;
showCalendar();
}
, error: function () {
if (typeof console == "object") {
console.log(xhr.status + "," + xhr.responseText + "," + textStatus + "," + error);
}
}
});
/* create the calendar */
showCalendar = function () {
$('#calendar').fullCalendar({
...
});
}
/* checkbox on click */
$('.resource').click(function() {
var resourceId = $(this).val();
var hideResource = !$(this)[0].checked;
$('.status:checkbox:checked').each(function () {
});
if(hideResource) {
$.each(planningResources, function(index, value){
if( value && value.id == resourceId ) {
value.ndx = index;
removedResource[resourceId] = value;
planningResources.splice(index,1);
return false;
}
});
$('#planningoverview').fullCalendar(
'removeResource',
resourceId
);
}
else {
planningResources.splice(removedResource[resourceId].ndx, 0, removedResource[resourceId]);
$('#planningoverview').fullCalendar('refetchResources');
}
});
showCalendar();
It probably doesn't get first price in a beauty contest, but it works for me ...
Cheers
You can use the resourceColumns option for this. In the column objects you can set the width property to a number of pixels or a percentage. If you pass a function here you can easily handle the width property someplace else. Your hide/show function can then set the width to 0 to hide the column. After that you can trigger reinitView to update the view: $('#calendar').fullCalendar("reinitView");

Meteor nested publications

I have two collections A and B in Meteor. For A I have a publication where I filter out a range of documents in A. Now I want to create a publications for B where I publish all documents in B that have a field B.length matching A.length.
I have not been able to find any example where this is shown but I feel it must be a standard use case. How can this be done in Meteor?
This is a common pattern for reywood:publish-composite
import { publishComposite } from 'meteor/reywood:publish-composite';
publishComposite('parentChild', {
const query = ... // your filter
find() {
return A.find(query, { sort: { score: -1 }, limit: 10 });
},
children: [
{
find(a) {
return B.find({length: a.length });
}
}
]
});
This is a quite different pattern than serverTransform as on the client you end up with two collections, A and B, as opposed to a synthetic single collection A that has some fields of of B. The latter is more like a SQL JOIN.
Use serverTransform
Meteor.publishTransformed('pub', function() {
const filter = {};
return A.find(filter)
.serverTransform({
'B': function(doc) {
return B.find({
length: doc.length
}); //this will feed directly into miniMongo as if it was a seperate publication
}
})
});

Firebase denormalizing many to many

I have a pretty basic data structure
events
topics
I would like to be able to easily show (query)
what topics are owned by an event
what events cover a topics
what are the most popular topics this month
I am pretty comfortable with my events structure like
/events/880088/topics.json *
["Firebase", "Cloud"]
but I struggle with how to structure the /topics nodes. I partially get the idea of going with something like
/topics/Firebase
{"12345":true,"88088":true}
and then if when I update an events's topic collection I would have to iterate over all the /topics/ nodes and update /topics/{{topic}}/{{eventid}} to {true | null}. Which seems rather ham fisted.
ALSO, then I am still at a loss of how to query say, what are the topics covered by events this month.
Example JSBin from comments below http://jsbin.com/dumumu/edit?js,output
* I know, I know, arrays are evil, https://www.firebase.com/blog/2014-04-28-best-practices-arrays-in-firebase.html, but I think they fit in this scenaris
Here's one way to add an event:
function addEvent(title, topics) {
var event =ref.child('events').push({ title: title });
topics.forEach(function(topic) {
event.child('topics').child(topic).set(true);
ref.child('topics').child(topic).child(event.key()).set(true);
});
}
Seems pretty simple for me. For an interesting twist, you can use the new multi-location updates we launched yesterday (September 2015):
function addEvent(title, topics) {
var updates = {};
var eventId = ref.push().key();
updates['events/'+eventId+'/title'] = title;
topics.forEach(function(topic) {
updates['events/'+eventId+'/topics/'+topic] = true;
updates['topic/'+topic+'/'+eventId] = true;
});
ref.update(updates);
}
The latter is a bit more code. But it's a single write operation to Firebase, so there's no chance of the user closing the app between write operations.
You invoke both the same of course:
addEvent('Learn all about Firebase', ['Firebase']);
addEvent('Cloudspin', ['Firebase', 'Google', 'Cloud']);
And the data structure becomes:
{
"events": {
"-K-4HCzj_ziHkZq3Fpat": {
"title": "Learn all about Firebase",
"topics": {
"Firebase": true
}
},
"-K-4HCzlBFDIwaA8Ajb7": {
"title": "Cloudspin",
"topics": {
"Cloud": true,
"Firebase": true,
"Google": true
}
}
},
"topic": {
"Cloud": {
"-K-4HCzlBFDIwaA8Ajb7": true
},
"Firebase": {
"-K-4HCzj_ziHkZq3Fpat": true,
"-K-4HCzlBFDIwaA8Ajb7": true
},
"Google": {
"-K-4HCzlBFDIwaA8Ajb7": true
}
}
}
Querying/reporting
With Firebase (and most NoSQL databases), you typically have to adapt your data structure for the reporting you want to do on it.
Abe wrote a great answer on this recently, so go read that for sure: Firebase Data Structure Advice Required
Update: change the topics for an event
If you want to change the topics for an existing event, this function is once way to accomplish that:
function updateEventTopics(event, newTopics) {
newTopics.sort();
var eventId = event.key();
var updates = {};
event.once('value', function(snapshot) {
var oldTopics = Object.keys(snapshot.val().topics).sort();
var added = newTopics.filter(function(t) { return oldTopics.indexOf(t) < 0; }),
removed = oldTopics.filter(function(t) { return newTopics.indexOf(t) < 0; });
added.forEach(function(topic) {
updates['events/'+eventId+'/topics/'+topic] = true;
updates['topic/'+topic+'/'+eventId] = true;
});
removed.forEach(function(topic) {
updates['events/'+eventId+'/topics/'+topic] = null;
updates['topic/'+topic+'/'+eventId] = null;
});
ref.update(updates);
});
}
The code is indeed a bit long, but that's mostly to determine the delta between the current topics and the new topics.
In case you're curious, if we run these API calls now:
var event = addEvent('Cloudspin', Date.now() - month, ['Firebase', 'Google', 'Cloud']);
updateEventTopics(event, ['Firebase', 'Google', 'GCP']);
The changeEventTopics() call will result in this update():
{
"events/-K-93CxuCrFDxM6k0B14/topics/Cloud": null,
"events/-K-93CxuCrFDxM6k0B14/topics/GCP": true,
"topic/Cloud/-K-93CxuCrFDxM6k0B14": null,
"topic/GCP/-K-93CxuCrFDxM6k0B14": true
}

How to query two types of records in CouchDB

I’m having issues getting two dependant types of data from a PouchDB database.
I have a list of cars that I get like so:
localDB.query(function(doc) {
if (doc.type === ‘list’) {
emit(doc);
}
}, {include_docs : true}).then(function(response) {
console.log(“cars”, response);
// Save Cars List to app
for(var i = 0; i < response.rows.length; i++) {
addToCarsList(response.rows[i].id, response.rows[i].carNumber);
}
console.log(“Cars List: " + carsListToString());
return response;
}).then(function(listRecord) {
listRecord.rows.forEach(function(element, index){
console.log(index + ' -> ', element);
localDB.query(function(doc) {
console.log("filtering with carNb = " + element.carNb);
if (doc.type === 'defect' && doc.listId == getCurrentListId() && doc.carNb == element.carNb ) {
emit(doc);
}
}, {include_docs : false}).then(function(result){
console.log("defects", result);
}).catch(function(err){
console.log("an error has occurred", err);
});
});
}).catch(function(err) {
console.log('error', err);
});
Here's what happens. After getting the list of cars, then for each cars I would like to query the defects and store then in some arrays. Then when all that querying is done, I want to build the UI with the data saved.
But what's happening is that the forEach gets processed quickly and does not wait for the inner async'd localDb.query.
How can I query some documents based on an attribute from a parent query? I looked into promises in the PouchDB doc but I can't understand how to do it.
(please forget about curly quotes and possible lint errors, this code was anonymized by hand and ultra simplified)
The method you are looking for is Promise.all() (execute all promises and return when done).
However, your query is already pretty inefficient. It would be better to create a persistent index, otherwise it has to do a full database scan for every query() (!). You can read up on the PouchDB query guide for details.
I would recommend installing the pouchdb-upsert plugin and then doing:
// helper method
function createDesignDoc(name, mapFunction) {
var ddoc = {
_id: '_design/' + name,
views: {}
};
ddoc.views[name] = { map: mapFunction.toString() };
return ddoc;
}
localDB.putIfNotExists(createDesignDoc('my_index', function (doc) {
emit([doc.type, doc.listId, doc.carNb]);
})).then(function () {
// find all docs with type 'list'
return localDB.query('my_index', {
startkey: ['list'],
endkey: ['list', {}],
include_docs: true
});
}).then(function (response) {
console.log("cars", response);
// Save Cars List to app
for(var i = 0; i < response.rows.length; i++) {
addToCarsList(response.rows[i].id, response.rows[i].carNumber);
}
console.log("Cars List: " + carsListToString());
return response;
}).then(function (listRecord) {
return PouchDB.utils.Promise.all(listRecord.rows.map(function (row) {
// find all docs with the given type, listId, carNb
return localDB.query('my_index', {
key: ['defect', getCurrentListId(), row.doc.carNb],
include_docs: true
});
}));
}).then(function (finalResults) {
console.log(finalResults);
}).catch(function(err){
console.log("an error has occurred", err);
});
I'm using a few tricks here:
emit [doc.type, doc.listId, doc.carNb], which allows us to query by type or by type+listId+carNb.
when querying for just the type, we can do {startkey: ['list'], endkey: ['list', {}]}, which matches just those with the type "list" because {} is the "higher" than strings in CouchDB object collation order.
PouchDB.utils.Promise is a "hidden" API, but it's pretty safe to use if you ask me. It's unlikely we'll change it.
Edit Another option is to use the new pouchdb-find plugin, which offers a simplified query API designed to replace the existing map/reduce query() API.
Another approach would be to pull both the list docs and the defect docs down at the same time then merge them together using a reduce like method that will convert them into an array of objects:
{
_id: 1,
type: 'list',
...
defects: [{
type: 'defect'
listId: 1
...
}]
}
By pulling the list and the defects down in one call you save a several calls to the pouchdb query engine, but you do have to iterate through every result to build your collection of lists objects with and embedded array of defects.
// This is untested code so it may not work, but you should get the idea
var _ = require('underscore');
// order documents results by list then defect
var view = function (doc) {
if (doc.type === 'list') {
emit([doc._id, doc.carNumber, 1);
} else if (doc.type === 'defect') {
emit([doc.listId, doc.carNb, 2])
}
}
localDB.query(view, { include_docs: true })
.then(function(response) {
return _(response.rows)
.reduce(function(m, r) {
if (r.key[2] === 1) {
// initialize
r.doc.defects = [];
m.push(r.doc)
return m;
}
if (r.key[2] === 2) {
var list = _(m).last()
if (list._id === r.key[0] && list.carNumber === r.key[1]) {
list.defects.push(r.doc);
}
return m;
}
}, []);
})
.then(function(lists) {
// bind to UI
});
With couch, we found reducing calls to the couch engine to be more performant, but I don't know if this approach is better for PouchDB, but this should work as a solution, especially if you are wanting to embed several collections into one list document.

In Collection.find, how to format .limit, .sort, fieldlist, and variable column names

In non Meteor Server-Side calls to mongodb it is possible make the following chained-option call to the database
collection.find( { myField: { $gte: myOffset } ).limit( myLimit ).sort( { mySortField : 1 } );
where myField, myOffset, myLimit and mySortField may be resolved from elsewhere at run-time.
This pattern is very useful to create such a run-time generated generic query.
Meteor seems to insist on the non-chained options pattern of
collection.find( { { myField: { $gte: myOffset } }, { limit: myLimit, sort: { mySortField : 1 }} );
and I am having problems 'building up' a working Find Query as required above from js objects as described
in previous questions 17362401 and 10959729
Would anyone like to help?
Edited to show usage of variable:
I do it this way. You send two hashes, where the first is the where clause, and all else are peer level keys.
var locations;
var myfield = 'gps';
search = {
sureties: {
$in: sureties
}
}
search[myfield] = {
$near: this.gps,
$maxDistance: kilometers
};
locations = Agents.find(search, {
fields: {
name: 1,
phone: 1
},
limit: limit,
sort: { field1 : 1 }
}).fetch();
The chained pattern is not possible in Meteor, neither server side nor on the client. But the params pattern is as universal, you should be able to create any query you need with those params.

Resources