I'm trying to do this relatively complex operation in BaconJs.
Basically, the idea is keep trying each check until you have a 'pass' status or they all fail. The catch is that 'pending' statuses have a list of Observables (built from jquery ajax requests) that will resolve the check. For performance reasons, you need to try each Observable in order until either they all pass or one fails.
Here's the full pseudo algorithm:
Go thru each check. A check contains an id and status = fail/pass/pending. If pending, it contains a list of observables.
If status = pass, then return the id (you're done!)
if status = fail, then try the next check
if status = pending
try each observable in order
if observable result is 'false', then try the next check
if reach end of observable list and result is 'true', then return the id (you're done!)
Here's the Bacon code. It doesn't work when the Observables are Ajax requests.
Basically, what happens is that it skips over pending checks....it doesn't wait for the ajax calls to return. If I put a log() right before the filter(), it doesn't log pending requests:
Bacon.fromArray(checks)
.flatMap(function(check) {
return check.status === 'pass' ? check.id :
check.status === 'fail' ? null :
Bacon.fromArray(check.observables)
.flatMap(function(obs) { return obs; })
.takeWhile(function(obsResult) { return obsResult; })
.last()
.map(function(obsResult) { return obsResult ? check.id : null; });
})
.filter(function(contextId) { return contextId !== null; })
.first();
UPDATE: the code works when the checks look like this: [fail, fail, pending]. But it doesn't work when the checks look like this: [fail, pending, pass]
I am more familiar with RxJS than Bacon, but I would say the reason you aren't seeing the desired behavior is because flatMap waits for no man.
It passes [fail, pending, pass] in quick succession, fail returns null and is filtered out. pending kicks off an observable, and then receives pass which immediately returns check.id (Bacon may be different, but in RxJS flatMap won't accept a single value return). The check.id goes through filter and hits first at which point it completes and it just cancels the subscription to the ajax request.
A quick fix would probably be to use concatMap rather than flatMap.
In RxJS though I would refactor this to be (Disclaimer untested):
Rx.Observable.fromArray(checks)
//Process each check in order
.concatMap(function(check) {
var sources = {
//If we pass then we are done
'pass' : Rx.Observable.just({id : check.id, done : true}),
//If we fail keep trying
'fail' : Rx.Observable.just({done : false}),
'pending' : Rx.Observable.defer(function(){ return check.observables;})
.concatAll()
.every()
.map(function(x) {
return x ? {done : true, id : check.id} :
{done : false};
})
};
return Rx.Observable.case(function() { return check.status; }, sources);
})
//Take the first value that is done
.first(function(x) { return x.done; })
.pluck('id');
What the above does is:
Concatenate all of the checks
Use the case operator to propagate instead of nested ternaries.
Fail or pass fast
If pending create a flattened observable out of check.observables, if they are all true then we are done, otherwise continue to the next one
Use the predicate value of first to get the first value returned that is done
[Optionally] strip out the value that we care about.
I agree with #paulpdaniels Rx-based answer. The problem seems to be that when using flatMap, Bacon.js won't wait for your first "check-stream" to complete before launching a new one. Just replace flatMap with flatMapConcat.
Thanks to #raimohanska and #paulpdaniels. The answer is to use #flatMapConcat. This turns what is basically a list of async calls done in parallel into a sequence of calls done in order (and note that the last "check" is programmed to always pass so that this always outputs something):
Bacon.fromArray(checks)
.flatMapConcat(function(check) {
var result = check();
switch(result.status) {
case 'pass' :
case 'fail' :
return result;
case 'pending' :
return Bacon.fromArray(result.observables)
.flatMapConcat(function(obs) { return obs; })
.takeWhile(function(obsResult) { return obsResult.result; })
.last()
.map(function (obsResult) { return obsResult ? {id: result.id, status: 'pass'} : {status: 'fail'}; });
}
})
.filter(function(result) { return result.status === 'pass'; })
.first()
.map('.id');
Related
I'm using the Nodejs library for talking to Jira called jira-connector. I can get all of the boards on my jira instance by calling
jira.board.getAllBoards({ type: "scrum"})
.then(boards => { ...not important stuff... }
the return set looks something like the following:
{
maxResults: 50,
startAt: 0,
isLast: false,
values:
[ { id: ... } ]
}
then while isLast === false I keep calling like so:
jira.board.getAllBoards({ type: "scrum", startAt: XXX })
until isLast is true. then I can organize all of my returns from promises and be done with it.
I'm trying to reason out how I can get all of the data on pages with Ramda, I have a feeling it's possible I just can't seem to sort out the how of it.
Any help? Is this possible using Ramda?
Here's my Rx attempt to make this better:
const pagedCalls = new Subject();
pagedCalls.subscribe(value => {
jira.board.getAllBoards({ type:"scrum", startAt: value })
.then(boards => {
console.log('calling: ' + value);
allBoards.push(boards.values);
if (boards.isLast) {
pagedCalls.complete()
} else {
pagedCalls.next(boards.startAt + 50);
}
});
})
pagedCalls.next(0);
Seems pretty terrible. Here's the simplest solution I have so far with a do/while loop:
let returnResult = [];
let result;
let startAt = -50;
do {
result = await jira.board.getAllBoards( { type: "scrum", startAt: startAt += 50 })
returnResult.push(result.values); // there's an array of results under the values prop.
} while (!result.isLast)
Many of the interactions with Jira use this model and I am trying to avoid writing this kind of loop every time I make a call.
I had to do something similar today, calling the Gitlab API repeatedly until I had retrieved the entire folder/file structure of the project. I did it with a recursive call inside a .then, and it seems to work all right. I have not tried to convert the code to handle your case.
Here's what I wrote, if it will help:
const getAll = (project, perPage = 10, page = 1, res = []) =>
fetch(`https://gitlab.com/api/v4/projects/${encodeURIComponent(project)}/repository/tree?recursive=true&per_page=${perPage}&page=${page}`)
.then(resp => resp.json())
.then(xs => xs.length < perPage
? res.concat(xs)
: getAll(project, perPage, page + 1, res.concat(xs))
)
getAll('gitlab-examples/nodejs')
.then(console.log)
.catch(console.warn)
The technique is pretty simple: Our function accepts whatever parameters are necessary to be able to fetch a particular page and an additional one to hold the results, defaulting it to an empty array. We make the asynchronous call to fetch the page, and in the then, we use the result to see if we need to make another call. If we do, we call the function again, passing in the other parameters needed, the incremented page number, and the merge of the current results and the ones just received. If we don't need to make another call, then we just return that merged list.
Here, the repository contains 21 files and folders. Calling for ten at a time, we make three fetches and when the third one is complete, we resolve our returned Promise with that list of 21 items.
This recursive method definitely feels more functional than your versions above. There is no assignment except for the parameter defaulting, and nothing is mutated along the way.
I think it should be relatively easy to adapt this to your needs.
Here is a way to get all the boards using rubico:
import { pipe, fork, switchCase, get } from 'rubico'
const getAllBoards = boards => pipe([
fork({
type: () => 'scrum',
startAt: get('startAt'),
}),
jira.board.getAllBoards,
switchCase([
get('isLast'),
response => boards.concat(response.values),
response => getAllBoards(boards.concat(response.values))({
startAt: response.startAt + response.values.length,
})
]),
])
getAllBoards([])({ startAt: 0 }) // => [...boards]
getAllBoards will recursively get more boards and append to boards until isLast is true, then it will return the aggregated boards.
I have an effect that needs to return two actions. First, there is some logic involved before returning the first action. The second action is supposed to be dispatched after the first action, because it gets the data from the first action's state.
I have something like this:
return Observable.combineLatest(dataRequests).pipe(
map(res => {
const rows = *some logic done here*
};
return {
type: types.SET_DATA,
payload: rows;
}
})
merge(observableOf({
type: types.SAVE_DATA
}))
)
The problem is that SAVE_DATA is getting dispatched before SET_DATA, therefore it's not able to get the data from the state. How can I modify this so SAVE_DATA is always dispatched after SET_DATA?
fixed by replacing "merge" with "concat"
I'm using Saga's takeLatest to abort all requests except the latest. This works fine, but now I want to only abort requests which don't have identical url, params, and method.
I know Saga uses the type attribute to compare actions (just like vanilla Redux does), but I've also added url, params, and method to my actions because I was hoping there there was some way to do something like
yield takeLatestIf((action, latestAction) => {
const sameType = action.type === latestAction.type;
const sameUrl = action.url === latestAction.type;
const sameParams = areEqual(action.params, lastAction.params);
const sameMethod = action.method === lastAction.method;
return sameType && sameUrl && sameParams && sameMethod;
});
which should only abort requests if all 4 of those attribute comparisons are false.
How can I accomplish this?
If I get it right from your question, you want this:
Like standard takeLatest().
But when a duplicate request is made, ignore it and wait for the one already executing (a reasonable use case).
So I took takeLatest() implementation provided in the docs and adapted it to your scenario:
const takeLatestDeduped = (patternOrChannel, compareActions, saga, ...args) => fork(function*() {
let lastTask
let lastAction
while (true) {
const action = yield take(patternOrChannel)
// New logic: ignore duplicate request
if (lastTask && lastTask.isRunning() && !compareActions(lastAction, action)) {
continue
}
if (lastTask) {
yield cancel(lastTask)
}
lastTask = yield fork(saga, ...args.concat(action))
// New logic: save last action
lastAction = action
}
})
We have three cases:
No running task: start the new one - standard behavior
Running task, got non-duplicate: cancel old one, start new one - standard behavior
Running task, got duplicate: ignore - new custom hehavior
So I added case #3 logic:
Ignoring duplicate request (nothing should be done in this case, so I continue to handling next action).
Saving last action for future duplicate check.
I have a Meteor Helper that does a GET request and am supposed to get response back and pass it back to the Template, but its now showing up the front end. When I log it to console, it shows the value corerctly, for the life of mine I can't get this to output to the actual template.
Here is my helper:
UI.registerHelper('getDistance', function(formatted_address) {
HTTP.call( 'GET', 'https://maps.googleapis.com/maps/api/distancematrix/json? units=imperial&origins=Washington,DC&destinations='+formatted_address+'&key=MYKEY', {}, function( error, response ) {
if ( error ) {
console.log( error );
} else {
var distanceMiles = response.data.rows[0].elements[0].distance.text;
console.log(response.data.rows[0].elements[0].distance.text);
return distanceMiles;
}
});
});
In my template I pass have the following:
{{getDistance formatted_address}}
Again, this works fine and shows exactly what I need in the console, but not in the template.
Any ideas what I'm doing wrong?
I posted an article on TMC recently that you may find useful for such a pattern. In that article the problem involves executing an expensive function for each item in a list. As others have pointed out, doing asynchronous calls in a helper is not good practice.
In your case, make a local collection called Distances. If you wish, you can use your document _id to align it with your collection.
const Distances = new Mongo.collection(); // only declare this on the client
Then setup a function that either lazily computes the distance or returns it immediately if it's already been computed:
function lazyDistance(formatted_address){
let doc = Distances.findOne({ formatted_address: formatted_address });
if ( doc ){
return doc.distanceMiles;
} else {
let url = 'https://maps.googleapis.com/maps/api/distancematrix/json';
url += '?units=imperial&origins=Washington,DC&key=MYKEY&destinations=';
url += formatted_address;
HTTP.call('GET',url,{},(error,response )=>{
if ( error ) {
console.log( error );
} else {
Distances.insert({
formatted_address: formatted_address,
distanceMiles: response.data.rows[0].elements[0].distance.text
});
}
});
}
});
Now you can have a helper that just returns a cached value from that local collection:
UI.registerHelper('getDistance',formatted_address=>{
return lazyDistance(formatted_address);
});
You could also do this based on an _id instead of an address string of course. There's a tacit assumption above that formatted_address is unique.
It's Meteor's reactivity that really makes this work. The first time the helper is called the distance will be null but as it gets computed asynchronously the helper will automagically update the value.
best practice is not to do an async call in a helper. think of the #each and the helper as a way for the view to simply show the results of a prior calculation, not to get started on doing the calculation. remember that a helper might be called multiple times for a single item.
instead, in the onCreated() of your template, start the work of getting the data you need and doing your calculations. store those results in a reactive var, or reactive array. then your helper should do nothing more than look up the previously calculated results. further, should that helper be called more times than you expect, you don't have to worry about all those additional async calls being made.
The result does not show up because HTTP.call is an async function.
Use a reactiveVar in your case.
Depending on how is the formated_address param updated you can trigger the getDistance with a tracker autorun.
Regs
Yann
I'm trying to perform a custom sort using a comparator function from within a template helper in Meteor.
Here is my template helper:
Template.move_list.helpers({
assets() {
return Assets.find({}, { sort: sortFunction });
}
});
And here is the comparator function:
const sortFunction = function (doc1, doc2) {
const barcodes = Session.get('barcodesArray');
if (barcodes.indexOf(doc1.barcode) === -1 || barcodes.indexOf(doc2.barcode) === -1) {
return 0;
}
let last = null;
_.each(barcodes, function (barcode) {
if (barcode === doc1.barcode) last = doc1.barcode;
if (barcode === doc2.barcode) last = doc2.barcode;
});
return last === doc1.barcode ? 1 : -1;
}
Error
When the page loads, the following error is returned:
Exception in template helper: Error: Match error: Failed Match.OneOf, Match.Maybe or Match.Optional validation
I put a breakpoint in chrome into the sortFunction, however the function was never entered and the breakpoint never reached.
Of course, the error is not throw when I remove sort.
References
This feature is not very well documented, however here is the relevant part of the docs:
For local collections you can pass a comparator function which receives two document objects, and returns -1 if the first document comes first in order, 1 if the second document comes first, or 0 if neither document comes before the other. This is a Minimongo extension to MongoDB.
And the commit by mitar adding the functionality, with example code from the test:
var sortFunction = function (doc1, doc2) {
return doc2.a - doc1.a;
};
c.find({}, {sort: sortFunction})
Can anyone make sense of this error?
Edit:
This issue should be resolved in Meteor >= v1.3.3.1.
Local collections (i.e, client-side and in-memory server-side collections) will allow to pass a function as the sort clause.
The error comes from the mongo package, where the spec does not allow sort to be a function.
#mitar changed LocalCollection in the minimongo package. LocalCollection is part of the Mongo.Collection object on the client (its _collection attribute), but queries are still checked according to the original mongo spec. I believe this to be a bug, as the spec was not updated to reflect the change.
To overcome this (in the meantime), either have the function accept a sub-field, such that the sort value is an object:
var sortFunction = function (x, y) {
return x - y;
};
c.find({}, {sort: {a: sortFunction}});
or use the c._collection.find() instead, which will work (as far as I can tell), except it will not apply any transformations defined for the collection.
var sortFunction = function (doc1, doc2) {
return doc2.a - doc1.a;
};
c._collection.find({}, {sort: sortFunction});