I have some slow part of my code that does a http request then waits for the answer and calls an adf pipeline so it can update a database parameter at the end. Since this took 2 minutes and my client app seemed so slow for my clients, I wrapped this call inside Task.Run() and set my logic inside. This works good but the last part, update a database parameter, doesn't get executed at all.
Task.Run(async () =>
{
await client.ExecuteAsync(request, CancellationToken.None);
await RunADFPipeline(pipeline, _config);
await _projectRepository.UpdateValue(id, "status", "unlock");
});
I found this example online and I applied it in my code. The first two awaits seem to execute correctly but the third one doesn't execute at all. If I set it outside of Task.Run(), it executes. The third part literally just calls a stored procedure
return await _context.Database.ExecuteSqlRawAsync("EXEC core.UpdateValue #ProjectId, #Name, #Value", parameters);
I also tried by adding ContinueWith() at the end of the Task.Run() but still no luck.
Task.Run(async () =>
{
await client.ExecuteAsync(request, CancellationToken.None);
await _adfHelper.RunADFPipeline(pipeline, _config);
}).ContinueWith((result) => _projectRepository.UpdateValue(id, "status", "unlock"));
All this code is inside a controller. Any thoughts and ideas on how can I achieve my goal and update this parameter after Task.Run() finishes or the functions inside finish?
Related
Keeping a cron job pub/sub function (functions.pubsub.schedule), within a cloud function (functions.https.OnRequest) and exporting it, does not execute.
A complete example is as follows:
export const sayHelloWhen = functions.https.onRequest((request, response) => {
cors(request, response, () => {
const scheduleExpression = request.body.data.scheduleExpression;
functions.logger.log(`Called sayHelloWhen with ${scheduleExpression}`);
functions.pubsub.schedule(scheduleExpression).onRun((context) => {
functions.logger.log(`Executed sayHelloWhen with ${scheduleExpression}`)
});
response.send({
status: "success",
data: `scheduled at ${scheduleExpression}`
})
})
})
The problem is pub/sub does not trigger. Other codes are executed.
I would like to have HTTP request body scheduleExpression bring into pubsub.schedule's parameter. I don't want a static schedule expression in corn job.
In client, I would like to define a schedule expression in client side as follows:
function scheduleFunction() {
const functions = getFunctions();
const sayHello = httpsCallable(functions, "sayHelloWhen");
sayHello({ scheduleExpression: "every 1 minute" }).then((result) => {
// const data = result.data;
console.log("Result:", result);
});
}
The example below works only for a static schedule expression, meaning that a cloud function itself has a fixed schedule expression:
exports.scheduledFunction = functions.pubsub.schedule('every 5 minutes').onRun((context) => {
console.log('This will be run every 5 minutes!');
return null;
});
It can be exported as cron job trigger and it executes.
But keeping pub/sub cron job function, within onRequest cloud function, as in the first code example, does not execute.
I think it is very interesting what you are trying to do, but I would like to point out that you are missing some steps that could cause your app not to execute the way you need it to.
First, you need to Terminate your HTTP functions
Always end an HTTP function with send(), redirect(), or end(). Otherwise, your function might continue to run and be forcibly terminated by the system. See also Sync, Async and Promises.
When you do not terminate them, you might end up in a deeper level in which the following code will not execute unless the previous code has finished. I would also like to say I have not found any application with nested functions like you are doing.
In the Sync, async, and promises page you can find a very explicative video to understand the lifecycle of your functions, and in the How promises work with functions section we have:
When you return a JavaScript promise to a function, that function keeps running until the promise is resolved or rejected. To indicate that a function has completed its work successfully, the promise should be resolved. To indicate an error, the promise should be rejected. This means you only need to handle errors that you want to.
In addition to all this, I would suggest using a separate file to Organize multiple functions for a more organized code, but this is only a suggestion not really necessary.
Finally, I am concerned about the scheduledExpression parameter, I think if none of the above works, you might want to check and share what this value is.
I'm still struggeling with the async/await pattern so I'm here to ask you some precisions.
I saw this page explaining the async/await pattern pretty well. I'm posting here the example that bother me :
import 'dart:async';
Future<String> firstAsync() async {
await Future<String>.delayed(const Duration(seconds: 2));
return "First!";
}
Future<String> secondAsync() async {
await Future<String>.delayed(const Duration(seconds: 2));
return "Second!";
}
Future<String> thirdAsync() async {
await Future<String>.delayed(const Duration(seconds: 2));
return "Third!";
}
void main() async {
var f = await firstAsync();
print(f);
var s = await secondAsync();
print(s);
var t = await thirdAsync();
print(t);
print('done');
}
In this example, each async method is called one after another, so the execution time for the main function is 6 seconds (3 x 2 seconds). However, I don't understand what's the point of asynchronous function if they are executed one after another.
Are async functions not supposed to execute in the background ? Is it not the point of multiple async functions to fastens the process with parrallel execution ?
I think I'm missing something about asynchronous functions and async/await pattern in flutter so if you could explain me that, it would be very appreciated.
Best
Waiting on multiple Futures to complete using Future.wait()
If the order of execution of the functions is not important, you can use Future.wait().
The functions get triggered in quick succession; when all of them complete with a value, Future.wait() returns a new Future. This Future completes with a list containing the values produced by each function.
Future
.wait([firstAsync(), secondAsync(), thirdAsyncC()])
.then((List responses) => chooseBestResponse(responses))
.catchError((e) => handleError(e));
or with async/await
try {
List responses = await Future.wait([firstAsync(), secondAsync(), thirdAsyncC()]);
} catch (e) {
handleError(e)
}
If any of the invoked functions completes with an error, the Future returned by Future.wait() also completes with an error. Use catchError() to handle the error.
Resource:https://v1-dartlang-org.firebaseapp.com/tutorials/language/futures#waiting-on-multiple-futures-to-complete-using-futurewait
The example is designed to show how you can wait for a long-running process without actually blocking the thread. In practice, if you have several of those that you want to run in parallel (for example: independent network calls), you could optimize things.
Calling await stops the execution of the method until the future completes, so the call to secondAsync will not happen until firstAsync finishes, and so on. If you do this instead:
void main() async {
var f = firstAsync();
var s = secondAsync();
var t = thirdAsync();
print(await f);
print(await s);
print(await t);
print('done');
}
then all three futures are started right away, and then you wait for them to finish in a specific order.
It is worth highlighting that now f, s, and t have type Future<String>. You can experiment with different durations for each future, or changing the order of the statements.
If anyone new in this problem use the async . Dart has a function called FutureGroup. You can use it to run futures in parallel.
Sample:
final futureGroup = FutureGroup();//instantiate it
void runAllFutures() {
/// add all the futures , this is not the best way u can create an extension method to add all at the same time
futureGroup.add(hello());
futureGroup.add(checkLocalAuth());
futureGroup.add(hello1());
futureGroup.add(hello2());
futureGroup.add(hello3());
// call the `.close` of the group to fire all the futures,
// once u call `.close` this group cant be used again
futureGroup.close();
// await for future group to finish (all futures inside it to finish)
await futureGroup.future;
}
This futureGroup has some useful methods which can help you ie. .future etc.. check the documentation to get more info.
Here's a sample usage Example One using await/async and Example Two using Future.then.
you can always use them in a single future
final results = await Future.wait([
firstAsync();
secondAsync();
thirdAsync();
]);
results will be an array of you return type. in this case array of strings.
cheers.
Try this resolve.
final List<Future<dynamic>> featureList = <Future<dynamic>>[];
for (final Partner partner in partnerList) {
featureList.add(repository.fetchAvatar(partner.uid));
}
await Future.wait<dynamic>(featureList);
If want parallel execution you should switch to multi thread concept called Isolates
mix this with async/await concepts . You can also check this website for more
https://buildflutter.com/flutter-threading-isolates-future-async-and-await/
Using async / await like that is useful when you need a resource before executing the next task.
In your example you don't do really useful things, but imagine you call firstAsync, that gives you a stored authorization token in your phone, then you call secondAsync giving this token get asynchronously and execute an HTTP request and then checking the result of this request.
In this case you don't block the UI thread (user can interact with your app) and other tasks (get token, HTTP request...) are done in background.
i think you miss understood how flutter works first flutter is not multi threaded.....!
second if it isn't multi threaded how can it executes parallel tasks, which doesnt happen....! here is some links that will help you understand more https://webdev.dartlang.org/articles/performance/event-loop
https://www.dartlang.org/tutorials/language/futures
flutter doesn't put futures on another thread but what happens that they are added to a queue the links that i added are for event loop and how future works. hope you get it , feel free to ask me :)
I have the following scenario:
Client side has a button clicking it will execute Meteor.call method on the server-side which will call API and fetch products, During this time I wan't to disable this button + block this method from executing again basically nothing stops you from clicking the button 100x times and server will keep on executing same method again and again.
Few ideas I had in my mind: Use sessions to disable button (Problem: can still using the console Meteor.call and abuse it)
I also looked at Meteor.apply in the docs with wait:true didn't seems to stop from method execution. I honestly not sure how this kind of thing is handled with no hacks.
Client-side:
'click .button-products': function(e){
Meteor.call('getActiveProducts', function(error, results){
if (error)
return Alerts.add(error.reason, 'danger', {autoHide: 5000});
if (results.success)
return Alerts.add('Finished Importing Products Successfully', 'success', {autoHide: 5000});
})
}
Server-side
Meteor.methods({
getActiveProducts: function(){
var user = Meteor.user();
var api = api.forUser(user);
importProducts = function(items){
nextPage = items.pagination.next_page;
items.results.forEach(function(product){
var sameproduct = apiProducts.findOne({listing_id: product.listing_id});
if (sameproduct) {
return;
}
var productExtend = _.extend(product, {userId: Meteor.userId()});
apiProducts.insert(productExtend);
});
};
var products = api.ProductsActive('GET', {includes: 'Images', limit: 1});
importProducts(products);
while (nextPage !== null) {
products = api.ProductsActive('GET', {includes: 'Images', page: nextPage, limit: 1});
importProducts(products);
}
return {success: true};
}
});
From the Meteor docs:
On the server, methods from a given client run one at a time. The N+1th invocation from a client won't start until the Nth invocation returns. However, you can change this by calling this.unblock. This will allow the N+1th invocation to start running in a new fiber.
What this means is that subsequent calls to the method won't actually know that they were made while the first call was still running, because the first call will have already finished running. But you could do something like this:
Meteor.methods({
getActiveProducts: function() {
var currentUser = Meteor.users.findOne(this.userId);
if (currentUser && !currentUser.gettingProducts) {
Meteor.users.update(this.userId, {$set: {gettingProducts: true}});
// let the other calls run, but now they won't get past the if block
this.unblock();
// do your actual method stuff here
Meteor.users.update(this.userId, {$set: {gettingProducts: false}});
}
}
});
Now subsequent calls may run while the first is still running, but they won't run anything inside the if block. Theoretically, if the user sends enough calls, the first call could finish before all of the others have started. But this should at least significantly limit the number of etsy calls that can be initiated by a user. You could adapt this technique to be more robust, such as storing the last time a successful call was initiated and making sure X seconds have passed, or storing the number of times the method has been called in the last hour and limiting that number, etc.
A package I wrote a while back might come in handy for you. Essentially it exposes the Session api on the server side (hence the name), meaning you can do something like ServerSession.set('doingSomethingImportant', true) within the call, and then check this session's value in subsequent calls. The session can only be set on the server, and expires upon connection close (so they could spam calls, but only as fast as they can refresh the page).
In the event of error, you can just reset the session. There shouldn't be any issues related to unexpected errors either because the session will just expire upon connection close. Let me know what you think :)
I am having trouble figuring out how to use wrapAsync properly using Meteor. I am currently working with node-apac and the Amazon Product Advertising API.
If am trying to run the following code, how do I run it asynchronously:
opHelper.execute('ItemSearch', {
'SearchIndex': 'Books',
'Keywords': 'harry potter',
'ResponseGroup': 'ItemAttributes, Offers'
}, function(err, results) {
console.log(results);
});
I have tried to watch several videos, but am having trouble
Meteor.wrapAsync takes an asynchronous method like opHelper.execute and makes it synchronous. It can do this so long as the last param taken by the method returns a callback where the first param is the error and second the result. This is just like your method!
You make a new method that is synchronous:
var opExecuteSynchronous = Meteor.wrapAsync(opHelper.Execute, opHelper);
The first param opHelper.Execute is the method you want to make asynchronous, and the second param is the context of the method (opHelper).
You can use this synchronously now:
var results = opExecuteSynchronous('ItemSearch', {
'SearchIndex': 'Books',
'Keywords': 'harry potter',
'ResponseGroup': 'ItemAttributes, Offers'
})
This will throw an error if err is called instead of results in the callback.
My actual program is more sophisticated than this, but I've tried to simplify things.
So let's say I'm reading a file with a list of URLs. I want to download the HTML from each URL and process it. The processing may be a bit complex, so I'd like it to be done on a separate thread.
The basic problem is how to tell when all the processing is done. For example, if the user tries to close the program before all URLs are processed, I want to give him a message and not exit the program. Alternatively, I want to terminate the program (perhaps with a MsgBox("Done") message) as soon as all the URLs are processed.
I'd like my code to look something as follows (assuming I've got an outer loop reading the URLs and calling this routine)...
List<Task> TaskList = new List<Task>();
async void ProcessSingleUrl(string url) {
var web = new HttpClient();
var WebPageContents = await web.GetStringAsync(url);
Task t = Task.Run(() => ProcessWebPage(WebPageContents);
TaskList.Add(t);
}
The above code should run very quickly (Async methods run pretty well instantly) and return to the caller almost immediately.
But at that point, I may well have no entries whatsoever in TaskList, since a task isn't defined until the GetStringAsync is completed, and none (or maybe just a few) may have finished by then. So
Task.WaitAll(TaskList.ToArray());
doesn't work the way I need it to.
If absolutely necessary, I could first read in all the URLs and know how many Tasks to expect, but I'm hoping for a more elegant solution.
I suppose I could increment a counter just before the await, but that feels a bit kludgy.
I assume I'm structuring things incorrectly, but I'm not sure how to reorganize things.
Note: I'm not wedded to Task.Run. Good ol' QueueWorkItem is a possibility, but I think it has pretty well the same problems.
I assume I'm structuring things incorrectly, but I'm not sure how to reorganize things.
I think that is true. Here is a possible solution: Store the whole computation as a Task in your list, not just the second part:
async Task ProcessSingleUrlInner(string url) {
var web = new HttpClient();
var WebPageContents = await web.GetStringAsync(url);
Task t = Task.Run(() => ProcessWebPage(WebPageContents);
await t;
}
void ProcessSingleUrl(string url) {
var t = ProcessSingleUrlInner(url);
TaskList.Add(t);
}
Waiting on all tasks of this list will guarantee that everything is done. Probably, you need to adapt this idea to your exact needs.
I'm assuming that you're getting the list of urls as an IEnumerable<string> or some such.
You can use LINQ to convert each url into a Task, and then await them all to complete:
async Task ProcessUrls(IEnumerable<string> urls)
{
var tasks = urls.Select(async url =>
{
var web = new HttpClient();
var WebPageContents = await web.GetStringAsync(url);
await Task.Run(() => ProcessWebPage(WebPageContents);
});
await Task.WhenAll(tasks);
}
Note that if you use this solution and there are multiple different urls that have errors, then Task.WhenAll will only report one of those errors.