Let me introduce the scenario:
I need to test an AzureFunction with a queue trigger:
[FunctionName("AFunction")]
public async Task DispatchAction([QueueTrigger("queuename")] string message)
{
await DoMyLogicAsync();
}
The test needs to be run by the "functional-test-container" in my docker-compose testing env, which is made up by:
a) functional-test-container: a .net core container running an nUnit test suite
b) azure-function-container: this container hosts the azure function
c) azurite-container: this container hosts the queue server
d) sql-server-container
e) wiremock-container
The test logic is the following:
Clear the sql database, the queue and wiremock status
Prepare the wiremock stubs
Somehow trigger the function
wait for the function to end
make assertions on what the function produced in sql server, in the queue and on what wiremock's stubs have been called
As far as I know I have 2 ways of triggering the function:
a) pushing a message in the queue
b) using azure function's admin API /admin/functions/afunction
the problem is that both of them don't give any hint on when the function ends its execution.
Here it is my question: is there a way to call the function in a "sync" way (so that I can know when the execution ends)?
I don't think it can be implemented. The queue trigger function runs as an instance in azure server, we can just trigger it to run. It doesn't response any data like HttpTrigger function. So it can't be executed synchronously in your entire process.
To solve your problem, I think you can just add some code to do an operation at the end of your function. The operation is used to let you know the function execution is completed. Or another solution is move the steps after function into your function code.
Related
Really bizarre that Firebase doesn't seem to work quite like typical Express app. Whatever I write in Express and copy-paste to Firebase Functions I typically get error. There is one that I can't figure out on my own though.
This endpoint is designed to start a function and live long enough to finish even longer task. That request is a webhook (send docs, we will transform them and ping you when it's done to specified another webhook). Very simplified example below:
router.post('/', (req, res) => {
try {
generateZipWithDocuments(data) // on purpose it's not async so request can return freely
res.sendStatus(201)
} catch (error) {
res.send({ error })
}
})
On my local machine it works (both pure Express app and locally emulated Firebase Functions), but in the cloud it has problems and even though I put a cavalcade of console.log() I don't get much information. No error from Firebase.
If generateZipWithDocuments() is not asynchronous res.sendStatus() will be immediately executed after it, and the Cloud Function will be terminated (and the job done by generateZipWithDocuments() will not be completed). See the doc here for more details.
You have two possibilities:
You make it asynchronous and you wait its job is completed before sending the response. You would typically use async/await for that. Note that the maximum execution time for a Cloud Function is 9 minutes.
You delegate the long time execution job to another Cloud Function and, then, you send the response. For delegating the job to another Cloud Function, you should use Pub/Sub. See Pub/Sub triggers, the sample quickstart, and this SO thread for more details on how to implement that. In the Pub/Sub triggered Function, when the job is done you can inform the user via an email, a notification, the update of a Firestore document on which you have set a listener, etc... If generateZipWithDocuments() takes a long time, it is clearly the most user friendly option.
I have an app service in Azure running the front end from my MVC5 app, and another app service for web jobs. The app has several endpoints (GET Actions) which do some processing, send some emails or other simple task. Previously when we were hosted on a VPS, we used the Windows Task Scheduler to call each URL on a custom schedule. In Azure, the way we're currently doing this is with a Powershell script which uses CURL to fetch the URL and trigger the processing.
It seems messy though - as each powershell script has to be uploaded individually, and can't be viewed or changed after uploading. I've found various guides on deploying a .NET Core console app, but from what I can tell each job would need it's own project, deployed with it's own pipeline.
Is there a nicer way of doing this, are Webjobs even the right tool for this job, given the seemingly simple task we're performing.
As far as I understand your use case, you can use Azure Function App Timer Trigger to accomplish it.
A timer trigger lets you run a function on a schedule.
The following example shows a C# function that is executed each time the minutes have a value divisible by five (eg if the function starts at 18:57:00, the next performance will be at 19:00:00). The TimerInfo object is passed into the function.
[FunctionName("TimerTriggerCSharp")]
public static void Run([TimerTrigger("0 */5 * * * *")]TimerInfo myTimer, ILogger log)
{
if (myTimer.IsPastDue)
{
log.LogInformation("Timer is running late!");
}
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
}
The attribute's constructor takes a CRON expression or a TimeSpan. You can use TimeSpan only if the function app is running on an App Service plan. TimeSpan is not supported for Consumption or Elastic Premium Functions.
CRON (NCRONTAB expressions)
My workflow is quite simple, I have two script, first script is ASYNC and the second is SYNC. In each script I have a loop from 0 to Integer.MAX_VALUE as follow
for(int i=0;i<Integer.MAX_VALUE;i++)
System.out.println("value is "+i);
When I run my process, it starts working and I can see in my log file that it is being filled. But when I want to stop it, I find nothing in my active process instances, neither in completed process or even in aborted. even if I check my data base, I have nothing related to this process in the ProcessInstanceInfo or even ProcessInstanceLog. So weird isn't it? what could be the reason?
The goal from creating this workflow is to see the difference between ASYNC and SYNC tasks, because as I know that ASYNC tasks when they start running, the workflow don't have to wait until this task finish, but what I have is that my task ASYNC is still running and it didn't go to next task. So my second question is can any one give me the difference between ASYNC and SYNC with a good example to learn. I would appreciate if I'll get at least one answer on one of my two questions. thanks
What do you stop? Do you abort the process instance ?
In the scripts you can populate the process variables with kcontext.setVariable("variable_name","variable_value"). This will reflect in DB if you have defined the process variable persistent in the process model.
The tasks, the sync one will return the flow control to the process when is completed. In contrast to the async one, process flow will continue immediately after it sends the async tasks to execute.
Is there any way to manually trigger a scheduled function and/or a Firestore trigger function? I have two scenarios I need to solve:
A cloud function that is listening to a Firestore document (onCreate) didn't fire - it failed on 3 of about 1,000 invocations, so I need to manually trigger it for these 3 documents. Is this possible (to manually trigger this function)?
I have a scheduled function that runs hourly, but threw an error b/c of a map in the Firestore document when the code expected an array. Any way I can manually run the scheduled function once rather than waiting an hour before it runs again?
-- firebase console
-- functions
-- "..." at right side of cron job
-- "view in cloud scheduler"
-- "run now" at right side of function
You can run a firestore scheduled function via the FirebaseTools and running it locally. Starting the shell command eg npm run build && firebase functions:shell will allow you to invoke a Scheduled Function eg:
export const parseGarminHealthAPIActivityQueue = functions.region('europe-west2').runWith({
timeoutSeconds: TIMEOUT_IN_SECONDS,
memory: MEMORY
}).pubsub.schedule('every 10 minutes').onRun(async (context) => {
await parseQueueItems(ServiceNames.GarminHealthAPI);
});
It's not possible to manually trigger a function from the Firebase console. Your best bet is to use the methods shown in the Cloud documentation, which involve using gcloud's call command or the Cloud console's Testing tab. Neither of these are very easy, as you will have to construct the JSON payload to the function manually.
If I may make a suggestion - if your functions are failing due to errors, you should consider enabling retry on your functions, and making sure that your functions only generate errors for situations that should be retried. Depending on manual invocation in the event of a failure will not scale very well - errors should be handled by code as much as possible.
I need to use a Service which starts a Task more than once (= the same Service must run several parallelised Task). I read the JavaFX documentation, and they seem to say that a Service can run only one Task at once.
So if I call twice start with my Service object, the first Task returned by its createTask method would be stopped, as if I used restart after the first start.
However, that's not clear. As I told you, the documentation seems to tell that.
Indeed :
A Service creates and manages a Task that performs the work on the background thread.
Note that I could think they also say that a Service can have several Task started at the same time. Indeed :
a Service can be constructed declaratively and restarted on demand.
My question is : if I use N start in a row, will N Tasks be created AND KEEP EACH RUNNING ?
"If I use N start in a row, will N Tasks be created AND KEEP EACH RUNNING ?
In short, no.
"If I call start twice with my Service object..."
From the Javadocs:
public void start()
Starts this Service. The Service must be in the READY state to succeed in this call.
So if you call start() a second time without previously calling reset(), you will just get an exception. You can only call reset() if the Service is not in a RUNNING or SCHEDULED state. You can call restart(), which will have the effect of first canceling any current task, and then restarting the service. (This is what is meant by the documentation that says the "service can be restarted on demand".)
The net result of this is that a service cannot have two currently running tasks at the same time, since there is no sequence of calls that can get to that situation without throwing an IllegalStateException.
If you want multiple tasks running at once, simply create them yourself and submit them to an executor (or run each in its own thread, but an executor is preferred):
private final Executor exec = Executors.newCachedThreadPool(runnable -> {
Thread t = new Thread(runnable);
t.setDaemon(true);
return t ;
});
// ...
private void launchTask() {
Task<MyDataType> task = new Task<MyDataType>(){
#Override
protected Something call() {
// do work...
return new MyDataType(...);
}
};
task.setOnSucceeded(e -> { /* update UI ... */ });
task.setOnFailed(e -> { /* handle error ... */ });
exec.execute(task);
}