I'am working with Durable functions having 3 activities that executes one by one.All 3 activity executed one by one with any delayed, but since few days activities are not running at all and got stopped from executing.I see no exceptions and logs except below one,
[FunctionName("StartProcess")]
public static async Task Run([OrchestrationTrigger] DurableOrchestrationContext context)
{
//number of activities
// RandomNumberGeneration
// RandomNumberValidation
// DatabaseInsertion
}
My StartProcess OrchestrationTrigger is executing always but not able to invoke RandomNumberGeneration() Activity trigger.So I got below log when StartProcess() OrchestrationTrigger is executing and want to know more about it
0ee5eaa5ddb240d28df0f1077a563cc6: Function 'StartProcess (Orchestrator)', version '' started. IsReplay: False. Input: (42 bytes). State: Started. HubName: DurableFunctionsHub. AppName: schedulars. SlotName: Production. ExtensionVersion: 1.0.0.0.
Function started (Id=b8abdfcc-1794-4bea-b5b7-5a35bffacb4b)
0ee5eaa5ddb240d28df0f1077a563cc6: Function 'DatabaseInsertion (Activity)', version '' scheduled. Reason: StartProcess. IsReplay: False. State: Scheduled. HubName: DurableFunctionsHub. AppName: schedulars. SlotName: Production. ExtensionVersion: 1.0.0.0.
Function completed (Success, Id=b8abdfcc-1794-4bea-b5b7-5a35bffacb4b, Duration=18ms)
Below is log when StartProcess execute always and also invokes other activities.
Function started (Id=4e61d349-b6e5-4c04-9d85-8d490f2a4a4e)
9573f926bf884bb0a519e5cd1434621c: Function 'StartProcess (Orchestrator)', version '' started. IsReplay: True. Input: (42 bytes). State: Started. HubName: DurableFunctionsHub. AppName: schedulars. SlotName: Production. ExtensionVersion: 1.0.0.0.
9573f926bf884bb0a519e5cd1434621c: Function 'StartProcess (Orchestrator)', version '' completed. ContinuedAsNew: False. IsReplay: True. Output: (null). State: Completed. HubName: DurableFunctionsHub. AppName: schedulars. SlotName: Production. ExtensionVersion: 1.0.0.0.
Function completed (Success, Id=4e61d349-b6e5-4c04-9d85-8d490f2a4a4e, Duration=3270ms)
Gone through below Diagnostics in Durable Functions : https://learn.microsoft.com/en-us/azure/azure-functions/durable-functions-diagnostics
But not able to get it clearly
Does StartProcess() is unable to start another activity because DataBaseInsertion() ActivityTrigger is already in scheduled state and waiting for completion of it?
please help in understanding What is issue and Why it is not running?
Related
Step 1: Automatically create a new Next.js project using the new beta app directory:
npx create-next-app#latest --experimental-app
pages/api/hello.ts
// Next.js API route support: https://nextjs.org/docs/api-routes/introduction
import type { NextApiRequest, NextApiResponse } from 'next'
type Data = {
name: string
}
export default function handler(
req: NextApiRequest,
res: NextApiResponse<Data>
) {
res.status(200).json({ name: 'John Doe' })
}
This file is identical to the one created automatically created by npx - there are no modifications.
I am trying to build a simple home page, which fetches data from the api which gets data from my database. Either way an await/async will be required. I am following the instructions from here.
In this example I will demonstrate that even awaiting the supplied hello api can't seem to run in production, and I can't work out why.
app/page.tsx
async function getHelloAsync() {
const res = await fetch('http://localhost:3000/api/hello', { cache: 'no-store' });
// The return value is *not* serialized
// You can return Date, Map, Set, etc.
// Recommendation: handle errors
if (!res.ok) {
// This will activate the closest `error.js` Error Boundary
throw new Error('Failed to fetch data');
}
return res.json();
}
export default async function Page() {
const hello = await getHelloAsync();
return (
<main>
<h1>Hello: {hello.name}</h1>
</main>
)
}
To test the hello api works, I confirm that running pn run dev and then curl http://localhost:3000/api/hello the following successful response is received:
{"name":"John Doe"}
Next up we exit the dev server and run:
pn run build
The first headache is that the build will completely fail to build unless one adds { cache: 'no-store' } to the fetch command:
const res = await fetch('http://localhost:3000/api/hello', { cache: 'no-store' });
or adds this to the top of app/page.tsx:
export const fetchCache = 'force-no-store';
I am actually not sure how one would even build this if you wanted to cache the response or use revalidate instead and provide an initial optimistic response, because without cache: no-store it refuses to build outright. Ideally instead it should just cache the result from /api/hello and not fail. Running the dev server at the same idea as doing the build does allow the build to work, but then as soon as you exit the dev server and run pn run start then all the api calls fail anyway. So that is not a good idea.
This leads us to the next problem - why are the api calls not working in production (i.e. when calling pn run start).
Step 2:
pn run build
pn run start
Confirm that the following still works and yes it does:
curl http://localhost:3000/api/hello
Result:
{"name":"John Doe"}
Now we visit http://localhost:3000 in a browser but, surprise! We get the following error:
> next start
ready - started server on 0.0.0.0:3000, url: http://localhost:3000
warn - You have enabled experimental feature (appDir) in next.config.js.
warn - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use at your own risk.
info - Thank you for testing `appDir` please leave your feedback at https://nextjs.link/app-feedback
(node:787) ExperimentalWarning: The Fetch API is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
TypeError: fetch failed
at Object.fetch (node:internal/deps/undici/undici:11118:11)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async getHelloAsync (/Users/username/nextjstest/.next/server/app/page.js:229:17)
at async Page (/Users/username/nextjstest/.next/server/app/page.js:242:19) {
cause: Error: connect ECONNREFUSED ::1:3000
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1300:16)
at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
errno: -61,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '::1',
port: 3000
}
}
[Error: An error occurred in the Server Components render. The specific message is omitted in production builds to avoid leaking sensitive details. A digest property is included on this error instance which may provide additional details about the nature of the error.] {
digest: '3567993178'
}
Why is it saying that the connection is refused when we know the API is available? I can't get this to run at all. I know this is beta but surely the code should actually run right? How do I make this code work?
Also if anyone knows where where the logs are that I'm supposed to be accessing to see digest '3567993178' please let me know.
I'm implementing the ClassyTaxiAppKotlin + ClassyTaxiServer project but without success.
I followed all the steps in the tutorial.
Apparently the ClassyTaxiAppKotlin android app is functional, processes the subscription purchase order and sends the information to the CLOUD FUNCTIONS, but does not receive the purchase registration/confirmation.
In my understanding, upon receiving the payment confirmation, PLAY STORE must send this confirmation to the ClassyTaxiServer server so that the server returns the registration/confirmation of the purchase and grants access to the ClassyTaxiAppKotlin application.
So when confirming payment in ClassyTaxiAppKotlin app, I notice that I get some logs in firebase functions from (instanceId_register_v2, realtime_notification_listener AND subscription_register_v2), but the registration/confirmation of payment doesn't complete due to a server error (500) and doesn't release the access in the ClassyTaxiAppKotlin app.
Note: service-account.json is already configured
Does anyone have any idea why this error is occurring and indicate how I can solve it?
Code where the error points:
private async querySubscriptionPurchaseWithTriggerV2(packageName: string, product: string, purchaseToken: string, triggerNotificationType?: NotificationType): Promise<SubscriptionPurchaseV2> {
// STEP 1. Query Play Developer API to verify the purchase token
const apiResponseV2 = await new Promise((resolve, reject) => {
this.playDeveloperApiClient.purchases.subscriptionsv2.get({ // <<=== Server error: Cannot read property 'get' of undefined
packageName: packageName,
token: purchaseToken
}, (err, result) => {
if (err) {
reject(this.convertPlayAPIErrorToLibraryError(err));
} else {
resolve(result.data);
}
})
});
LOGS Firebase Functions
6:55:28.798 PM instanceId_register_v2 Function execution started
6:55:31.025 PM instanceId_register_v2 Instance id is ddR1Hi...NOO2Z
6:55:31.122 PM instanceId_register_v2 Instance Id specified and verified
6:55:31.122 PM instanceId_register_v2 Instance verification passed
6:55:31.774 PM instanceId_register_v2 Function execution took 2976 ms, finished with status code: 200
6:55:53.623 PM realtime_notification_listener ========> purchase: null PACKAGE_NAME: com.example.subscriptions
6:55:53.624 PM realtime_notification_listener Function execution took 5 ms, finished with status: 'ok'
6:55:57.537 PM subscription_register_v2 Function execution started
6:55:59.817 PM subscription_register_v2 Server error: Cannot read property 'get' of undefined
6:55:59.825 PM subscription_register_v2 Function execution took 2289 ms, finished with status code: 500
LOGS Android Studio
D/OkHttp: --> PUT https://us-central1-postosgnv.cloudfunctions.net/subscription_register_v2 http/1.1 (437-byte body)
D/OkHttp: <-- 500 https://us-central1-postosgnv.cloudfunctions.net/subscription_register_v2 (2661ms, 86-byte body)
E/RemoteServerFunction: Failed to call API (Error code: 500) - {"status":500,"error":"not-found","message":"Cannot read property 'get' of undefined"}
As per the answer found HERE, the issue was solved by updating the GoogleApis package in the package.json from "googleapis": "^67.0.0" -> "googleapis": "105.0.0"
I have a Durable Function that always seems to be in a Running state. Even when it completed successfully.
It seems to complete, then it updates the RunningStatus back to Running for some reason.
This is the end of my logs:
[2021-07-22T08:52:59.211Z] Executing 'SavingsOrchestrator' (Reason='(null)', Id=36d509f3-4655-4382-9b72-ccb6fc39f413)
[2021-07-22T08:52:59.230Z] Executed 'SavingsOrchestrator' (Succeeded, Id=36d509f3-4655-4382-9b72-ccb6fc39f413, Duration=51ms)
[2021-07-22T08:53:02.206Z] SaveCustomerSavings stored 34 records.
[2021-07-22T08:53:02.210Z] Orechestrator_SaveCustomerSavings: Function 'SavingsOrchestrator (Orchestrator)' awaited. IsReplay: False. State: Awaited. HubName: TestHubName. AppName: . SlotName: . ExtensionVersion: 2.5.0. SequenceNumber: 24.
[2021-07-22T08:53:02.214Z] Orechestrator_SaveCustomerSavings: Function 'SavingsOrchestrator (Orchestrator)' completed. ContinuedAsNew: False. IsReplay: False. Output: (null). State: Completed. HubName: TestHubName. AppName: . SlotName: . ExtensionVersion: 2.5.0. SequenceNumber: 25. TaskEventId: -1
[2021-07-22T08:53:02.214Z] Orechestrator_SaveCustomerSavings: Orchestration 'SavingsOrchestrator' awaited and scheduled 0 durable operation(s).
[2021-07-22T08:53:02.254Z] Orechestrator_SaveCustomerSavings: Appended 3 new events to the history table in 35ms
[2021-07-22T08:53:02.290Z] Orechestrator_SaveCustomerSavings: Updated Instances table and set the runtime status to 'Running'
[2021-07-22T08:53:02.292Z] Orechestrator_SaveCustomerSavings: Deleting [TaskCompleted#3] message from testhubname-control-00
What I basically do in my code is the following:
I start my Orchestrator (as a singleton):
await client.StartNewAsync<Task>("SavingsOrchestrator", _instanceId, null);
My Orchestrator function basically looks like this:
[FunctionName(nameof(SavingsOrchestrator))]
public async Task SavingsOrchestrator([OrchestrationTrigger] IDurableOrchestrationContext context, ILogger logger)
{
var a1 = await context.CallActivityAsync<List<MyModel>>("Activity1", null);
if (a1?.Any() != true)
{
return;
}
// Run as 4 parallel tasks.
var batchSize = a1.Count / 4 + 1;
var tasks = new List<Task<int>>();
foreach (var batch in a1.Batch(batchSize))
{
tasks.Add(context.CallActivityAsync<int>("Activity2", batch));
}
await Task.WhenAll(tasks);
var total = tasks.Sum(x => x.Result);
logger.LogInformation($"Done: {total}");
}
Why is my Durable Function still in a Running state, even after it successfully ran?
Update
In my code I do a "Fan out" by using a Task.WhenAll. When I remove that code and simply call Activity2 in a single line, passing all the items to it instead of batches:
// Pass all items to the Activity function
var total = await context.CallActivityAsync<int>("Activity2", a1);
Then it updates the RunningStatus to Completed.
There seems to be an issue using Task.WhenAll. But I got this idea directly from the Microsoft Documentation
This could be because there may be larger amount of data or return values get serialized into queue messages and Azure storage queues only support 64 KB messages.
For further information refer Durable Function
This trigger is used for detecting sequence in schedule has been updated, and help to update the schedule's overview status and finished time.
But it didn't always work when an internal error was occurred as below:
Error: 13 INTERNAL: An internal error occurred. at Object.exports.createStatusError
(/srv/node_modules/grpc/src/common.js:91:15) at Object.onReceiveStatus
(/srv/node_modules/grpc/src/client_interceptors.js:1204:28) at InterceptingListener._callNext
(/srv/node_modules/grpc/src/client_interceptors.js:568:42) at InterceptingListener.onReceiveStatus
(/srv/node_modules/grpc/src/client_interceptors.js:618:8) at callback
(/srv/node_modules/grpc/src/client_interceptors.js:845:24)
Here is my code:
export const calc_status = function.firestore.document("users/{userid}/schedule/{scheduledid}").onUpdate(async (change, context) => {
// before error occurred ...
const data = change.after.data();
let curStatus = data.status;
...
...
// after getting occurred ...
if(data.status !== curStatus ) {
data.status = curStatus;
if(curStatus === 'finished') {
data.finish_time = new Date().toISOString();
}
if(curStatus !== 'expired'){
data.update_time = data.expired_time;
data.finish_time = data.expired_time;
} else {
data.update_time = new Date().toISOString();
}
await change.after.ref.update(data);
return Status.SUCCEEDED;
}
return Status.SUCCEEDED;
}
I'm very confused why the error occurred because this function works fine at most time.
Does anyone met the same problem as mine?
Why the error happened? And what's your solution?
Thank you.
This appears to be long standing framework bug github.com/firebase/firebase-functions/issues/536 with no resolution as of yet.
Though you can't get around the error which anecdotally and very intermittently happens on a cold start you can work around it by enabling retries for the function via the full console see Retry Cloud Functions for Firebase until it succeeds for instructions.
This assumes you handle internal errors in your code very well as it will retry for any failure but in my case the functions onCreate handler was just queuing up some later processing via PubSub so any failure meant it should retry.
Oct 2020 Update
Since v3.11 of firebase-functions you can now set the retry mode in your function code by setting failurePolicy to true
module.exports = functions.runWith({ failurePolicy: true }).firestore.document('collection/doc').onWrite(async (change, context) => {
//do function stuff
});
Here are two simple examples. No unnecessary. They do not work. Error is output to the console.
No asynchronous tests are working. These - not.
Tests write for meteor application.
About jasmine.DEFAULT_TIMEOUT_INTERVAL I already know, is the 5000.
describe("Jasmine async tests", function(){
it("First test with timeout", function(done){
Meteor.setTimeout(function(){
done();
}, 300);
});
it("Second test, request to google.com", function(done){
HTTP.get("http://google.com/", {}, function(){
done();
});
});
});
Output to the console:
I20141021-21:08:35.178(3)? Async Login Test response to google.com
I20141021-21:08:35.179(3)? Error: Error: Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.
I20141021-21:08:35.180(3)? at Timer.listOnTimeout [as ontimeout] (timers.js:110:15)
In unit test mode most of the Meteor API is stubbed out. That means that calling Meteor.setTimeout or HTTP.get will just execute an empty function and therefore will do nothing. Your callback is never called.
You can find the used stubs here: https://github.com/meteor-velocity/meteor-stubs/blob/master/index.js
Also relevant: https://github.com/Sanjo/meteor-jasmine/issues/55#issuecomment-61026304