I've got a Google Cloud app with several cloud functions that call an API, then save the response in Firebase. I have this scheduled function set up to retry the API on error and it works great. But I want to retry the call if the data I need isn't there yet. Calling again right away could work if I throw an error, but it's highly unlikely that the missing data will be available seconds later, so I'd rather check once an hour after that until I have valid data.
Below is a shortened version of my function. I can imagine adding a setTimeout and having the function call itself again, but I'm not sure I would do that or if it's a good idea since it would keep the function alive for a long time. Is there a way to automatically retry this scheduled function on an arbitrary time interval?
exports.fetchData= functions.pubsub
.schedule('every Tuesday 6:00')
.timeZone('America/New_York')
.onRun(async context => {
const response = fetch(...)
.then(res => {
if (res.status < 400) {
return res;
} else {
throw new Error(`Network response was not ok. ${res}`);
}
})
.then(res => res.json());
const resObj = await response;
resObj.map(x => {
// check response for valid data
})
if (// data is valid) {
// save to Firebase
} else {
// retry in 1 hour
}
});
});
Scheduled functions only run on the schedule you specify. There is no "arbitrary" scheduling. If you think that the function might frequently fail, consider just increasing the frequency of the schedule, and bail out of function invocations that don't need to run because of recent success.
If you enable retries, and the function generates an error by throwing an exception, returning a rejected promise, or timing out, then Cloud Functions will automatically retry the function on a schedule that you can't control.
setTimeout is not a feasible option to keep a function alive for longer than its configured timeout. Cloud Functions will terminate the function and all of its ongoing work after the timeout expires (and you would be paying for the time the function sits idle, which is kind of a waste).
Related
I'm testing a react app with cypress, and cypress is really good at checking if atleast n many are calls are made on load, and not so good at checking if only those calls were made. The app itself is a pretty standard react app that uses window.fetch for api calls. Is there another service, like cypress something similar, that's got the ability to test that in an automated fashion?
For instance, if you visit the site logged in we have three config calls. If a developer commits code that accidentally calls one of those twice, ideally I'd like an automated test to catch that.
I don't want to do it in Cypress because it seems like that's not what the product is really intended to do unless you hack it with an undocumented solution.
Seems like you need a hard wait to ensure you check after all calls have been issued.
Also recommend stubbing so that there's no latency waiting for server response.
it('check for excess calls', () => {
cy.intercept('https://jsonplaceholder.typicode.com/todos/1', {}).as('fetch')
cy.visit('/').then(() => {
// wait for expected calls
cy.wait('#fetch')
cy.wait('#fetch')
cy.wait('#fetch')
cy.wait(500)
cy.get('#fetch.all') // check the call count
.should(calls => {
expect(calls).to.have.length(3) // no extra calls
})
})
})
Tested with this app HTML
<script>
fetch('https://jsonplaceholder.typicode.com/todos/1')
fetch('https://jsonplaceholder.typicode.com/todos/1')
fetch('https://jsonplaceholder.typicode.com/todos/1')
</script>
You may want to burn-test to get the right wait time, see Burning Tests with cypress-grep
Alternatively you can wait four times and catch the error
it('check for excess calls', () => {
cy.intercept('https://jsonplaceholder.typicode.com/todos/1', {}).as('fetch')
cy.visit('html/call-limit.html').then(() => {
cy.wait('#fetch')
cy.wait('#fetch')
cy.wait('#fetch')
Cypress.once('fail', (error) => {
// catch ignore failure on 4th cy.wait('#fetch')
if (error.message.includes('No request ever occurred')) {
return
}
throw error // rethrow to fail for another reason
})
cy.wait('#fetch') // should fail and trigger listener above
cy.get('#fetch.all')
.should(calls => {
expect(calls).to.have.length(3)
})
})
})
In the case of network connectivity loss, the following code just loops endlessly and keeps making API calls. Is there a way to cancel with a timeout (for example, 5000 ms) using Firebase API? Or would I have to make my own Coroutine to handle this?
fun updateUserFieldInDB(
collectionPath: String,
strArr: ArrayList<String>,
onSuccess: (() -> Unit),
onFail: (() -> Unit)
) {
val fbUser = Firebase.auth.currentUser
if (fbUser == null) {
Log.i(TAG, "user is null....")
return
}
val db = Firebase.firestore
when (strArr.size) {
2 -> {
db.collection(collectionPath).document(fbUser.uid).update(strArr[0], strArr[1])
.addOnSuccessListener {
onSuccess()
}
.addOnFailureListener {
onFail()
}
}
}
}
The onSuccess ad onFail completion handlers for Firestore only fire once the write operation has been committed or rejected on the server. You should only use them if you're interested in detecting that situation, in which case the looping is to be expected.
If you only care whether the write operation was recorded by the Firestore client (in its local cache), the best way to detect that is when the update(strArr[0], strArr[1]) call completes.
So pretty much: when the next line of code executes, the write has been recorded locally; when the completion listeners fire, the write has been handled on the server.
I'm using Firebase Functions with https triggers, and I was wondering how long after sending the response to the client, the functions keeps executing. I want to send a response to the client and then perform another operation (send a mail).
Currently I'm doing this as following:
module.exports.doSomeJob = functions.https.onRequest((req, res) => {
doSomeAsyncJob()
.then(() => {
res.send("Ok");
})
.then(() => {
emailSender.sendEmail();
})
.catch(...);
});
The above code is working for me, but I'm suspecting that the code only works because sending the mail has finished before the res.send has completed, so I was wondering how exactly the termination process is working to make sure the code will not break.
You should expect that the HTTP function terminates the moment after you send the response. Any other behavior is some combination of luck or a race condition. Don't write code that depends on luck.
If you need to send a response to the client before the work is fully complete, you will need to kick off a second function to continue where the HTTP function left off. It's common to use a pub/sub function to do with. Have the HTTP function send a pub/sub message to another function, then terminate the HTTP function (by sending a response) only after the message is sent.
If the expected response is not pegged to the outcome of the execution, then you can use
module.exports.doSomeJob = functions.https.onRequest((req, res) => {
res.write('SUCCESS')
return doSomeAsyncJob()
.then(() => {
emailSender.sendEmail();
})
.then(() => {
res.end();
})
.catch(...);
});
This sends back a response a soon as a request is received, but keeps the function running until res.end() is called
Your client can end the connection as soon as a response is received back, but the cloud function will keep running in the background
Not sure if this helps, but it might be a workaround where the client needs a response within a very limited time, considering that executing pub/sub requires some extra processing on its own and takes time to execute
TL;DR
While https functions will terminate shortly after res.send(), it is not guaranteed that 0 lines of code after res.send() will be executed.
I think a fuller answer has 2 components:
as Doug pointed out, do not put any additional code you expect to be executed after res.send()
cloud functions will terminate shortly after res.send(), but don't expect that exactly 0 lines of code will be executed
I ran into a situation where for a db maintenance script, if no records met my criteria, I said goodbye with res.send() and had additional logic after it. I was expecting that piece not to be run, since I've already terminated the request.
Example producing unexpected results:
exports.someFunction = functions.https.onRequest((req, res) => {
if (exitCriteria === true) {
// we can exit the function, nothing to do
console.log('Exit criteria met')
res.status(200).send()
}
// code to handle if someCriteria was falsy
console.log('Exit criteria not met, continue executing code')
})
In the above example, I was expecting res.send() to terminate the function immediately - this is not so, the second console.log may also be hit - along with any other code you may have. This is not guaranteed, however, so execution may abruptly stop at some point.
Example producing correct results:
exports.someFunction = functions.https.onRequest((req, res) => {
if (exitCriteria === true) {
// we can exit the function, nothing to do
console.log('Exit criteria met')
res.status(200).send()
}
else {
// code to handle if someCriteria was falsy
console.log('Exit criteria not met, continue executing code')
}
})
In this version, you will see exactly 1 line of console.logs - as I was originally intending.
I'm using Firebase Functions with https triggers, and I was wondering how long after sending the response to the client, the functions keeps executing. I want to send a response to the client and then perform another operation (send a mail).
Currently I'm doing this as following:
module.exports.doSomeJob = functions.https.onRequest((req, res) => {
doSomeAsyncJob()
.then(() => {
res.send("Ok");
})
.then(() => {
emailSender.sendEmail();
})
.catch(...);
});
The above code is working for me, but I'm suspecting that the code only works because sending the mail has finished before the res.send has completed, so I was wondering how exactly the termination process is working to make sure the code will not break.
You should expect that the HTTP function terminates the moment after you send the response. Any other behavior is some combination of luck or a race condition. Don't write code that depends on luck.
If you need to send a response to the client before the work is fully complete, you will need to kick off a second function to continue where the HTTP function left off. It's common to use a pub/sub function to do with. Have the HTTP function send a pub/sub message to another function, then terminate the HTTP function (by sending a response) only after the message is sent.
If the expected response is not pegged to the outcome of the execution, then you can use
module.exports.doSomeJob = functions.https.onRequest((req, res) => {
res.write('SUCCESS')
return doSomeAsyncJob()
.then(() => {
emailSender.sendEmail();
})
.then(() => {
res.end();
})
.catch(...);
});
This sends back a response a soon as a request is received, but keeps the function running until res.end() is called
Your client can end the connection as soon as a response is received back, but the cloud function will keep running in the background
Not sure if this helps, but it might be a workaround where the client needs a response within a very limited time, considering that executing pub/sub requires some extra processing on its own and takes time to execute
TL;DR
While https functions will terminate shortly after res.send(), it is not guaranteed that 0 lines of code after res.send() will be executed.
I think a fuller answer has 2 components:
as Doug pointed out, do not put any additional code you expect to be executed after res.send()
cloud functions will terminate shortly after res.send(), but don't expect that exactly 0 lines of code will be executed
I ran into a situation where for a db maintenance script, if no records met my criteria, I said goodbye with res.send() and had additional logic after it. I was expecting that piece not to be run, since I've already terminated the request.
Example producing unexpected results:
exports.someFunction = functions.https.onRequest((req, res) => {
if (exitCriteria === true) {
// we can exit the function, nothing to do
console.log('Exit criteria met')
res.status(200).send()
}
// code to handle if someCriteria was falsy
console.log('Exit criteria not met, continue executing code')
})
In the above example, I was expecting res.send() to terminate the function immediately - this is not so, the second console.log may also be hit - along with any other code you may have. This is not guaranteed, however, so execution may abruptly stop at some point.
Example producing correct results:
exports.someFunction = functions.https.onRequest((req, res) => {
if (exitCriteria === true) {
// we can exit the function, nothing to do
console.log('Exit criteria met')
res.status(200).send()
}
else {
// code to handle if someCriteria was falsy
console.log('Exit criteria not met, continue executing code')
}
})
In this version, you will see exactly 1 line of console.logs - as I was originally intending.
I'm using Firebase Functions with https triggers, and I was wondering how long after sending the response to the client, the functions keeps executing. I want to send a response to the client and then perform another operation (send a mail).
Currently I'm doing this as following:
module.exports.doSomeJob = functions.https.onRequest((req, res) => {
doSomeAsyncJob()
.then(() => {
res.send("Ok");
})
.then(() => {
emailSender.sendEmail();
})
.catch(...);
});
The above code is working for me, but I'm suspecting that the code only works because sending the mail has finished before the res.send has completed, so I was wondering how exactly the termination process is working to make sure the code will not break.
You should expect that the HTTP function terminates the moment after you send the response. Any other behavior is some combination of luck or a race condition. Don't write code that depends on luck.
If you need to send a response to the client before the work is fully complete, you will need to kick off a second function to continue where the HTTP function left off. It's common to use a pub/sub function to do with. Have the HTTP function send a pub/sub message to another function, then terminate the HTTP function (by sending a response) only after the message is sent.
If the expected response is not pegged to the outcome of the execution, then you can use
module.exports.doSomeJob = functions.https.onRequest((req, res) => {
res.write('SUCCESS')
return doSomeAsyncJob()
.then(() => {
emailSender.sendEmail();
})
.then(() => {
res.end();
})
.catch(...);
});
This sends back a response a soon as a request is received, but keeps the function running until res.end() is called
Your client can end the connection as soon as a response is received back, but the cloud function will keep running in the background
Not sure if this helps, but it might be a workaround where the client needs a response within a very limited time, considering that executing pub/sub requires some extra processing on its own and takes time to execute
TL;DR
While https functions will terminate shortly after res.send(), it is not guaranteed that 0 lines of code after res.send() will be executed.
I think a fuller answer has 2 components:
as Doug pointed out, do not put any additional code you expect to be executed after res.send()
cloud functions will terminate shortly after res.send(), but don't expect that exactly 0 lines of code will be executed
I ran into a situation where for a db maintenance script, if no records met my criteria, I said goodbye with res.send() and had additional logic after it. I was expecting that piece not to be run, since I've already terminated the request.
Example producing unexpected results:
exports.someFunction = functions.https.onRequest((req, res) => {
if (exitCriteria === true) {
// we can exit the function, nothing to do
console.log('Exit criteria met')
res.status(200).send()
}
// code to handle if someCriteria was falsy
console.log('Exit criteria not met, continue executing code')
})
In the above example, I was expecting res.send() to terminate the function immediately - this is not so, the second console.log may also be hit - along with any other code you may have. This is not guaranteed, however, so execution may abruptly stop at some point.
Example producing correct results:
exports.someFunction = functions.https.onRequest((req, res) => {
if (exitCriteria === true) {
// we can exit the function, nothing to do
console.log('Exit criteria met')
res.status(200).send()
}
else {
// code to handle if someCriteria was falsy
console.log('Exit criteria not met, continue executing code')
}
})
In this version, you will see exactly 1 line of console.logs - as I was originally intending.