Google Cloud Functions with ECONNRESET errors until I redeploy - firebase

I'm using Google Cloud Functions to:
Watch for a new Firebase entry
Download a file that's referenced in the Firebase entry
Generate a thumbnail based on that file.
Upload the thumbnail to the cloud bucket.
Unfortunately I'm getting ECONNRESET errors repeatedly on step 4, and the only way to fix it seems to be to redeploy the function. Any ideas how to further debug this?
Edit:
It seems like many times when this happens, when I try to deploy the function again, it errors, and I have to run the deploy twice. Is something hanging or something?

Update May 9 2017
According to this thread, the google cloud nodejs API developers have made some changes to the defaults that are used when initializing that should solve these ECONNRESET socket issues.
From #stephen++ on github GoogleCloudPlatform/google-cloud-node issue 2254:
We have disabled the forever agent by default in Cloud Function
environments. If you un- and re-install #google-cloud/storage, you
will pick up the new behavior automatically. Thanks for all of the
helpful debugging, everyone!
Older Post Follows:
The solution for me to similar ECONNRESET issues using storage on the cloud functions platform was to use npm:promise-retry, but set up your own retry strategy because the default of 10 retries is too many.
I reported an ECONNRESET issue with cloud functions to Google Support (which you might star if you are also getting ECONNRESET in this context but not in other contexts) and they replied with a "won't fix" that the behavior is expected. Google support said the socket that the API client library uses to connect times out after a few minutes, and then when your cloud function tries to use it again you get ECONNRESET. They recommended adding autoRetry:true when initializing the storage API, but that did not help.
The ECONNRESETs happen on the read side too. In both read and write cases promise-retry helps, and most of the time with only 1 retry needed to reset the bad socket.
So I wrote npm:pipe-to-storage to return a promise to do the retries, check md5, etc., but I haven't tested it with binary data, only text, so I don't know if you could use it with image files. The calls would look like this:
const fs = require('fs');
const storage = require('#google-cloud/storage')();
const pipeToStorage = require('pipe-to-storage')(storage);
const source = ()=>(fs.createReadStream("/path/to/your/file/to/upload"));
pipeToStorage(source, bucketName, fileNameInBucket).then(//do next step);
See also How do I read the contents of a new cloud storage file of type .json from within a cloud function?

You can directly report a bug to the Firebase Support team, or open a support ticket with Firebase to troubleshoot a specific issue.
You may also report a Cloud Functions specific issue in the Google Issue Tracker, which is similar to Stack Overflow in that it is accessible by the public (but specifically used for filing issue reports).

Related

Firestore has already been started and cannot be changed - Nuxt 2, Firebase Emulator

EDIT (ANSWER): There were two issues with my code. One was that I was passing "http://localhost" instead of just "localhost" to the connectFirestoreEmulator line. The second was that I needed to check db._settingsFrozen before running the emulator line in order to prevent it from being double-initialized. I believe this has something to do with how Nuxt runs its code on the server side vs client side: I think Nuxt shares context/state across sessions on the server side, leading to the emulator line getting run more than once.
I have a Nuxt 2 app that I've connected to Firebase production successfully for a while now. I'm trying to set it up now to use the emulator instead of production when in local environment (indicated by an env variable), but I'm getting an error with the Firebase module specifically. There doesn't seem to be any issues using the auth emulator (which I've also set up). Code (plugins/firebase.js):
const app = initializeApp(firebaseConfig);
const auth = getAuth(app);
const db = getFirestore(app);
export const services = { auth, db };
export default (context) => {
if (context.env.appEnv === 'dev') {
console.log('dev', parseInt(context.env.firestorePort))
connectFirestoreEmulator(db, 'http://localhost', parseInt(context.env.firestorePort));
connectAuthEmulator(auth, `http://localhost:${context.env.authPort}`);
}
}
I have narrowed it down to the "connectFirestoreEmulator" line which when commented out, there are no errors. When it's included, this is the error I get:
Firestore has already been started and its settings can no longer be changed. You can only modify settings before calling any other methods on a Firestore object.
Finally, when I set the plugin to run on client-side only, it no longer errors out which is odd. I'll probably just keep working with this client-only for now, but I would like to be able to run the firebase stuff on the server too ideally.
Any ideas/guidance is appreciated. Thanks in advance!
I've Googled the relevant terms and found one other question that was similar to my question ("Firebase Error : Firestore has already been started and its settings can no longer be changed." connecting Firebase v9 with Firestore Emulator). However, because I don't have enough reputation on SO, I can't comment to ask the OP if he ever found out what was happening (which I would normally do before asking my own question, but I'm not being given a choice here).
I also even looked at the nuxt-firebase npm package's source code (https://github.com/nuxt-community/firebase-module) to see how they may have set up the emulator, but their plugin/code is written so differently from mine that it was unhelpful.

(firebase functions) Error: Forbidden Your client does not have permission to get URL /

I have problem when invoking deployed function in firebase. I have an editor role in the firebase project and when I deployed functions, didn't have any problem with invoking them. When I deployed a new function yesterday, I got the error message that says
Error: Forbidden
Your client does not have permission to get URL / < Function Name > from this server.
Nothing has been changed to my role. It is strange that since yesterday, whatever function I deployed, threw those errors.
In gcp console/cloud functions, where you can see permissions of the function that was selected, I've noticed that "cloud functions invoker" was not assigned to that function. I thought this should be added to any function by default as long as I have an editor access but strangely it does not add them anymore. other functions that were deployed since yesterday have the same issue
any suggestions or advices will be appreciated. Thank you
Please Review Allowing unauthenticated function invocation
As of January 15, 2020, HTTP functions require authentication by default. You can specify whether a function allows unauthenticated invocation at or after deployment.
So here's the answer from the firebase team
The issue you are experiencing is likely caused by the fact that after January 15, 2020, Google Cloud Functions automatically creates HTTP functions to be >private by default.
Please, update the CLI, by running the following command:
npm install -g firebase-tools
This will ensure that future HTTP functions that are created will be publicly accessible.
Lastly, for the existing functions that has the permission issues, you will need >to manually set a function to public using Cloud Console or gcloud CLI.
If you have any questions or you are still facing this issue, please, don’t >hesitate to write back.
edited*
There could be several reasons to cause this issue.
check your function endpoint url make sure there's no typo or space
In the gcp console, make sure you have permission to invoke function https://console.cloud.google.com/functions/list?project=<YOUR_PROJECT_ID>
If the above two are checked, delete your function and redeploy your them again
Unfortunately, you can't do this in Firebase, you have to go into the Google Cloud project which 'hosts' your firebase project. You can follow this guide by Google, and have a look at the screenshots below:
You should see Allow unauthenticated now
To allow unauthenticated invocation of a function, you add the special allUsers member id to the function and grant it the Cloud Functions Invoker role:
You can limit domain access in your function, for example:
exports.myTest= async(req, res) => {
res.set('Access-Control-Allow-Origin', 'foo.com');
...etc
I defined my Firebase cloud functions in typescript/JS and deploy using Firebase CLI. I got this error after customizing the deployment settings, and I fixed it by specifying invoker="public" - I did not need to dig into IAM settings or use the console or CLI to fix.
export const serve = functions
.region("us-west2")
.runWith({
invoker: "public", // this is the magic line
})
.https.onRequest(
async (request: functions.Request, response: functions.Response) => {
// ...
})
updating firebase-tools wasn't enough in my case because i already deployed that function and updating it didn't fix the issue, i had to delete it and deploy again

Cloud Functions times out when calling an external HTTP with a POST request

I have a Google Cloud Function and within that I called two external APIs (or URLs), using the Python requests library, one requests.get and another requests.post. Please note that these APIs are tested and working using Postman.
The requests.get is downloading an MP3 file and it is working in Cloud Functions as I was able to see the downloaded file in Cloud Storage:
download_url = "https://some.url.com/music.mp3"
resp = requests.get(download_url)
[get the resp.content, put to storage bucket]
Aside from storing in Cloud Storage, I also send this mp3 file over to Google Cloud Speech-to-Text API and get the transcribed text, which I want to post to the another URL.
[transcribe the audio using Speech-to-Text API, get the transcibed text]
data = { "download_url": download_url, "transcription": transcribed_text }
upload_api = https://another.url.com/api"
resp = requests.post(upload_api, data = data)
Likewise, the transcription is working because I print/log the text and it shows in the View Logs console of Cloud Functions. In the requests.post however, I am getting a time out, also according to the logs.
requests.post(upload_api, data = data, timeout=120)
I even lengthen the timeout setting but that still happens. What could be the explanation for this? Is there a Google Cloud configuration that I miss to set somewhere that might be causing this?
This is because GCP cloud functions has a 1 minute timeout, to fix it is necessary to declare the timeout limit when you create your function
if you are using Gcloud CLI commands to deploy your function, you need to add the flag --timeout
for example
gcloud functions deploy FUNCTION_NAME --timeout=TIMEOUT FLAGS
if you are using the GCP console (Web UI) you need to follow the steps on this link

Why do I get "Client is not yet ready to issue requests" - Firestore get() fails when called by Schedule function

In my Firebase project I have a functions.pubsub.schedule().onRun() that runs every 5 minutes to perform some calendar related tasks. It needs to look up in my Firestore collections and does so with a .get() query.
This has been working fine until sometime 2020-03-13 in the morning where the function started to throw
2020-03-13 09:36:02.326 CET scheduledHooks 1042277797598294
Error: INTERNAL ERROR: Client is not yet ready to issue requests.
at Firestore.get projectId [as projectId] (/srv/functions/node_modules/#google-cloud/firestore/build/src/index.js:401:19)
at Query.toProto (/srv/functions/node_modules/#google-cloud/firestore/build/src/reference.js:1556:42) at Query._get (/srv/functions/node_modules/#google-cloud/firestore/build/src/reference.js:1466:30)
at Query.get (/srv/functions/node_modules/#google-cloud/firestore/build/src/reference.js:1457:21)
at FirebaseActivitiesCollection.<anonymous> (/srv/functions/lib/collections/activities/FirebaseActivitiesCollection.js:32:40)
at Generator.next (<anonymous>) at /srv/functions/lib/collections/activities/FirebaseActivitiesCollection.js:8:71
at new Promise (<anonymous>) at __awaiter (/srv/functions/lib/collections/activities/FirebaseActivitiesCollection.js:4:12)
at FirebaseActivitiesCollection.getActivitiesByInterval (/srv/functions/lib/collections/activities/FirebaseActivitiesCollection.js:27:16)
I can't track that I have changed anything, could it be that Firebase made some changes that I should be aware of or am I missing an obvious clue in this error message?
My other Cloud Functions and the Firebase JavaScript SDK still work fine.
Extra info: The same code works on my other environment where I have not deployed. This of course led me to search for changes in the deployed code, but I can't find any!?
It seems by the error, that the project was not connected and initialized correctly. As per the official API here, this error occurs when:
Returns the Project ID for this Firestore instance. Validates that
initializeIfNeeded() was called before.
I would recommend you to check that. In case you are still facing, I would say for you to contact the Firebase Free Support. Since you mentioned that it was working until yesterday morning, it might be some change on their side.
Hope this helps!
This turned out to be a very simple matter of initialization of Firebase:
Before I had:
// Initialize firebase
admin.initializeApp(functions.config().firebase)
..And it worked fine until it didn't anymore.
Now I have:
// Initialize firebase
admin.initializeApp()
... And it works again
Got the same error while running some jest tests involving firebase. Problem was that I didn't have GOOGLE_APPLICATION_CREDENTIALS environment variable set.
Since I was running them locally on my machine, exporting this variable in terminal before running the tests was sufficient for my case:
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials/file.json

AngularFire/Firestore Using maximum backoff delay to prevent overloading the backend

Today after a simple deploy for adjusts (without install new features) my angularfire application stay lot delay. When checked the console log see this message, in login screen, without any operation or request.
Somebody can give some trick?
Firestore (4.9.0) 2018-05-22T22:05:09.569Z: FirebaseError: [code=resource-exhausted]: Quota exceeded.
Firestore (4.9.0) 2018-05-22T22:04:17.606Z: Using maximum backoff delay to prevent overloading the backend.
Problem solved after unistall old unused functions
npm uninstall firabase-functions
I am also facing the same error in angularfire 2. It looks like there are two possible options to solve this issue
1) User firebase cloud functions. Move your code there and call the function from angular. Retrieve the data.
2) Write a python script and execute it using tools like PyCharm.

Resources