I have a (very slightly) modified version of the generateThumbnail Firebase Cloud Function found in the Firebase Github repo. The function was working correctly at one point, and now it will time out every time it is called. I haven't made any changes to the function or to the rules for my storage bucket. The rules are the default ones that check for authentication. After adding some logging I can see that it never makes it past this line:
await file.download({destination: tempLocalFile});
The image file I am testing with is a 15.21KB PNG. The timeout of the function happens after ~60000 ms (default). There is no error in the logs, only the timeout.
Any suggestions as to why it started timing out all of the sudden? Or how to debug this single call further?
Node: 14
Firebase Admin: 9.8.0
Firebase Functions: 3.14.1
EDITS
I have deployed a minimum reproducable function and am seeing the same results.
exports.newGenerateThumbnail = functions.storage.object().onFinalize(async (object) => {
const filePath = object.name;
const tempLocalFile = path.join(os.tmpdir(), filePath);
const tempLocalDir = path.dirname(tempLocalFile);
// Cloud Storage files.
const bucket = admin.storage().bucket(object.bucket);
const file = bucket.file(filePath);
functions.logger.log('Creating Temp Directory');
await mkdirp(tempLocalDir);
functions.logger.log('Temp Directory Created');
functions.logger.log('Downloading File');
await file.download({ destination: tempLocalFile });
functions.logger.log('File Downloaded');
functions.logger.log('Removing File');
fs.unlinkSync(tempLocalFile);
functions.logger.log('File Deleted');
return true;
});
The logs show this
I tried to reproduce the issue using your code, but it is working fine for me without any timeout as shown in the screenshot.
I cross checked my versions with yours, I am using Node: 16.6, Firebase: 10.2.1, Firebase Functions: 3.16
It may be because of the version as it is seen in the past as well, as shown in the stackoverflow thread. So I suggest you upgrade your version and try again, it might help you resolve the issue.
Related
I am new to Firebase Functions and backend dev in general. I have written a function (called convertFile) that uses the node package "node-7z" to unzip a file called "example.7z" (located in the root of my functions directory). This is my functions code:
const functions = require("firebase-functions");
const admin = require("firebase-admin");
const sevenZipBin = require("7zip-bin");
const { extractFull } = require("node-7z");
admin.initializeApp();
const pathTo7za = sevenZipBin.path7za;
exports.convertFile = functions.https.onCall((data, context) => {
const seven = extractFull('./example.7z', './example', {
$bin: pathTo7za
});
seven.on('error', function (err) {
console.log(err.message);
});
seven.on('end', function () {
console.log('END');
});
});
When I use Firebase's local emulator to run the function, the function runs successfully. A new directory called "example" containing the contents of "example.7z" is created within my "functions" directory (this is the desired behavior). When I deploy the function to firebase, and run the function in production, I get the following error in the function log (image of full log shown at the bottom):
spawn /workspace/node_modules/7zip-bin/linux/x64/7za EACCES
Note that "/workspace/node_modules/7zip-bin/linux/x64/7za" is the filepath stored in the variable "sevenZipBin.path7za".
I can understand how there might be a permissions issue when the function is run on firebase as opposed to my local system, but I don't know how to solve the issue. I tried using a child process (within the function) to change the file permissions using the "chmod" command, but I get this error in the functions log:
chmod: changing permissions of '/workspace/node_modules/7zip-bin/linux/x64/7za': Read-only file system
I assume this is not a valid solution. I tried searching Google, stackoverflow, Github, etc. for solutions but I still don't know how to proceed from here. If anyone has suggestions, I would greatly appreciate it.
Additional info:
node-7z docs - https://github.com/quentinrossetti/node-7z
7zip-bin docs - https://github.com/develar/7zip-bin
Screenshot of functions log
I'm trying to deploy one of my functions from firebase CLI (version 8.12.1) and it keeps failing.
The function hasn't changed in weeks, so I am a bit confused as to why it's failing now.
Error from the CLI
functions[http-api-(europe-west1)]: Deployment error.
Build failed: Build error details not available. Please check the logs at https://console.cloud.google.com/logs/viewer?project=&advancedFilter=resource.type%3Dbuild%0Aresource.labels.build_id%3Dfeb2697d-29b4-4ab7-9b84-90d9f847be42%0AlogName%3Dprojects%2Fvestico-dev%2Flogs%2Fcloudbuild
Logs from the cloud console
Step #3 - "restorer": Restoring data for "google.nodejs.functions-framework:functions-framework" from cache
Step #3 - "restorer": \u001b[31;1mERROR: \u001b[0mfailed to restore: restoring data: GET https://storage.googleapis.com/eu.artifacts..appspot.com/containers/images/sha256:484d08dfc6a8f356c34a86fa4440fedf86f4fc398967eea66e4aab4e9ee81e3d?access_token=REDACTED: unsupported status code 404; body: NoSuchKeyThe specified key does not exist.No such object: eu.artifacts..appspot.com/containers/images/sha256:484d08dfc6a8f356c34a86fa4440fedf86f4fc398967eea66e4aab4e9ee81e3d
Finished Step #3 - "restorer"
ERROR: build step 3 "eu.gcr.io/fn-img/buildpacks/nodejs10/builder:nodejs10_20201005_20_RC00" failed: step exited with non-zero status: 46
The interesting piece is probably the error from above:
<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Details>No such object: eu.artifacts.<project-id>.appspot.com/containers/images/sha256:484d08dfc6a8f356c34a86fa4440fedf86f4fc398967eea66e4aab4e9ee81e3d</Details>
</Error>
What key is builder referring to? <project-id>#appspot.gserviceaccount.com has accesss to the cloud function with roles Cloud Functions Admin and Editor.
EDIT
Deploying through firebase deploy --only functions:<my-api>
That function uses #google-cloud/storage to get a public url for a storage resource.
I'm loading the service account configs like this:
const devServiceAccount = require("../../service-accounts/dev.json");
const prodServiceAccount = require("../../service-accounts/prod.json");
export const getAdminConfig = (): (AppOptions | undefined) => {
const baseConfigEnv = process.env.FIREBASE_CONFIG;
if (!baseConfigEnv) {
console.error("no firebase config environment");
return undefined;
}
const app = functions.config().app;
if (app === undefined) {
console.error("no firebase app config");
return undefined;
}
const serviceAccount = app.environment === 'dev' ? devServiceAccount : prodServiceAccount;
const adminConfig = JSON.parse(baseConfigEnv) as AppOptions;
adminConfig.credential = credential.cert(serviceAccount);
return adminConfig;
}
The cloud storage is used here.
const options = {
action: 'read',
expires: Date.now() + 1000 * 60 * 60 //1 hour
} as GetSignedUrlConfig;
const file = bucket.file(path);
filePathPromises.push(file.getSignedUrl(options))
});
My folder structure is as follows.
+ functions
+ lib
+ function.js
+ service-accounts
+ dev.json
+ prod.json
+ src
+ function.ts
I was ruling out that the service account files are the issue given that the files are loaded in getAdminConfig() for all functions in the project.
Update 10/13/20
I've verified the files uploaded to the GCF storage container. The JSON keys are there and in the right location. The paths match, so they should be found when the GCF is running.
Adding a hint for the next soul running into this problem. It seems to be caused by missing/inaccessible file in the restore/rollback process.
I was successfully removing the problem by simply:
Deleting my functions using the web firebase console.
Deploying normally again >firebase deploy
It seems there was an intermittent issue in Firebase Cloud Functions or GCF. I just ran firebase deploy --only functions again and it deployed successfully.
I've been attempting to get to the bottom of issues with a Firebase function I'm using to update some aggregate data in Firestore. I set up a simple test bed and found that any attempt to access the data triggers the error:
> Error: Could not load the default credentials. Browse to https://cloud.google.com/docs/authentication/getting-started for more information.
> at GoogleAuth.getApplicationDefaultAsync (/Users/michael/Documents/htdocs/vue/mjf20/functions/node_modules/google-auth-library/build/src/auth/googleauth.js:160:19)
I've attempted to just copy the entire Firebase config into the initializeApp() function, but it still generates the same error. Here's the entire test code:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const db = admin.firestore();
exports.postCache = functions.https.onRequest((request, response) => {
console.log("Starting test function");
db.collection('myCollection').doc('myDoc').get()
.then(snap => {
if (!snap.exists) {
console.log('Document not found');
} else {
console.log(snap.data());
response.send(snap.data());
}
})
.catch(error => {
console.log('In catch block');
console.log(error);
response.send(error);
});
});
If I take out the db stuff, the function will work fine. As soon as I add even the simplest firestore request, it generates the error. I've seen this issue asked about before, but the situations seem different, and none of the solutions seems to work. I'm stumped.
The code is perfectly fine.
I have implemented it exactly on my testing project using this tutorial.
I have added your code to index.js and in my firestore I have added collection myCollection with myDoc with just one testing field and deployed using firebase deploy.
Everything works fine. You should focus on your environment and deployment steps to figure out what is the problem. This link that you have in the error is mentioning environment variables. Maybe a case as well.
I hope it will help!
I have an app having firebase back-end. and when i made i didn't thought about the dimensions of images as if they ll be useful in future so i just kept the images as they are and kept their URLs in firestore.
But now i m in need of dimensions of images before showing them to user so i have thought of making a function that i ll execute only once in order to set the files with their dimension in firestore and i ll also add some client side code in order to get the dimensions before uploading them.
So i have tried almost everything to get the file dimensions in functions but couldn't do it.
sample[abstract code]
this code works in node.js but fails in firebase functions
const fs = require('fs')
const request = require('request')
import sizeOf from 'image-size'
const FIRE = 'https://firebasestorage.googleapis.com/file....'
const FILE = 'file.jpg';
request.head(FIRE, (err, res, body) => {
request(FIRE)
.pipe(fs.createWriteStream(FILE))
.on('close', () => {
sizeOf(FILE, (err1, dimensions) => {
const result = {
"width": dimensions.width,
"height": dimensions.height
}
console.log(dimensions.width, dimensions.height);
fs.unlinkSync(FILE);
response.setHeader('Content-Type', 'application/json');
const responseData = {
'Error': false,
'Message': "result : " + result
};
response.send(JSON.stringify(responseData));
})
})
})
help me if someone knows something about this!
and moreover also tell me about how firebase keeps images, i mean in what manner ? whenever i open the url it doesnt show me the image instead it just downloads the image unline other urls on random websites.
I have got a trick to do it. It is quite prone to error, but will work for sure :
get all the urls of images using an api and do the stuff locally using node.js and post the result to another api, which will then feed the data to firestore ?
Your code is trying to write to:
const FILE = 'file.jpg';
Which is a file in the same directory as where your index.js is stored. This is (as the error message says) a read-only directory in the Cloud Functions container. If you want to write any files, they must be in /tmp (also known as tempfs). See Write temporary files from Google Cloud Function
I'm able to authorize the Firebase app from my existing Electron app using firebase.auth().signInWithCustomToken. The promise for this method resolves and I'm able to obtain the current authorized user with firebase.auth().currentUser.uid.
At this point I must technically be able to write to /users/<currentUser>. However calling the userRef.set() and userRef.update() methods does not update the database reference and fails silently (both the callback and the promise from these methods do not resolve and there is no error thrown).
What is strange is that the exact same code works in a different, newly created Electron app. My code looks like below:
const writeToFirebase = (customToken) => {
syncApp.auth().signInWithCustomToken(customToken).then(user => {
const userId = firebase.auth().currentUser.uid; // this is successfull
const userRef = firebase.database().ref("/users/" + userId);
userRef.set({data: data}, () => { //callback does not trigger });
userRef.update({data: data})
.then(() => {//promise does not resolve})
.catch(err) => {// promise is not rejected either! }
});
}
Any pointers on how to go about debugging this would be helpful.
I discovered the problem. It's unlikely anybody else would have the same issue, but if you do, take a look at the userAgent value in your browserWindow.loadURL in Electron.
Mine was set to an Android mobile device & Firebase was not setting/updating due to this reason. I presume the Firebase server reacts differently when it sees a mobile userAgent and I was using the Firebase JS SDK and not the Android SDK which caused the issue.