Google Cloud Functions Datastore Emulator local connection with functions - google-cloud-datastore

I've launched the local Datastore emulator and although wrote and test GCF with the remote Datastore instance (not emulated one). Now I'm trying to use the locally launched Datastore instance for testing purposes, but all requests still going to the cloud instance of Datastore.
Here is the code.
const db = require("#google-cloud/datastore")();
exports.signUp = (req, res) => {
if(!req.body.firstName || !req.body.lastName || !req.body.email) {
res.status(400).send("Incorrect user data passed");
} else {
let key = db.key("User");
console.log("KEY: ", key);
db.insert({
key: key,
data: {
firsName: req.body.firsName,
lastName: req.body.lastName,
email: req.body.email
}
}, (err, apiResponse) => {
console.log(apiResponse);
if(err) {
res.status(400).json({
message: "Error occured during creation"
});
} else {
res.status(200).json({
message: `Created under ${apiResponse}`
});
}
});
}
};
I know about the apiEndpoint (link on documentation) parameter in the Datastore instance configuration object. But should it actually be passed explicitly in the code? I though that there should be some environment variable that will tell default configuration to search for the Datastore emulator first, and then try to use the cloud one.

Related

Firebase Functions ENOENT: no such file or directory, open 'HttpsErrorImpl'

I'm fighting with this issue for over 3 days. I have no idea what is happening.
Firebase Functions throws error only when I try to use Emulator. I try to execute this function in useEffect hook. Again, when I call deployed Cloud Functions everything seems fine, unfortunately in case of using Emulator things don't want to go so well.
const resolvePromise = async () => {
functions.useEmulator("https://0.0.0.0:5001");
const query = functions.httpsCallable("helloWorld");
query()
.then((result) => console.log(result))
.catch((err) => console.log(err));
};
I receive this useless (for me) error.
Error: ENOENT: no such file or directory, open 'HttpsErrorImpl#http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false'
at Object.openSync (node:fs:585:3)
at Object.readFileSync (node:fs:453:35)
at getCodeFrame (Z:\repo\PTCG_Marketplace\node_modules\metro\src\Server.js:1296:18)
at Z:\repo\PTCG_Marketplace\node_modules\metro\src\Server.js:1367:24
at Generator.next (<anonymous>)
at asyncGeneratorStep (Z:\repo\PTCG_Marketplace\node_modules\metro\src\Server.js:146:24)
at _next (Z:\repo\PTCG_Marketplace\node_modules\metro\src\Server.js:168:9)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
internal
at HttpsErrorImpl#http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:197178:29 in <unknown>
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:197273:29 in _errorForResponse
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:197751:39 in <unknown>
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:170357:26 in step
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:170287:21 in <unknown>
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:170241:31 in fulfilled
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:31526:15 in tryCallOne
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:31627:26 in <unknown>
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:31955:16 in _callTimer
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:31994:16 in _callImmediatesPass
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:32211:32 in callImmediates
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:3457:34 in __callImmediates
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:3236:33 in <unknown>
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:3440:14 in __guard
at http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false:3235:20 in flushedQueue
This is all the text which I can see after visiting
http://192.168.0.104:19000/index.bundle -- https://pastebin.com/ggsCMN0W
http://192.168.0.104:19000/index.bundle?platform=android&dev=true&hot=false&minify=false -- https://pastebin.com/LSeufs8H
It don't make any sense for me. At second address it seems like it's correlated to metro dependency so I updated it, it didn't work.
Any ideas? Thanks in advance :D
Edit 1: All errors are logged at client site, it seems like client can't even call emulator.
Edit 2:
I tried to update entire firebase to v.9 aswell as Expo to 44 SKD with react-native to 0.64.3
This is how my Request function looks now:
const requestApi = () => {
const functions = getFunctions(app);
connectFunctionsEmulator(functions, "127.0.0.1", 5001);
const helloWorld = httpsCallable(functions, "helloWorld");
helloWorld()
.then((result) => {
console.log(result);
})
.catch((error) => {
console.log(error.message, error.code, error.details);
});
};
I receive only this from catch block :(
internal functions/internal undefined
I also receive warning about Timer after executing that function
Setting a timer for a long period of time, i.e. multiple minutes, is a
performance and correctness issue on Android as it keeps the timer
module awake, and timers can only be called when the app is in the
foreground. See https://github.com/facebook/react-native/issues/12981
for more info. (Saw setTimeout with duration 70000ms)
It don't work only when I try to use Emulator
Couple of potential issues here:
Assuming you're using the latest version of Firebase, functions is actually a method: firebase.functions().useEmulator("localhost", 5001); - Note the () after functions. See docs for more info.
Maybe you've already done this, but have you made sure that the functions emulator is actually running and connectable on port 5001)? Could be useful to test it via Postman or similar.
Make sure you're using the correct IP address for the functions emulator given your setup. 0.0.0.0 probably doesn't map where you want it to... assuming the app is running locally and the functions emulator is too, try 127.0.0.1 or "localhost" ... this answer has more options to troubleshoot.
I am not sure if it is your case, but I had a function:
exports.findUserInAuth = functions.https.onCall((data, context) => {
let field = data.field;
let value = data.value;
if (!field || !value) {
return false;
}
if (field === "email") {
return admin.auth().getUserByEmail(value);
}
});
This one returns a promise, I had to change it to wait for the result before doing a return and problem solved...
exports.findUserInAuth = functions.https.onCall((data, context) => {
let field = data.field;
let value = data.value;
if (!field || !value) {
return false;
}
if (field === "email") {
admin.auth().getUserByEmail(value).then((result) => {
return result;
})
.catch((error) => {
if (error.code === "auth/user-not-found") {
return "Email or Password is incorrect";
}
return `${error.code} ${error.message}`;
});
}
return false;
});
Ok, so after almost a week of fighting with this sh!t.
When you use Expo Go like me. You should copy the host address on which you are emulating your app, and use the same address you emulate your functions (or other tools).
app.json
{
"firestore": {
"rules": "firestore.rules",
"indexes": "firestore.indexes.json"
},
"emulators": {
"functions": {
"host": "192.168.0.104",
"port": 5001
}
}
}
and final code of requestApi function
const requestApi = async () => {
const functions = firebase.functions()
functions.useEmulator("192.168.0.104", 5001); <--- ADDRESS!!!
const helloWorld = functions.httpsCallable("helloWorld");
helloWorld()
.then((result) => {
console.log(result);
})
.catch((error) => {
console.log(error.message, error.code, error.details);
});
};

How to fix the error : your application id may be incorrect. If the error persists, contact support#algolia.com

I want to send cloud firestore data to algolia to enable full-text search. Firebase cloud function log is showing an error about application id. I am not able to understand this error and how to fix this.
name: 'RetryError',
message: 'Unreachable hosts - your application id may be incorrect. If the error persists, contact support#algolia.com.'
This is my index.js file
exports.addFirestoreDataToAlgolia =
functions.https.onRequest((req, res) => {
var arr = [];
admin.firestore().collection("tags").get()
.then((docs) => {
docs.forEach((doc) => {
let user = doc.data();
user.objectID = doc.id;
arr.push(user);
})
const client = algoliasearch(ALGOLIA_ID, ALGOLIA_ADMIN_KEY);
const index = client.initIndex(ALGOLIA_INDEX_NAME);
return index.saveObjects(arr, (err, content) => {
if (err) {
res.status(500);
}
else {
res.status(200).send(content);
}
})
})
.catch( err => {
console.log(err);
})
})
Outbound requests (outside of Google services) can only be made from functions on a paid plan (https://firebase.google.com/pricing).
Reason for the wrong appID error is that the Algolia is trying to resolve a dns using your appID, which fails. See https://github.com/algolia/algoliasearch-client-javascript/issues/587#issuecomment-407397688
You have to move off of the free Spark plan in order to call out to Algolia from your function..
I also got this error with NextJS, it was working fine with react but then when I moved to NextJs I got the error.
Turns out it was my .env variables that were not being passed correctly to the client/browser. Renaming the variables from REACT_APP_<variable name> to NEXT_PUBLIC_<variable name> to make them available to the browser as per the NextJs documentation fixed the issue.
NEXT_PUBLIC_ALGOLIA_APP_ID=xxxxxx
NEXT_PUBLIC_ALGOLIA_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
NEXT_PUBLIC_ALGOLIA_ADMIN_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxx

Unable to access URLs other than googleapis.com from Firebase emulator using fetch()

I'm trying to create some Firebase Cloud Functions and test them locally using
firebase emulators:start --only functions
These functions supposed to call some external services using fetch.
I find that I can call these external services when the functions are deployed to the Firebase cloud, but cannot invoke them when running locally in the emulator:
import 'cross-fetch/polyfill';
export const fetchTest = functions
.region(config.firebaseRegion)
.https.onRequest((request: Request, response: Response) => {
fetch("https://www.wikipedia.org/", {
method: 'GET',
}).then(value => {
console.log("Fetched: ", value);
}).catch(reason => {
console.log("Fetch failed: ", reason);
});
fetch("https://googleapis.com/foo", {
method: 'GET',
}).then(value => {
console.log("Fetched: ", value);
}).catch(reason => {
console.log("Fetch failed: ", reason);
});
response.send("Done");
});
This is the output I get when invoking fetchTest in the emulator:
⚠ Unknown network resource requested!
- URL: "https://www.wikipedia.org/"
⚠ Google API requested!
- URL: "https://googleapis.com/foo"
- Be careful, this may be a production service.
Looking at the source code there seems to be some filtering implemented in the emulator:
https://github.com/firebase/firebase-tools/blob/0586ee1e23adc64b0fe8607a026ba472a6bd7d2e/src/emulator/functionsEmulatorRuntime.ts
if (href && !history[href]) {
history[href] = true;
if (href.indexOf("googleapis.com") !== -1) {
new EmulatorLog("SYSTEM", "googleapis-network-access", "", {
href,
module: bundle.name,
}).log();
} else {
new EmulatorLog("SYSTEM", "unidentified-network-access", "", {
href,
module: bundle.name,
}).log();
}
}
Are there any reason for such restrictions? And is there a workaround for this?
Thanks!
When you see "Unknown network requested" logs like this one:
⚠ Unknown network resource requested!
- URL: "https://www.wikipedia.org/"
They are just warnings. The actual request is allowed through. The log is meant to tell you that the emulators are accessing resources outside your machine. This is normally something people want to avoid because local testing is meant to be hermetic, but sometimes it's what you want to do and you can ignore the warnings safely!

Cloud Functions with Cloud SQL - Property 'socketPath' does not exist on type

I'm trying to access my Google Cloud SQL database on my Firebase Functions. I followed the Google documentation but it's not very good and leaves out a lot of information.
I added the code from the docs to my src\index.ts file and filled in my data:
const mysql = require('mysql');
const connectionName =
process.env.INSTANCE_CONNECTION_NAME || 'test';
const dbUser = process.env.SQL_USER || 'test';
const dbPassword = process.env.SQL_PASSWORD || 'test';
const dbName = process.env.SQL_NAME || 'test';
const mysqlConfig = {
connectionLimit: 1,
user: dbUser,
password: dbPassword,
database: dbName,
};
if (process.env.NODE_ENV === 'production') {
mysqlConfig.socketPath = `/cloudsql/${connectionName}`;
}
// Connection pools reuse connections between invocations,
// and handle dropped or expired connections automatically.
let mysqlPool;
exports.mysqlDemo = (req, res) => {
// Initialize the pool lazily, in case SQL access isn't needed for this
// GCF instance. Doing so minimizes the number of active SQL connections,
// which helps keep your GCF instances under SQL connection limits.
if (!mysqlPool) {
mysqlPool = mysql.createPool(mysqlConfig);
}
mysqlPool.query('SELECT NOW() AS now', (err, results) => {
if (err) {
console.error(err);
res.status(500).send(err);
} else {
res.send(JSON.stringify(results));
console.log("Connected!")
}
});
// Close any SQL resources that were declared inside this function.
// Keep any declared in global scope (e.g. mysqlPool) for later reuse.
};
I then tried publishing this to see if it would work, and I got the error:
Module 'mysql' is not listed as dependency in package.json
So I installed mysql by doing npm install mysql
and I ran it again and this time I get the error:
src/index.ts(23,15): error TS2339: Property 'socketPath' does not exist on type '{ connectionLimit: number; user: string; password: string; database: string; }'
What's going on? Have I installed this correctly? I'm very lost and the docs don't seem to offer much of a solution. Thanks!
This seems like a TypeScript error, where the interface for your config object is being inferred and then you're trying to add an extra field on that object that wasn't declared. In other words, the example code from Google is JavaScript, you're using TypeScript and running into a TS error.
I think that you should be able to adjust the declaration of mysqlConfig to include a socketPath property (set to null) and all should be OK. In other words, change the mysqlConfig definition to:
const mysqlConfig = {
connectionLimit: 1,
user: dbUser,
password: dbPassword,
database: dbName,
socketPath: null
};

Log 'jsonPayload' in Firebase Cloud Functions

TL;DR;
Does anyone know if it's possible to use console.log in a Firebase/Google Cloud Function to log entries to Stack Driver using the jsonPayload property so my logs are searchable (currently anything I pass to console.log gets stringified into textPayload).
I have a multi-module project with some code running on Firebase Cloud Functions, and some running in other environments like Google Compute Engine. Simplifying things a little, I essentially have a 'core' module, and then I deploy the 'cloud-functions' module to Cloud Functions, 'backend-service' to GCE, which all depend on 'core' etc.
I'm using bunyan for logging throughout my 'core' module, and when deployed to GCE the logger is configured using '#google-cloud/logging-bunyan' so my logs go to Stack Driver.
Aside: Using this configuration in Google Cloud Functions is causing issues with Error: Endpoint read failed which I think is due to functions not going cold and trying to reuse dead connections, but I'm not 100% sure what the real cause is.
So now I'm trying to log using console.log(arg) where arg is an object, not a string. I want this object to appear in Stack Driver under the jsonPayload but it's being stringified and put into the textPayload field.
It took me awhile, but I finally came across this example in firebase functions samples repository. In the end I settled on something a bit like this:
const Logging = require('#google-cloud/logging');
const logging = new Logging();
const log = logging.log('my-func-logger');
const logMetadata = {
resource: {
type: 'cloud_function',
labels: {
function_name: process.env.FUNCTION_NAME ,
project: process.env.GCLOUD_PROJECT,
region: process.env.FUNCTION_REGION
},
},
};
const logData = { id: 1, score: 100 };
const entry = log.entry(logMetaData, logData);
log.write(entry)
You can add a string severity property value to logMetaData (e.g. "INFO" or "ERROR"). Here is the list of possible values.
Update for available node 10 env vars. These seem to do the trick:
labels: {
function_name: process.env.FUNCTION_TARGET,
project: process.env.GCP_PROJECT,
region: JSON.parse(process.env.FIREBASE_CONFIG).locationId
}
UPDATE: Looks like for Node 10 runtimes they want you to set env values explicitly during deploy. I guess there has been a grace period in place because my deployed functions are still working.
I ran into the same problem, and as stated by comments on #wtk's answer, I would like to add replicating all of the default cloud function logging behavior I could find in the snippet below, including execution_id.
At least for using Cloud Functions with the HTTP Trigger option the following produced correct logs for me. I have not tested for Firebase Cloud Functions
// global
const { Logging } = require("#google-cloud/logging");
const logging = new Logging();
const Log = logging.log("cloudfunctions.googleapis.com%2Fcloud-functions");
const LogMetadata = {
severity: "INFO",
type: "cloud_function",
labels: {
function_name: process.env.FUNCTION_NAME,
project: process.env.GCLOUD_PROJECT,
region: process.env.FUNCTION_REGION
}
};
// per request
const data = { foo: "bar" };
const traceId = req.get("x-cloud-trace-context").split("/")[0];
const metadata = {
...LogMetadata,
severity: 'INFO',
trace: `projects/${process.env.GCLOUD_PROJECT}/traces/${traceId}`,
labels: {
execution_id: req.get("function-execution-id")
}
};
Log.write(Log.entry(metadata, data));
The github link in #wtk's answer should be updated to:
https://github.com/firebase/functions-samples/blob/2f678fb933e416fed9be93e290ae79f5ea463a2b/stripe/functions/index.js#L103
As it refers to the repository as of when the question was answered, and has the following function in it:
// To keep on top of errors, we should raise a verbose error report with Stackdriver rather
// than simply relying on console.error. This will calculate users affected + send you email
// alerts, if you've opted into receiving them.
// [START reporterror]
function reportError(err, context = {}) {
// This is the name of the StackDriver log stream that will receive the log
// entry. This name can be any valid log stream name, but must contain "err"
// in order for the error to be picked up by StackDriver Error Reporting.
const logName = 'errors';
const log = logging.log(logName);
// https://cloud.google.com/logging/docs/api/ref_v2beta1/rest/v2beta1/MonitoredResource
const metadata = {
resource: {
type: 'cloud_function',
labels: {function_name: process.env.FUNCTION_NAME},
},
};
// https://cloud.google.com/error-reporting/reference/rest/v1beta1/ErrorEvent
const errorEvent = {
message: err.stack,
serviceContext: {
service: process.env.FUNCTION_NAME,
resourceType: 'cloud_function',
},
context: context,
};
// Write the error log entry
return new Promise((resolve, reject) => {
log.write(log.entry(metadata, errorEvent), (error) => {
if (error) {
return reject(error);
}
resolve();
});
});
}
// [END reporterror]

Resources