as part of my firebase app, I'm using a cloud function to get data from google places API.
for some reason, I'm getting 403 errors when trying to retrieve data, even though the service account I'm using is the default one (App Engine default service account with Editor role) which seems to exist on the API credentials list and also on the specific cloud function I'm using.
here's the code I'm using to retrieve data from the API -
class GoogleMapsRestApiClass {
client = new Client({});
getPlaceInfo(placeId: string) {
return this.client.placeDetails({
params: {
place_id: placeId,
fields: ["name", "rating", "geometry", "photo"],
key: environment.googleMapsJsApi.apiKey
}
} as PlaceDetailsRequest);
}
}
export const GoogleMapsRestApi = new GoogleMapsRestApiClass();
and the cloud function itself -
export const place = functions.https.onRequest(async (request, response) => {
const placeId = request.query.place_id as string;
const resp: AxiosResponse = await GoogleMapsRestApi.getPlaceInfo(placeId);
const result = resp.data.result;
response.send({result});
});
any ideas what I'm missing here?
Update -
if I'm not restricting the API key I do manage to retrieve the data (restricted it to my host address).
how should I protect the API key being used by cloud function?
Related
I've deployed a small HTTP endpoint via Google Cloud Run. It is working fine when I turn off the authentication.
I now want to turn it on so that it is only callable by my Firebase Cloud Function. If I understand it right, I just have to add the correct service account mail address in the IAM settings of the Cloud Run as "Cloud Run invoker".
But which address is the correct one?
I've tried all addresses that I have found in Firebase Console -> Project Settings -> Service Accounts.
I think you can check the specific firebase function. In the UI, the service account used should be listed.
By default, GCF functions all use <project_id>#appspot.gserviceaccount.com
Thanks to #AhmetB - Google and #whlee's answer I got it working. Basically it is enough adding an Authorization Bearer token to the request, which you can get from a special endpoint: https://cloud.google.com/run/docs/authenticating/service-to-service#nodejs
Then you just have to add the service account of the function to the IAM list of the Cloud Run container: <project_id>#appspot.gserviceaccount.com
The nodejs example is using the deprecated request library, so here is my version using axios:
const getOAuthToken = async (receivingServiceURL: string): Promise<string> => {
// Set up metadata server request
const metadataServerTokenURL = 'http://metadata/computeMetadata/v1/instance/service-accounts/default/identity?audience=';
const uri = metadataServerTokenURL + receivingServiceURL;
const options = {
headers: {
'Metadata-Flavor': 'Google'
}
};
return axios.get(uri, options)
.then((res) => res.data)
.catch((error) => Promise.reject(error));
}
Then you can just use the token in the actual request:
const url = `...`;
const token = await getOAuthToken(url);
axios.post(url, formData, {
headers: {
Authorization: `Bearer ${token}`,
}
}).then(...).catch(...);
#luhu 's answer was really helpful. I'd like to add just one note for those whose are willing to test with the emulators locally first. The metadata server (which is actually http://metadata.google.internal now) as they state
does not work outside of Google Cloud, including from your local machine.
As a workarund, you can use the google-auth-library and then get the token directly if you prefer sticking with axios. Remember to set the GOOGLE_APPLICATION_CREDENTIALS env variable pointing to a service account secret first as it's the only way to make it work (I've tested setting the credential field during admin.initializeApp() but didn't seem to like it).
const {GoogleAuth} = require('google-auth-library');
const auth = new GoogleAuth();
const url_origin = '....'
const client = await auth.getIdTokenClient(url_origin);
const token = (await client.getRequestHeaders()).Authorization;
const url = '....'
const response = await axios.get(
url,
{
headers: {
Authorization: token,
},
}
);
I'm moving a MERN project into React + MongoDB Stitch after seeing it allows for easy user authentication, quick deployment, etc.
However, I am having a hard time understanding where and how can I call a site scraping function. Previously, I web scraped in Express.js with cheerio like:
app.post("/api/getTitleAtURL", (req, res) => {
if (req.body.url) {
request(req.body.url, function(error, response, body) {
if (!error && response.statusCode == 200) {
const $ = cheerio.load(body);
const webpageTitle = $("title").text();
const metaDescription = $("meta[name=description]").attr("content");
const webpage = {
title: webpageTitle,
metaDescription: metaDescription
};
res.send(webpage);
} else {
res.status(400).send({ message: "THIS IS AN ERROR" });
}
});
}
});
But obviously with Stitch no Node & Express is needed. Is there a way to fetch another site's content without having to host a node.js application just serving that one function?
Thanks
Turns out you can build Functions in MongoDB Stitch that allows you to upload external dependencies.
However, there're limitation, for example, cheerio didn't work as an uploaded external dependency while request worked. A solution, therefore, would be to create a serverless function in AWS's lambda, and then connect mongoDB stitch to AWS lambda (mongoDB stitch can connect to many third party services, including many AWS lambda cloud services like lambda, s3, kinesis, etc).
AWS lambda allows you to upload any external dependencies, if mongoDB stitch allowed for any, we wouldn't need lambda, but stitch still needs many support. In my case, I had a node function with cheerio & request as external dependencies, to upload this to lambda: make an account, create new lambda function, and pack your node modules & code into a zip file to upload it. Your zip should look like this:
and your file containing the function should look like:
const cheerio = require("cheerio");
const request = require("request");
exports.rss = function(event, context, callback) {
request(event.requestURL, function(error, response, body) {
if (!error && response.statusCode == 200) {
const $ = cheerio.load(body);
const webpageTitle = $("title").text();
const metaDescription = $("meta[name=description]").attr("content");
const webpage = {
title: webpageTitle,
metaDescription: metaDescription
};
callback(null, webpage);
return webpage;
} else {
callback(null, {message: "THIS IS AN ERROR"})
return {message: "THIS IS AN ERROR"};
}
});
};
and in mongoDB, connect to a third party service, choose AWS, enter the secret keys you got from making an IAM amazon user. In rules -> actions, choose lambda as your API, and allow for all actions. Now, in your mongoDB stitch functions, you can connect to Lambda, and that function should look like this in my case:
exports = async function(requestURL) {
const lambda = context.services.get('getTitleAtURL').lambda("us-east-1");
const result = await lambda.Invoke({
FunctionName: "getTitleAtURL",
Payload: JSON.stringify({requestURL: requestURL})
});
console.log(result.Payload.text());
return EJSON.parse(result.Payload.text());
};
Note: this slowed down performances big time though, generally, it took twice extra time for the call to finish.
I am trying to develop a server-side validation of my users' in-app purchases and subscriptions as recommended, and I want to use Firebase Functions for that. Basically it has to be an HTTP trigger function that receives a purchase token, calls the Play Developer API to verify the purchase, and then does something with the result.
However, calling many of the Google APIs (including Play Developer API) requires non-trivial authorization. Here's how I understand the required setup:
There has to be a GCP project with Google Play Developer API v2 enabled.
It should be a separate project, since there can be only one linked to Play Store in the Google Play Console.
My Firebase Functions project must somehow authenticate to that other project. I figured that using a Service Account is most suitable in this server-to-server scenario.
Finally, my Firebase Functions code must somehow obtain authentication token (hopefully JWT?) and finally make an API call to get a subscription status.
The problem is that absolutely no human-readable documentation or guidance on that is existent. Given that ingress traffic in Firebase is included in the free plan (so I assume they encourage using Google APIs from Firebase Functions), that fact is pretty disappointing. I've managed to find some bits of info here and there, but having too little experience with Google APIs (most of which required simply using an api key), I need help with putting it together.
Here's what I figured out so far:
I got a GCP project linked to the Play Store and with the API enabled. For some reason though, trying to test it in APIs Explorer results in an error "The project id used to call the Google Play Developer API has not been linked in the Google Play Developer Console".
I made a Service Account and exported a JSON key, which contains the key to produce a JWT.
I also set up read permissions for that Service Account in Play Console.
I found a Node.JS client library for Google APIs, which is in alpha and has very sparse documentation (e.g. there's no obvious documentation on how to authenticate with JWT, and no samples on how to call the android publisher API). At the moment I'm struggling with that. Unfortunately I'm not super-comfortable with reading JS library code, especially when the editor doesn't provide the possibility to jump to highlighted functions' sources.
I'm pretty surprised this hasn't been asked or documented, because verifying in-app purchases from Firebase Functions seems like a common task. Has anyone successfully done it before, or maybe the Firebase team will step in to answer?
I figured it out myself. I also ditched the heavyweight client library and just coded those few requests manually.
Notes:
The same applies to any Node.js server environment. You still need the key file of a separate service account to mint a JWT and the two steps to call the API, and Firebase is no different.
The same applies to other APIs that require authentication as well — differing only in scope field of the JWT.
There are a few APIs that don't need you to exchange the JWT for an access token — you can mint a JWT and provide it directly in Authentication: Bearer, without a round trip to OAuth backend.
After you've got the JSON file with the private key for a Service Account that's linked to Play Store, the code to call the API is like this (adjust to your needs). Note: I used request-promise as a nicer way to do http.request.
const functions = require('firebase-functions');
const jwt = require('jsonwebtoken');
const keyData = require('./key.json'); // Path to your JSON key file
const request = require('request-promise');
/**
* Exchanges the private key file for a temporary access token,
* which is valid for 1 hour and can be reused for multiple requests
*/
function getAccessToken(keyData) {
// Create a JSON Web Token for the Service Account linked to Play Store
const token = jwt.sign(
{ scope: 'https://www.googleapis.com/auth/androidpublisher' },
keyData.private_key,
{
algorithm: 'RS256',
expiresIn: '1h',
issuer: keyData.client_email,
subject: keyData.client_email,
audience: 'https://www.googleapis.com/oauth2/v4/token'
}
);
// Make a request to Google APIs OAuth backend to exchange it for an access token
// Returns a promise
return request.post({
uri: 'https://www.googleapis.com/oauth2/v4/token',
form: {
'grant_type': 'urn:ietf:params:oauth:grant-type:jwt-bearer',
'assertion': token
},
transform: body => JSON.parse(body).access_token
});
}
/**
* Makes a GET request to given URL with the access token
*/
function makeApiRequest(url, accessToken) {
return request.get({
url: url,
auth: {
bearer: accessToken
},
transform: body => JSON.parse(body)
});
}
// Our test function
exports.testApi = functions.https.onRequest((req, res) => {
// TODO: process the request, extract parameters, authenticate the user etc
// The API url to call - edit this
const url = `https://www.googleapis.com/androidpublisher/v2/applications/${packageName}/purchases/subscriptions/${subscriptionId}/tokens/${token}`;
getAccessToken(keyData)
.then(token => {
return makeApiRequest(url, token);
})
.then(response => {
// TODO: process the response, e.g. validate the purchase, set access claims to the user etc.
res.send(response);
return;
})
.catch(err => {
res.status(500).send(err);
});
});
These are the docs I followed.
I think I found a slightly quicker way to do this... or at least... more simply.
To support scaling and keep index.ts from growing out of control... I have all the functions and globals in the index file but all the actual events are handled by handlers. Easier to maintain.
So here's my index.ts (I heart type safety):
//my imports so you know
import * as functions from 'firebase-functions';
import * as admin from "firebase-admin";
import { SubscriptionEventHandler } from "./subscription/subscription-event-handler";
// honestly not 100% sure this is necessary
admin.initializeApp({
credential: admin.credential.applicationDefault(),
databaseURL: 'dburl'
});
const db = admin.database();
//reference to the class that actually does the logic things
const subscriptionEventHandler = new SubscriptionEventHandler(db);
//yay events!!!
export const onSubscriptionChange = functions.pubsub.topic('subscription_status_channel').onPublish((message, context) => {
return subscriptionEventHandler.handle(message, context);
});
//aren't you happy this is succinct??? I am!
Now... for the show!
// importing like World Market
import * as admin from "firebase-admin";
import {SubscriptionMessageEvent} from "./model/subscription-message-event";
import {androidpublisher_v3, google, oauth2_v2} from "googleapis";
import {UrlParser} from "../utils/url-parser";
import {AxiosResponse} from "axios";
import Schema$SubscriptionPurchase = androidpublisher_v3.Schema$SubscriptionPurchase;
import Androidpublisher = androidpublisher_v3.Androidpublisher;
// you have to get this from your service account... or you could guess
const key = {
"type": "service_account",
"project_id": "not going to tell you",
"private_key_id": "really not going to tell you",
"private_key": "okay... I'll tell you",
"client_email": "doesn't matter",
"client_id": "some number",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "another url"
};
//don't guess this... this is right
const androidPublisherScope = "https://www.googleapis.com/auth/androidpublisher";
// the handler
export class SubscriptionEventHandler {
private ref: admin.database.Reference;
// so you don't need to do this... I just did to log the events in the db
constructor(db: admin.database.Database) {
this.ref = db.ref('/subscriptionEvents');
}
// where the magic happens
public handle(message, context): any {
const data = JSON.parse(Buffer.from(message.data, 'base64').toString()) as SubscriptionMessageEvent;
// if subscriptionNotification is truthy then we're solid here
if (message.json.subscriptionNotification) {
// go get the the auth client but it's async... so wait
return google.auth.getClient({
scopes: androidPublisherScope,
credentials: key
}).then(auth => {
//yay! success! Build android publisher!
const androidPublisher = new Androidpublisher({
auth: auth
});
// get the subscription details
androidPublisher.purchases.subscriptions.get({
packageName: data.packageName,
subscriptionId: data.subscriptionNotification.subscriptionId,
token: data.subscriptionNotification.purchaseToken
}).then((response: AxiosResponse<Schema$SubscriptionPurchase>) => {
//promise fulfilled... grandma would be so happy
console.log("Successfully retrieved details: " + response.data.orderId);
}).catch(err => console.error('Error during retrieval', err));
});
} else {
console.log('Test event... logging test');
return this.ref.child('/testSubscriptionEvents').push(data);
}
}
}
There are few model classes that help:
export class SubscriptionMessageEvent {
version: string;
packageName: string;
eventTimeMillis: number;
subscriptionNotification: SubscriptionNotification;
testNotification: TestNotification;
}
export class SubscriptionNotification {
version: string;
notificationType: number;
purchaseToken: string;
subscriptionId: string;
}
So that's how we do that thing.
TL;DR;
Does anyone know if it's possible to use console.log in a Firebase/Google Cloud Function to log entries to Stack Driver using the jsonPayload property so my logs are searchable (currently anything I pass to console.log gets stringified into textPayload).
I have a multi-module project with some code running on Firebase Cloud Functions, and some running in other environments like Google Compute Engine. Simplifying things a little, I essentially have a 'core' module, and then I deploy the 'cloud-functions' module to Cloud Functions, 'backend-service' to GCE, which all depend on 'core' etc.
I'm using bunyan for logging throughout my 'core' module, and when deployed to GCE the logger is configured using '#google-cloud/logging-bunyan' so my logs go to Stack Driver.
Aside: Using this configuration in Google Cloud Functions is causing issues with Error: Endpoint read failed which I think is due to functions not going cold and trying to reuse dead connections, but I'm not 100% sure what the real cause is.
So now I'm trying to log using console.log(arg) where arg is an object, not a string. I want this object to appear in Stack Driver under the jsonPayload but it's being stringified and put into the textPayload field.
It took me awhile, but I finally came across this example in firebase functions samples repository. In the end I settled on something a bit like this:
const Logging = require('#google-cloud/logging');
const logging = new Logging();
const log = logging.log('my-func-logger');
const logMetadata = {
resource: {
type: 'cloud_function',
labels: {
function_name: process.env.FUNCTION_NAME ,
project: process.env.GCLOUD_PROJECT,
region: process.env.FUNCTION_REGION
},
},
};
const logData = { id: 1, score: 100 };
const entry = log.entry(logMetaData, logData);
log.write(entry)
You can add a string severity property value to logMetaData (e.g. "INFO" or "ERROR"). Here is the list of possible values.
Update for available node 10 env vars. These seem to do the trick:
labels: {
function_name: process.env.FUNCTION_TARGET,
project: process.env.GCP_PROJECT,
region: JSON.parse(process.env.FIREBASE_CONFIG).locationId
}
UPDATE: Looks like for Node 10 runtimes they want you to set env values explicitly during deploy. I guess there has been a grace period in place because my deployed functions are still working.
I ran into the same problem, and as stated by comments on #wtk's answer, I would like to add replicating all of the default cloud function logging behavior I could find in the snippet below, including execution_id.
At least for using Cloud Functions with the HTTP Trigger option the following produced correct logs for me. I have not tested for Firebase Cloud Functions
// global
const { Logging } = require("#google-cloud/logging");
const logging = new Logging();
const Log = logging.log("cloudfunctions.googleapis.com%2Fcloud-functions");
const LogMetadata = {
severity: "INFO",
type: "cloud_function",
labels: {
function_name: process.env.FUNCTION_NAME,
project: process.env.GCLOUD_PROJECT,
region: process.env.FUNCTION_REGION
}
};
// per request
const data = { foo: "bar" };
const traceId = req.get("x-cloud-trace-context").split("/")[0];
const metadata = {
...LogMetadata,
severity: 'INFO',
trace: `projects/${process.env.GCLOUD_PROJECT}/traces/${traceId}`,
labels: {
execution_id: req.get("function-execution-id")
}
};
Log.write(Log.entry(metadata, data));
The github link in #wtk's answer should be updated to:
https://github.com/firebase/functions-samples/blob/2f678fb933e416fed9be93e290ae79f5ea463a2b/stripe/functions/index.js#L103
As it refers to the repository as of when the question was answered, and has the following function in it:
// To keep on top of errors, we should raise a verbose error report with Stackdriver rather
// than simply relying on console.error. This will calculate users affected + send you email
// alerts, if you've opted into receiving them.
// [START reporterror]
function reportError(err, context = {}) {
// This is the name of the StackDriver log stream that will receive the log
// entry. This name can be any valid log stream name, but must contain "err"
// in order for the error to be picked up by StackDriver Error Reporting.
const logName = 'errors';
const log = logging.log(logName);
// https://cloud.google.com/logging/docs/api/ref_v2beta1/rest/v2beta1/MonitoredResource
const metadata = {
resource: {
type: 'cloud_function',
labels: {function_name: process.env.FUNCTION_NAME},
},
};
// https://cloud.google.com/error-reporting/reference/rest/v1beta1/ErrorEvent
const errorEvent = {
message: err.stack,
serviceContext: {
service: process.env.FUNCTION_NAME,
resourceType: 'cloud_function',
},
context: context,
};
// Write the error log entry
return new Promise((resolve, reject) => {
log.write(log.entry(metadata, errorEvent), (error) => {
if (error) {
return reject(error);
}
resolve();
});
});
}
// [END reporterror]
We have a large SPA using Firebase v2. We would like to upgrade to the new API, but we experience the following problem:
As the app is quite large, we have developed many integration tests, and for these tests we always need to reset the database and initialize it to a state, where some users exist. However, we found out there really is no such thing as creating a user on server anymore ( Firebase createUserWithEmailAndPassword method is undefined in node.js ), and we are quite unsure, how to upgrade the API and yet be able to reset and initialize the database from server.
Moreover, we are quite forced to do this upgrade, because we noticed that the Firebase v2, is still using the deprecated Graph API v2.0 for Facebook OAuth, and is not recommended for use after 8.8.2016. We understand that the Firebase v2 will probably not upgrade the calls to the Graph API, as the v2 is legacy. This, however, leaves us quite cornered for now.
Any help on this topic, please?
As of Firebase v3.3.0 you are able to create user accounts using Node, but the documentation isn't great on how to expose these methods.
In order to use the user management methods, you need to initialize an application in node using your Web API key, and not the Service Account config that is walked through in the setup guide.
// The Usual Service Account Init
// This will not contain any user management methods on firebase.auth()
this.app = firebase.initializeApp(
{
serviceAccount: 'path/to/serviceaccount/file.json',
databaseURL: 'https://mydbfb.firebaseio.com'
},
'MyAppName');
// Web Client Init in Node.js
// firebase.auth() will now contain user management methods
this.app = firebase.initializeApp(
{
"apiKey": "my-api-key",
"authDomain": "somedomain.firebaseapp.com",
"databaseURL": "https://mydbfb.firebaseio.com",
"storageBucket": "myfbdb.appspot.com",
"messagingSenderId": "SomeId"
},
'MyAppName');
You can grab your client api key from your Firebase console from the Web Setup guide
https://firebase.google.com/docs/web/setup
This is the only reference I could find that explicitly referenced the need to init with api key to get this to work.
https://groups.google.com/forum/#!msg/firebase-talk/_6Rhro3zBbk/u8hB1oVRCgAJ
Given below is a working example of creating Firebase user through Node.js
exports.addUser = function(req, res) {
var wine = req.body;
var email = req.body.email;
console.log(req.body);
var password = req.body.password;
var name = req.body.name;
console.log(“Creating user for -“+email+”-“+password);
var defaultAuth = admin.auth();
admin.auth().createUser({
email: email,
emailVerified: false,
password: password,
displayName: name,
disabled: false
})
.then(function(userRecord) {
console.log(“Created Firebase User successfully with id :”, userRecord.uid);
var wine = req.body;
wine.userId = userRecord.uid;
wine.timestamp = Date.now();
delete wine.password;
status = “201”;
var reply = JSON.stringify(wine);
db.collection(‘collname’, function(err, collection) {
collection.insert(wine, {safe:true}, function(err, result) {
if (err) {
wine.status = “200”;
wine.message = “An error occured”;
reply.set(‘status’,”201″);
res.status(201).send(wine);
} else {
console.log(‘Success: ‘ + JSON.stringify(result[0]));
status= “200”;
wine.status = “200”;
wine.message = “Account created Successfully”;
res.status(200).send(wine);
}
});
});
})
.catch(function(error) {
wine.message = “An error occured—“;
wine.status = “201”;
console.log(“User Creation onf Firebase failed:”, error);
res.status(201).send(wine);
});
}
For details you can see the following blog post
http://navraj.net/?p=53
Thanks