Optimizing firebase functions cold start with expressjs - firebase

I was trying to figure out how I could optimize cold start times for my firebase functions. After reading this article, I wanted to try it out but I realized that the article specifically targets the base usage of the http onRequest function and doesn't give an example using express.
A similar question popped up here but doesn't seem like there's a clear answer. I saw the author of the article Doug actually commented on the question and he mentions to create a dynamic import for each route in the app since onRequest() only allows for passing the app as its only argument, but I wasn't understanding exactly what he meant by that other than to use the base API without the express app. Ideally I'd be able to use express so I can have finer control over the api url paths and use some of utility that express offers.
Can anyone give me an example of how to use express with Doug's example? Even if I have to define a new express app for each route, I'm okay with that. Just don't see how to configure it that way.
EDIT: To be clear, the goal is to optimize cold starts across all function invocations, not just the http routed ones. From my understanding, Doug's example eliminates the imports being preloaded with single routes declared using onRequest, but it doesn't show how that is possible when defining routes through express.

Assuming each router you split out is defined in it's own file like so:
// $FUNCTIONS_DIR/routes/some-route-handler.js
import express from "express";
const router = express.Router();
/* ... define routes ... */
export default router;
You could then use this middleware to load each route handler module only when it's needed.
function lazyRouterModule(modulePath) {
return async (req, res, next) {
let router;
try {
router = (await import(modulePath)).default;
} catch (err) {
// error loading module, let next() handle it
next(err);
return;
}
router(req, res, next);
}
}
In your sub-function file, you'd use that middleware to create your express app and connect the routes.
// $FUNCTIONS_DIR/fn/my-express.js
import express from "express";
const app = express();
app.use('/api', lazyRouterModule('./routes/api.js'));
app.use('/profiles', lazyRouterModule('./routes/profiles.js'));
export default app;
Then in your main functions file, you'd connect up your subfunction files on-demand:
// $FUNCTIONS_DIR/index.js
import * as functions from 'firebase-functions'
export const myExpress = functions.https
.onRequest(async (request, response) => {
await (await import('./fn/my-express.js')).default(request, response)
});
export const newUserData = functions.firestore.document('/users/{userId}')
.onCreate(async (snap, context) => {
await (await import('./fn/new-user-data.js')).default(snap, context)
});
When lazy-loading modules like this, you will want to lazy-load firebase-admin from a common file so you don't end up calling initializeApp() multiple times.
// $FUNCTIONS_DIR/common/firebase-admin.js
import * as admin from "firebase-admin";
admin.initializeApp();
export = admin;
In any function that wants to use "firebase-admin", you'd import it from here using:
// $FUNCTIONS_DIR/fn/some-function.js OR $FUNCTIONS_DIR/routes/some-route-handler.js
import * as admin from "../common/firebase-admin";
// use admin as normal, it's already initialized

Related

Modular Firebase Admin Database

Is there or will there be modular Firebase Admin Database functions i.e. update(), get(), set(), ref() etc.? Maybe there is a workaround?
Else I would have to code equal functions twice, like "craftBuilding()" on server (admin) and one for client.
Tried to:
import { getDatabase, ref, set, update, child, onChildAdded } from 'firebase-admin/database';
Error:
The requested module 'firebase-admin/database' does not provide an export named 'ref'
I expected to be able to use firebase rtdb functions like on client-side, to not code identical functions twice.
The Admin SDK is not totally modular yet but has some function such a getDatabase() starting V10.
import { getDatabase } from 'firebase-admin/database';
// TODO: Initialize Admin SDK
const db = getDatabase();
// usage
const ref = db.ref('../../..')
I would recommend using latest version of the SDK along with the existing syntax but use newer modular imports.
Since Firebase Admin is namespaced, ref is available after you initialize.
import { ref, update, set } from '#path/yourAdminFunctions';
var admin = require("firebase-admin");
// Initialize Admin SDK
var db = admin.database();
export db;
const dbReference = ref("your reference here");
const IDSnapFromQuery = query(dbReference, ["ID", "==", 10]);
You could modularize it yourself by creating functions for the operations you require in a separate file:
import { db } from '#path/yourMainFile';
export function ref(databaseReference) {
return db.ref(databaseReference);
};
// for query pass in the db.ref instance you're using
export function query(dbReference, queryParams) {
// identify what query params are e.g. ID == 10 in this example
// note that you will have to code the queryParams filter, I have left it out
dbReference
.orderByChild('ID')
.equalTo(10).on('child_added',
(snapshot) => {
return snapshot;
});
}
// do the same for update, set etc
You can read more on the admin SDK documentation.

nextjs-vercel api routes not working in production and development [duplicate]

I'm new to Next.js and I'm trying to understand the suggested structure and dealing with data between pages or components.
For instance, inside my page home.js, I fetch an internal API called /api/user.js which returns some user data from MongoDB. I am doing this by using fetch() to call the API route from within getServerSideProps(), which passes various props to the page after some calculations.
From my understanding, this is good for SEO, since props get fetched/modified server-side and the page gets them ready to render. But then I read in the Next.js documentation that you should not use fetch() to all an API route in getServerSideProps(). So what am I suppose to do to comply to good practice and good SEO?
The reason I'm not doing the required calculations for home.js in the API route itself is that I need more generic data from this API route, as I will use it in other pages as well.
I also have to consider caching, which client-side is very straightforward using SWR to fetch an internal API, but server-side I'm not yet sure how to achieve it.
home.js:
export default function Page({ prop1, prop2, prop3 }) {
// render etc.
}
export async function getServerSideProps(context) {
const session = await getSession(context)
let data = null
var aArray = [], bArray = [], cArray = []
const { db } = await connectToDatabase()
function shuffle(array) {
var currentIndex = array.length, temporaryValue, randomIndex;
while (0 !== currentIndex) {
randomIndex = Math.floor(Math.random() * currentIndex);
currentIndex -= 1;
temporaryValue = array[currentIndex];
array[currentIndex] = array[randomIndex];
array[randomIndex] = temporaryValue;
}
return array;
}
if (session) {
const hostname = process.env.NEXT_PUBLIC_SITE_URL
const options = { headers: { cookie: context.req.headers.cookie } }
const res = await fetch(`${hostname}/api/user`, options)
const json = await res.json()
if (json.data) { data = json.data }
// do some math with data ...
// connect to MongoDB and do some comparisons, etc.
But then I read in the Next.js documentation that you should not use fetch() to all an API route in getServerSideProps().
You want to use the logic that's in your API route directly in getServerSideProps, rather than calling your internal API. That's because getServerSideProps runs on the server just like the API routes (making a request from the server to the server itself would be pointless). You can read from the filesystem or access a database directly from getServerSideProps. Note that this only applies to calls to internal API routes - it's perfectly fine to call external APIs from getServerSideProps.
From Next.js getServerSideProps documentation:
It can be tempting to reach for an API Route when you want to fetch
data from the server, then call that API route from
getServerSideProps. This is an unnecessary and inefficient approach,
as it will cause an extra request to be made due to both
getServerSideProps and API Routes running on the server.
(...) Instead, directly import the logic used inside your API Route
into getServerSideProps. This could mean calling a CMS, database, or
other API directly from inside getServerSideProps.
(Note that the same applies when using getStaticProps/getStaticPaths methods)
Here's a small refactor example that allows you to have logic from an API route reused in getServerSideProps.
Let's assume you have this simple API route.
// pages/api/user
export default async function handler(req, res) {
// Using a fetch here but could be any async operation to an external source
const response = await fetch(/* external API endpoint */)
const jsonData = await response.json()
res.status(200).json(jsonData)
}
You can extract the fetching logic to a separate function (can still keep it in api/user if you want), which is still usable in the API route.
// pages/api/user
export async function getData() {
const response = await fetch(/* external API endpoint */)
const jsonData = await response.json()
return jsonData
}
export default async function handler(req, res) {
const jsonData = await getData()
res.status(200).json(jsonData)
}
But also allows you to re-use the getData function in getServerSideProps.
// pages/home
import { getData } from './api/user'
//...
export async function getServerSideProps(context) {
const jsonData = await getData()
//...
}
You want to use the logic that's in your API route directly in
getServerSideProps, rather than calling your internal API. That's
because getServerSideProps runs on the server just like the API routes
(making a request from the server to the server itself would be
pointless). You can read from the filesystem or access a database
directly from getServerSideProps
As I admit, what you say is correct but problem still exist. Assume you have your backend written and your api's are secured so fetching out logic from a secured and written backend seems to be annoying and wasting time and energy. Another disadvantage is that by fetching out logic from backend you must rewrite your own code to handle errors and authenticate user's and validate user request's that exist in your written backend. I wonder if it's possible to call api's within nextjs without fetching out logic from middlewars? The answer is positive here is my solution:
npm i node-mocks-http
import httpMocks from "node-mocks-http";
import newsController from "./api/news/newsController";
import logger from "../middlewares/logger";
import dbConnectMid from "../middlewares/dbconnect";
import NewsCard from "../components/newsCard";
export default function Home({ news }) {
return (
<section>
<h2>Latest News</h2>
<NewsCard news={news} />
</section>
);
}
export async function getServerSideProps() {
let req = httpMocks.createRequest();
let res = httpMocks.createResponse();
async function callMids(req, res, index, ...mids) {
index = index || 0;
if (index <= mids.length - 1)
await mids[index](req, res, () => callMids(req, res, ++index, ...mids));
}
await callMids(
req,
res,
null,
dbConnectMid,
logger,
newsController.sendAllNews
);
return {
props: { news: res._getJSONData() },
};
}
important NOTE: don't forget to use await next() instead of next() if you use my code in all of your middlewares or else you get an error.
Another solution: next connect has run method that do something like mycode but personally I had some problems with it; here is its link:
next connet run method to call next api's in serverSideProps
Just try to use useSWR, example below
import useSWR from 'swr'
import React from 'react';
//important to return only result, not Promise
const fetcher = (url) => fetch(url).then((res) => res.json());
const Categories = () => {
//getting data and error
const { data, error } = useSWR('/api/category/getCategories', fetcher)
if (error) return <div>Failed to load</div>
if (!data) return <div>Loading...</div>
if (data){
// {data} is completed, it's ok!
//your code here to make something with {data}
return (
<div>
//something here, example {data.name}
</div>
)
}
}
export default Categories
Please notice, fetch only supports absolute URLs, it's why I don't like to use it.
P.S. According to the docs, you can even use useSWR with SSR.

Firebase Functions: when to call initializeApp/deleteApp

I have a variety of functions for Cloud Functions that use things derived from firebase-admin/app's initializeApp(). Since multiple functions use this, I had the idea of calling initializeApp() close to the beginning of the Typescript/Javascript. Like this:
import { initializeApp } from 'firebase-admin/app';
...
const app = initializeApp();
...
export const activeUser = https.onCall(
async (email: string, ctx: CallableContext) => {
...
const auth = getAuth(app);
const firestore = getFirestore(app);
try {
const user = await auth.getUserByEmail(email);
const snpsht = await firestore.collection('users').doc(user.uid).get();
...
export const userExists = https.onCall(
async (email: string, ctx: CallableContext) => {
if (!emulating() && forbiddenOrigin(ctx)) {
return `${ERRORFLAG}: forbidden`;
}
const auth = getAuth(app);
...
A testing headache showed me that I forgot about calling deleteApp(app).
Is deleteApp() necessary, or will the app be deleted as a side effect of Cloud tearing down the function?
Should initializeApp()/deleteApp() be called within each function where an app is needed, ie within activeUser and userExists in my above example?
Is there any way to tell Cloud Functions to run setup/teardown code before running a given function?
A call to deleteApp is not necessary. When a Cloud Functions instance terminates, all memory gets shut down with it (which is what deleteApp would do). The entire server instance is completely gone.
You only need to call deleteApp if you are controlling a process that needs to continue after the call to deleteApp with the Firebase resources freed from memory. Cloud Functions does not meet this criteria since you don't control the process startup or shutdown.

Firebase Storage with Google Actions

I am having some issues connecting my firebase storage with my google action. I need to be able to "download" the json files inside in order to be able to read and pick out what a user may need given data that they provide when they call the action.
Below is the code that I currently have, complied from the different APIs and other stackoverflow questions I have found.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
const Firestore = require('#google-cloud/firestore');
const firestore = new Firestore();
var storage = require('#google-cloud/storage');
const gcs = storage({projectId: 'aur-healthcare-group'});
const bucket = gcs.bucket('gs://aur-healthcare-group');
admin.storage().bucket().file('aur-healthcare-group/aur_members.json').download(function(errr, contents){
if(!err){
var jsObjext = JSON.parse(contents.toString('utf8'));
}
});
The current error I am receiving is "code":3,"message":"Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause. When I check the logs I only get the above mentioned message again.
I believe that I am not accessing my firebase storage correctly and have trouble finding a good resource on how to access this correctly. Would somebody be able to give me an example of how to access the storage correctly so I will be able to apply it to my project?
Since you're running in Firebase Functions, you shouldn't need to require the #google-cloud/storage dependency directly. Rather, you can get the correctly authenticated storage component via admin.storage()
Following that, you shouldn't download the file to your function, as you would be better off reading directly into memory via a readStream.
With regards to your existing code error, it may be because you're checking if (!err) when the callback variable is errr.
I've done this in the past and here's a code snippet of how I achieved it. It's written in Typescript specifically, but I think you should be able to port it to JS if you're using that directly.
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin'
import { Bucket } from '#google-cloud/storage';
admin.initializeApp()
const db = admin.firestore()
const bucket = admin.storage().bucket('project-id.appspot.com') // Use your project-id here.
const readFile = async (bucket: Bucket, fileName: string) => {
const stream = bucket.file(fileName).createReadStream();
return new Promise((resolve, reject) => {
let buffer = '';
stream.on('data', function(d: string) {
buffer += d;
}).on('end', function() {
resolve(buffer)
});
})
}
app.handle('my-intent-handler', async (conv) => {
const contents = await readArticle(bucket, 'filename.txt')
conv.add(`Your content is ${contents}`)
})
exports.fulfillment = functions.https.onRequest(app)

Firebase: How to run 'HTTPS callable functions' locally using Cloud Functions shell?

I couldn't find a solution for this use case in Firebase official guides.
They are HTTPS callable functions
Want to run Functions locally using Cloud Functions shell to test
Functions save received data to Firestore
The 'auth' context information is also needed
My code as below. Thanks in advance.
Function :
exports.myFunction = functions.https.onCall((data, context) => {
const id = context.auth.uid;
const message = data.message;
admin.firestore()...
// Do something with Firestore //
});
Client call :
const message = { message: 'Hello.' };
firebase.functions().httpsCallable('myFunction')(message)
.then(result => {
// Do something //
})
.catch(error => {
// Error handler //
});
There is an api exactly for this use case, see here.
I used it in javascript(Client side) as follows -
button.addEventListener('click',()=>{
//use locally deployed function
firebase.functions().useFunctionsEmulator('http://localhost:5001');
//get function reference
const sayHello = firebase.functions().httpsCallable('sayHello');
sayHello().then(result=>{
console.log(result.data);
})
})
where sayHello() is the callable firebase function.
When the client is an android emulator/device. Use 10.0.2.2 in place of localhost.
Also the code for flutter would be like so -
CloudFunctions.instance.useFunctionsEmulator(origin: 'http://10.0.2.2:5000')
.getHttpsCallable(functionName: 'sayHello')
Cloud functions have emulators for that. Check this link it can suite your case. Its not functions shell, but for testing purposes i think it can still works for you
In newer versions of firebase, this is the way:
import firebaseApp from './firebaseConfig';
import { getFunctions, httpsCallable, connectFunctionsEmulator } from 'firebase/functions';
const functions = getFunctions(firebaseApp);
export async function post(funcName, params) {
connectFunctionsEmulator(functions, 'localhost', '5001'); // DEBUG
let func = httpsCallable(functions, funcName);
let result = await func(params);
return result.data;
}

Resources