Deploying to Firebase Functions with a monorepo - firebase

As per Firebase Functions with Yarn workspaces, I'd like to deploy a monorepo to Firebase using Yarn Workspaces. I can successfully deploy the "web" aspect to Firebase Hosting, which references a shared "core" workspace. However, attempts to do the same with the "functions" workspace on Firebase Functions fails.
I'm using Yarn Workspaces with the following folder structure:
packages/
core/ // name: firebase-monorepo-core: custom core package
functions/ // name: firebase-monorepo-functions: firebase functions package
web/ // name: firebase-monorepo-web: react package
These have been configured in the root package.json file:
"workspaces": {
"packages": [
"packages/*"
],
"nohoist": [
"**/firebase-monorepo-core"
]
}
In an attempt to enable Yarn workspaces in Firebase (and therefore share my core package) I've used the nohoist feature to create symlinks to the core workspace in functions and web as per twiz's Stackoverflow answer.
The core package also exists as a dependency in functions and web:
"dependencies": {
"firebase-monorepo-core": "*"
}
There are no problems when any of this runs locally, and in fact deployment of the web package to Firebase hosting works fine. However, deployment of the functions package to Firebase functions throws an error:
I've created a Github repository https://github.com/cjmyles/firebase-monorepo to demonstrate this, and the resultant web workspace can be viewed at https://fir-monorepo.firebaseapp.com.
I can get the functions package to deploy if I yarn pack the core workspace and reference it as a tarball in the package.json file, but I really don't like that solution and it will require more overhead to automate it.
Any advice would be appreciated to resolve this issue.

The issue is that firebase-tools assumes that all packages used by your function are available in npm.
A quick solution is to use firelink.
cd ./packages/functions
yarn add -D #rxdi/firelink
Add an equivalent section to your functions package.json for all shared dependencies.
"fireDependencies": {
"#org/common": "../../packages/common"
},
Change your deploy script to use firelink binary
"scripts": {
"build": "tsc --build tsconfig.build.json",
"predeploy": "yarn build",
"deploy:functions": "firelink deploy --only functions"
}
Just a heads up does not work with "devDependencies", but you should not be needing them in production anyway.
Update your firebase.json to run the predeploy target.
"functions": {
"predeploy": ["npm --prefix \"$RESOURCE_DIR\" run predeploy"],
"source": "packages/functions"
},
Lastly remove any local yarn.lock files in the workspace package. If you wish to lock the dependencies add npm install to your predeploy step. If anyone has a better solution to this let me know. It seem unfortunate that yarn does not have the ability to generate lock files on demand in a workspace.

one potential solution is to use Lerna.
You can then publish your core package to a private GitHub package and use Lerna to create a symlink to the core package for local development.

I got this to work but it was a pain and hacky. So far it's working well though. I'm using NPM Workspaces, but the concept should apply to any project where node modules are hoisted to the root folder. I'm also using GitHub Workflows / Actions for CI and TypeScript, so all of this works within those environments.
Build my dependent packages using TypeScript's tsc
Copy the compiled files into the functions folder (functions/src/domain to be exact) so that they get uploaded by Firebase CLI
cp -r packages/domain firebase/functions/src/domain
Insert local dependency inside functions/package.json. I'm doing it with sed, which GitHub actions supports:
sed -i 's/"dependencies": {/"dependencies": { "#mycompany\/domain": "file:src\/domain",/' package.json
Before that command, my package.json looks like this:
"dependencies": {
"lodash": "^4.17.20",
...
After that command, it looks like this:
"dependencies": { "#mycompany/domain": "file:src/domain",
"lodash": "^4.17.20",
...
Then deploy: firebase deploy --only functions
Not pretty, but it works with full automation. 😁

Related

"firebase use" - projects out of sync between command line and when invoked from npm

I have been doing firebase functions development for a while now and was used to be able to switch the target project from the command line via firebase use test, firebase use staging etc.
In my npm package scripts in the project I also invoke the firebase tools to do things like generating a .runtimeconfig.json via:
"scripts": {
"genruntime" : "firebase functions:config:get > .runtimeconfig.json",
}
In general, this worked fine - I'd change the target project from the command line, and the same target project would be used when I ran firebase commands by npm scripts.
Over the last few days though, I've found that sometimes the target projects weren't synced as I'd set the target environment to test but the find that the npm command was getting functions from the staging environment.
My firebase tools are the same version (global via project) and it's almost as if the target project is not being shared between the global firebase tools and the the npm ones.
Has anyone else seen this issue?
I can change my npm scripts to be explicit about the project being used but this has worked in the past and I'm curious about what might have happened.

Lerna on AWS CodeBuild & CodeDeploy Not Installing Local Dependencies

So I have a Typescript project called backend with a depedency like such in its package.json:
"dependencies": {
"#the-couple-game/helpers": "^1.0.0",
}
And helpers (also Typescript) is in another folder with its package.json like such:
{
"name": "#the-couple-game/helpers",
}
So running lerna bootstrap should link the two and install #the-couple-game/helpers in backend's node_modules which it does locally.
However, doing the same (with --no-ci because I don't want npm ci) using Codebuild using the below buildspec.yml does not add a #the-couple-game/helpers in backend's node_modules. So if I run backend's transpiled index.js it would complain about the missing module.
version: 0.1
phases:
install:
commands:
- npm install -g lerna
pre_build:
commands:
- lerna bootstrap --no-ci --concurrency 4
build:
commands:
- lerna run build --concurrency 4
artifacts:
files:
- "**/*"
For now, I have to resort to manually doing a lerna bootstrap after deployment to CodeDeploy (using a script called from appspec.yml) so it installs the missing module but shouldn't Codebuild have covered that part?
Thanks.
So it turns out AWS CodePipeline doesn't support symlinks in source (Codebuild in my case). Reference
So after deployment, I'll have to lerna link using a shell script to create the links which is not an expensive job for my micro ec2 instance to do.

Error: Functions did not deploy properly

Functions deploy had errors. To continue deploying other features (such as database), run:
firebase deploy --except functions
Error:
Functions did not deploy properly.
Try running deploy with --debug. For example: firebase --debug deploy.
In my case the function looked like this:
exports.test = functions.firestore.document('chats').onCreate((snap, context) => {
...
});
The debug log contained the following error message:
fieldViolations=[field=event_trigger, description=Expected value chats to match regular expression [^/]+/[^/]+(/[^/]+/[^/]+)*]]
And that's when I realized that chats is not a document but a collection. I changed the path to that of the collection and everything worked fine.
EDIT:
To view more logs you can also try:
firebase functions:log
or
firebase functions:log --only <FUNCTION_NAME>
Link to documentation.
I had the same issue with Firestore ,my problem was that the path to the document was bad. I had a slash at the beginning and the end of the path to my document like I used to do with Firebase Realtime Database path.
Hope it helps someone.
Just close whatever IDE you're using and then reopen it. Then on node redeploy your functions. This should resolve the issue.
I got this error as well, I had the problem that a dependency was missing in the package.json file.
By running
firebase --debug deploy it returned an error on the user code.
firebase functions:log gave then the specifics, that a package was not included.
I deleted yarn.lock then error goes away. I think package-lock.json and yarn.lock should not exist together.
Check your json modules dependencies inside functions folder.
There's an specific json package there.
If some module you are including in the project and is not there (in json) you probably will have the error mentioned.
To install just run:
npm install <module> --save
or
yarn add <module> --save
and then run your deploy again:
firebase deploy --only functions
In my case firebase deploy failed with the error "Error: Functions did not deploy properly." and no further explanation. I noticed that no functions could be deployed even the ones I didn't touch.
So I checked for dependencies and I found that a library was not present in the "package.json".
So I execute this command:
cd functions && rm -rf node_modules && rm package-lock.json && npm install && npm install [specific Librairy not in package.json] && firebase deploy
After that the deployment succeeded and the missing dependency appeared in the package.json
Recently Firebase lets you use Functions only if you have the Blaze-Pay as you go plan. So you cannot deploy functions as well, and you will get this same error message without any further clarifications.
Mine was about;
"Function deployment failed because the billing account is not available."
So I had pending payment, after paying, it worked as expected.
I was running into the same generic error message, and ended up running (as mentioned in this question):
firebase --debug deploy --only functions:[functionName]
This revealed the actual issue (my project did not have a default GCP resource location, fix for this described here: Firebase Project Initialization Error: Cloud resource location is not set for this project)
Once I set the GCP resource location, function deployed without issue!
In my case, After a long time of searching, I found the reason to be that I had disabled billing for the project for debugging.
It worked fine now after I enabled it in the Google cloud console.
It's really sad how firebase can't show any useful error message to suggest the problem. I found this after many trials.
I faced a similar problem, but I was using Typescript.
It was basically targeting a non-supported javascript version (ES2020). I changed the tsconfig.json and it deployed normally
"lib": [
"es2019",
"es2020.bigint",
"es2020.string",
"es2020.symbol.wellknown"
],
"module": "commonjs",
"target": "es2019"
After getting the above deploy error, I edited the index.js file and changed the name of the function from "helloWorld" to "heyWorld" and suddenly the firebase deploy command worked.
If anyone gets here trying to deploy ssr functions in angular universal with firebase, it is because firebase is not included in your functions.
1.) cd functions
2.) npm i firebase#latest
3.) npm run deploy
4.) cd ..
5.) now when you run ng deploy it should work correctly... The key here is to add firebase, and deploy with npm run deploy first...
try to add this
"runtime": "nodejs12"
in your firebase,json
example :
"functions": {
"source": ".",
"ignore": [
".firebase/**",
"**/node_modules/**",
"**/public/**",
".firebaserc",
"firebase.json",
"postcss.config.js",
"README.md",
"tailwind.config.js"
],
"runtime": "nodejs12"
}
was with the same error, I ended up solving it by updating package.json
from:
"engines": {
"node": "12"
},
to:
"engines": {
"node": "14"
},
$ in path causes Error: Functions did not deploy properly.
before:
`export const newMessage = functions.firestore.document('chats/${chatId}/messages/${messageId}').onCreate(handleNewMessage())`
after:
`export const newMessage = functions.firestore.document('chats/{chatId}/messages/{messageId}').onCreate(handleNewMessage())`
In my case, the location for the Firestore was not correctly set. From the Functions Documentation:
By default, functions run in the us-central1 region.
In my case, Firestore was set up on a different location. You can find this under Project Settings, at Default GCP resource location.
To fix it, simply make the following change:
// Before
functions.firestore.document("...").onCreate(...)
// After
functions.region("asia-east2").firestore.document("...").onCreate(...)
Follow the steps below:
Deploy with debug enabled:
firebase --debug deploy
You will see the error printed in your console.
In most cases it is something similar to the line below:
npm install before continuing.\nnpm ERR! \nnpm ERR! \nnpm ERR! Invalid: lock file's body-parser#1.19.0 does not satisfy body-parser#~1.0.1\nnpm ERR!
If the error is similar to "lock file's body-parse" in 1) above:
Go to the views directory in your project and edit the dependencies section in your package.json file as below:
Change the body-parser version to the latest version shown in the error printed in 1) above. In my case it was from body-parser#~1.0.1 to body-parser#~1.19.0
Run: npm install
Re-deploy with: firebase deploy
This should work.
If you get a 403 error from your hosting URL after re-deploying, your CLI version could be out of date:
Login to you google cloud account and go your the google cloud console.
Delete your Cloud Functions
Update Firebase CLI using: npm install -g firebase-tools
Re-deploy: firebase deploy
My firebase couldnt find the modules firebase-functions and firebase-admin on deploy because I wrote them in devDependencies instead of just dependencies. But after it ... I still had an issue. But this solved it for me: https://github.com/nuxt/nuxt.js/issues/2714#issuecomment-423048343.
Just notice node 8 as engine object in your function's folder package.json and add the babel snippet in your nuxt config.
I had same problem with Firebase project deployment. Then figure out error and find that some module paths are cause problem. Then i changed the path of that module.
To try redeploying those functions, run:
firebase deploy --only functions:api,functions:createScream
To continue deploying other features (such as database), run:
firebase deploy --except functions
Error: Functions did not deploy properly.
The error shows that could not find modules.
Did you list all required modules in the package.json dependencies?
Detailed stack trace: Error: Cannot find module (../key/your secret key.json) '
Then i changed the path of my secret key that is required for login credentials and put it in different folder. Then run firebase deploy.
That work for me.
I had the same problem. The problem came from the fact that I had an index.js file and package.json file in the root folder and, after running the firebase functions init command, also an index.js and package.json in the functions folder. The dependencies are only listed in the package.json file in the root folder, not in the functions folder. I copied the dependencies to the package.json file in the functions folders and everything worked like a charm.
Deleting the build functions folder worked to me
firebase functions:log
This allowed me to find out the problem in my case: one of the packages was not listed among the dependencies in package.json
This worked for me today: Delete Function and Redeploy.
It happened the same to me but unfortunately I had not any error with --debug.
I then deleted all my deployed functions in the firebase console and
cd functions
rm -rf node_modules
npm install
firebase deploy --only functions
and it worked like magic.
I got the same error
Functions were working in emulator, but couldn't be deployed.
When I deployed functions using debug, I got no more specific info.
Then I found out, that my IDE accidentally imported lib (that I wasn't even using) from parent folder where is my Svelte project located.
Removing that import fixed my error, obviously.
I deleted the recently added 3rd party library (from package/package-lock.json as well). And checked the name of the function to not be a duplicate.
This worked for me.
If u are using SSR you need declare your node version In my case my node version is 14 and firebase was deploying with 10.0 I set this config on angular.json
"functionsNodeVersion": 14
"serve-ssr": {
"builder": "#nguniversal/builders:ssr-dev-server",
"options": {
"browserTarget": "Rioja-web:build",
"serverTarget": "Rioja-web:server"
},
"configurations": {
"production": {
"browserTarget": "Rioja-web:build:production",
"serverTarget": "Rioja-web:server:production"
}
}
},
"prerender": {
"builder": "#nguniversal/builders:prerender",
"options": {
"browserTarget": "Rioja-web:build:production",
"serverTarget": "Rioja-web:server:production",
"routes": [
"/"
]
},
"configurations": {
"production": {}
}
},
"deploy": {
"builder": "#angular/fire:deploy",
"options": {
"ssr": true,
"functionsNodeVersion": 14
}
}
}
I had the same problem.
updating node engine to 16 in package.json solved the issue.
My nodejs local version was 16.
{
...
"engines": {
"node": "16"
},
"main": "lib/index.js",
"dependencies": { ... }
}

How to install NPM package from GitHub for Meteor?

I have forked an npm package on GitHub, made changes to it, and now want to install the package in my Meteor app directly from GitHub.
My package.json looks like so:
{
"dependencies": {
"semantic-ui-react": "git+https://git#github.com/nolandg/Semantic-UI-React.git",
}
}
I then run
meteor npm install
Which appears to work fine and tells me it's installed the package:
semantic-ui-react#0.61.6 (git+https://git#github.com/nolandg/Semantic-UI-React.git#f27d5c736e5de1eed0acf7769f18caee57578526)
And indeed the package appears in the node_modules folder. But when I try to start my Meteor app, I get this error:
Cannot set property '/my-website/node_modules/semantic-ui-react/package.json' of undefined
at Resolver._resolvePkgJsonMain (/tools/isobuild/resolver.js:320:9)
Has anyone successfully install an npm package in a Meteor app directly from GitHub? I can't figure this one out. Thanks!
Meteor version: 1.4.2.3
The main reason why the package does not work when fetching from git is because it is not configured to work that way. This is not a Meteor specific problem, but a problem that a JS developer may face sometimes.
For this particular case there are two problems:
The whitelist files field in package.json only contains src and dist folder. That means when you fetch it by npm almost all config files needed to build the code are gone.
Code for this package requies to be built in order to work with your code. This is done when the author publish it to npm, but you fetch it directly from github so this step is undone.
Because you already folked and modified the package, so let modify the package.json as below (remove all the comments I added them to give you some explanation), push it to github, and fetch it again by npm:
// remove the "files" field
// ...
"scripts": {
// this script is used to build the package
"postinstall": "postinstall-build dist \"npm run build:commonjs\""
// ...
},
"dependencies": {
// this helps build the package
"postinstall-build": "^2.1.3"
},
// ...
Packages are not usually installed from github, they are published, which means that many versions of a package are available, you can choose which one you get. I'm not sure if what you are doing is possible, but it's certainly inadvisable.
If you want to make changes to a github package, you can download the it to your local machine and do npm link, so that it uses your local package instead of the one on npm. Read more about it at https://docs.npmjs.com/cli/link
Why do you not use simple command?
meteor npm install https://github.com/nolandg/Semantic-UI-React.git
I did:
meteor create test
cd test
meteor npm install
meteor add react react-dom
meteor npm install https://github.com/nolandg/Semantic-UI-React.git
meteor
And no errors (-:

Ignore "Local NPM module not found" warning in Grunt

I'm using matchdep to read dependencies from my package.json file into grunt.
require('matchdep').filterAll('grunt-*').forEach(grunt.loadNpmTasks);
I have my dependencies split between dependencies (for everyone) and devDependencies (for front-end developers.)
Our back-end devs will run the following to get a build of the static assets without requiring jasmine, phantomJS, etc (things that will be run by front-end devs and the CI server)
$ npm install --production
$ grunt build
However, when using the --production build, grunt.loadNpmTasks() will emit a warning for any missing packages.
>> Local Npm module "grunt-contrib-watch" not found. Is it installed?
Is there a way to supress this warning?
You have to question why your "back-end devs" would have to actually build your package - put otherwise, why do they need grunt but NOT devDependencies. This is kind of backwards (requiring users to build your package is certainly an anti-pattern).
That being said, using matchdep, you can / should use:
require('matchdep').filter inside your "production" target
require('matchdep').filterAll inside your "development" target
Certainly, that would require you to specialize your grunt build (eg: have grunt builddev and grunt buildproduction - or maybe use environment variables) - but again, see above...
You can use CLI flags to pass options into grunt. For consistency, I am using a --production flag, just as I do with npm.
So, from the CLI:
$ grunt build --production
And then in the Gruntfile:
var dependencies;
// test for the production flag
if (grunt.option('production')) {
// scan dependencies but ignore dev
dependencies = require('matchdep').filter('grunt-*');
} else {
// scan all dependencies
dependencies = require('matchdep').filterAll('grunt-*');
}
// load only relevant dependencies
dependencies.forEach(grunt.loadNpmTasks);
This is done at the top of the module before any custom tasks are registered.

Resources