Each dev on our team is setting up a Firebase project to work with 'locally'. Due to the outbound requests restriction on the free tier, the implication here is that non-Google services that are being called from our functions, cannot actually be called to validate the function works as expected.
Right now the best I can come up with is determining which environment a cloud function is running in (e.g. local, master, prod, etc.) and, if not on a paid tier, fake the outbound service response.
Is there a better way to do this? Ideally we would like to be able to have a fully functional cloud function for each dev.
As you said faking the 3rd party service is one solution.
Another could be using Google's 12 month 300$ credit at https://cloud.google.com/free/
That makes it possible for you to have a single Google account with 300$ credits, then you can change your Firebase free plans into Blaze.
If needed, you can also set up budget alerts which notifies you when you spend a certain amount of credit.
Related
As far as I understand, my Google Cloud Functions are globally accessible. If I want to control access to them, I need to implement authorization as a part of the function itself. Say, I could use Bearer token based approach. This would protect the resources behind this function from unauthorized access.
However, since the function is available globally, it can still be DDoS-ed by a bad guy. If the attack is not as strong as Google's defence, my function/service may still be responsive. This is good. However, I don't want to pay for those function calls made by the party I didn't authorize to access the function. (Since the billing is per number of function invocations). That's why it's important for me to know whether Google Cloud Functions detect DDoS attacks and enable counter-measures before I'm being responsible for charges.
I think the question about DDOS protection has been sufficiently answered. Unfortunately the reality is that, DDOS protection or no, it's easy to rack up a lot of charges. I racked up about $30 in charges in 20 minutes and DDOS protection was nowhere in sight. We're still left with "I don't want to pay for those function calls made by the party I didn't authorize to access the function."
So let's talk about realistic mitigation strategies. Google doesn't give you a way to put a hard limit on your spending, but there are various things you can do.
Limit the maximum instances a function can have
When editing your function, you can specify the maximum number of simultaneous instances that it can spawn. Set it to something your users are unlikely to hit, but that won't immediately break the bank if an attacker does. Then...
Set a budget alert
You can create budgets and set alerts in the Billing section of the cloud console. But these alerts come hours late and you might be sleeping or something so don't depend on this too much.
Obfuscate your function names
This is only relevant if your functions are only privately accessed. You can give your functions obfuscated names (maybe hashed) that attackers are unlikely to be able to guess. If your functions are not privately accessed maybe you can...
Set up a Compute Engine instance to act as a relay between users and your cloud functions
Compute instances are fixed-price. Attackers can slow them down but can't make them break your wallet. You can set up rate limiting on the compute instance. Users won't know your obfuscated cloud function names, only the relay will, so no one can attack your cloud functions directly unless they can guess your function names.
Have your cloud functions shut off billing if they get called too much
Every time your function gets called, you can have it increment a counter in Firebase or in a Cloud Storage object. If this counter gets too high, your functions can automatically disable billing to your project.
Google provides an example for how a cloud function can disable billing to a project: https://cloud.google.com/billing/docs/how-to/notify#cap_disable_billing_to_stop_usage
In the example, it disables billing in response to a pub/sub from billing. However the price in these pub/subs is hours behind, so this seems like a poor strategy. Having a counter somewhere would be more effective.
I have sent an email to google-cloud support, regarding cloud functions and whether they were protected against DDoS attacks. I have received this answer from the engineering team (as of 4th of April 2018):
Cloud Functions sits behind the Google Front End which mitigates and absorbs many Layer 4 and below attacks, such as SYN floods, IP fragment floods, port exhaustion, etc.
I have been asking myself the same question recently and stumbled upon this information. To shortly answer your question: Google does still not auto-protect your GCF from massive DDOS-attacks, hence: unless the Google infrastructure crashes from the attack attempts, you will have to pay for all traffic and computing time caused by the attack.
There is certain mechanisms, that you should take a closer look at as I am not sure, whether each of them also applies to GCF:
https://cloud.google.com/files/GCPDDoSprotection-04122016.pdf
https://projectshield.withgoogle.com/public/
UPDATE JULY 2020: There seems to be a dedicated Google service addressing this issue, which is called Google Cloud Armor (Link to Google) as pointed out by morozko.
This is from my own, real-life, experience: THEY DON'T. You have to employ your own combo of rules, origin-detection, etc to protect against this. I've recently been a victim of DDoS and had to take the services down for a while to implement my own security wall.
from reading the docs at https://cloud.google.com/functions/quotas and https://cloud.google.com/functions/pricing it doesn't seem that there's any abuse protection for HTTP functions. you should distinguish between a DDoS attack that will make Google's servers unresponsive and an abuse that some attacker knows the URL of your HTTP function and invokes it millions of times, which in the latter case is only about how much you pay.
DDoS attacks can be mitigated by the Google Cloud Armour which is in the beta stage at the moment
See also related Google insider's short example with GC Security Rules and the corresponding reference docs
I am relatively new to this world, but from my little experience and after some research, it's possible to benefit from Cloudflare's DDOS protection on a function's http endpoint by using rewrites in your firebase.json config file.
In a typical Firebase project, here's how I do this :
Add cloud functions and hosting to the project
Add a custom domain (with Cloudflare DNSs) to the hosting
Add the proper rewrites to your firebase.json
"hosting": {
// ...
// Directs all requests from the page `/bigben` to execute the `bigben` function
"rewrites": [ {
"source": "/bigben",
"function": "bigben",
"region": "us-central1"
} ]
}
Now, the job is on Cloudflare's side
One possible solution could be the API Gateway, where you can use firebase authentication. After successful authentication to the api gw it can call your function that deployed with --no-allow-unauthenticated flag.
However I'm confused if you are charged for unauthenticated requests to api gw too..
I'm currently working in a social network app and I need to do a search feature. Firestore does not support these kind of queries, so I need to use an external service like Algolia.
The problem is that the free plan does not support connecting to external websites/APIs other than Google's own ones, so I can't connect to Algolia to get my search system working.
I have read multiple stories about devs paying high bills because of loops or errors in their code, and as the Blaze plan is a pay-to-go plan, they get charged what they used. If a loop generated 10TB of files they will get charged for that.
I also know that Blaze plan's features are free as long as each of them (individually) stay below the limits of the free Spark plan.
So as my question says, is there a way to set limits? For example, I would like to tell Firebase to limit my cloud functions invocations to 100k per month. That way it would be free and I would never be able to get over 100k as it's limited, which means I'll never get billed for that.
Take into account that the only thing I need right now from a paid plan is the connection to external networks. I don't need anything else as we're just starting and the app is not in production, so there's no need for huge limits.
Every Firebase project is also a Google Cloud Platform project. This means that many of the advanced features of Google Cloud Platform are also available for your Firebase project.
For example, you can set up billing alert for your Firebase project, so that you are alerted when the usage reaches a certain level. While you can't configure it to switch off the project at some point, the alert should typically be quite good for alerting you to unusual usage patterns.
For more on this see:
Tracking your spending with budgets in a recent blog post.
The GCP documentation on how to set budget alerts, which is what Firebase uses under the hood.
The GCP documentation now also has a section on capping (disabling) billing to stop usage. This is a brute force approach though and may lead to data being lost, so I'd recommend investigating all other options first.
Update (December 2020): Firebase's Todd Kerpelman just released a series of videos where he disables billing using the process from the documentation mentioned above.
You cannot set spending limits to your app now.
As of December 12, 2019, you can no longer create spending limits, but
you can change or remove existing spending limits.
https://cloud.google.com/appengine/pricing#spending_limit
You can create budgets, which will alert you when reaching the budget. But it won't stop the usage when hitting the budget.
https://cloud.google.com/billing/docs/how-to/budgets#add-new-budget
The screenshot here seems to show a Spending Limit setting for Firebase projects: Firebase: Budget and Daily Spending Limit
That settings page is located here (the Spending Limit setting apparently only shows up once you set up billing for the project): https://console.cloud.google.com/appengine/settings
It's disabled in the poster's case, but I think that's only because he connected it up to a "NodeJS App Engine app", which isn't the case for many Firebase developers.
I haven't tried it yet myself, but will do so once I start a paid plan.
EDIT: Yep, the setting shows up once you switch to a paid plan. (in my case, Blaze) I don't have enough traffic yet to confirm that it works as expected, but if I find later that it doesn't, I'll give an update here.
"This example shows you how to cap costs and stops usage for a project by disabling Cloud Billing. This will cause all Google Cloud services to terminate non-free tier services for the project."
Google Cloud Source
I have created a free trial account on smooch that gives me access to their WeChat API in the sandbox. However, it is still not clear to me what happens after the 14 day trial period is over. Will I be forced to upgrade to a paid plan even for using the sandbox or is the sandbox always free. Note: My intended use is to develop and test our app against their API and hence I will always need API access in their sandbox. Our production usage will be on a different account with the relevant paid plan. Will really appreciate someone sharing their knowledge around how it works.
The trial period applies only to usage of the Smooch Public REST API and Webhooks, so if you intend to receive and reply to messages programmatically, then the trial will not be sufficient for testing or development beyond the 14 day period. If you intend to receive and respond to messages using one of the built-in Business System integrations like Slack or Zendesk, you can continue to use those integrations after the trial period expires.
If you have a paid account already, you could also create a separate Smooch App for testing under that same account and take advantage of the API access afforded by your plan.
I'm working with Firebase and quite enjoying it so far.
I'm working with DEV, PREPROD and PROD environments for each of my projects. For each env I've had to create a distinct firebase project.
Since my app is using Algolia and Cloud vision API, I apparently have to be on the Blaze plan because Spark plan doesn't allow outbound requests and Cloud vision API calls (if I'm correct).
The thing is we're limited with the numbers of Blaze projects we can have at the same time. Above a certain amount (6 or 7, I think) we have to request a "billing quota increase" and explain why we need more (sounds odd but ok).
So I did, but now Firebase is asking for a $50 transaction to increase the number of Blaze projects I can have.
So I have several questions:
- Am I right to think that in Spark plan I can't call the Algolia API in my cloud functions or call Cloud vision API ?
- Are these $50 a payment to unlock new projects slots or just credits that will be available if needed ?
- If I need even more projects in the future will I have to pay even more credits ?
- How am I supposed to handle separate environments on Firebase without creating a different project each time ?
Thanks a lot
On the Spark plan, with Cloud Functions, you can only make outgoing connections to services that Google fully controls. Algolia will not work.
Please read the FAQ regarding the number of projects you may have and the payment being asked to create a new project:
Why am I being asked to make a payment for more projects?
You may be asked to make a payment if your request for more projects
indicates that you need projects that will use paid cloud services.
The payment can be applied to any charges you incur in the future and
will be visible as a credit in your account.
This payment is required to ensure paid services will be available for
the projects you requested in the quota increase request form. This is
a common requirement, because Google Cloud Platform services are paid
(e.g., Compute Engine, Cloud SQL, and BigQuery).
The payment required varies depending on your billing history, the use
cases described in your request form, the number of projects you
request, and other factors.
So, the $50 you are being asked to pay will apply as credit to your project billing.
You should definitely create new projects for each environment.
What costs are involved with developing and/or releasing a Google Assistant App?
eg: Can you develop an app using DialogFlow and a backend (say Firebase) without having to pay while you learn?
First of all - you don't need to use Dialogflow or Firebase to develop your action. Both are suggested, but neither are required. You can use any NLP you want, or none at all if you use the Actions SDK (but you want an NLP). You can use any backend at all, including running it off your local machine and tunneling to it via ngrok, but you don't want to do that for production.
But, during development (and even during a light deployment before your action becomes massively popular and a stand-out hit), you have lots of solutions that will be free.
Dialogflow is free for use with the Google Assistant. Period. There is an Enterprise edition which offers additional services and support for a cost, but you won't need them. There are restrictions, but you won't bump into them until you hit 3 requests per second - which you shouldn't during development.
Firebase's free tier (the Spark Plan) is good for very simple experimentation, but once you start doing network calls to outside Google's network (if you are trying to call the network API for other services), you will be blocked. No worries! The "Blaze Plan" paid tier does require a way to bill you, but they don't start billing you until you get quite a bit of usage: 2 million function calls / month and similarly scaled usage of CPU, memory, and network. So even the "Blaze Plan" will be free during development (and for basic usage).
Updated, December 2020
Things have changed a bit since the original answer was posted, but the underlying basics remain true - there is no charge to develop for Actions on Google.
Dialogflow now has an "Essentials" edition and a more advanced "CX" edition. While you can still use both to build Actions, they're not really intended for this purpose anymore.
Instead, Google has included the Actions Builder into the Actions Console to handle the NLP work. The Actions SDK works with this, but can also just pass along all the STT information to your webhook. Both are also free to use.
Dialogflow is for free if you don't use it as an enterprise:
https://dialogflow.com/pricing/
And Firebase free tier should be enough if you not using firebase
already for other projects. enter link description here
But of course you have to calculate your own time so in case of the
spent time probably not.
For everything else yes it is, as long as you not using it already somewhere. You can for development also host your server local and use an ngrok tunnel as sever address for Dialogflow.
As an addition to shortQuestion's answer:
The free plan in Firebase should be enough if you're just using it for learning and developing apps for personal use. If you want to go a bit further you'll need to upgrade the plan.
You can sign up with a free trial for actions on Google to get 300$ of credits during a 12 month period which would be more than enough to do anything you want.
The costs of Firebase/Actions on Google on a higher plan aren't anything to worry about though, you'd be talking about a few cent per multiple hundred thousand requests.