Problem:
We are using Https Functions that are taking longer than 60 sec to respond (this is a necessary requirement for our use case and cannot be replaced with async execution). We configured the functions timeout, such that they succeed after e.g. 75 sec, as indicated by direct execution from Chrome or Postman through e.g. https://us-central1-<project-id>.cloudfunctions.net/test.
We have also configured Hosting proxy to rewrite all requests for /api to these functions. However, when submitting identical requests through these "hosting" endpoints (e.g. https://<project-id>.firebaseapp.com/api/test), we receive a 504 (Gateway timeout) error after only 60 sec, both in Chrome and Postman. This suggests that the Hosting proxy itself times out before a function succeeds (which still happens, as indicated in Firebase Console log).
Question:
Is it possible to set Hosting timeout to a higher value (e.g. 120 sec)?
Possible solutions/workarounds:
tried to set Connection: keep-alive and Keep-Alive: timeout=120 headers in client requests, however that didn't seem to have any effect on the Hosting proxy; or maybe we're not doing it right.
the only workaround for now is to not use rewrite rules, instead relying on the "direct" URLs (from cloudfunctions.net). However, that entails dealing with CORS in browser clients and more importantly, changing our URL resolution scheme and organization of functions' code (e.g. we cannot use a long path for each function, such as /api/some/path/to/test, because only the first part of that path, such as api, will be considered as the function name).
Thanks!
It is possible to write your function in such a way as to maintain a persistent connection:
https://firebase.google.com/docs/functions/networking#https_requests
const http = require('http');
const functions = require('firebase-functions');
const agent = new http.Agent({keepAlive: true});
exports.function = functions.https.onRequest((request, response) => {
req = http.request({
host: '',
port: 80,
path: '',
method: 'GET',
agent: agent,
}, res => {
let rawData = '';
res.setEncoding('utf8');
res.on('data', chunk => { rawData += chunk; });
res.on('end', () => {
response.status(200).send(`Data: ${rawData}`);
});
});
req.on('error', e => {
response.status(500).send(`Error: ${e.message}`);
});
req.end();
});
Just ran into this very same problem with a Cloud Scheduler Job that hits my API endpoint (which is connected via Firebase Hosting). It takes around 80 seconds to complete.
But it seems that there is no workaround to this. At least not while using Firebase Hosting. Maybe you can hit the functions' URL directly instead of doing it through Firebase Hosting. But if those request are coming from the browser, you'd have to deal with configuring CORS.
From: https://firebase.google.com/docs/hosting/functions
Text version:
Note: Firebase Hosting is subject to a 60-second request timeout. Even if you configure your HTTPS function with a longer request timeout, you'll still receive an HTTPS status code 504 (request timeout) if your function requires more than 60 seconds to run. To support dynamic content that requires longer compute time, consider using an App Engine flexible environment.
Related
I know this question has been asked a bunch of times, but I tried most of the answers and still can't get it to work.
I have a Golang API with net/http package and a JS frontend. I have a function
func SetCookie(w *http.ResponseWriter, email string) string {
val := uuid.NewString()
http.SetCookie(*w, &http.Cookie{
Name: "goCookie",
Value: val,
Path: "/",
})
return val
}
This function is called when the user logs in, and I expect it to be sent to all the other endpoints. This works as expected with Postman. However, when it comes to the browser, I can't seem to get it to remember the cookie or even send it to other endpoints.
An example of JS using an endpoint
async function getDataWithQuery(query, schema){
let raw = `{"query":"${query}", "schema":"${schema}"}`;
let requestOptions = {
method: 'POST',
body: raw,
redirect: 'follow',
};
try{
let dataJson = await fetch("http://localhost:8080/query/", requestOptions)
data = await dataJson.json();
}catch(error){
console.log(error);
}
return data;
}
I tried answers like setting SameSite attribute in Golang, or using credential: "include" in JS with no luck.
Thanks to the discussion in the comments, I found some hints about the problem.
Saving cookies (both API and frontend on the same host)
I used document.cookie to save the cookie. I set the options by hand since calling res.cookie on the response of the API fetch only returned the value. An example is document.cookie = `goCookie=${res.cookie}; path=/; domain=localhost;.
Sending cookies
This has been answered before in previous questions and answered again in the comments. The problem was that I used credential:'include' instead of the correct credentials:'include' (plural).
CORS and cookies
In case the API and the frontend are not on the same host you will have to modify both the API and the frontend.
frontend
The cookie has to have the domain of the API since it's the API that requires it, not the frontend. So, for security reasons, you can't set a cookie for a domain (API) from another domain (frontend). A solution would be redirect the user to an API endpoint that returns Set-Cookie header in the response header. This solution signals the browser to register that cookie with the domain attached to it (the API's domain, since the API sent it).
Also, you still need to include credentials:'include' in the frontend.
API
You will need to set a few headers. The ones I set are
w.Header().Set("Access-Control-Allow-Origin", frontendOrigin)
w.Header().Set("Access-Control-Allow-Credentials", "true")
w.Header().Set("Access-Control-Allow-Headers", "Content-Type, withCredentials")
w.Header().Set("Access-Control-Allow-Methods", method) // use the endpoint's method: POST, GET, OPTIONS
You need to expose the endpoint where the frontend will redirect the user and set the cookie in the response. Instead of setting the domain of the API by hand, you can omit it, the browser will fill it with the domain automatically.
To handle the CORS and let JS send the cookie successfully, you will have to set the SameSite=None and Secure attributes in the cookie and serve the API over https (I used ngrok to make it simple).
Like so
func SetCookie(w *http.ResponseWriter, email string) string {
val := uuid.NewString()
http.SetCookie(*w, &http.Cookie{
Name: "goCookie",
Value: val,
SameSite: http.SameSiteNoneMode,
Secure: true,
Path: "/",
})
// rest of the code
}
I recommend you also read the difference between using localStorage and document.cookie, it was one of the problems I had.
Hope this helps.
I have a Cloud Function that calls to Chargebee. In index.ts:
const chargeBee = new ChargeBee();
...
chargeBee.configure({
site,
api_key: apiKey
});
...
export const finalizeSignup = https.onCall(
async (info: SignupInfo, ctx: CallableContext) => {
const cbCmd = chargeBee.hosted_page.retrieve(info.cbHostedPage);
const callbackResolver = new Promise<any>((resolve, reject) => {
// cbCmd.request returns a Promise that seems to do nothing.
// The callback works, however.
// Resolve/reject the Promise with the callback.
void cbCmd.request((err: any, res: any) => {
if (err) {
reject(err);
}
resolve(res);
});
});
// Calling Promise.resolve subscribes to the Promise.
return Promise.resolve(callbackResolver);
}
);
I am testing this function using the Firebase emulators, started via firebase emulators:start --only functions. Chargebee is responding strangely. They require the domain of their incoming requests to be whitelisted: my first guess is that the domain being used by my locally emulated Cloud Function is not whitelisted on the Chargebee side.
How do I see outgoing HTTP information sent by my locally emulated Cloud Function?
The connection is actually HTTPS, not HTTP.
The emulators provide no functionality to intercept network traffic of any form.
For HTTP: you have to apply your own tooling to monitor the HTTP traffic (ie Wireshark).
For HTTPS: possible to monitor using Wireshark, but impossible to analyze without knowing the SSL key. And in the setup above, where a third-party library is handling the request, there is currently no way to obtain the SSL key. I entered a feature request with Firebase to gauge the interest of developing a way to define an SSL key log when starting the Functions emulator, similar to Chrome. A user only identifying themselves as 'Oscar' told me in a private email that "I've already filed a feature regarding this topic to our engineering team regarding this matter, which will be discussed internally." So that tells us that (1) Firebase is aware that the feature is currently lacking, and (2) there is no progress to report on the feature.
I'm using Meteor v1.9 Webapp API to have my app listen to HTTP requests, specifically from a link to the app itself from a website, let's say example.org.
The documentation says to use
WebApp.connectHandlers.use([path], handler)
Where the [path] is defined as such:
path - an optional path field. This handler will only be called on
paths that match this string. The match has to border on a / or a ..
For example, /hello will match /hello/world and /hello.world, but not
/hello_world.
My question:
Let's say my meteor application is hosted on abc.com and the POST data being sent over to it is from example.org (where the link to abc.com is as well).
For the [path] argument mentioned above, in this case, should I have it as "/example" since example.org is where my app is listening to requests from (getting the POST data)? Or does it have to be a different format? I tried the former, but it doesn't seem to be working for it, so I'm trying to find the root of the issue.
Additional information that might be useful: I know it says 'optional' so I tried omitting it, but when I tested it out via 'meteor run' where it runs off of localhost:3000, it just yielded a blank page, with no errors and a success sent back, probably because it uses a GET request instead of POST.
My code for the webapp in my meteor application is as follows:
WebApp.connectHandlers.use("/[example]", async (req, res, next) => {
userName = req.body;
res.writeHead(200);
res.end();
});
Also technically my meteor application is built/bundled and deployed as a Node.js application on the website, but that shouldn't affect anything regarding this as far as I could tell.
That path is the path (part of the URL) on your meteor server. So in your example, for instance,
WebApp.connectHandlers.use("/example", async (req, res, next) => {
userName = req.body;
res.writeHead(200);
res.end();
});
means that you will need to send your POST requests to abc.com/example.
I am using functions and hosting from firebase.
I have defined one function as shown below.
const functions = require("firebase-functions")
const cors = require('cors')
exports.hello = functions.https.onRequest((request, response) => {
functions.logger.info("Hello logs!", {structuredData: true})
return cors()(request, response, () => {
response.send({data: 'hello fire functions'})
})
})
And in hosting call the function like this:
import firebase from "firebase/app"
import "firebase/functions"
const config = { ... }
firebase.initializeApp( config )
const test = firebase.functions().httpsCallable('hello')
test().then( result => console.log(result) )
Then the functions log will be written twice as follows:
2:37:07.548 PM hello: Function execution started
2:37:07.599 PM hello: Hello logs!
2:37:07.600 PM hello: Function execution took 53 ms, finished with status code: 204
2:37:07.809 PM hello: Function execution started
2:37:07.816 PM hello: Hello logs!
2:37:07.817 PM hello: Function execution took 8 ms, finished with status code: 200
It is also displayed twice in the usage graph.
This behavior means I have to pay twice as much usage. This is not normal.
If cors is not used, the log and usage graph will show that it has been executed only once.
But if you don't use cors: When you call a function in the browser, the function is executed, but the browser gets a CORS error.
How can I solve this problem? I couldn't find a solution in the official documentation. (This is a problem after hosting and functions deploying. It is not a localhost environment.)
Firstly, you are mixing up HTTP requests on the client with callable functions. That's not what you're supposed to do. Please review the documentation for both HTTP functions and callable functions to see how they are different. If you're using the callable SDK on the client, you should use a callable function on the backend.
Second, this is the normal expected behavior. Callable functions use CORS between the client and server. CORS clients issue a preflight request which causes the first request and first log. Then, the actual request which causes the second request and second log. You cannot avoid this when using CORS - that's simply how the protocol works.
See also:
Firebase Cloud Function executing twice when triggered over HTTP
Enabling CORS in Cloud Functions for Firebase
Cloud Functions for Firebase triggering function on CORS preflight request
If you're using Firebase Hosting, you probably don't need CORS.
You can avoid preflight requests by adding rewrites to firebase.json.
However, there are conditions such as the position of the function must be us-central1.
https://firebase.google.com/docs/hosting/functions
I have a Google Assistant Action that uses Cloud Functions and in that Action I need to send GET/POST requests to my private server.
We're using Thingworx platform for development and it is hosted on a private cloud behind a firewall.
I've also tried Postman to send these requests and it worked perfectly, but when I use the same on my Google Assistant Action, it just doesn't work and all I get is an empty response.
My request code is below:
function callMeAPI(agent) {
var request = require("request");
var options = { method: 'GET',
url: 'https://{IP:Port}/Thingworx/Things/{Path}',
qs: { appKey: 'AppKey Used Here' },
headers:
{ 'cache-control': 'no-cache',
Accept: 'application/json' } };
request(options, function (error, response, body) {
if (error) throw new Error(error);
console.log(body);
agent.add("body");
});
So, are there any ports and IPs/URLs we need to add to our firewall rules to add for requests originating from Cloud Functions to go through the firewall onto our Thingworx Application?
You should not make any assumptions about the IP address of outbound traffic originating from Cloud Functions. The originating IP could change over time, depending on Google infrastructure, and also based on which region your function was deployed to.
If you need to ensure that access to your network originated from your function, you should implement some sort of authentication in the request so that your own application can validate the request. Typically, there is some shared secret that only both sides know that indicate they are aware of each other.