I am trying to create a firebase project with a firestore instance programatically. I've been using the firebase-tools cli and have managed to create a new project, a web app and get the app config, but I still need to manually enter the console and click the "Create database" button. Is it possible to automate this process?
Given that Firestore depends on an implementation of Google App Engine, I was able to programatically create a Firestore database using the Google App Engine api:
package main
import (
"context"
"google.golang.org/api/appengine/v1"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"log"
)
// CreateFirestoreDatabase uses the Google App Engine API
// to create a Firestore database
func CreateFirestoreDatabase(ctx context.Context) error {
// instantiate service
service, err := appengine.NewService(context.Background())
if err != nil {
log.Fatalf("appengine.NewService: %v", err)
}
// create application
op, err := service.Apps.
Create(&appengine.Application{
DatabaseType: "CLOUD_FIRESTORE",
Id: "your-google-project-id",
LocationId: "europe-west",
}).
Context(ctx).Do()
if err != nil {
return status.Errorf(
codes.Internal,
"service.Apps.Create: %s", err,
)
}
// check the status of the longrunning operations
// TODO: loop until op.Done == true
_ = op
return nil
}
The same process is used by the gcloud SDK:
gcloud firestore databases create [--region=REGION]
It is useful to have a look a the underlying python code backing the gcloud bin folder. Here is a screenshot of the gcloud firestore create command confirming this:
The process can be automated using terraform and the work around to configure an app engine application resource.
resource "google_app_engine_application" "app" {
project = "your-project-id"
location_id = "us-central"
database_type = "CLOUD_FIRESTORE"
}
As stated in the answers before firestore seems to rely on app engine even in the gcloud SDK.
Ugly but works.
Related
I'm using localStorage on the server and it works fine locally. But when I deployed my code to Deno deploy is not defined
Do I need to import the localStorage? I Deno.com I couldn't find any docs talking about localStorage so maybe that feature is not supported yet. In that case, where can I deploy my code to use it? Thanks
import {Handlers, PageProps} from "$fresh/server.ts";
interface Data {
email: string[]
}
export const handler: Handlers<Data> = {
GET(_req, ctx) {
const emailsStorage = localStorage.getItem("email");
const email = emailsStorage ? JSON.parse(emailsStorage) : [];
console.log(email);
return ctx.render({ email });
},
};
export default function EmailPage({ data }: PageProps<Data>) {
const { email } = data;
return (
<main>
<h1>Emails</h1>
<ul>
{email.map((email) => (
<li>{email}</li>
))}
</ul>
</main>
);
}
The full list of available APIs is here (note that localStorage is not listed).
Deploy does not offer any persistent data storage mechanism. After your deployed code finishes executing in response to a request, all of the JS memory is destroyed, so if you want to work with mutable data that persists between requests, then you'll have to store that data yourself elsewhere — e.g. by sending the data in a network request to another server / hosted database / etc. and then requesting it when you need it.
The docs include several "persist data" tutorials that you can use as a guide/reference in order to learn.
You can persist data in local storage by creating a virtual local Storage by using this code.
import { installGlobals } from "https://deno.land/x/virtualstorage#0.1.0/mod.ts";
installGlobals();
localStorage.getItem("email") will work on Deno Deploy also.
I have a node.js application that creates Cloud HTTP tasks with authentication. I'd like to handle these tasks viaFirebase HTTP function (also in JS). I understand that I need to use oidcToken when creating a task, but I don't understand how to validate such a token on the Firebase HTTP function end. Docs are not very helpful. I was expecting to find some utility in #google-cloud/tasks or in googleapis/google-auth-library-nodejs, but nothing jump out at me.
All you have to do is assosiacte your Cloud Function with a service account. This is called Function Identity. Please note that whenever you deploy a Cloud Function either in GCP or Firebase, the same function appears in both platforms.
You can create a new service account for this purpose with:
gcloud iam service-accounts create [YOUR_NEW_SERVICE_ACCOUNT_NAME] \
--display-name "Service Account Test"
And assign the required IAM role with:
gcloud projects add-iam-policy-binding ${PROJECT_ID} \
--member serviceAccount:[YOUR_NEW_SERVICE_ACCOUNT_NAME]#${PROJECT_ID}.iam.gserviceaccount.com \
--role roles/cloudfunctions.invoker
Your function deployment should look like something like this:
gcloud functions deploy hello_world \
--trigger-http \
--region us-central1 \
--runtime nodejs14 \
--service-account [YOUR_NEW_SERVICE_ACCOUNT_NAME]#${PROJECT_ID}.iam.gserviceaccount.com \
--no-allow-unauthenticated
Once you have all that, your Cloud HTTP task should use this service account as the OICD token, but I believe you already know how to do so.
You can find more information in this guide (although it uses the Cloud Scheduler instead of Cloud Tasks, the idea is pretty much the same).
You don't need to validate the token in the function, when you create the function so only authenticated users can call it this verification will automatically be done on Google's side.
So, to make what you want you need to:
1. Create the Task Queue
Detailed steps in this how-to.
2. Create the service account
You will need to create a SA with these IAM permissions:
Cloud Functions Invoker
Cloud Tasks Enqueuer
Service Account User
"Cloud Functions Invoker" is necessary to be able to call the function and "Cloud Tasks Enqueuer" to add tasks to the queue. If you want to use a different SA for each of those steps you can separate these permissions.
3. Create the Firebase Functions function
When creating the function, make sure that it required authentication.
4. Create the task using the SA
There's a section for this in the documentation, which you can find here. The code below is copied below:
// Imports the Google Cloud Tasks library.
const {CloudTasksClient} = require('#google-cloud/tasks');
// Instantiates a client.
const client = new CloudTasksClient();
async function createHttpTaskWithToken() {
const project = '[PROJECT_NAME]';
const queue = '[QUEUE_NAME]';
const location = '[LOCATION]';
const url = '[FUNCTION_TREIGGER_URL]';
const serviceAccountEmail = '[SA]';
const payload = 'Hello, World!';
// Construct the fully qualified queue name.
const parent = client.queuePath(project, location, queue);
const task = {
httpRequest: {
httpMethod: 'POST',
url,
oidcToken: {
serviceAccountEmail,
},
},
};
if (payload) {
task.httpRequest.body = Buffer.from(payload).toString('base64');
}
console.log('Sending task:');
console.log(task);
// Send create task request.
const request = {parent: parent, task: task};
const [response] = await client.createTask(request);
const name = response.name;
console.log(`Created task ${name}`);
}
createHttpTaskWithToken();
I am using the Go Firebase Admin SDK and listening to changes in the Realtime Database.
The problem is that the listener is ONLY triggered if I manually update the data from the Firebase Console, if I change data from another app (in this case Flutter), the listener is NOT triggered even though the changes can be seen in the Firebase Console (so the data definitely changed).
I even tried performing an update via Firebase Database REST API https://firebase.google.com/docs/reference/rest/database#section-streaming
Which had the same result: Data changes are viewable in the Console, but still don't trigger the listener.
Here the way I'm listening to changes in Go:
func listenToFirebase(ref *db.Ref, ctx context.Context) {
iter, err := ref.Listen(ctx)
if err != nil {
fmt.Printf(" Error: failed to create Listener %v\n", err)
return
}
defer iter.Stop()
for {
if iter.Done() {
break
}
event, err := iter.Next()
if err != nil {
// Handle error here based on specific usecase
// We can continue Listening
log.Printf("%v\n", err)
continue
}
fmt.Printf("Listener | Ref Path: %s | event.Path %s | event.Snapshot() = %v\n", ref.Path, event.Path, event.Snapshot())
fmt.Printf("\n")
}
}
The fact that the listener is triggered by updating data from the Console, indicates that the listener is working properly.
P.S.:
The Listen-Method has not yet been integrated to the Go Firebase Admin SDK and is from https://github.com/firebase/firebase-admin-go/issues/229
Turned out there was no problem regarding the Go code.
The problem was the way I updated the Realtime Database from Flutter.
This is what I used to perform updates:
await ref.update({"value": value});
What fixed the problem was to use "set" instead of "update" the following way:
await ref.child("value").set(value);
This way my Go-Listener method then was finally triggered and everything was working as expected.
Background
Consider the function func NewApp(ctx context.Context, config *Config, opts ...option.ClientOption) (*App, error) .
Theoretically it should be possible to override the project ID via the config parameter.
However, the problem is that when the credentials are fetched, the project ID is also determined, and this happens before the override via config (or via GOOGLE_CLOUD_PROJECT/GCLOUD_PROJECT env. variables for that matter.)
As a result, the token will be for the "default" project ID, not the overridden one.
Detailed steps
Suppose we use the "default service account" as per here:
If the environment variable isn't set, ADC uses the default service account that Compute Engine, Kubernetes Engine, App Engine, and Cloud
Functions provide, for applications that run on those services.
So this is what will happen in that case:
creds, err := transport.Creds(ctx, o...)
return internal.Creds(ctx, &ds)
cred, err := google.FindDefaultCredentials(ctx, ds.Scopes...)
id, _ := metadata.ProjectID() // (x) at this point we get the default project ID
func ProjectID() (string, error) { return defaultClient.ProjectID() }
func (c *Client) ProjectID() (string, error) { return projID.get(c) }
v, err = cl.Get(c.k)
val, _, err := c.getETag(suffix)
res, err := c.hc.Do(req)
In short, we get the credentials with the default project ID.
Question: Am I missing something, or is there really no way to override the project ID, when using the default service account?
Addendum: Maybe the problem is not the projectID in the credentials, but getting the bearer token with the default project ID configured there. The base question ("how to override the project") is still valid, though.
Over here the Firebase docs explain how you can retrieve a token required to make requests to the Remote Config Rest API.
It provides example code for Python, Java and Node.js. Because there is no code for Go, it sends me to the Google Client Library (for Go). You might be able to understand why I am getting lost there...
The examples use GoogleCredential in Java, ServiceAccountCredentials in Python and google.auth.JWT in Node.js. I was not able to find any of those here. I do not know why there are no clear naming conventions.
I have found
firebaseremoteconfig-gen.go: The code looks like it already implements what the Firebase documentation page tries to achieve "manually". Comparison: doc, package.
Help
Because the "Usage example" of the package ends strangely abrupt and is the opposite of extensive, I do not understand how to make use of it.
I would be helped if someone could tell me how I can use this:
firebaseremoteconfigService, err := firebaseremoteconfig.New(oauthHttpClient)
I could not figure out where I would get oauthHttpClient from. There is an oauth2 package in the repository, but there I face the same problem:
oauth2Service, err := oauth2.New(oauthHttpClient)
I need oauthHttpClient again, so this cannot be a solution.
http.Client could be anything, but I need to authenticate with a service-account.json file, like shown in the three example snippets here.
Tags explanation
I hope that someone has either had experience with integrating Firebase Remote Config with Go, someone knows how Google Client API authentication works or someone is good enough with Go to get how the usage works.
There are a couple of main ways of authenticating with the google APIs, they are documented here:
Link to docs
The ways documented are "3-legged OAuth", "Using API Keys" and finally "Service Accounts".
From the links that you've included in the question; you are looking at the Python / Java / Node examples of "Service Accounts".
Using Service Accounts in go
The oauthHttpClient that you are referring to, is an http client that will attach the authentication information to the requests automatically.
You can create one using this package:
https://godoc.org/golang.org/x/oauth2/google
The examples linked in other languages use a "service account json key file".
Using the method linked below, you can read that keyfile and create a jwt.Config struct that will give you access to the client that you need.
https://godoc.org/golang.org/x/oauth2/google#JWTConfigFromJSON
The go equivalent of the other language examples linked is;
data, err := ioutil.ReadFile("/path/to/your-project-key.json")
if err != nil {
log.Fatal(err)
}
conf, err := google.JWTConfigFromJSON(data, "https://www.googleapis.com/auth/firebase.remoteconfig")
if err != nil {
log.Fatal(err)
}
// Initiate an http.Client. The following GET request will be
// authorized and authenticated on the behalf of
// your service account.
client := conf.Client(oauth2.NoContext)
client.Get("...")
I just started using the same library (from an AppEngine Standard project). This is how I am creating the service client:
import (
"context"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"golang.org/x/oauth2/google"
fb "google.golang.org/api/firebaseremoteconfig/v1"
"google.golang.org/appengine"
"google.golang.org/appengine/log"
)
const (
// Name of our service account file
saFileName = "my-firebase-sa.json"
// OAuth scopes used for remote config API
scopeRemoteConfig = "https://www.googleapis.com/auth/firebase.remoteconfig"
)
func createFirebaseService(ctx context.Context) (*fb.Service, error) {
data, err := ioutil.ReadFile(saFileName)
if err != nil {
return nil, err
}
conf, err := google.JWTConfigFromJSON(data, scopeRemoteConfig)
if err != nil {
return nil, err
}
return fb.New(conf.Client(ctx))
}
And I call it as such:
func fetchConfig(ctx context.Context) (*fb.RemoteConfig, error) {
s, err := createFirebaseService(ctx)
if err != nil {
log.Errorf(ctx, "Failed to create firebase service: %v", err)
return nil, fmt.Errorf("Failed to initialize Firebase service")
}
projectID := "projects/" + appengine.AppID(ctx)
cfg, err := s.Projects.GetRemoteConfig(projectID).Do()
if err != nil {
log.Errorf(ctx, "Failed to call Firebase remote config API: %v", err)
return nil, err
}
return cfg, nil
}
The code is using the Project ID to form its path; after reading through the lib code I noticed it was missing /projects/ from that path; so I just prepended that to my project ID and it works ;-) At least until they fix that and my code stops working..
Hopefully this helps someone.