I'm trying to figure out if it's possible to export all the Firebase Analytics data to an excel spreadsheet, similar to how you can do it with Google Analytics. From what I can find the only way to go about doing it is to link with BigQuery then do some SQL statements to build a table and export.
Unfortunately for us this is not going to work (due to client budget and capabilities). Is there any other way to export this data that I'm missing?
Update: You can now export the analytics reports as CSV from the Firebase console by clicking the Download CSV option from the ⠇ overflow menu.
In the meantime, you really should give BigQuery another look. The pricing is very reasonable and there is a free query tier of 1 TB/mo.
Steve Ganem
Product Manager, Firebase Analytics
As our company uses AWS for it's projects, BigQuery is not an option for now so I have moved on to scrape the data from Firebase.
You can use Selenium and Beautifulsoup in python to scrape the data from Firebase.
You could easily select and copy UI blocks on the Analytics web page in Firebase, and then just paste the copied data into an Excel sheet. You'll see fancy tables.
There is a YouTube video that explains and demonstrates the procedure here:
Copy Your Analytics Data to a Spreadsheet with this One WEIRD Trick! - Firecasts
For all those who use AWS and want an automated pipeline, BigQuery allows you to export CSV if you have no arrays and JSON if you do. You can then automate exports to google cloud storage. Finally, you can use Airflow with AWS or an EC2 with a CRON or some other orchestration or scheduling system to schedule the merging of the google data to your AWS pipeline.
Related
We accidentally unlinked our Google analytics account from Firebase and linked it to another project in another organisation.
Status quo is that in current project we do have new data from the time we proceeded the unlinking and in the old project we have old data before the time we did the unlinking. Is there any way how to merge those projects to have all data in one place ?
Your analytics data resides within Google Analytics property and not in the Firebase project. If you delete or unlink the property, then your data is unlinked from your project as well. BQ integration is disabled during this period and sometimes the exporting apps are not getting enabled after relinking.
I have to Firebase projects, one is for dev and another is for production. I create a bunch of collections and after they successfully pass the test, I'll need to move them into production database. How can I do this without using Cloud Shell, or are there any alternative suggestions of database?
Thank you!
You can export the collection using cloud shell
Manage export ad import
In case you want to move collections between your dev and your production databases, without using Cloud Shell, there is an alternative that you can follow.
For you to achieve that, you will need to follow the below steps.
Create a Cloud Storage bucket to hold the data from your source project.
Export the data from your source project to the bucket.
Give your destination project permission to read from the bucket.
Import the data from the bucket into your destination project.
With these steps, you should be able to migrate data between your projects and have this way, the structure you want of a Development database and a Production database, where you can easily transfer the data. I would recommend you to check the official documentation Move Data Between Projects, to get the whole tutorial on how to achieve the above steps, in case you have doubts on how to achieve them.
Let me know if the information helped you!
I'd like to write python script which manages my google data fusion pipelines and instances (creates new, deletes, starts, etc). For that purpose I use airflow installed as library. I've read some tutorials and documentations but I still can't make that script connect with data fusion instance. I've tried to use next string:
export AIRFLOW_CONN_GOOGLE_CLOUD_DEFAULT='google-cloud-platform://?extra__google_cloud_platform__key_path=%2Fkeys%2Fkey.json&extra__google_cloud_platform__scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform&extra__google_cloud_platform__project=airflow&extra__google_cloud_platform__num_retries=5'
with my data json key file and Project id but it still doesn't work. Can you give me an example of creating that connection?
You can find an example python script here:
https://airflow.readthedocs.io/en/latest/_modules/airflow/providers/google/cloud/example_dags/example_datafusion.html
This page provides a breakdown for each Data Fusion Operator if you would like to learn more about them:
https://airflow.readthedocs.io/en/latest/howto/operator/gcp/datafusion.html
My preview works and has data but my deployment has no data. I'm using the (Recommended) DEFAULT CLOUD SQL database configuration.
Note: This is only day 4 with Google App Maker. Finding answers to App Maker-specific questions has been super difficult, but I'm making rapid progress on my application, so overall tired but good. :{)
As written in the documentation,
App Maker deployments can use the same Cloud SQL instance, but have separate databases on that instance. Data that you had in preview mode is not available in other deployments. You have a few options for how to handle this situation:
To use data from the preview instance in your published deployment, export the deployment data from the preview instance and import it to the published deployment.
To share a database across all deployments (preview and published), use a custom Cloud SQL database.
When you deploy your app, AppMaker create a new database in your google cloud SQL instance for the deployment. All the data create in previews is in another database.
To use the same database as the preview mode you have to go in the settings of your app in the tab "DATABASE" and copy the Database Key. Then go to your cloud sql instance in google cloud platforme and on the details of the instance in the overview tab just copy the instance connexion name.
then edit your deployments and select "Use Custom Cloud SQL database" and copy with the format
"instanceConnexionName/DatabaseKey" then save and appmaker should ask you to enter you username and password of your google cloud sql insatnce.
On app settings, database page you should see
Databas key: iTIJQaCj491a4111
(Actually this is the name of the mySQL instance)
In GCP console, go to SQL, click on Instance ID, and on the Instance ID overview page is the instance connection name, e.g., MyProject-123456:us-central1:instancename
Back in app settings
Select Switch to custom database and enter the full connection string
projectname:instancename/schema as
MyProject-123456:us-central1:instancename/iTIJQaCj491a4111
Provide username and password
and follow the steps to confirm existing database
Turns out the issue is when you publish it doesn't push the data, you have to manually re-upload the data into the live version. This is actually a good thing, but I wish it'd been explicitly documented. I found it, after figuring it out on my own, in some early release notes from a few years back. I guess I wasn't the only one this stupid.
I have exported analytics data from big query.It was working till the new version is released. After release of new version i could not see any data in BigQuery, but I can see the data in Firebase report/console, but not in BigQuery.
Could you please help me to understand, why I don't see the data for new versions? What are the conditions of data export?
Attached Screenshot
Loading data from Cloud Firestore exports has its limitations. BigQuery and Cloud Storage also require some permissions.
You need to make sure options' specifications is needed to change how BigQuery parses Cloud Firestore export data, and data type conversion.
After reviewing other similar issues that occurred with the release of new versions of apps, make sure that the app still has Analytics-linked to BigQuery in the integrations section in the Firebase console.