I'm trying to analyze an app user data now have access to the Firebase and Google BigQuery data. To do some analysis, I need to link these two database together by identify the users. In the database of Firebase, I have a field named user and I hope I could find a field in the BigQuery database which contains the same information to link these two database. But I only find a field named app instance id in BigQuery, which is not the same as user field in Firebase and I don't know how to join these two data source by a common field. Is anyone can help? Thanks!
Firebase does not automatically record User identification in its Analytics data. If you want that, you should set the relevant value yourself in by calling setUserId().
Related
The scenario here is that I have my Firebase( just Analytics-Data) project linked to BigQuery. but when i check on the bigQuery to see the dataset it's not appear there and i don't know which name/id has.
I highly appreciate your support. Thanks
According to the documentation, when you link your Firebase Project to BigQuery,a corresponding dataset will be created. This dataset can be found, in BigQuery, under your project id and it will be named as your app. In case, that you have both IOS and Android versions of your app, two datasets will be created as follows:
The above image was taken from the documentation, here.
Furthermore, in addition to your app_events table, under your app's name, you will have apps_events_intraday, which will receive data near real time from Firebase. In other words, as soon as Firebase receives the data from the app it will transfer it to the intraday table in BigQuery. Whereas, the app_events table will be uploaded once per day, link.
Lastly, keep in mind that the data generated by your app can take up to 1 hour to be sent to Firebase which then will be nearly instantly sent to BigQuery. You can read more about the latency here.
I am trying to extract near real-time analytics data from Firebase Analytics. The data is exported to BigQuery in real-time to intraday table. In order to sync real-time data to my database. I need to know when the record has logged to the table to avoid reading same records multiple times.
However, According to the Firebase document below, event_timestamp field is the timestamp when it is logged in a client device. event_server_timestamp_offset field is the offset between the collection time and upload time.
https://support.google.com/firebase/answer/7029846?hl=en
So, I assumed the server logged time can be found by event_timetamp + event_server_timestamp_offset.
But I've found that event_server_timestamp_offset have both negative and positive number.
Does anyone know what the event_server_timestamp_offset is for? Unloading data before Collecting data is impossible.
It is possible to erase a customer with InstanceID to comply with GDPR: https://godoc.org/firebase.google.com/go/iid#Client.DeleteInstanceID
However we do not have historical Firebase Instance IDs. BigQuery has a field app_info.app_instance_id but this is not a valid instance ID.
Is it possible to erase a customer with app_instance_id?
An app instance ID identifies (as its name implies) an app instance. It does not identify a specific user. While it is quite common to associate IIDs with users, Firebase has nothing built in for that. This means that, unless you have the data in your database, there is no way to find out the associated IIDs for a user by calling the API.
I want to use google's Data Studio tool to make visualizations for the data I have in the Datastore. But it is not available as a source of data on the interface. How can I approach doing this?
You can write your own Community Connector to fetch data using the Cloud Datastore API. One thing to keep in mind here is that Data Studio only accepts tabular data so you will need to reshape your data using the connector code if necessary.
Your options are pretty limited. You'll probably have to move/convert your datastore entities into a database that can act as a data source for Data Studio. The following link will help you get started:
https://support.google.com/datastudio/topic/6370347?hl=en&ref_topic=7441382
You can export datastore data to BigQuery, which is available for data studio. There is more detail on this here:
https://cloud.google.com/bigquery/docs/loading-data-cloud-datastore
Open the BigQuery web UI in the Cloud Console.
Go to the BigQuery web UI
In the navigation panel, in the Resources section, expand your Google Cloud project and select a dataset. Click Create table. The process for loading data is the same as the process for creating an empty table.
On the Create table page, in the Source section:
For Create table from, select Cloud Storage.
In the source field, enter the Cloud Storage URI. The Cloud Storage bucket must be in the same location as the dataset that contains the table you're creating. The URI for your Datastore export file should end with [KIND_NAME].export_metadata or export[NUM].export_metadata. For example: default_namespace_kind_Book.export_metadata. In this example, Book is the kind name, and default_namespace_kind_Book is the file name generated by Datastore.
For File format, select Datastore Backup
On the Create table page, in the Destination section:
For Dataset name, choose the appropriate dataset.
In the Table name field, enter the name of the table you're creating in BigQuery.
Verify that Table type is set to Native table.
In the Schema section, no action is necessary. The schema is inferred from a Datastore export.
Select applicable items in the Advanced options section and then click Create table. For information on the available options, see Datastore options.
I have a question in regards to the capabilities of Firebase in equivalence to MySQL features like:
Events
Triggers
Stored Procedures
In my case I want to migrate off of MySQL to Firebase but I need to know if the usecase can be replicated in Firebase.
My current MySQL DB I have a table that has a column called status which once it gets changed to 'Final' it triggers the execution of a stored procedure to do a calculation on an entire table.
So in other words I would have to be able to add a 'trigger' on the actual firebase data to then perform a 'stored procedure' to calculate something; is this possible with Firebase?!
You now have the capability to use Cloud Functions for Firebase to write code that triggers in response to changes in your Firebase project. There is lots of sample code that illustrates how to respond to writes to a particular location in your database, among many other types of events.