I have tried finding the answer to this but have only found anecdotal references. Does Google encrypt the data that is stored on BigQuery? If so, what encryption mechanisms are used?
All new data being added to BigQuery tables are encrypted using strong, industry-standard encryption methods. All old data will be encrypted over time, but currently with no specific timeline. If you'd like more detail on security across the Google Cloud Platform you might want to check out this blog post:
http://googlecloudplatform.blogspot.com/2014/08/googles-cloud-is-secure-but-you-dont.html
BigQuery is a part of Google's Cloud Platform Offering. As part of utilizing BigQuery you need to first load the data into it.
You can load Data 2 ways.
Load jobs support two data sources:
Objects in Google Cloud Storage
Data sent with the job or streaming insert
Data sourced from Google Cloud Storage is always encrypted as per the link below. Streaming data into BigQuery probably depends on how you stream it (I'm haven't found any absolute data on this point)
Loading data into BigQuery
Description of Compute Engine Disks
Yes, data stored in BigQuery is encrypted.
Google Cloud Platform encrypts data stored in BigQuery, without any action required from the user, using one or more encryption mechanisms. Data stored in Google Cloud Platform is encrypted at the storage level using either AES256 or AES128.
There are more details in this whitepaper:
https://cloud.google.com/security/encryption-at-rest/
Related
I am going to build a mobile app for my dad which will have four tables namely- buyer table , seller table , buyer balance payment table and seller due payment table. I have thought of using Reactive Native and SQLite due to its offline data storage capability.
Transaction limit per day will be between 2-10 rows in buyer and seller table.
Can anyone advise whether the offline data storage is safe?
Can anyone read the data from phone storage or its hidden inside the app?
Is it possible to export/import the data weekly to personal cloud storage like google drive or Microsoft one-drive and vice versa?
Is it advisable to publish such apps to app store or can i use it without publishing?
If the app is for personal use, u don't have to publish in play store. You can create apk and install in your phone.
SQLite storage is adequate for offline storage but u might lose data if backup of the data is not been taken.
Encrypt your data before you enter it in the database. As far as I know, the SQLite database is kept in a single file somewhere in the /data/ directory. What is more, your data is kept in plain text format. This means that it will always be possible for someone to extract that data by rooting the phone, obtaining the .db SQLite file and opening it with a text editor.
If the data is sensitive, you should consider encryption.
Yes, you can take backup and storage as encrypted file in cloud.
In my personal opinion, you can use firebase or other free cloud websites to host your data. It will be more simple and secure.
I am researching of a way to regularly sync Firebase data to BigQuery, then display that data to Data Studio. I saw this instruction in the documentation:
https://support.google.com/firebase/answer/6318765?hl=en
According to the above instruction, it says once Firebase is linked to BigQuery, the data from Firebase is being streamed to BigQuery real-time.
Let's say I have initial export of Firebase data to BigQuery (before linking) and I made a Data Studio visualization out of that initial data, we call it Dataset A. Then I started linking Firebase to BigQuery. I want Dataset A to be in sync with Firebase every 3 hours.
Based on the documentation, does this mean I don't have to use some external program to synchronize Firebase data every 3 hours to BigQuery, since it is streaming real-time already? After linking, does the streamed data from Firebase automatically goes to Dataset A?
I am asking because I don't want to break the visualization if the streaming behaves differently than the expected (expected means that Firebase streams to BigQuery's Dataset A consistent with the original schema). Because if it does (break the original dataset or it doesn't stream to the original dataset), I might as well write a program that does the syncing.
Once you link your Firebase project to BigQuery, Firebase will continuously export the data to BigQuery, until you unlink the project. As the documentation says, the data is exported to daily tables, and a single fixed intraday table. There is no way for you to control the schedule of the data export beyond enabling/disabling it.
If you're talking about Analytics data, schema changes to the exported data are very rare. So far there's been a schema change once, and there are currently no plans to make any more schema changes. If a schema change ever were to happen again though, all collaborators on the project will be emailed well in advance of the change.
I’m looking for some advice with regards GDPR and firebase, I’ve looked online and apparently firebase is GDPR compliant as it stores data securely during transit and the firebase servers themselves are encrypted. So if that’s the case would I even need to encrypt personal data locally before sending it to firebase?
Currently I’m encrypting all personal data but the issue is that I want to have an autocomplete function that searches for customers as the user types. Now this wouldn’t work currently as a search for ‘sha’ would not find ‘shaun’ because shaun is currently encrypted. So I’m wondering whether I even need to encrypt customer details at all as firebase is encrypted itself. The only issue is that me as a database admin can see everyone’s details in the database but surely that’s the case with all database admins in most apps as you can query the data in there to your hearts content and see everything about everyone?
As long as you are not sharing this data for any purposes outside of the apps terms and conditions this should be ok shouldn't it?
So a few things first , when you encrypt data at the DB level , any usage of that data at the application layer has to go through a level of decryption .
So in the example that you outline the autocomplete feature ideally will be served by an API and the API at the back-end will take the encrypted data through the decryption process to keep it in memory and return to the client appropriately.
Also it is not OK to assume that admin cannot go rouge , most of the data breaches and leeks happen from internal source. So it is better to keep the PII as encrypted in the DB.
According to Google Cloud all customer data is encrypted automatically. Then why isn't my data encrypted when I export it to a google storage and download it from there? Do I need to enable some service anyway for the encryption to work?
I really appreciate your help, thanks!
Your data is always encrypted by default on server-side. What it means is that, once Google receives the data, it encrypts it. It is recommended that you always sent your data over HTTPS or TLS. Data is automatically and transparently decrypted when read by an authorised user.
You can use the other two options for server side encryption, and you can also encrypt the data on your side, before sending it. I think this option suits your concern, but you'll have to manage your own encryption from your side, and ensure you never less your keys. Nevertheless, GCP will encrypt your data again once received.
In my Android application I have an idea to store in database some serial key. If user enters correct key he gets full version of application and the key is disabled on the server to avoid multiply usage of the same key, otherwise he can buy app in Google Play without a key.
For this I thought to use Firebase Database but after read this I have some doubts
Firebase Realtime Database
Store and sync data with our NoSQL cloud database. Data is synced across all clients in realtime, and remains available when your app goes offline.
Does it mean that firebase will duplicate the table with all available keys to all application users and some smart user can read the list from this copy at his phone?
Not all data is automatically duplicated to all clients. Only data that the client subscribes to is received by that client.
You can control what data each client can see through Firebase's server-side security rules. For example, you'll typically want to ensure that each user can only read their own data.
It probably isn't a good idea to store super-sensitive data like social security numbers or credit card numbers, but if you see https://firebase.google.com/docs/database/security/ you can see, that you can control access to data, & use validation, especially since you can regenerate the keys if they become compromised, it wouldn't be the worst option. If you look at https://firebase.google.com/docs/database/security/user-security you can see, that it's possible to write an app that uses it like google drive with a smartphone-based client.
Personally the answer would no. You may want to think about Google Play Subscriptions and In-App Purchases.
If you really have to then:
Create a key as a user buys the upgrade (server-side).
Store the device id/account id (hashed) and timestamp with the key.
Credit card details and expiry dates should be combined into one hash.
Just encrypt everything.
It's better to have a banned list than a list of approved key. Eventually you have to create more keys and it's easier just to maintain a list of banned keys.