I have a Xamarin.Forms app that uses a local SqLite database as its source for data. The data is proprietary, so I want to protect it so that if someone gets access to the database file, they would have to decrypt it to access the data.
I also want to limit the number of queries users can make against the database so that at a certain point they have to purchase the ability to use more of the data (in-app purchase).
I want to avoid making network calls as much as possible to minimize impact to the user's data plan and allow the app to work well in conditions where there is poor or no connectivity. So, I want the data stored in a local database (perhaps in SqLite).
I'm curious how different people would approach this problem to protect the data and at the same time minimize network usage.
Here is kind of what I was thinking (if it's possible):
1) Let the user download/install the app.
2) On first load, the app will upload a key based on the device id and the user's current purchase information. Then it will download a SqLite database file that has been encrypted using the uploaded key.
3) When the user reaches their limit of queries, the database file is deleted. If they purchase more data, then a new key is uploaded and a new encrypted database is downloaded to be used.
Thoughts? Is there a better way?
I would suggest SQLCipher! It is a Component within Xamarin (http://components.xamarin.com/view/sqlcipher-for-xamarin-ios) but can also be built from source as it is Open Source (https://www.zetetic.net/sqlcipher/open-source/)
That will totally secure your database :)
UPDATE 8/2/2018 - SQL Cipher is now free and easy to implement thanks to the greatness of Frank Krueger. sqlite-net (https://github.com/praeclarum/sqlite-net) is the defacto sqlite library for Xamarin now (if you're still using the Sqlite.Net fork I recommend going back to sqlite-net as soon as possible as Sqlite.Net has been abandoned) and it now includes SQL Cipher support completely free of charge.
As clb mentioned, SQLCipher is open source. So if you don't want to pay for the component you can download and build the source yourself, then wrap it for use in Xamarin. This is, admittedly, a technically challenging task.
If that's not an option, I would recommend two other options:
Reevaluate your need to store data locally. It's extremely unlikely that you need to transfer enough data to even cause a blip on a user's data plan. And between cellular and wifi, it's not that common anymore for users to be without a connection. It certainly does happen, and there are certain apps where this is very important, but you may have to make concessions if the data is that sensitive.
If you absolutely have to store the data locally, and you can't use SQLCipher, your last real option is to use a cryptography library and encrypt the data itself, rather than the database file. This is less than ideal, typically, for a variety of reasons, but it may be your last resort. PCL Crypt is a PCL capable crypto library that you can look into.
https://github.com/aarnott/pclcrypto
Related
I'm reading on hash table and data structure, and one question come to mind. Where is hash table implemented? Is it on server code or database?
The resource I've read seems to implement them on the server code, but isnt storing data the job of database? PS: I've havent get to a point of knowing non-sql database yet, maybe that's where my knowledge lack.
Many applications need to store some data internally, even if they're also using or updating data in a database at times. Often they'll even retrieve related data from a remote (across the network) database and have it available in RAM on the local machine for the application to access quickly.
Other times, an application may use a data structure such as a hash table to support some application behaviours that are not part of the business data model, and therefore don't belong in the database. For example, a GUI application might keep help strings to display when the mouse hovers over a widget/button/whatever - they might be stored in a hash table keyed on some GUI object identifier, screen region or whatever the GUI library finds useful to help it display the tooltips at the right time. Another application might keep a table of usernames and activity statistics that it generated by scraping some website - it might display them to the user on demand, or aggregate them or something, without ever saving them down to a database (historic data may be of no value, and it can scrape the website again).
In summary - non-trivial programs tend to use hash tables to provide quick access to the data they consult or manipulate, whether the programs are themselves databases, applications that do also use databases, or applications that run without any database support.
Right now the only way to encrypt a Cassandra database at rest seems to be with their enterprise edition which costs thousands of dollars: How to use Cassandra with TDE (Transparent Data Encryption)
Another solution is to encrypt every value before it enters the database, but then the key will be stored somewhere on every server in plaintext and would be easy to find.
I understand they offer "free" use for certain companies, but this is not an option and I am not authorized to pay $2000/server. How do traditional companies encrypt their distributed databases?
Thanks for the advice
I took the approach of encrypting the data disk on AWS. I added a new volume to the instance and checked the option to encrypt the volume. Then I edited cassandra.yaml to point to the encrypted volume.
We have done similar requirement in one of our project. Basically, I made use of trigger feature in Cassandra with custom implementation to perform encryption. It seems to be working fine for us.
You can refer below docs on how to create trigger and sample implemention of ITrigger interface
https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cqlCreateTrigger.html
https://github.com/apache/cassandra/blob/2e5847d29bbdd45fd4fc73f071779d91326ceeba/examples/triggers/src/org/apache/cassandra/triggers/AuditTrigger.java
Encrypting before inserting is a good way. The keys will either be on each application or on each cassandra node. There isnt much difference really, either way you should use filesystem permissions to restrict access to key just the apps user. Theres steps to get more secure from there like requiring entering of passphrase on startup vs storing on disk, but it makes operational tasks horrific.
I have an application that needs to send data to a cloud database (DynamoDb).
The app runs on a computer that can lose internet connectivity or be switched off at any time, but I must ensure that all data eventually gets to the cloud database.
I can assume the application will eventually be switched on, and will eventually get internet access back.
The app is written in VB .NET
What are some schemes for achieving this, and are there any ready-made products that already achieve this?
You could implement a write-through cache using a local DynamoDB instance (or even using SQLite). But without getting specific details about what kind of data you'd be storing into the database, and what data should be made available "offline" it's hard to say exactly how you should structure your application. You'll definitely want to not keep everything local, unless the volume of data is really small overall.
Then there is the problem of resolving conflicts that may occur during network partitions (ie. a client goes offline and makes some database modifications, while other clients also make modifications to the database; these need to be reconciled and it's up to you, and your users to determine how)
It's not a simple problem to solve.
Edit: see my answer for the solution
Currently working on a Hybrid App with Ionic where there is a requirement to store an authentication Token in order to keep the user logged in, and also guarantee that this data cannot be accessed outside the App context.
There is of course plenty of solutions for this task, each one with different pros-and-cons so it's confusing (for me) to locate the one technology that fits.
I've been looking at angular-localForage and other candidates:
LocalStorage
Obvious choice for small data.
Data gets wiped in iOS when the system is low on memory.
IndexedDB
Buggy support in iOS (IndexedDB support)
WebSQL (SQLite)
Apparently a good option for small data and decent support (WebSQL suppport) but it's deprecated.
SQLite
There are related issues with Cordova in iOS.
LokiJS
Looks like an overkill for this scenario but is definitely a strong candidate. Is the any security concerns I should be aware of (as I read it locally persist data to JSON files)?
PouchDB + SQLite
Well, it's a JS client to work with CouchDB or Couchbase databases wich can also work with SQLite but again I only need to store a Token..
So apparently the best option for Android/iOS cross-compatibility should ironically be WebSQL, but is no longer being developed and I have to guarantee long-term support.
So my question is: are there any other options I'm missing to securely store an access Token? If don't, wich of the above ones should be the best choice for this task?
After doing some research I will share my conclusion.
Funny enough, none of the above candidates are suitable for securely storing an access Token. The approach should be using a native solution for both Android (Shared Preferences) and iOS (Keychain).
In the particular case of Ionic, a broadcaster plugin for Cordova could be used to communicate JS with Native so you can access the stored data.
The only secure way is using "httponly cookie". However, since april 2020 Apple uses wkwebview which has cookie problem.
SQLite is the best option to go with as the content of DB will be encrypted and saved. Also native apps rely on SQLite to save data.
To make CRUD easier with SQLite, I have created a wrapper library. Please check it here
Go for LocalStorage, it is the best way to store
I am trying to create an app that receives an Sqlite database from a server for offline use but cloud synchronization. The server has a postgres database with information from many clients.
1) Is it better to delete the sql database and create a new one from a query, or try to synchronize and update the existing separate sqlite files (or another better solution). The refreshes will be a few times a day per client.
2) if it is the latter, could you give me any leads to resources on how I could do this?
I am pretty new to database applications so please excuse my ignorance and let me know if there is any way I could clarify.
There is no one size fits all approach here. You need to carefully consider exactly what needs to be done, what you are replicating, how much data is involved, and what your write models are, all before you build a solution. Along the way you have to decide how to handle write conflicts and more.
In general the one thing I would say is that such synchronization works best with append-only write models (i.e. inserts, no deletes, no updates), and one way to do it is to log changes that need to be made and replicate those changes.
However, master-master replication is difficult on the best of days and with the best of tools available. Jumping between databases with very different capabilities will introduce a number of additional problems. You are in for a big job.
Here's an open source product that claims to solve this for many database types including Postgres. I have no affiliation or commercial interest in this company.
https://github.com/sqlite-sync/SQLite-sync.com
http://sqlite-sync.com/
If you're able and willing to step outside relational databases to use an object store you might want to have a look at CouchDb and perhaps PouchDb that use a MVCC based replication protocol designed to support multi-master replication including conflict resolution. Under the covers, PouchDb uses adaptors for Sqlite, IndexDb, Local storage or a remote CouchBb instance to persist client side data. It auto selects the best client side storage option for the given desktop or mobile browser. The Sqlite engine can be either WebSQL or a Cordova Sqlite plugin.
http://couchdb.apache.org/
https://pouchdb.com/