I am exploring options to implement encryption at rest for RocksDB data which I am using in one application (that is, I don't have to store the key in the data, I can calculate it in runtime). Ideally, it should be DES.
Are there easy plugins or libraries specifically for RocksDB, or I will have to improvise?
There is nothing that I know of. In terms of implementation you have a few options:
Handle the encryption in your app. Rocks doesn't care what you store in it, just that keys are comparable. So you just need to design a sensible key encoding.
You could use the StackableDB feature of Rocks to implement something between your app and Rocks to handle the encryption. You would still have to design a key encoding for your data.
Look at how compression is implemented in Rocks try to implement something at that level if appropriate
Related
I have a Java application (an ESB - Enterprise Service Bus) which deals with customer sensitive data and have a requirement of supporting Data encryption at rest in order to prevent data abuse.
The application stores customer data for processing on the file system and application interacts with it through java.io.File API. So basically, I need to encrypt the file before it is persisted on the file system by the application and then decrypt it before application reads it so that it can be parsed and processed.
Are there any good frameworks or libraries that can help me implement Data encryption at rest? I am planning to use PGP encryption/decryption for implementing Data encryption at rest.
I am looking for best and recommended approach for implementing Data encryption at rest within my Java application and any help shall be appreciated.
Why on Earth would you think pgp is the right tool for this? Seems to me that you only need a symmetric key solution, so pgp feels like the wrong answer.
Cryptography in Java is a minefield. Easy to do things wrong, hard to do things right.
Having said that, you have a better chance to not screwing up if you use BouncyCastle rather than something else. They have example code that shows you how to do various operations.
For encrypting data at rest, I recommend you use AES in either CBC mode or CTR mode. If using CBC mode, make sure you choose your IV in a cryptographic secure way (unpredictable). Also, never re-use an IV for any mode of operation.
You should also consider whether you need message integrity. General guidance about symmetric encryption here.
Even though people often get crypto wrong, the bigger problem is key management. That's a whole new can of worms (and don't be fooled into thinking pgp provides a solution to this: all it does is shifts the problem to somewhere else).
I am currently comparing multiple data persistence frameworks for iOS.
One important requirement we got is the possibility to encrypt the data we are working with. When using Realm there is an option to use encryption with the consequence of a 10% performance hit as you can read here
When using Core Data there are basically 3 Ways to encrypt your Data
Using iOS-Level Data Protection
SQLCipher
Transformable decryption
I would prefer to encrypt the Data on iOS Data Protection Level using the NSFileProtectionComplete Option on the File Manager.
My Question is:
How big is the performance Impact when using iOS-Level encryption? Are there any Statistics out there or do you have experienced a major decrease in performance? It would be really interesting to here about your experiences.
Thanks in Advance!
How big is the performance Impact when using iOS-Level encryption?
This would be a great thing to benchmark. Just like Realm's 10% overhead number, your mileage will vary based on the type of work you do, how many reads vs writes, the size of the database, the device's storage controller, etc.
I suspect that Apple has gone to great lengths to minimize the overhead of NSFileProtection crypto and would be surprised if it came close to Realm's 10% number.
More than numbers from micro-benchmarks, you should probably be more concerned with the features and limitations of each approach.
For example, NSFileProtectionComplete won't let you access a file when the device is locked, whereas SQLCipher and Realm's encryption let you access it as long as you have the encryption key. Also, if you export the file out of iOS, NSFileProtection won't apply there, so if you need to export an encrypted file, you should use SQLCipher or Realm's encryption.
If an encryption function requires a key, how do you obfuscate the key in your source so that decompilation will not reveal the key and thereby enable decryption?
The answer to large extent depends on the platform and development tool, but in general there's no reliable solution. Encryption function is the point at which the key must be present in it's "natural" form. So all the hacker needs to do is to put the breakpoint there and dump the key. There's no need to even decompile anything. Consequently any obfuscation is only good for newbies or when debugging is not possible for whatever reason. Using the text string that exists in the application as the key is one of variants.
But the best approach is not to have the key inside, of course. Depending on your usage scenario you sometimes can use some system information (eg. smartphone's IMEI) as the key. In other cases you can generate the key when the application is installed and store that key as an integral part of your application data (eg. use column names of your DB as the key, or something similar).
Still, as said, all of this is tracked relatively easy when one can run the debugger.
There's one thing to counteract debugging -- offload decryption to third-party. This can be done by employing external cryptodevice (USB cryptotoken or smartcard) or by calling a web service to decrypt certain parts of information. Of course, there methods are also suitable only for a limited set of scenarios.
Encryption is built into the .NET configuration system. You can encrypt chunks of your app/web.config file, including where you store your private key.
http://www.dotnetprofessional.com/blog/post/2008/03/03/Encrypt-sections-of-WebConfig-or-AppConfig.aspx
source
I am going to write my own encryption, but would like to discuss some internals. Should be employed on several mobile platforms - iOS, Android, WP7 with desktop serving more or less as a test platform.
Let's start first with brief characteristics of existing solutions:
SQLite standard (commercial) SEE extension - I have no idea how it works internally and how it co-operates with mentioned mobile platforms.
System.data.sqlite (Windows only): RC4 encyption of the complete DB, ECB mode. They encrypt also DB header, which occasionally (0.01% chance) leads to DB corruption.*) Additional advantage: They use SQLite amalgamation distribution.
SqlCipher (openssl, i.e. several platforms): Selectable encryption scheme. They encrypt whole DB. CBC mode (I think), random IV vector. Because of this, they must modify page parameters (size + reserved space to store IV). They realized the problems related to unencrypted reading of the DB header and tried to introduce workarounds, yet the solution is unsatisfactory. Additional disadvantage: They use SQLite3 source tree. (Which - on the other hand - enables additional features, i.e. fine tuning of the encryption parameters using special pragmas.)
Based on my own analysis I think the following could be a good solution that would not suffer above mentioned problems:
Encrypting whole DB except the DB header.
ECB mode: Sounds risky, but after briefly looking at the DB format I cannot imagine how this could be exploited for an attack.
AES128?
Implementation on top of the SQLite amalgamation (similarly as system.data.sqlite)
I'd like to discuss possible problems of this encryption scheme.
*) Due to SQLite reading DB header without decryption. Due to RC4 (a stream cipher) this problem will manifest at the very first use only. AES would be a lot more dangerous as every "live" DB would sooner or later face this problem.
EDITED - case of VFS-based encryption
Above mentioned methods use codec-based methodology endorsed by sqlite.org. It is a set of 3 callbacks, the most important being this one:
void *(*xCodec)(void *iCtx, void *data, Pgno pgno, int mode)
This callback is used at SQLite discretion for encrypting/decrypting data read from/written to the disk. The data is exchanged page by page. (Page is a multiple of 512 By.)
Alternative option is to use VFS. VFS is a set of callbacks used for low-level OS-services. Among them there are several file-related services, e.g. xOpen/xSeek/xRead/xWrite/xClose. In particular, here are the methods used for data exchange
int (*xRead)(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst);
int (*xWrite)(sqlite3_file*, const void*, int iAmt, sqlite3_int64 iOfst);
Data size in these calls ranges from 4 By (frequent case) to the DB page size. If you want to use a block cipher (what else to use?), then you need to organize underlying block cache. I cannot imagine an implementation that would be as safe and as efficient as SQLite built-in transactions.
Second problem: VFS implementation is platform-dependent. Android/iOS/WP7/desktop all use different sources, i.e. VFS-based encryption would have to be implemented platform-by-platform.
Next problem is a more subtle: Platform may use VFS calls to realize file locks. These uses must not be encrypted. More over, shared locks must not be buffered. In other words, encryption at the VFS level might compromise locking functionality.
EDITED - plaintext attack on VFS-based encryption
I realized this later: DB header starts with fixed string "SQLite format 3" and the header contains a lot of other fixed byte values. This opens the door for known plaintext attacks (KPA).
This is mainly the problem of VFS-based encryption as it does not have the info that the DB header is being encrypted.
System.data.sqlite has also this problem as it encrypts (RC4) also the DB header.
SqlCipher overwrites hdr string with salt used to convert password to the key. Moreover, it uses by default AES, hence KPA attack presents no danger.
You don't need to hack db format or sqlite source code. SQLite exposes virtual file-system (vfs) API, which can be used to wrap file system (or another vfs) with encryption layer which encrypts/decrypts pages on the fly. When I did that it turned out to be very simple task, just hundred lines of code or so. This way whole DB will be encrypted, including journal file, and it is completely transparent to any client code. With typical page size of 1024, almost any known block cipher can be used. From what I can conclude from their docs, this is exactly what SQLCipher does.
Regarding the 'problems' you see:
You don't need to reimplement file system support, you can wrap around the default VFS. So no problems with locks or platform-dependence.
SQLite's default OS backend is also VFS, there is no overhead for using VFS except that you add.
You don't need block cache. Of course you will have to read whole block when it asks for just 4 bytes, but don't cache it, it will never be read again. SQLite has its own cache to prevent that (Pager module).
Didn't get much response, so here is my decision:
Own encryption (AES128), CBC mode
Codec interface (same as used by SqlCipher or system.data.sqlite)
DB header unencrypted
Page headers unencrypted as well and used for IV generation
Using amalgamation SQLite distribution
AFAIK this solution should be better than either SqlCipher or system.data.sqlite.
I am looking for a programmatic way to generate consumer key/secrets for HMAC-SHA1 to be used by clients invoking our API over OAuth. Any pointers to existing implementations would be highly helpful.
Secrets are best when generated from random data. That way there is no external data which could help an attacker deduce or guess part or the entire key. Of course, it depends on how much protection your secret key needs. Java includes some random number generators in java.util.Random (since JDK1.0). If you don't have backward compatibility issues, Java 6 has java.security.SecureRandom which meets FIPS 140-2 requirements. The Java libraries are not truly random, but it is probably good enough for most applications. If you need better random data, you should go for a hardware-based random generator.