I have a Java application (an ESB - Enterprise Service Bus) which deals with customer sensitive data and have a requirement of supporting Data encryption at rest in order to prevent data abuse.
The application stores customer data for processing on the file system and application interacts with it through java.io.File API. So basically, I need to encrypt the file before it is persisted on the file system by the application and then decrypt it before application reads it so that it can be parsed and processed.
Are there any good frameworks or libraries that can help me implement Data encryption at rest? I am planning to use PGP encryption/decryption for implementing Data encryption at rest.
I am looking for best and recommended approach for implementing Data encryption at rest within my Java application and any help shall be appreciated.
Why on Earth would you think pgp is the right tool for this? Seems to me that you only need a symmetric key solution, so pgp feels like the wrong answer.
Cryptography in Java is a minefield. Easy to do things wrong, hard to do things right.
Having said that, you have a better chance to not screwing up if you use BouncyCastle rather than something else. They have example code that shows you how to do various operations.
For encrypting data at rest, I recommend you use AES in either CBC mode or CTR mode. If using CBC mode, make sure you choose your IV in a cryptographic secure way (unpredictable). Also, never re-use an IV for any mode of operation.
You should also consider whether you need message integrity. General guidance about symmetric encryption here.
Even though people often get crypto wrong, the bigger problem is key management. That's a whole new can of worms (and don't be fooled into thinking pgp provides a solution to this: all it does is shifts the problem to somewhere else).
Related
I am writing a site that needs to use multiple encryptions through out the entire process. I started to use CryptoJS v3.1.2 vSHA3 (512) and am able to encrypt the data no problem. Where I'm having trouble is decrypting the data once it's been encrypted. I have been searching the web for the last week or 2 and have only been able to find ways to encrypt the data but no way to decrypt it. If there is no way to decrypt the data, I will have to look for a different encryption. Any help here would be gratefully appreciated.
SHA3 is a hash function, and hash functions are not reversible -- that is, not decryptable. You may instead need a symmetric algorithm such as AES.
That being said, getting the nuances of cryptography right to avoid vulnerabilities is notoriously hard even for veterans, much less someone new to these concepts. Try very, very, VERY hard to rely on standard and ready-made solutions. Use things like HTTPS, for example, or your database's built-in security features.
My case is that I want to make the data protected even from people who have access to the back-end (the keys store), so they couldn't read it without the user's (represented by the client app, in my case the browser) assistance.
One option is to have the decryption keys stored on the client and passed with each request which sounds pretty messy to me and i'm not sure I want my keys to wander around the net like this. What I imagine though is that the client will keep some token (it might be a password the user knows) and the decryption can't happen without it.
I thought about using the purpose string for this, I have the feeling it is not a good idea since its main purpose is isolation. On the other hand it is part of the additional authenticated data used for subkey derivation. (based on this article https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/implementation/subkeyderivation?view=aspnetcore-2.1#additional-authenticated-data-and-subkey-derivation).
I came across some examples that create their own symmetric encryption with a lower level classes. (like this post Encrypt and decrypt a string in C#?). Since I'm not an expert in this area I would like to use as much build in classes as possible.
What is the recommended way to achieve what I need with the classes from the Data Protection API? (I'm using .net core 1.1 on Ubuntu)
I am exploring options to implement encryption at rest for RocksDB data which I am using in one application (that is, I don't have to store the key in the data, I can calculate it in runtime). Ideally, it should be DES.
Are there easy plugins or libraries specifically for RocksDB, or I will have to improvise?
There is nothing that I know of. In terms of implementation you have a few options:
Handle the encryption in your app. Rocks doesn't care what you store in it, just that keys are comparable. So you just need to design a sensible key encoding.
You could use the StackableDB feature of Rocks to implement something between your app and Rocks to handle the encryption. You would still have to design a key encoding for your data.
Look at how compression is implemented in Rocks try to implement something at that level if appropriate
I am currently using Solr to perform search services over some sensitive records.
As Solr/lucene provides fast searching by storing inverted indexes of the sensitive information in plain text on a disk there is a requirement to encrypt these index files so that unauthorized people can't have access to them by bypassing the system's security.
I found there are similar patches open on Apache JIRA AES encrypted directory and Codec for index-level encryption.
AES encrypted directory looks promising but this patch has been implemented for lucene 3.1 as I am using the newer version, I am not sure if this patch can be used with lucene version 5 or higher.
I was wondering if there is a way to implement a security measure that encrypts the indexes or if it is possible to write some custom plugin which can encrypt/decrypt the indexes on I/O level(i.e FsDirectory)?
The discussion in the comment section of LUCENE-6966 you have shared is really interesting. I would reason with this quote of Robert Muir that there is nothing baked into Solr and probably will never be.
More importantly, with file-level encryption, data would reside in an unencrypted form in memory which is not acceptable to our security team and, therefore, a non-starter for us.
This speaks volumes. You should fire your security team! You are wasting your time worrying about this: if you are using lucene, your data will be in memory, in plaintext, in ways you cannot control, and there is nothing you can do about that!
Trying to guarantee anything better than "at rest" is serious business, sounds like your team is over their head.
So you should consider to encrypt the storage Solr is using on OS level. This should be transparent for Solr. But if someone comes into your system, he should not be able to copy the Solr data.
This is also the conclusion the article Encrypting Solr/Lucene indexes from Erick Erickson of Lucidwors draws in the end
The short form is that this is one of those ideas that doesn't stand up to scrutiny. If you're concerned about security at this level, it's probably best to consider other options, from securing your communications channels to using an encrypting file system to physically divorcing your system from public networks. Of course, you should never, ever, let your working Solr installation be accessible directly from the outside world, just consider the following: http://server:port/solr/update?stream.body=<delete><query>*:*</query></delete>!
If an encryption function requires a key, how do you obfuscate the key in your source so that decompilation will not reveal the key and thereby enable decryption?
The answer to large extent depends on the platform and development tool, but in general there's no reliable solution. Encryption function is the point at which the key must be present in it's "natural" form. So all the hacker needs to do is to put the breakpoint there and dump the key. There's no need to even decompile anything. Consequently any obfuscation is only good for newbies or when debugging is not possible for whatever reason. Using the text string that exists in the application as the key is one of variants.
But the best approach is not to have the key inside, of course. Depending on your usage scenario you sometimes can use some system information (eg. smartphone's IMEI) as the key. In other cases you can generate the key when the application is installed and store that key as an integral part of your application data (eg. use column names of your DB as the key, or something similar).
Still, as said, all of this is tracked relatively easy when one can run the debugger.
There's one thing to counteract debugging -- offload decryption to third-party. This can be done by employing external cryptodevice (USB cryptotoken or smartcard) or by calling a web service to decrypt certain parts of information. Of course, there methods are also suitable only for a limited set of scenarios.
Encryption is built into the .NET configuration system. You can encrypt chunks of your app/web.config file, including where you store your private key.
http://www.dotnetprofessional.com/blog/post/2008/03/03/Encrypt-sections-of-WebConfig-or-AppConfig.aspx
source