I am using a AES encryption/decryption class that needs a key value and vector value encrypt and decrypt data in an MVC3 application.
On saving the record I am encrypting the data then storing in a database. When i retrieve the record i am decrypting in the controller and passing the unencrypted value to the view.
The concern is not protecting data as it traverses the network but to protect the database should it be compromised.
I have read many posts that say dont put the keys for encryption in your code.
Ok so where should they be kept? File system? Another Database?
Looking for some direction.
Common sense says, if an intruder gets access to your database, they will most likely also have access to your file system. It really comes down to you. For one, you can try to hide it. In configuration files, in plain files somewhere in file system, encrypt it with another key that is within the application ... and so on and so forth.
Configuration files are a logical answer, but why take a chance - mix it. Feel free to mix keys with multi-level encryptions - one requiring something from the record itself and being unique to every record, other one requiring a configuration value, third one requiring an application-specific value, and perhaps a fourth one from a library hidden well within your application's references? This way, even if one layer somehow gets compromised, you will have several others protecting it.
Yes, it adds overhead. Yes, it is relatively expensive. But is it worth it if you have sensitive data like user credit card details? You bet it is.
I'm using similar encryption and hashing techniques in one of my personal pet projects that is highly security focused and carefully controlled. It depends how much data you need to display at any one time - for example, mine will ever fetch only 10 records at a time, most likely even less.
... To specify what I mean by mixing: Encrypt once. Then encrypt that data again with different key and suggestedly different algorithm.
I would use Registry Keys protected by ACL, so only the account under which your app pool is running can read them.
Related
If an encryption function requires a key, how do you obfuscate the key in your source so that decompilation will not reveal the key and thereby enable decryption?
The answer to large extent depends on the platform and development tool, but in general there's no reliable solution. Encryption function is the point at which the key must be present in it's "natural" form. So all the hacker needs to do is to put the breakpoint there and dump the key. There's no need to even decompile anything. Consequently any obfuscation is only good for newbies or when debugging is not possible for whatever reason. Using the text string that exists in the application as the key is one of variants.
But the best approach is not to have the key inside, of course. Depending on your usage scenario you sometimes can use some system information (eg. smartphone's IMEI) as the key. In other cases you can generate the key when the application is installed and store that key as an integral part of your application data (eg. use column names of your DB as the key, or something similar).
Still, as said, all of this is tracked relatively easy when one can run the debugger.
There's one thing to counteract debugging -- offload decryption to third-party. This can be done by employing external cryptodevice (USB cryptotoken or smartcard) or by calling a web service to decrypt certain parts of information. Of course, there methods are also suitable only for a limited set of scenarios.
Encryption is built into the .NET configuration system. You can encrypt chunks of your app/web.config file, including where you store your private key.
http://www.dotnetprofessional.com/blog/post/2008/03/03/Encrypt-sections-of-WebConfig-or-AppConfig.aspx
source
Bitcasa's claim its to provide infinite storage for a fixed fee.
According to a TechCrunch interview, Bitcasa uses client-side convergent encryption. Thus no unencrypted data ever reaches the server. Using convergent encryption, the encryption-key gets derived from the be encrypted source-data.
Basically, Bitcasa uses a hash function to identify identical files uploaded by different users to store them only once on their servers.
I wonder, how the provider is able to ensure, that no two different files get mapped to the same encrypted file or the same encrypted data stream, since hash functions aren't bijective.
Technical question: What do I have to implement, so that such a collision may never happen.
Most deduplication schemes make the assumption that hash collisions are so unlikely to happen that they can be ignored. This allows clients to skip reuploading already-present data. It does break down when you have two files with the same hash, but that's unlikely to happen by chance (and you did pick a secure hash function to prevent people from doing it intentionally, right?)
If you insist on being absolutely sure, all clients must reupload their data (even if it's already on the server), and once this data is reuploaded, you must check that it's identical to the currently-present data. If it's not, you need to pick a new ID rather than using the hash (and sound the alarm that a collision has been found in SHA1!)
We wish to make a desktop application that searches a locally packaged text database that will be a few GB in size. We are thinking of using lucene.
So basically the user will search for a few words and the local lucene database will give back a result. However, we want to prevent the user from taking a full text dump of the lucene index as the text database is valuable and proprietary. A web application is not the solution here as the Customer would like for this desktop application to work in areas where the internet is not available.
How do we encrypt lucene's database so that only the client application can access lucene's index and a prying user can't take a full text dump of the index?
One way of doing this, we thought, was if the lucene index could be stored on an encrypted file system within a file (something like truecrypt). So the desktop application would "mount" the file containing the lucene indexes.
And this needs to be cross platform (Linux, Windows)...We would be using Qt or Java to write the desktop application.
Is there an easier/better way to do this?
[This is for a client. Yes, yes, conceptually this is bad thing :-) but this is how they want it. Basically the point is that only the Desktop application should be able to access the lucene index and no one else. Someone pointed that this is essentially DRM. Yeah, it resembles DRM]
How do we encrypt lucene's database so
that only the client application can
access lucene's index and a prying
user can't take a full text dump of
the index?
You don't. The user would have the key and the encrypted data, so they could access everything. You can bury the key in an obfuscated file, but that only adds a slight delay. It certainly will not keep out prying users. You need to rethink.
The problem here is that you're trying to both provide the user with data and deny it from em, at the same time. This is basically the DRM problem under a different name - the attacker (user) is in full control of the application's environment (hardware and OS). No security is possible in such situation, only obfuscation and illusion of security.
While you can make it harder for the user to get to the unencrypted data, you can never prevent it - because that would mean breaking your app. Probably the closest thing is to provide a sealed hardware box, but IMHO that would make it unusable.
Note that making a half-assed illusion of security might be sufficient from a legal standpoint (e.g. DMCA's anti-circumvention clauses) - but that's outside SO's scope.
Technically, there is little you can do. Lucene is written in Java and Java code can always be decompiled or run in a debugger to get the key which you need to store somewhere (probably in the license key which you sell the user).
Your only option is the law (or the contract with the user). The text data is copyrighted, so you can sue the user if they use it in any way that is outside the scope of the license agreement.
Or you can write your own text indexing system.
Or buy a commercial one which meets your needs.
[EDIT] If you want to use an encrypted index, just implement your own FSDirectory. Check the source for SimpleFSDirectory for an example.
Why not building an index that contains only the data that user can access and ship that index with the desktop app?
True-crypt sounds like a solid plan to me. You can mount volumes and encrypt them in all sorts of crazy overkill ways, and access them just as any other file.
No, it isn't entirely secure, but it should work well enough.
One-way hash function.
You don't store the plaintext, you store hashes. When you want to search for a term, you push the term through the function and then search for the hash. If there's a match in the database, return thumbs up.
Are you willing to entertain false positives in order to save space? Bloom filter.
A few years ago, when first being introduced to ASP.net and the .NET Framework, I built a very simple online file storage system.
This system used Rijndael encryption for storing the files encrypted on the server's hard drive, and an HttpHandler to decrypt and send those files to the client.
Being one of my first project with ASP.net and databases, not understanding much about how the whole thing works (as well as falling to the same trap described by Jeff Atwood on this subject), I decided to store freshly generated keys and IVs together with each file entry in the database.
To make things a bit clearer, encryption was only to protect files from direct access to the server, and keys were not generated by user-entered passwords.
My question is, assuming I don't want to keep one key for all files, how should I store encryption keys for best security? What is considered best practice? (i.e: On a different server, on a plain-text file, encrypted).
Also, what is the initialization vector used for in this type of encryption algorithm? Should it be constant in a system?
Keys should be protected and kept secret, simple as that. The implementation is not. Key Management Systems get sold for large amounts of money by trusted vendors because solving the problem is hard.
You certainly don't want to use the same key for each user, the more a key is used the "easier" it comes to break it, or at least have some information leaks. AES is a block cipher, it splits the data into blocks and feeds the results of the last block encryption into the next block. An initialization vector is the initial feed into the algorithm, because at the starting point there is nothing to start with. Using random IVs with the same key lowers the risk of information leaks - it should be different for every single piece of data encrypted.
How you store the keys depends on how your system is architected. I've just finished a KMS where the keys are kept away from the main system and functions to encrypt and decrypt are exposed via WCF. You send in plain text and get a reference to a key and the ciphered text back - that way the KMS is responsible for all cryptography in the system. This may be overkill in your case. If the user enters a password into your system then you could use that to generate a key pair. This keypair could then be used to encrypt a key store for that user - XML, SQL, whatever, and used to decrypt each key which is used to protect data.
Without knowing more about how your system is configured, or it's purpose it's hard to recommend anything other than "Keys must be protected, keys and IVs must not be reused."
There's a very good article on this one at http://web.archive.org/web/20121017062956/http://www.di-mgt.com.au/cryptoCreditcard.html which covers the both the IV and salting issues and the problems with ECB referred to above.
It still doesn't quite cover "where do I store the key", admittedly, but after reading and digesting it, it won't be a huge leap to a solution hopefully....
As a pretty good soltution, you could store your Key/IV pair in a table:
ID Key IV
skjsh-38798-1298-hjj FHDJK398720== HFkjdf87923==
When you save an encrypted value, save the ID and a random Salt value along with it.
Then, when you need to decrypt the value, lookup the key/iv pair using the id and the salt stored with the data.
You'd want to make sure you have a good security model around the key storage. If you went with SQL server, don't grant SELECT rights to the user that accesses the database from the application. You wouldn't want to give someone access to the whole table.
What if, you simply just generated a key for each user, then encrypted it with a "master key"? Then, make sure to have random ivs and as long as you keep the master key secret, no one should be able to make much use of any amount of keys. Of course, the encryption and decryption functions would have to be server-side, as well as the master key not being exposed at all, not even to the rest of the server. This would be a decent way to go about it, but obviously, there are some issues, namely, if you have stored your master key unsafely, well there goes your security. Of course, you could encrypt the master key, but then your just kicking the can down the road. Maybe, you could have an AES key, encrypted with a RSA key, and the RSA key is then secured by a secret passprase. This would mitigate the problem, as if you have a decent sized RSA key, you should be good, and then you could expose the encryption functions to the client (though still probably shouldn't) and since the key encryption uses a public key, you can have that taken. For added security, you could cycle the RSA key every few months or even weeks if need be. These are just a few ideas, and I know that it isn't bulletproof, but is more secure than just stuffing it in a sql database.
Say you have a bunch of files.
Say you can store meta data to these files.
Say, one of these meta attributes were called "encryption"
Say everyone was allowed to look at these files, but since they are encrypted, only people who know how to decrypt them can actually read the contents.
Say, for every given value of "encryption", a group of people share the knowledge on how to decrypt files marked with that value.
Say you want to be able to do this programmatically, in an OS agnostic way (if possible)
What are the values you would use for "encryption"?
How would you store the keys?
How would you organize access to the keys?
I am currently leaning towards following implementation:
the value of the field "encryption" contains the name of a key, possibly also denoting the algorithm used
each user has access to a bunch of keys. This could be defined by roles the user has in an LDAP/ActiveDirectory like structure, or they could just be files in a secure directory in the users profile/home directory.
on viewing a file, the viewer (I'm trying to build a document management system) checks the users keys and decrypts the file if a matching key was found.
What encryption would you use? Symmetric (AES)? Or Asymmetric (what are the good ones)?
Using asymmetric keys would have the additional benefit of making a difference between reading a file and writing a file: Access to the private key is necessary for writing the file, access to the public key (only semi public, as only certain roles have access to it) would allow reading the file. Am I totally mistaken here?
What are common systems to solve these problems used in small to medium sized businesses?
EDIT: It seems there are no universal sollutions. So, I will state the problem I am trying to solve a little more clearly:
Imagine a Document Management System that operates in a distributed fashion: Each document is copied to various nodes in a (company controlled, private) P2P network. An algorithm for assuring redundancy of documents is used to ensure backups of all documents (including revisions). This system works as a service / daemon in the background and shovels documents to and fro.
This means, that users will end up with documents probably not meant for them to see on their local workstation (a company controlled PC or a laptop or something - the setting is such that a SME IT guy sets this all up and controls who is part of the P2P network).
This rules out directory access based schemes, as the user will probably be able to get to the data. Am I mistaken here? Could a local folder be encrypted such that it can only be accessed by a Domain user? How secure is that?
I am aware of users sharing decrypted versions of files - and that that is hard to suppress technically. This is not a problem I am trying to solve.
The encryption isn't the hard part, here. Understanding the business needs, and especially, what threats you're trying to protect against, is the hard part. Key management isn't a trivial thing.
I highly recommend the book "Applied Cryptography" to help you understand the protocol-level issues better.
This is a hard problem. If this is something really serious, you should not use the advice of amateur cryptographers on the internet.
That said, here's my musings:
I'd encrypt each file with a random symmetric key using AES. This encryption would be on a job that runs overnight, so the key changes overnight.
I'd encrypt the key of each file with the public key of everyone who has access to the file.
If someone loses access to files, they'd be unable to read the new copies the next day (they could still have copies locally of old versions).
I'd use gpg (runs on nearly all OS-es happily).
You misunderstand asymmetric crypto. Public key is given to everyone, Private key you keep yourself. If Alice encrypts something with Bob's Public key, only Bob can decrypt it. If Bob encrypts something with his Private key - everyone can decrypt it, and everyone knows it came from Bob cause only he has his Private Key.
EDIT: However, if you ignored everything I said and went a different route, and gave every FILE it's own pub/priv keypair... then you would rely on the public key be available ONLY to those you want to read the file, and the private key available to those you want to r/w. But that's a bit trickier, and relies heavily on people not being able to distribute keys. Overnight jobs to change keys could mitigate that problem, but then you have the problem of distributing new keys to users.
If I understand you correctly, you could use GNU Privacy Guard. It's cross-platform and open source. Basically, every user has a copy of GPG and a local "keychain" with their "private keys" and "public keys". When you want to encrypt something, you use the person's public key, and the results can only be decrypted with their associated private key. A user can have more than one keypair, so you could give all administrators access to the "administrator role" private key, and each hold of they private key could decrypt documents encrypted with the "administrator role" public key.
The cool part is that you can encrypt a file with multiple public keys, and any one of the corresponding private keys could then be used to decrypt it.
The difficulty of this problem is why many businesses default to using OS-specific solutions, such as Active Directory.
For OS-agnostic, you have to re-create a lot of user-management stuff that the specific OS and/or Network vendors have already built.
But it can be done. For the encryption itself - go with AviewAnew's answer.
I have to agree with Mark here:
Understanding the business needs, and especially, what threats you're trying to protect against, is the hard part
For example; are you worried that unauthorized users may gain access to sensitive files? You can use file-level access control on virtually any operating system to restrict users or groups from accessing files/directories.
Are you worried that authorized users may copy the files locally and then lose their laptop? There are a number of os-level encryption facilities that provide varying degrees of protection. I personally recommend TrueCrypt for thumb drives and other portable media, and Windows Vista now include BitLocker which provides a different level of protection.
Another variation of the lost-laptop theme is the lost-backup theme, and many backup vendors now include encryption schemes for your tape backups for just this reason.
Finally, if you're worried that authorized users may share the files with unauthorized users then you may be trying to solve the wrong problem. Authorized users who can decrypt these files can just as easily share a new unencrypted version of the same document.
What you need is public-key encryption using either OpenPGP or X.509 certificates. In both cases you can encrypt the single block of data for multiple "recipients" using their OpenPGP keys or X.509 certificates respectively. In X.509 the standards for encrypting the data this way are PKCS#7 and CMS (defined in some RFC, I forgot the number). You would need to employ some key revocation checking in order to prevent access for those people, who were given access before but don't have it now.