I'm designing an API that uses application level encryption to protect sensitive information in a database.
A simple example of the problem is a user table with the following fields:
UserID
Name
Address
Telephone Number
ClientIdentifier
In the above table all fields are sensitive and should be encrypted apart from UserID which is just the primary key of the table. While the primary key exists for foreign key constraints the actual identifying value for the record is ClientIdentifier. This is an ID for the user controlled by the consumer of the API.
When a consumer of the API wishes to create a user record they pass all the details (including the ClientIdentifier) to us which we store. When they want to retrieve those details they again pass us the ClientIdentifier. For this use case, they cannot use the UserID.
It's likely that the ClientIdentifier could be public knowledge e.g. an email address or account number and can be traced back to a real person. Therefore we have to secure ClientIdentifier as the existence of a record would imply that the person exists on our system.
I see a couple of options.
Hash the ClientIdentifier. The downside to this is that the ClientIdentifier is likely to follow a fixed format and be vulnerable to brute force attacks.
Encrypt all the data but in the case of ClientIdentifier use a fixed IV. The downside here is that an attacker who had access to both the API and the database could execute plain text attacks on the system.
I'm leaning towards to the second option as the plain text attack can probably be mitigated with monitoring of the encryption service, whereas the hashing option could be broken reasonably quickly if a snapshot of the database was lost.
So my question is, do you think I'm on the right track or are there any better alternatives?
Edit: It's possible that we could have multiple records in the database with the same ClientIdentifier. Given a plaintext ID we should be able to select all those records.
If you're set on encrypting both the user id and the password, you might want to take a look at bcrypt. Its an adaptive hashing algorithm, and you can up the encryption complexity whenever you want. You'll have to use a system-wide salt, which will still make this vulnerable to a dictionary attack. Your best bet is to randomly generate an account # (with numbers and letters) to use for the client id, that should negate any dictionary attack. Be sure to make it long enough to be secure.
In general, the user id isn't encrypted because there is no way to salt the hash of the user id, which makes it vulnerable to attacks. Before you jump through flaming hoops, I'd make sure that you NEED to encrypt both the client id and password.
Related
There are so many articles on stack-overflow and security.stackexchange on storing encryption keys, but I am still confused, so that's why I decided to ask again here.
Basically, I am creating an Email client for education purpose, in that Users can create account where they enter there Email-ID and Password. I am looking for secure way to save the information.
I will be
Encrypting the Email-ID and Password
and storing the encryption key on the user PC because I don't want the user to type in password every time he sends and Email
From reading I have understood that,
I need to store the encryption key in a separate location, so that it will be difficult to find by an hacker, But the problem here is that my application is written in Python and it will be open source application, so hacker can view the source code and get the path of the directory where the key is stored.
Second solution is that I can have a master password which will be used as a key, when the user opens the application for the first time after starting the computer, the application will ask for the master password, then I can store the key in RAM.
Looking at all the articles on internet on this topic this is a repetition, but I am sill learning to make applications and for the last two days I going in a loop with no success.
OS: Linux Ubuntu 14.04
Programming Language/Framework: Python/Gtk+
Your understanding is correct.
It's impossible to prevent a attacker with access to the local key from accessing the password. Obscuring the path where it is stored provides virtually zero additional security - any attacker with the know-how necessary to perform the decryption will easily bypass such a mechanism.
The only secure way to do this is storing the key (or a key to the key) out of the computer - in the user's mind, in the case of the master password mechanism.
If you end up using a master password, don't forget to use a proper key derivation function, ideally with a key-stretching mechanism, such as PBKDF2 or bcrypt. Never use a password as a key directly (or even a simple hash of the password.
I'm asking at a purely abstract level (no code required).
Some data is to be stored on a web-server/database on behalf of a user. The data is encrypted using the user's password as the key, ensuring that if the server or database is compromised, it is relatively useless to the attacker.
The problem is that the standard process of resetting a user's password (sending them a link with a token) won't work, because there is no way to decrypt and re-encrypt the data without knowing the previous password.
Are there any other ways around this?
When you say that you are willing to reset the password using some piece of automated authentication, then what you're really saying is that there are two passwords: the "normal" password and the "authentication" password. The solution to your problem is to encrypt the file using a random key and then encrypt the key using each of the passwords.
As a concrete example:
User provides a password "Aw3som1"
User also provides his high school mascot: "the Chipmunks"
Just to make it really complete, let's assume your authentication scheme is very forgiving like many are. You would accept just "chipmunks" (or "CHIPMUNKS" or maybe even "chipmunk") rather than "the Chipmunks." But whatever it is, your scheme must be deterministic. Every possible answer you will accept must resolve to the same hash. In this case, I'm going to assume that you lowercase the security answer, remove articles, and reduce to singular if it's plural. Then you prepend the class of question. So your secondary password is something like "mascot:chipmunk".
You now make up a random 16-bytes and use that key to encrypt the data. You then use normal password-based encryption techniques (e.g. PBKDF2) to encrypt the key with each of your passwords above. Then you throw away the key and the passwords.
When you need to do a reset, decrypt the real key with the provided password ("mascot:chipmunk") and re-encrypt the key with the new password (and "mascot:chipmunk" again).
The one usability issue is that a password reset invalidates all the other security answers, and the user must reconfigure them all. If that's a problem, you could put all the security answers into a bundle and encrypt that using the same technique as the data. (i.e. the security answers are encrypted against all of the security answers.)
This approach of course creates two (or more) passwords that can unlock the data, and so dramatically drops brute-force search time. You should consider that when scaling things. That said, your safety margins should generally be several orders of magnitude, so even a few passwords should be workable for many situations. Remember also that the security questions live in a tiny key space, particularly things like "mascot" or "make of car" which probably only have a few dozen likely values (unless you went to my high school which had a truly bizarre mascot…) That just means that aggressively tuning PBKDF2 is even more important.
Of course the existence of any kind of password reset system is going to make targeted attacks much easier. But that's true no matter how you implement the encryption.
But what if there isn't even a security question? What if you'll reset the password based on an email address alone? Well, then the email address is the password. That's problematic in the face of a stolen database, and it's hard to fix because there is no real secret (and encryption requires a secret). But all is not completely lost. One thing you can do is to never include the actual email address in the database. Think of the email address as a password (since it is in this case). Store a hash. You can't use a random salt on it, but you could still use some single salt for the whole database.
So my email is test#example.com. You treat that as a password, salt it with "mygreatservice" and run it through PBKDF2. To login, just treat it like a password again.
An attacker, though, has to guess email addresses one at a time. That's not incredibly hard, but he can't just decrypt the entire database in one go, and he'll never get the data from emails he doesn't guess. (And you did choose a really easy reset mechanism.)
Why do you need to un-encrypt the password? If they have forgotten then you will have to verify them based on the fact they had access to the email account where the reset token was sent. Ideally you would include some other form of question/proof of identity on the reset page like a secret question.
Also, you're best not storing the password using reversible encryption but instead using a strong salted hash which is one way.
We have a typical web-based login system. We want customers to have the ability to generate a "login link" that doesn't ever expire and includes their password.
We want to therefore create a link which includes their password in encrypted form.
NOTE: I know the best way is a lookup table where each link has a unique key... I won't go into why that won't work for us; it won't.
I'm not familiar with public key encryption.. Maybe that's what I need?
Here's what I'm thinking. This still might not be enough, so please let me know:
Definition: user-password refers to the user's password that we want in encrypted form in the link.
We cannot use one encryption key for all user-passwords because it may be possible to derive the encryption key by generating tons of links, so...
Use a standard form of symmetric encryption.
The server has a text file with 1000 complex encryption keys.
When a link is generated (using the PHP script that has that list), one of the 1000 encyption keys is chosen to encrypt their plaintext password (chosen in sequence, not at random to prevent the same from being chosen close to the same time).
Before encrypting the user's password, add something like "s345lm34l5k342342343534432324sdfs" to the start of it, to "salt" it. (Ex: password1 becomes s345lm34l5k342342343534432324sdfspassword1). "Salting" makes it harder to decrypt against dictionary attacks. This salt is kept private. But, of course, there's the risk it can be compromised and it's one salt for all passwords, so...
In addition, there's a second randomly generated salt added to the password. This salt is encrypted with a single strong password. Because both the salt and the password encrypting it is a random pattern of bytes, it makes it harder to determine the salt.
The link maker tool only lets you generate 15 links every 10 minutes, and then locks out the IP.
The link maker tool doesn't not generate links unless the user/password provided to it is actual a functioning user/password combination. That way, if someone is just try to generate links to determine the encryption info, it won't work. Still, theoretically, they could obtain a valid user/password and try to brute force.
Is this secure?
No, because the encrypted password has become the password. Have you tried SSL? SSL should solve your problems, you can just use a plain URL on HTTP level.
I want to encrypt user's personal data then save them in database .
the encryption must be done in application ( I can't do that in sql server side )
now I wonder if it's possible to use each user's password to encrypt and later decrypt their data ? what are pros and cons of this approach /
One big 'con': what if the user changes his/her password? Then you would need to re-encrypt all data!
You've said that you want to store secure personal data of a user. Doing this unless the personal info. is extremely sensitive is generally NOT recommended for a number of reasons. What is commonly done however is hashing + salting of the user's password.
This page has a good explanation on how hashing and salting works and why it's better than encrypting, and then decrypting the password.
http://net.tutsplus.com/tutorials/php/understanding-hash-functions-and-keeping-passwords-safe/
As for encrypting the user's personal information, just like a password we can use a custom salt + hashing algorithm that's quite simple but effective on our application to use the custom hash equivalent of the userID which is expected to be permanent, static and persistent forever.
Since the uID (or a specialized unique string for every user) can be hidden from normal public and we ensure that our custom shared function cannot be accessed from unauthorized sources, we have a solid secured system.
This means, we hash+salt personal info based on a unique string such as a userID and a hash+salt the user's password aswell. For the personal information to be decrypted, both the userID hash and password hash should match with the database.
A better approach would just be to use known encryption protocols within your program. Data sent via HTTPS TLS for example is quite secure if implemented right.
An organization I work for has a few different websites they use on a daily basis. I've been asked to develop a web application (using ASP.NET) that can access/synthesize information from these and display it in one location. Unfortunately, one of the websites does not support OAuth or anything similar, so I need to store their login credentials in a database.
My first thought was to use their credentials for my site as a key to encrypt their credentials for the remote site. For example: Bob logs in to my website with the password hunter2. Using that password, I decrypt Bob's credentials for www.example.com and log in as Bob there. Since I don't need to access example.com unless Bob is on my site, I can discard the decrypted credentials once he's done.
My assumption that simply using hunter2 (or whatever Bob's password is) isn't enough and that there is a "standard" way that I haven't been able to find on Google or Stack Overflow.
If you can't avoid storing the passwords on the server, then encrypting with the user's "master" password (e.g. "hunter2") is your best bet. No other approach offers protection in the event that the server is compromised. Now... how much protection you get hinges entirely on the complexity of the user's master password. I'll offer my analysis of the security of this scheme at the end, but before that, let me review the pitfalls to avoid.
First—and I assume that you already know this, but—you must not store the user's master password anywhere. Ok, with assumptions out of the way...
Do not use the user's actual password as the key to the encryption function.
Consider what would be possible if you did: what if an attacker managed to download your entire users table, complete with encrypted example.com passwords? We all know that user chosen passwords are easy to guess. What would stop the attacker from repeatedly decrypting the encrypted example.com password, trying 40 million commonly used passwords as the key, discarding any result that doesn't look like a password (that is, the decrypted result does not appear in the wordlist)? AES is designed to be fast. While not an apples to apples comparison, a sense of the speed of AES should be imparted when you consider that an encrypted version of the aforementioned 500mb wordlist could be decrypted in about one second on modern hardware. Worse yet, the attacker would not only get the example.com password, they would also have the key used for encryption, or in other words, the user's master password!
That, in a nutshell, is why you need to use a key derivation function (KDF). A KDF will ideally protect you in three ways:
Require a non-trivial amount of time to compute each key. A user can wait one second for the server to turn their password into a key. An attacker may be less inclined to wait 40,000,000 seconds—see analysis below.
Salt the password. Without salt, an attacker could brute-force the entire users table in one pass, not to mention make a space-time tradeoff.
Prevent recovery of the master password, even if the attacker recovers the encryption key.
One such KDF that provides all three is PBKDF2. Conveniently, there is an implementation built in to .NET:
public static byte[] DeriveKey(int keyBitSize, string password, byte[] salt) {
const int iterations = 1<<12; // Once set, any change will break decryption.
using (var kdf = new Rfc2898DeriveBytes(password, salt, iterations)) {
return kdf.GetBytes(keyBitSize);
}
}
Analysis
40 million seconds is less than 500 days. Since wordlists are usually ordered with the most commonly used passwords first, the attacker has a good chance of finding the password in significantly less than half the time it takes to try the entire wordlist. As a final wrinkle, it is possible to try keys in parallel: a 500-node botnet could try the entire wordlist in a day.
That's the problem with relying on the user's password for encryption security. You can choose to accept this risk or you can decide not to store the user's password on the server. If you decide to store encrypted passwords on the server, you can mitigate the risk by increasing the complexity requirements for the user's master password.
Whatever approach you take will have issues that will leave you doubting yourself. You need to balance the solutions against your environment & see what best fits.
Will each user of your app have an account on the remote system? How are users authenticated by your system? Will your app run in a trusted environment (eg corporate network).
I wrote a similar app to front an internal system that had it's own username & passwords. My app used Windows Integrated auth to figure out who the user was. it then asked for their password to the remote system & encrypted that value using a hard-coded key & stored the value in the DB. It could then retrieve the value, decrypt it & supply it to the remote system when needed.
Now in a non-trusted environment, someone could obtain my binaries & work out what they key was & get all the passwords. That would be bad. But on a corporate network, if they did that, they should come & work for me.
You already have access to the user's clear-text secondary passwords, so regardless of whether or not you throw away the encryption key, you're still responsible for their safety when it is in the clear. Keep that in mind when you handle the passwords.
If you use the user's own password to encrypt, you are reducing the strength of all their secondary passwords to the strength of the primary one. This is probably bad since a) user passwords are notoriously weak already (I know, why do we even bother in the first place?) and b) even with the strongest of passwords, it still won't match the strength of a solidly random 256 bit AES key.
My suggestion is to consider having a single AES key that encrypts their clear-text secondary passwords. Then, guard the AES key well. It may make sense, instead to have a root AES key that encrypts many sub-keys, one for each user. You'll have to do a risk analysis, I suppose.