Is it recommended that I use an initialization vector to encrypt/decrypt my data? Will it make things more secure? Is it one of those things that need to be evaluated on a case by case basis?
To put this into actual context, the Win32 Cryptography function, CryptSetKeyParam allows for the setting of an initialization vector on a key prior to encrypting/decrypting. Other API's also allow for this.
What is generally recommended and why?
An IV is essential when the same key might ever be used to encrypt more than one message.
The reason is because, under most encryption modes, two messages encrypted with the same key can be analyzed together. In a simple stream cipher, for instance, XORing two ciphertexts encrypted with the same key results in the XOR of the two messages, from which the plaintext can be easily extracted using traditional cryptanalysis techniques.
A weak IV is part of what made WEP breakable.
An IV basically mixes some unique, non-secret data into the key to prevent the same key ever being used twice.
In most cases you should use IV. Since IV is generated randomly each time, if you encrypt same data twice, encrypted messages are going to be different and it will be impossible for the observer to say if this two messages are the same.
Take a good look at a picture (see below) of CBC mode. You'll quickly realize that an attacker knowing the IV is like the attacker knowing a previous block of ciphertext (and yes they already know plenty of that).
Here's what I say: most of the "problems" with IV=0 are general problems with block encryption modes when you don't ensure data integrity. You really must ensure integrity.
Here's what I do: use a strong checksum (cryptographic hash or HMAC) and prepend it to your plaintext before encrypting. There's your known first block of ciphertext: it's the IV of the same thing without the checksum, and you need the checksum for a million other reasons.
Finally: any analogy between CBC and stream ciphers is not terribly insightful IMHO.
Just look at the picture of CBC mode, I think you'll be pleasantly surprised.
Here's a picture:
http://en.wikipedia.org/wiki/Block_cipher_modes_of_operation
link text
If the same key is used multiple times for multiple different secrets patterns could emerge in the encrypted results. The IV, that should be pseudo random and used only once with each key, is there to obfuscate the result. You should never use the same IV with the same key twice, that would defeat the purpose of it.
To not have to bother keeping track of the IV the simplest thing is to prepend, or append it, to the resulting encrypted secret. That way you don't have to think much about it. You will then always know that the first or last N bits is the IV.
When decrypting the secret you just split out the IV, and then use it together with the key to decrypt the secret.
I found the writeup of HTTP Digest Auth (RFC 2617) very helpful in understanding the use and need for IVs / nonces.
Is it one of those things that need to be evaluated on a case by case
basis?
Yes, it is. Always read up on the cipher you are using and how it expects its inputs to look. Some ciphers don't use IVs but do require salts to be secure. IVs can be of different lengths. The mode of the cipher can change what the IV is used for (if it is used at all) and, as a result, what properties it needs to be secure (random, unique, incremental?).
It is generally recommended because most people are used to using AES-256 or similar block ciphers in a mode called 'Cipher Block Chaining'. That's a good, sensible default go-to for a lot of engineering uses and it needs you to have an appropriate (non-repeating) IV. In that instance, it's not optional.
The IV allows for plaintext to be encrypted such that the encrypted text is harder to decrypt for an attacker. Each bit of IV you use will double the possibilities of encrypted text from a given plain text.
For example, let's encrypt 'hello world' using an IV one character long. The IV is randomly selected to be 'x'. The text that is then encrypted is then 'xhello world', which yeilds, say, 'asdfghjkl'. If we encrypt it again, first generate a new IV--say we get 'b' this time--and encrypt like normal (thus encrypting 'bhello world'). This time we get 'qwertyuio'.
The point is that the attacker doesn't know what the IV is and therefore must compute every possible IV for a given plain text to find the matching cipher text. In this way, the IV acts like a password salt. Most commonly, an IV is used with a chaining cipher (either a stream or block cipher). In a chaining block cipher, the result of each block of plain text is fed to the cipher algorithm to find the cipher text for the next block. In this way, each block is chained together.
So, if you have a random IV used to encrypt the plain text, how do you decrypt it? Simple. Pass the IV (in plain text) along with your encrypted text. Using our fist example above, the final cipher text would be 'xasdfghjkl' (IV + cipher text).
Yes you should use an IV, but be sure to choose it properly. Use a good random number source to make it. Don't ever use the same IV twice. And never use a constant IV.
The Wikipedia article on initialization vectors provides a general overview.
Related
I am using node and the crypto module to encrypt and decrypt a large binary file. I encrypt the file using crypto.createCipheriv and decrypt it using crypto.createDecipheriv.
For the encryption I use a random IV as follows:
const iv = crypto.randomBytes(16);
const encrypt = crypto.createCipheriv('aes-128-cbc', key, iv)
What I don't understand, do I need to pass a random IV for createDecipheriv as well? The SO here says:
The IV needs to be identical for encryption and decryption.
Can the IV be static? And if it can't, is it considered to be a secret? Where would I store the IV? In the payload?
If I use different random IVs for the encryption and decryption, my payload gets decrypted but the first 16 bytes are corrupt. This means, it looks like the IV needs to be the same but from a security perspective there is also not much value as the payload is decrypted except 16 bytes.
Can anyone elaborate what the go-to approach is? Thanks for your help!
The Key+IV pair must never be duplicated on two encryptions using CBC. Doing so leaks information about the first block (in all cases), and is creates duplicate cipher texts (which is a problem if you ever encrypt the same message prefix twice).
So, if your key changes for every encryption, then your IV could be static. But no one does that. They have a key they reuse. So the IV must change.
There is no requirement that it be random. It just shouldn't repeat and it must not be predictable (in cases where the attacker can control the messages). Random is the easiest way to do that. Anything other than random requires a lot of specialized knowledge to get right, so use random.
Reusing a Key+IV pair in CBC weakens the security of the cipher, but does not destroy it, as in CTR. IV reused with CTR can lead to trivial decryptions. In CBC, it generally just leaks information. It's a serious problem, but it is not catastrophic. (Not all insecure configurations are created equal.)
The IV is not a secret. Everyone can know it. So it is typically prepended to the ciphertext.
For security reasons, the IV needs to be chosen to meet cryptographic randomness security requirements (i.e. use crypto.randomBytes( ) in node). This was shown in Phil Rogaway's research paper. The summary is in Figure 1.2 of the paper, which I transcribe here:
CBC (SP 800-38A): An IV-based encryption scheme, the mode is secure as a probabilistic encryption scheme, achieving indistinguishability from random bits, assuming a random IV. Confidentiality is not achieved if the IV is merely a nonce, nor if it is a nonce enciphered under the same key used by the scheme, as the standard incorrectly suggests to do.
The normal way to implement this is to include the IV prepended to the ciphertext. The receiving party extracts the IV and then decrypts the ciphertext. The IV is not a secret, instead it is just used to bring necessary security properties into the mode of operation.
However, be aware that encryption with CBC does not prevent people from tampering with the data. If an attacker fiddles with ciphertext bits within a block, it affects exactly two plaintext blocks, one of which is in a very controlled way.
To make a very long story short, GCM is a better mode to use to prevent such abuses. In that case, you do not need a random IV, but instead you must never let the IV repeat (in cryptography, we call this property a "nonce"). Luke Park gives an example of how to implement it, here. He uses randomness for the nonce, which achieves the nonce property for all practical purposes (unless you are encrypting 2^48 texts, which is crazy large).
But whatever mode you do, you must never repeat an IV for a given key, which is a very common mistake.
I'm looking at an C# AES asymmetric encryption and decryption example here and not sure if i should store the IV in a safe place (also encrypted??). Or i can just attach it to the encrypted text for using later when i with to decrypt. From a short reading about AES it seems it's not needed at all for decryption but i'm not sure i got it right and also the aes.CreateDecryptor(keyBytes, iv) need it as parameter.
I use a single key for all encryptions.
It's fairly standard to transmit the encrypted data as IV.Concat(cipherText). It's also fairly standard to put the IV off to the side, like in PKCS#5.
The IV-on-the-side approach matches more closely with how .NET wants to process the data, since it's somewhat annoying to slice off the IV to pass it separately to the IV parameter (or property), and then to have a more complicated slicing operation with the ciphertext (or recovered plaintext).
But the IV is usually transmitted in the clear either way.
So, glue it together, or make it a separate column... whatever fits your program and structure better.
Answer: IV is necessary for decryption as long as the content has been encrypted with it. You don't need to encrypt or hide the IV. It may be public.
--
The purpose of the IV is to be combined to the key that you are using, so it's like you are encrypting every "block of data" with a different "final key" and then it guarantees that the cipher data (the encrypted one) will always be different along the encryption (and decryption) process.
This is a very good illustration of what happens IF YOU DON'T use IV.
Basically, the encryption process is done by encrypting the input data in blocks. So during the encryption of this example, all the parts of the image that have the same color (let's say the white background) will output the same "cipher data" if you use always the same key, then a pattern can still be found and then you didn't hide the image as desired.
So combining a different extra data (the IV) to the key for each block is like you are using a different "final key" for each block, then you solve your problem.
I have a tcl/tk based tool, which uses network password for authentication. Issue is that, it is saving password in the logs/history. So objective is to encrypt the password.
I tried to use aes package. But at the very beginning aes::init asks for keydata and initialization vector (16 byte). So how to generate IV and keydata. Is is some Random number? I am a novice in encryption algorithms.
If you have the password in the logs/history, why not fix the bug of logging/storing it in the first place?
Otherwise there are distinct things you might want:
A password hashing scheme like PBKDF2, bcrypt, argon2 etc. to store a password in a safe way and compare some user input to it. This is typically the case when you need to implement some kind of authentication with passwords on the server side.
A password encryption and protection scheme like AES. You need a password to authenticate to some service automatically, and it requires some form of cleartext password.
You have some secret data and need to securly store it to in non cleartext form.
If you have case 1, don't use the aespackage, it is the wrong tool for the job. If you have case 2, the aes package might help you, but you just exchanged the problem of keeping the password secret with the other problem of keeping the key secret (not a huge win). So the only viable case where aes is an option might be 3.
Lets assume you need to store some secret data in a reversible way, e.g. case 3 from above.
AES has a few possible modes of operation, common ones you might see are ECB, CBC, OFB, GCM, CTR. The Tcllib package just supports ECB and CBC, and only CBC (which is the default) is really an option to use.
Visit Wikipedia for an example why you should never use ECB mode.
Now back to your actual question:
Initialization Vector (IV)
This is a random value you pick for each encryption, it is not secret, you can just publish it together with the encrypted data. Picking a random IV helps to make two encrypted blocks differ, even if you use the same key and cleartext.
Secret Key
This is also a random value, but you must keep it secret, as it can be used for encryption and decryption. You often have the same key for multiple encryptions.
Where to get good randomness?
If you are on Linux, BSD or other unixoid systems just read bytes from /dev/urandom or use a wrapper for getrandom(). Do NOT use Tcls expr {rand()} or similar pseudorandom number generators (PRNG). On Windows TWAPI and the CryptGenRandom function would be the best idea, but sadly there is no Tcl high level wrapper included.
Is that enough?
Depends. If you just want to hide a bit of plaintext from cursory looks, maybe. If you have attackers manipulating your data or actively trying to hack your system, less so. Plain AES-CBC has a lot of things you can do wrong, and even experts did wrong (read about SSL/TLS 1.0 problems with AES-CBC).
Final words: If you are a novice in encryption algorithms, be sure you understand what you want and need to protect, there are a lot of pitfalls.
If I read the Tcler's Wiki page on aes, I see that I encrypt by doing this:
package require aes
set plaintext "Some super-secret bytes!"
set key "abcd1234dcba4321"; # 16 bytes
set encrypted [aes::aes -dir encrypt -key $key $plaintext]
and I decrypt by doing:
# Assuming the code above was run...
set decrypted [aes::aes -dir decrypt -key $key $encrypted]
Note that the decrypted text has NUL (zero) bytes added on the end (8 of them in this example) because the encryption algorithm always works on blocks of 16 bytes, and if you're working with non-ASCII text then encoding convertto and encoding convertfrom might be necessary.
You don't need to use aes::init directly unless you are doing large-scale streaming encryption. Your use case doesn't sound like it needs that sort of thing. (The key data is your “secret”, and the initialisation vector is something standardised that usually you don't need to set.)
Right now, this is what I am doing:
1. SHA-1 a password like "pass123", use the first 32 characters of the hexadecimal decoding for the key
2. Encrypt with AES-256 with just whatever the default parameters are
^Is that secure enough?
I need my application to encrypt data with a password, and securely. There are too many different things that come up when I google this and some things that I don't understand about it too. I am asking this as a general question, not any specific coding language (though I'm planning on using this with Java and with iOS).
So now that I am trying to do this more properly, please follow what I have in mind:
Input is a password such as "pass123" and the data is
what I want to encrypt such as "The bank account is 038414838 and the pin is 5931"
Use PBKDF2 to derive a key from the password. Parameters:
1000 iterations
length of 256bits
Salt - this one confuses me because I am not sure where to get the salt from, do I just make one up? As in, all my encryptions would always use the salt "F" for example (since apparently salts are 8bits which is just one character)
Now I take this key, and do I hash it?? Should I use something like SHA-256? Is that secure? And what is HMAC? Should I use that?
Note: Do I need to perform both steps 2 and 3 or is just one or the other okay?
Okay now I have the 256-bit key to do the encryption with. So I perform the encryption using AES, but here's yet another confusing part (the parameters).
I'm not really sure what are the different "modes" to use, apparently there's like CBC and EBC and a bunch of others
I also am not sure about the "Initialization Vector," do I just make one up and always use that one?
And then what about other options, what is PKCS7Padding?
For your initial points:
Using hexadecimals clearly splits the key size in half. Basically, you are using AES-128 security wise. Not that that is bad, but you might also go for AES-128 and use 16 bytes.
SHA-1 is relatively safe for key derivation, but it shouldn't be used directly because of the existence/creation of rainbow tables. For this you need a function like PBKDF2 which uses an iteration count and salt.
As for the solution:
You should not encrypt PIN's if that can be avoided. Please make sure your passwords are safe enough, allow pass phrases.
Create a random number per password and save the salt (16 bytes) with the output of PBKDF2. The salt does not have to be secret, although you might want to include a system secret to add some extra security. The salt and password are hashed, so they may have any length to be compatible with PBKDF2.
No, you just save the secret generated by the PBKDF2, let the PBKDF2 generate more data when required.
Never use ECB (not EBC). Use CBC as minimum. Note that CBC encryption does not provide integrity checking (somebody might change the cipher text and you might never know it) or authenticity. For that, you might want to add an additional MAC, HMAC or use an encryption mode such as GCM. PKCS7Padding (identical to PKCS5Padding in most occurences) is a simple method of adding bogus data to get N * [blocksize] bytes, required by block wise encryption.
Don't forget to prepend a (random) IV to your cipher text in case you reuse your encryption keys. An IV is similar to a salt, but should be exactly [blocksize] bytes (16 for AES).
I am creating an encryption scheme with AES in cbc mode with a 256-bit key. Before I learned about CBC mode and initial values, I was planning on creating a 32-bit salt for each act of encryption and storing the salt. The password/entered key would then be padded with this salt up to 32 bits.
ie. if the pass/key entered was "tree," instead of padding it with 28 0s, it would be padded with the first 28 chars of this salt.
However, this was before I learned of the iv, also called a salt in some places. The question for me has now arisen as to whether or not this earlier method of salting has become redundant in principle with the IV. This would be to assume that the salt and the iv would be stored with the cipher text and so a theoretical brute force attack would not be deterred any.
Storing this key and using it rather than 0s is a step that involves some effort, so it is worth asking I think whether or not it is a practically useless measure. It is not as though there could be made, with current knowledge, any brute-force decryption tables for AES, and even a 16 bit salt pains the creation of md5 tables.
Thanks,
Elijah
It's good that you know CBC, as it is certainly better than using ECB mode encryption (although even better modes such as the authenticated modes GCM and EAX exist as well).
I think there are several things that you should know about, so I'll explain them here.
Keys and passwords are not the same. Normally you create a key used for symmetric encryption out of a password using a key derivation function. The most common one discussed here is PBKDF2 (password based key derivation function #2), which is used for PBE (password based encryption). This is defined in the latest, open PKCS#5 standard by RSA labs. Before entering the password need to check if the password is correctly translated into bytes (character encoding).
The salt is used as another input of the key derivation function. It is used to prevent brute force attacks using "rainbow tables" where keys are pre-computed for specific passwords. Because of the salt, the attacker cannot use pre-computed values, as he cannot generate one for each salt. The salt should normally be 8 bytes (64 bits) or longer; using a 128 bit salt would give you optimum security. The salt also ensures that identical passwords (of different users) do not derive the same key.
The output of the key derivation function is a secret of dkLen bytes, where dkLen is the length of the key to generate, in bytes. As an AES key does not contain anything other than these bytes, the AES key will be identical to the generated secret. dkLen should be 16, 24 or 32 bytes for the key lengths of AES: 128, 192 or 256 bits.
OK, so now you finally have an AES key to use. However, if you simply encrypt each plain text block with this key, you will get identical result if the plain text blocks are identical. CBC mode gets around this by XOR'ing the next plain text block with the last encrypted block before doing the encryption. That last encrypted block is the "vector". This does not work for the first block, because there is no last encrypted block. This is why you need to specify the first vector: the "initialization vector" or IV.
The block size of AES is 16 bytes independent of the key size. So the vectors, including the initialization vector, need to be 16 bytes as well. Now, if you only use the key to encrypt e.g. a single file, then the IV could simply contain 16 bytes with the value 00h.
This does not work for multiple files, because if the files contain the same text, you will be able to detect that the first part of the encrypted file is identical. This is why you need to specify a different IV for each encryption you perform with the key. It does not matter what it contains, as long as it is unique, 16 bytes and known to the application performing the decryption.
[EDIT 6 years later] The above part is not entirely correct: for CBC the IV needs to be unpredictable to an attacker, it doesn't just need to be unique. So for instance a counter cannot be used.
Now there is one trick that might allow you to use all zero's for the IV all the time: for each plain text you encrypt using AES-CBC, you could calculate a key using the same password but a different salt. In that case, you will only use the resulting key for a single piece of information. This might be a good idea if you cannot provide an IV for a library implementing password based encryption.
[EDIT] Another commonly used trick is to use additional output of PBKDF2 to derive the IV. This way the official recommendation that the IV for CBC should not be predicted by an adversary is fulfilled. You should however make sure that you do not ask for more output of the PBKDF2 function than that the underlying hash function can deliver. PBKDF2 has weaknesses that would enable an adversary to gain an advantage in such a situation. So do not ask for more than 256 bits if SHA-256 is used as hash function for PBKDF2. Note that SHA-1 is the common default for PBKDF2 so that only allows for a single 128 bit AES key.
IV's and salts are completely separate terms, although often confused. In your question, you also confuse bits and bytes, key size and block size and rainbow tables with MD5 tables (nobody said crypto is easy). One thing is certain: in cryptography it pays to be as secure as possible; redundant security is generally not a problem, unless you really (really) cannot afford the extra resources.
When you understand how this all works, I would seriously you to find a library that performs PBE encryption. You might just need to feed this the password, salt, plain data and - if separately configured- the IV.
[Edit] You should probably look for a library that uses Argon2 by now. PBKDF2 is still considered secure, but it does give unfair advantage to an attacker in some cases, letting the attacker perform fewer calculations than the regular user of the function. That's not a good property for a PBKDF / password hash.
If you are talking about AES-CBC then it is an Initialisation Vector (IV), not Salt. It is common practice to send the IV in clear as the first block of the encyphered message. The IV does not need to be kept secret. It should however be changed with every message - a constant IV means that effectively your first block is encrypted in ECB mode, which is not properly secure.