This is a pretty straightforward question which concerns the initialization vector (IV), when using symmetric encryption (e.g. AES) togheter with say CBC.
My question: Should the IV change for each new plaintext or does it suffice to create one for each new session?
At the moment im using java for my implementation and the Cipher Class, and i noticed that it indeed creates a new IV first time, however that same IV is also used for later plaintexts as well.
Perhaps there is some reading resources about this topic?
Thanks :)
In CBC, for maximum security the IV must be unpredictable.
Should the IV change for each new plaintext or
does it suffice to create one for each new session?
It's not totally clear to me the meaning you give to "plaintext" and to "session".
If you mean that for each session there is a new, fresh key, and that the data exchanged is chopped up in pieces and then encrypted, then CBC will be typically applied to the whole chain of pieces, meaning that the ciphertext block of piece Pn-1 can be used as IV for piece Pn. You therefore need only one IV for the whole session.
Perhaps there is some reading resources about this topic?
Sure, NIST SP 800-38A, section 6.2.
Note: this answer is only about the IV in CBC mode encryption.
You need to create a new IV for each separate encryption "session" with the same key. For an IV to be cryptographically secure it should be indistinquishable from a random number to the attacker. Java by default uses a zero IV, which means you should not reuse the key to create other cipher texts.
So no, you should not reuse the session without setting a new IV. Basically, the only thing that is secure for any protocol is the source below. There are ways of creating a secure IV using other information in a protocol, but I won't go into that.
Cipher aes = Cipher.getInstance("AES/CBC/PKCS5Padding");
// repeat this for each cipher text
byte[] ivBytes = new byte[aes.getBlockSize()];
SecureRandom rnd = new SecureRandom();
rnd.nextBytes(ivBytes);
aes.init(Cipher.ENCRYPT_MODE, sk, new IvParameterSpec(ivBytes));
// now prepend the ivBytes to the output, e.g. by writing it to a stream first
// remove and use as IV at the receiving side
[EDIT]: Forgot about the shorthand notation for this:
aes.init(Cipher.ENCRYPT_MODE, sk, new SecureRandom());
byte[] ivBytes = aes.getIV();
Note that the code above does not provide integrity protection.
Related
I am using node and the crypto module to encrypt and decrypt a large binary file. I encrypt the file using crypto.createCipheriv and decrypt it using crypto.createDecipheriv.
For the encryption I use a random IV as follows:
const iv = crypto.randomBytes(16);
const encrypt = crypto.createCipheriv('aes-128-cbc', key, iv)
What I don't understand, do I need to pass a random IV for createDecipheriv as well? The SO here says:
The IV needs to be identical for encryption and decryption.
Can the IV be static? And if it can't, is it considered to be a secret? Where would I store the IV? In the payload?
If I use different random IVs for the encryption and decryption, my payload gets decrypted but the first 16 bytes are corrupt. This means, it looks like the IV needs to be the same but from a security perspective there is also not much value as the payload is decrypted except 16 bytes.
Can anyone elaborate what the go-to approach is? Thanks for your help!
The Key+IV pair must never be duplicated on two encryptions using CBC. Doing so leaks information about the first block (in all cases), and is creates duplicate cipher texts (which is a problem if you ever encrypt the same message prefix twice).
So, if your key changes for every encryption, then your IV could be static. But no one does that. They have a key they reuse. So the IV must change.
There is no requirement that it be random. It just shouldn't repeat and it must not be predictable (in cases where the attacker can control the messages). Random is the easiest way to do that. Anything other than random requires a lot of specialized knowledge to get right, so use random.
Reusing a Key+IV pair in CBC weakens the security of the cipher, but does not destroy it, as in CTR. IV reused with CTR can lead to trivial decryptions. In CBC, it generally just leaks information. It's a serious problem, but it is not catastrophic. (Not all insecure configurations are created equal.)
The IV is not a secret. Everyone can know it. So it is typically prepended to the ciphertext.
For security reasons, the IV needs to be chosen to meet cryptographic randomness security requirements (i.e. use crypto.randomBytes( ) in node). This was shown in Phil Rogaway's research paper. The summary is in Figure 1.2 of the paper, which I transcribe here:
CBC (SP 800-38A): An IV-based encryption scheme, the mode is secure as a probabilistic encryption scheme, achieving indistinguishability from random bits, assuming a random IV. Confidentiality is not achieved if the IV is merely a nonce, nor if it is a nonce enciphered under the same key used by the scheme, as the standard incorrectly suggests to do.
The normal way to implement this is to include the IV prepended to the ciphertext. The receiving party extracts the IV and then decrypts the ciphertext. The IV is not a secret, instead it is just used to bring necessary security properties into the mode of operation.
However, be aware that encryption with CBC does not prevent people from tampering with the data. If an attacker fiddles with ciphertext bits within a block, it affects exactly two plaintext blocks, one of which is in a very controlled way.
To make a very long story short, GCM is a better mode to use to prevent such abuses. In that case, you do not need a random IV, but instead you must never let the IV repeat (in cryptography, we call this property a "nonce"). Luke Park gives an example of how to implement it, here. He uses randomness for the nonce, which achieves the nonce property for all practical purposes (unless you are encrypting 2^48 texts, which is crazy large).
But whatever mode you do, you must never repeat an IV for a given key, which is a very common mistake.
I'm using Googles Tink DeterministicAead. What I'd like to achieve, is to use the same key over a long time, but change the ciphertext for a certain plaintext every 24 hours (for GDPR reasons).
Can I pick a new initialization vector somehow once every 24 hours? The code I'm using is:
KeysetHandle keysetHandle = KeysetHandle.generateNew(
DeterministicAeadKeyTemplates.AES256_SIV);
DeterministicAead daead = keysetHandle.getPrimitive(DeterministicAead.class);
byte[] ciphertext = daead.encryptDeterministically(clearText.getBytes(), "".getBytes());
ciphertext is indeed deterministic, so at least that part is good :-)
I want to be able to decrypt the cyphertext in BigQuery, so custom primitives worry me a bit.
Note: this is not about financial data or such, it is about being able to update recommendation models on the behavior of the ciphertext while at the same time complying with privacy requirements for the user.
By definition (from what I'm reading in those docs) there doesn't seem to be a way to specify an IV (as then it wouldn't be deterministic).
The DeterministicAead primitive seems to be AES with a MAC, without IV.
So, I think the only option here would be to rotate the key, or, add your own prepended/appended salt to the plaintext prior to encryption and store it for decryption.
Remember deterministic schemes aren't semantically secure.
For
`BDK = "0123456789ABCDEFFEDCBA9876543210"` `KSN = "FFFF9876543210E00008"`
The ciphertext generated was below
"C25C1D1197D31CAA87285D59A892047426D9182EC11353C051ADD6D0F072A6CB3436560B3071FC1FD11D9F7E74886742D9BEE0CFD1EA1064C213BB55278B2F12"`
which I found here. I know this cipher-text is based on BDK and KSN but how this 128 length cipher text was generated? What are steps involved in it or algorithm used for this? Could someone explain in simple steps. I found it hard to understand the documents I got while googled.
Regarding DUKPT , there are some explanations given on Wiki. If that doesn't suffice you, here goes some brief explanation.
Quoting http://www.maravis.com/library/derived-unique-key-per-transaction-dukpt/
What is DUKPT?
Derived Unique Key Per Transaction (DUKPT) is a key management scheme. It uses one time encryption keys that are derived from a secret master key that is shared by the entity (or device) that encrypts and the entity (or device) that decrypts the data.
Why DUKPT?
Any encryption algorithm is only as secure as its keys. The strongest algorithm is useless if the keys used to encrypt the data with the algorithm are not secure. This is like locking your door with the biggest and strongest lock, but if you hid the key under the doormat, the lock itself is useless. When we talk about encryption, we also need to keep in mind that the data has to be decrypted at the other end.
Typically, the weakest link in any encryption scheme is the sharing of the keys between the encrypting and decrypting parties. DUKPT is an attempt to ensure that both the parties can encrypt and decrypt data without having to pass the encryption/decryption keys around.
The Cryptographic Best Practices document that VISA has published also recommends the use of DUKPT for PCI DSS compliance.
How DUKPT Works
DUKPT uses one time keys that are generated for every transaction and then discarded. The advantage is that if one of these keys is compromised, only one transaction will be compromised. With DUKPT, the originating (say, a Pin Entry Device or PED) and the receiving (processor, gateway, etc) parties share a key. This key is not actually used for encryption. Instead, another one time key that is derived from this master key is used for encrypting and decrypting the data. It is important to note that the master key should not be recoverable from the derived one time key.
To decrypt data, the receiving end has to know which master key was used to generate the one time key. This means that the receiving end has to store and keep track of a master key for each device. This can be a lot of work for someone that supports a lot of devices. A better way is required to deal with this.
This is how it works in real-life: The receiver has a master key called the Base Derivation Key (BDK). The BDK is supposed to be secret and will never be shared with anyone. This key is used to generate keys called the Initial Pin Encryption Key (IPEK). From this a set of keys called Future Keys is generated and the IPEK discarded. Each of the Future keys is embedded into a PED by the device manufacturer, with whom these are shared. This additional derivation step means that the receiver does not have to keep track of each and every key that goes into the PEDs. They can be re-generated when required.
The receiver shares the Future keys with the PED manufacturer, who embeds one key into each PED. If one of these keys is compromised, the PED can be rekeyed with a new Future key that is derived from the BDK, since the BDK is still safe.
Encryption and Decryption
When data needs to be sent from the PED to the receiver, the Future key within that device is used to generate a one time key and then this key is used with an encryption algorithm to encrypt the data. This data is then sent to the receiver along with the Key Serial Number (KSN) which consists of the Device ID and the device transaction counter.
Based on the KSN, the receiver then generates the IPEK and from that generates the Future Key that was used by the device and then the actual key that was used to encrypt the data. With this key, the receiver will be able to decrypt the data.
Source
First, let me quote the complete sourcecode you linked and of which you provided only 3 lines...
require 'bundler/setup'
require 'test/unit'
require 'dukpt'
class DUKPT::DecrypterTest < Test::Unit::TestCase
def test_decrypt_track_data
bdk = "0123456789ABCDEFFEDCBA9876543210"
ksn = "FFFF9876543210E00008"
ciphertext = "C25C1D1197D31CAA87285D59A892047426D9182EC11353C051ADD6D0F072A6CB3436560B3071FC1FD11D9F7E74886742D9BEE0CFD1EA1064C213BB55278B2F12"
plaintext = "%B5452300551227189^HOGAN/PAUL ^08043210000000725000000?\x00\x00\x00\x00"
decrypter = DUKPT::Decrypter.new(bdk, "cbc")
assert_equal plaintext, decrypter.decrypt(ciphertext, ksn)
end
end
Now, you're asking is how the "ciphertext" was created...
Well, first thing we know is that it is based on "plaintext", which is used in the code to verify if decryption works.
The plaintext is 0-padded - which fits the encryption that is being tested by verifying decryption with this DecrypterTest TestCase.
Let's look at the encoding code then...
I found the related encryption code at https://github.com/Shopify/dukpt/blob/master/lib/dukpt/encryption.rb.
As the DecrypterTEst uses "cbc", it becomes apparent that the encrypting uses:
#cipher_type_des = "des-cbc"
#cipher_type_tdes = "des-ede-cbc"
A bit more down that encryption code, the following solves our quest for an answer:
ciphertext = des_encrypt(...
Which shows we're indeed looking at the result of a DES encryption.
Now, DES has a block size of 64 bits. That's (64/8=) 8 bytes binary, or - as the "ciphertext" is a hex-encoded text representation of the bytes - 16 chars hex.
The "ciphertext" is 128 hex chars long, which means it holds (128 hex chars/16 hex chars=) 8 DES blocks with each 64 bits of encrypted information.
Wrapping all this up in a simple answer:
When looking at "ciphertext", you are looking at (8 blocks of) DES encrypted data, which is being represented using a human-readable, hexadecimal (2 hex chars = 1 byte) notation instead of the original binary bytes that DES encryption would produce.
As for the steps involved in "recreating" the ciphertext, I tend to tell you to simply use the relevant parts of the ruby project where you based your question upon. Simply have to look at the sourcecode. The file at "https://github.com/Shopify/dukpt/blob/master/lib/dukpt/encryption.rb" pretty much explains it all and I'm pretty sure all functionality you need can be found at the project's GitHub repository. Alternatively, you can try to recreate it yourself - using the preferred programming language of your choice. You only need to handle 2 things: DES encryption/decryption and bin-to-hex/hex-to-bin translation.
Since this is one of the first topics that come up regarding this I figured I'd share how I was able to encode the ciphertext. This is the first time I've worked with Ruby and it was specifically to work with DUKPT
First I had to get the ipek and pek (same as in the decrypt) method. Then unpack the plaintext string. Convert the unpacked string to a 72 byte array (again, forgive me if my terminology is incorrect).
I noticed in the dukpt gem author example he used the following plain text string
"%B5452300551227189^HOGAN/PAUL ^08043210000000725000000?\x00\x00\x00\x00"
I feel this string is incorrect as there shouldn't be a space after the name (AFAIK).. so it should be
"%B5452300551227189^HOGAN/PAUL^08043210000000725000000?\x00\x00\x00\x00"
All in all, this is the solution I ended up on that can encrypt a string and then decrypt it using DUKPT
class Encrypt
include DUKPT::Encryption
attr_reader :bdk
def initialize(bdk, mode=nil)
#bdk = bdk
self.cipher_mode = mode.nil? ? 'cbc' : mode
end
def encrypt(plaintext, ksn)
ipek = derive_IPEK(bdk, ksn)
pek = derive_PEK(ipek, ksn)
message = plaintext.unpack("H*").first
message = hex_string_from_unpacked(message, 72)
encrypted_cryptogram = triple_des_encrypt(pek,message).upcase
encrypted_cryptogram
end
def hex_string_from_unpacked val, bytes
val.ljust(bytes * 2, "0")
end
end
boomedukpt FFFF9876543210E00008 "%B5452300551227189^HOGAN/PAUL^08043210000000725000000?"
(my ruby gem, the KSN and the plain text string)
2542353435323330303535313232373138395e484f47414e2f5041554c5e30383034333231303030303030303732353030303030303f000000000000000000000000000000000000
(my ruby gem doing a puts on the unpacked string after calling hex_string_from_unpacked)
C25C1D1197D31CAA87285D59A892047426D9182EC11353C0B82D407291CED53DA14FB107DC0AAB9974DB6E5943735BFFE7D72062708FB389E65A38C444432A6421B7F7EDD559AF11
(my ruby gem doing a puts on the encrypted string)
%B5452300551227189^HOGAN/PAUL^08043210000000725000000?
(my ruby gem doing a puts after calling decrypt on the dukpt gem)
Look at this: https://github.com/sgbj/Dukpt.NET, I was in a similar situation where i wondered how to implement dukpt on the terminal when the terminal has its own function calls which take the INIT and KSN to create the first key, so my only problem was to make sure the INIT key was generated the same way on the terminal as it is in the above mentioned repo's code, which was simple enough using ossl encryption library for 3des with ebc and applying the appropriate masks.
When passing symetrically encrypted data in a URL or possibly storing encrypted data in a cookie, is it resonable and/or nessassary and/or possible to also pass the Symetric Encryption IV (Salt) in the same URL? Is the idea of using Salt even valid in a stateless environment such as the web?
(I understand how salt works in a database given a list of names or accounts etc. but we can't save the salt given that we are passing data in a stateless environment.
Assuming a server side password that is used to encrypt data and then decrypt data, how can Salt be used? I guess a separate IV could be passed in the query string but is publicly exposing the salt ok?
Or can one generate a key and IV from the hash of a "password". Assuming the IV and Key come from non-overlapping areas of the hash, is this ok? (I realize that the salt / key will always be the same for a given password.)
EDIT: Typically using AES.
It is encouraged to generate random IVs for each encryption routine, and they can be passed along safely with the cipher text.
Edit:
I should probably ask what type of information you're storing and why you're using a salt with AES encryption, since salts are typically used for hashing, not symmetric encryption. If the salt is publicly available, it defeats the purpose of having it.
What you really need to do is ensure the strength of your key, because if an attacker has the salt, IV, and cipher text, a brute-force attack can easily be done on weaker keys.
You should not generate an initialization vector from the secret key. The initialization vector should be unpredictable for a given message; if you generated it from the key (or a password used to generate a key), the IV will always be the same, which defeats its purpose.
The IV doesn't need to be secret, however. It's quite common to send it with the ciphertext, unprotected. Incorporating the IV in the URL is a lot easier than trying to keep track of the IV for a given link in some server-side state.
Salt and IVs have distinct applications, but they do act in similar ways.
Cryptographic "salt" is used in password-based key derivation algorithms; storing a hashed password for authentication is a special case of this function. Salt causes the same password to yield different hashes, and thwarts "dictionary attacks", where a hacker has pre-computed hash values for common passwords, and built a "reverse-lookup" index so that they can quickly discover a password for a given hash. Like an IV, the salt used is not a secret.
An initialization vector is used with block ciphers like DES and AES in a feedback mode like CBC. Each block is combined with the next block when it is encrypted. For example, under CBC, the previous block cipher text is XOR-ed with the plain text of the current block before encryption. The IV is randomly generated to serve as a dummy initial block to bootstrap the process.
Because a different IV is (or should be, at least) chosen for each message, when the same message is encrypted with the same key, the resulting cipher text is different. In that sense, an IV is very similar to a salt. A cryptographic random generator is usually the easiest and most secure source for a salt or an IV, so they have that similarity too.
Cryptography is very easy to mess up. If you are not confident about what you are doing, you should consider the value of the information you are protecting, and budget accordingly to get the training or consultation you need.
Is it recommended that I use an initialization vector to encrypt/decrypt my data? Will it make things more secure? Is it one of those things that need to be evaluated on a case by case basis?
To put this into actual context, the Win32 Cryptography function, CryptSetKeyParam allows for the setting of an initialization vector on a key prior to encrypting/decrypting. Other API's also allow for this.
What is generally recommended and why?
An IV is essential when the same key might ever be used to encrypt more than one message.
The reason is because, under most encryption modes, two messages encrypted with the same key can be analyzed together. In a simple stream cipher, for instance, XORing two ciphertexts encrypted with the same key results in the XOR of the two messages, from which the plaintext can be easily extracted using traditional cryptanalysis techniques.
A weak IV is part of what made WEP breakable.
An IV basically mixes some unique, non-secret data into the key to prevent the same key ever being used twice.
In most cases you should use IV. Since IV is generated randomly each time, if you encrypt same data twice, encrypted messages are going to be different and it will be impossible for the observer to say if this two messages are the same.
Take a good look at a picture (see below) of CBC mode. You'll quickly realize that an attacker knowing the IV is like the attacker knowing a previous block of ciphertext (and yes they already know plenty of that).
Here's what I say: most of the "problems" with IV=0 are general problems with block encryption modes when you don't ensure data integrity. You really must ensure integrity.
Here's what I do: use a strong checksum (cryptographic hash or HMAC) and prepend it to your plaintext before encrypting. There's your known first block of ciphertext: it's the IV of the same thing without the checksum, and you need the checksum for a million other reasons.
Finally: any analogy between CBC and stream ciphers is not terribly insightful IMHO.
Just look at the picture of CBC mode, I think you'll be pleasantly surprised.
Here's a picture:
http://en.wikipedia.org/wiki/Block_cipher_modes_of_operation
link text
If the same key is used multiple times for multiple different secrets patterns could emerge in the encrypted results. The IV, that should be pseudo random and used only once with each key, is there to obfuscate the result. You should never use the same IV with the same key twice, that would defeat the purpose of it.
To not have to bother keeping track of the IV the simplest thing is to prepend, or append it, to the resulting encrypted secret. That way you don't have to think much about it. You will then always know that the first or last N bits is the IV.
When decrypting the secret you just split out the IV, and then use it together with the key to decrypt the secret.
I found the writeup of HTTP Digest Auth (RFC 2617) very helpful in understanding the use and need for IVs / nonces.
Is it one of those things that need to be evaluated on a case by case
basis?
Yes, it is. Always read up on the cipher you are using and how it expects its inputs to look. Some ciphers don't use IVs but do require salts to be secure. IVs can be of different lengths. The mode of the cipher can change what the IV is used for (if it is used at all) and, as a result, what properties it needs to be secure (random, unique, incremental?).
It is generally recommended because most people are used to using AES-256 or similar block ciphers in a mode called 'Cipher Block Chaining'. That's a good, sensible default go-to for a lot of engineering uses and it needs you to have an appropriate (non-repeating) IV. In that instance, it's not optional.
The IV allows for plaintext to be encrypted such that the encrypted text is harder to decrypt for an attacker. Each bit of IV you use will double the possibilities of encrypted text from a given plain text.
For example, let's encrypt 'hello world' using an IV one character long. The IV is randomly selected to be 'x'. The text that is then encrypted is then 'xhello world', which yeilds, say, 'asdfghjkl'. If we encrypt it again, first generate a new IV--say we get 'b' this time--and encrypt like normal (thus encrypting 'bhello world'). This time we get 'qwertyuio'.
The point is that the attacker doesn't know what the IV is and therefore must compute every possible IV for a given plain text to find the matching cipher text. In this way, the IV acts like a password salt. Most commonly, an IV is used with a chaining cipher (either a stream or block cipher). In a chaining block cipher, the result of each block of plain text is fed to the cipher algorithm to find the cipher text for the next block. In this way, each block is chained together.
So, if you have a random IV used to encrypt the plain text, how do you decrypt it? Simple. Pass the IV (in plain text) along with your encrypted text. Using our fist example above, the final cipher text would be 'xasdfghjkl' (IV + cipher text).
Yes you should use an IV, but be sure to choose it properly. Use a good random number source to make it. Don't ever use the same IV twice. And never use a constant IV.
The Wikipedia article on initialization vectors provides a general overview.