I ran into an issue with my server that made me ask a question - is AES-256 encryption reliably consistent?
So my node server, to avoid DDOS vulnerabilities while keeping user data anonymous, takes an IP from the request, encrypts it, and stores the number of times it has made a request in the last 24 hours.
It checks if the user had made a request in the last 24 hours by re-encrypting the IP using the exact same password. It then, takes that and matches it with the first stored IP.
But then:
Error: Can't find data: /ipcount/d00c526612cec9e5d3e201af35993c75. Stopped at ipcount
d00c526612cec9e5d3e201af35993c75 should be the encrypted string, but the database is acting like it's never seen it before. So can I have it encrypt consistently with the same password?
Thanks in advance.
Here is the encryption function...
const crypto = require('crypto'),
algorithm = 'aes-256-ctr',
password = 'aSecureNotSharedOverAForumPassword'
function encrypt(text){
var cipher = crypto.createCipher(algorithm,password)
var crypted = cipher.update(text,'utf8','hex')
crypted += cipher.final('hex');
return crypted;
}
As per the Node.js documentation for crypto:
It is recommended that developers derive a key and IV on their own using crypto.pbkdf2() and to use crypto.createCipheriv() to create the Cipher object. Users should not use ciphers with counter mode (e.g. CTR, GCM or CCM) in crypto.createCipher()
So the behaviour you are experiencing isn't exactly a defined behaviour, per se. My guess is that the underlying cipher object is randomly generating a nonce for your CTR mode and thus the output is different each time.
To fix this, use crypto.createCipheriv as recommended by the documentation and generate the nonce yourself. This way you can store the nonce and use it to generate the same output stream.
For others who are looking, do not use the same nonce. It takes away all the security guarantees of AES. Take a look at this thread instead: CryptoJS AES encryption with ECB mode produces different results with the same params
Related
I'm using Googles Tink DeterministicAead. What I'd like to achieve, is to use the same key over a long time, but change the ciphertext for a certain plaintext every 24 hours (for GDPR reasons).
Can I pick a new initialization vector somehow once every 24 hours? The code I'm using is:
KeysetHandle keysetHandle = KeysetHandle.generateNew(
DeterministicAeadKeyTemplates.AES256_SIV);
DeterministicAead daead = keysetHandle.getPrimitive(DeterministicAead.class);
byte[] ciphertext = daead.encryptDeterministically(clearText.getBytes(), "".getBytes());
ciphertext is indeed deterministic, so at least that part is good :-)
I want to be able to decrypt the cyphertext in BigQuery, so custom primitives worry me a bit.
Note: this is not about financial data or such, it is about being able to update recommendation models on the behavior of the ciphertext while at the same time complying with privacy requirements for the user.
By definition (from what I'm reading in those docs) there doesn't seem to be a way to specify an IV (as then it wouldn't be deterministic).
The DeterministicAead primitive seems to be AES with a MAC, without IV.
So, I think the only option here would be to rotate the key, or, add your own prepended/appended salt to the plaintext prior to encryption and store it for decryption.
Remember deterministic schemes aren't semantically secure.
I am creating a program that is going to encrypt files in a folder and allow me to decrypt them later on.
I am using AES-GCM at the moment and I am wondering if using the same key but changing the IV for every encryption is the correct way to do this?
I am using mBedTLS to do this and I am able to successfully encrypt/decrypt a file using AES-GCM and hard coded key and IV. However for security reasons I want to make sure that I am changing what needs to be changed and for performance reasons, not changing something that doesn't need to be.
Yup,
Even a single AES-GCM nonce reuse can be catastrophic, with the same static key.
A single nonce reuse leaks the xor of plaintexts, so if one plaintext is known the adversary can completely decrypt the other.
So you can only use the same [nonce (IV), key] tuple once per message.
It's perfectly fine to use a new nonce/IV with the same static key.
AES-GCM operates with a 32-bit counter, so unfortunately with the same key, nonce (IV) pair you can only safely encrypt ~ 64GB of data (2^39-256 bits).
If you want to move to an even safer cipher, I recommend XSalsa20 or XChaCha20, which provide a 192-bit nonce size, effectively allowing a practically "unlimited" number of messages to be encrypted with the same key, nonce pair.
I have a tcl/tk based tool, which uses network password for authentication. Issue is that, it is saving password in the logs/history. So objective is to encrypt the password.
I tried to use aes package. But at the very beginning aes::init asks for keydata and initialization vector (16 byte). So how to generate IV and keydata. Is is some Random number? I am a novice in encryption algorithms.
If you have the password in the logs/history, why not fix the bug of logging/storing it in the first place?
Otherwise there are distinct things you might want:
A password hashing scheme like PBKDF2, bcrypt, argon2 etc. to store a password in a safe way and compare some user input to it. This is typically the case when you need to implement some kind of authentication with passwords on the server side.
A password encryption and protection scheme like AES. You need a password to authenticate to some service automatically, and it requires some form of cleartext password.
You have some secret data and need to securly store it to in non cleartext form.
If you have case 1, don't use the aespackage, it is the wrong tool for the job. If you have case 2, the aes package might help you, but you just exchanged the problem of keeping the password secret with the other problem of keeping the key secret (not a huge win). So the only viable case where aes is an option might be 3.
Lets assume you need to store some secret data in a reversible way, e.g. case 3 from above.
AES has a few possible modes of operation, common ones you might see are ECB, CBC, OFB, GCM, CTR. The Tcllib package just supports ECB and CBC, and only CBC (which is the default) is really an option to use.
Visit Wikipedia for an example why you should never use ECB mode.
Now back to your actual question:
Initialization Vector (IV)
This is a random value you pick for each encryption, it is not secret, you can just publish it together with the encrypted data. Picking a random IV helps to make two encrypted blocks differ, even if you use the same key and cleartext.
Secret Key
This is also a random value, but you must keep it secret, as it can be used for encryption and decryption. You often have the same key for multiple encryptions.
Where to get good randomness?
If you are on Linux, BSD or other unixoid systems just read bytes from /dev/urandom or use a wrapper for getrandom(). Do NOT use Tcls expr {rand()} or similar pseudorandom number generators (PRNG). On Windows TWAPI and the CryptGenRandom function would be the best idea, but sadly there is no Tcl high level wrapper included.
Is that enough?
Depends. If you just want to hide a bit of plaintext from cursory looks, maybe. If you have attackers manipulating your data or actively trying to hack your system, less so. Plain AES-CBC has a lot of things you can do wrong, and even experts did wrong (read about SSL/TLS 1.0 problems with AES-CBC).
Final words: If you are a novice in encryption algorithms, be sure you understand what you want and need to protect, there are a lot of pitfalls.
If I read the Tcler's Wiki page on aes, I see that I encrypt by doing this:
package require aes
set plaintext "Some super-secret bytes!"
set key "abcd1234dcba4321"; # 16 bytes
set encrypted [aes::aes -dir encrypt -key $key $plaintext]
and I decrypt by doing:
# Assuming the code above was run...
set decrypted [aes::aes -dir decrypt -key $key $encrypted]
Note that the decrypted text has NUL (zero) bytes added on the end (8 of them in this example) because the encryption algorithm always works on blocks of 16 bytes, and if you're working with non-ASCII text then encoding convertto and encoding convertfrom might be necessary.
You don't need to use aes::init directly unless you are doing large-scale streaming encryption. Your use case doesn't sound like it needs that sort of thing. (The key data is your “secret”, and the initialisation vector is something standardised that usually you don't need to set.)
For
`BDK = "0123456789ABCDEFFEDCBA9876543210"` `KSN = "FFFF9876543210E00008"`
The ciphertext generated was below
"C25C1D1197D31CAA87285D59A892047426D9182EC11353C051ADD6D0F072A6CB3436560B3071FC1FD11D9F7E74886742D9BEE0CFD1EA1064C213BB55278B2F12"`
which I found here. I know this cipher-text is based on BDK and KSN but how this 128 length cipher text was generated? What are steps involved in it or algorithm used for this? Could someone explain in simple steps. I found it hard to understand the documents I got while googled.
Regarding DUKPT , there are some explanations given on Wiki. If that doesn't suffice you, here goes some brief explanation.
Quoting http://www.maravis.com/library/derived-unique-key-per-transaction-dukpt/
What is DUKPT?
Derived Unique Key Per Transaction (DUKPT) is a key management scheme. It uses one time encryption keys that are derived from a secret master key that is shared by the entity (or device) that encrypts and the entity (or device) that decrypts the data.
Why DUKPT?
Any encryption algorithm is only as secure as its keys. The strongest algorithm is useless if the keys used to encrypt the data with the algorithm are not secure. This is like locking your door with the biggest and strongest lock, but if you hid the key under the doormat, the lock itself is useless. When we talk about encryption, we also need to keep in mind that the data has to be decrypted at the other end.
Typically, the weakest link in any encryption scheme is the sharing of the keys between the encrypting and decrypting parties. DUKPT is an attempt to ensure that both the parties can encrypt and decrypt data without having to pass the encryption/decryption keys around.
The Cryptographic Best Practices document that VISA has published also recommends the use of DUKPT for PCI DSS compliance.
How DUKPT Works
DUKPT uses one time keys that are generated for every transaction and then discarded. The advantage is that if one of these keys is compromised, only one transaction will be compromised. With DUKPT, the originating (say, a Pin Entry Device or PED) and the receiving (processor, gateway, etc) parties share a key. This key is not actually used for encryption. Instead, another one time key that is derived from this master key is used for encrypting and decrypting the data. It is important to note that the master key should not be recoverable from the derived one time key.
To decrypt data, the receiving end has to know which master key was used to generate the one time key. This means that the receiving end has to store and keep track of a master key for each device. This can be a lot of work for someone that supports a lot of devices. A better way is required to deal with this.
This is how it works in real-life: The receiver has a master key called the Base Derivation Key (BDK). The BDK is supposed to be secret and will never be shared with anyone. This key is used to generate keys called the Initial Pin Encryption Key (IPEK). From this a set of keys called Future Keys is generated and the IPEK discarded. Each of the Future keys is embedded into a PED by the device manufacturer, with whom these are shared. This additional derivation step means that the receiver does not have to keep track of each and every key that goes into the PEDs. They can be re-generated when required.
The receiver shares the Future keys with the PED manufacturer, who embeds one key into each PED. If one of these keys is compromised, the PED can be rekeyed with a new Future key that is derived from the BDK, since the BDK is still safe.
Encryption and Decryption
When data needs to be sent from the PED to the receiver, the Future key within that device is used to generate a one time key and then this key is used with an encryption algorithm to encrypt the data. This data is then sent to the receiver along with the Key Serial Number (KSN) which consists of the Device ID and the device transaction counter.
Based on the KSN, the receiver then generates the IPEK and from that generates the Future Key that was used by the device and then the actual key that was used to encrypt the data. With this key, the receiver will be able to decrypt the data.
Source
First, let me quote the complete sourcecode you linked and of which you provided only 3 lines...
require 'bundler/setup'
require 'test/unit'
require 'dukpt'
class DUKPT::DecrypterTest < Test::Unit::TestCase
def test_decrypt_track_data
bdk = "0123456789ABCDEFFEDCBA9876543210"
ksn = "FFFF9876543210E00008"
ciphertext = "C25C1D1197D31CAA87285D59A892047426D9182EC11353C051ADD6D0F072A6CB3436560B3071FC1FD11D9F7E74886742D9BEE0CFD1EA1064C213BB55278B2F12"
plaintext = "%B5452300551227189^HOGAN/PAUL ^08043210000000725000000?\x00\x00\x00\x00"
decrypter = DUKPT::Decrypter.new(bdk, "cbc")
assert_equal plaintext, decrypter.decrypt(ciphertext, ksn)
end
end
Now, you're asking is how the "ciphertext" was created...
Well, first thing we know is that it is based on "plaintext", which is used in the code to verify if decryption works.
The plaintext is 0-padded - which fits the encryption that is being tested by verifying decryption with this DecrypterTest TestCase.
Let's look at the encoding code then...
I found the related encryption code at https://github.com/Shopify/dukpt/blob/master/lib/dukpt/encryption.rb.
As the DecrypterTEst uses "cbc", it becomes apparent that the encrypting uses:
#cipher_type_des = "des-cbc"
#cipher_type_tdes = "des-ede-cbc"
A bit more down that encryption code, the following solves our quest for an answer:
ciphertext = des_encrypt(...
Which shows we're indeed looking at the result of a DES encryption.
Now, DES has a block size of 64 bits. That's (64/8=) 8 bytes binary, or - as the "ciphertext" is a hex-encoded text representation of the bytes - 16 chars hex.
The "ciphertext" is 128 hex chars long, which means it holds (128 hex chars/16 hex chars=) 8 DES blocks with each 64 bits of encrypted information.
Wrapping all this up in a simple answer:
When looking at "ciphertext", you are looking at (8 blocks of) DES encrypted data, which is being represented using a human-readable, hexadecimal (2 hex chars = 1 byte) notation instead of the original binary bytes that DES encryption would produce.
As for the steps involved in "recreating" the ciphertext, I tend to tell you to simply use the relevant parts of the ruby project where you based your question upon. Simply have to look at the sourcecode. The file at "https://github.com/Shopify/dukpt/blob/master/lib/dukpt/encryption.rb" pretty much explains it all and I'm pretty sure all functionality you need can be found at the project's GitHub repository. Alternatively, you can try to recreate it yourself - using the preferred programming language of your choice. You only need to handle 2 things: DES encryption/decryption and bin-to-hex/hex-to-bin translation.
Since this is one of the first topics that come up regarding this I figured I'd share how I was able to encode the ciphertext. This is the first time I've worked with Ruby and it was specifically to work with DUKPT
First I had to get the ipek and pek (same as in the decrypt) method. Then unpack the plaintext string. Convert the unpacked string to a 72 byte array (again, forgive me if my terminology is incorrect).
I noticed in the dukpt gem author example he used the following plain text string
"%B5452300551227189^HOGAN/PAUL ^08043210000000725000000?\x00\x00\x00\x00"
I feel this string is incorrect as there shouldn't be a space after the name (AFAIK).. so it should be
"%B5452300551227189^HOGAN/PAUL^08043210000000725000000?\x00\x00\x00\x00"
All in all, this is the solution I ended up on that can encrypt a string and then decrypt it using DUKPT
class Encrypt
include DUKPT::Encryption
attr_reader :bdk
def initialize(bdk, mode=nil)
#bdk = bdk
self.cipher_mode = mode.nil? ? 'cbc' : mode
end
def encrypt(plaintext, ksn)
ipek = derive_IPEK(bdk, ksn)
pek = derive_PEK(ipek, ksn)
message = plaintext.unpack("H*").first
message = hex_string_from_unpacked(message, 72)
encrypted_cryptogram = triple_des_encrypt(pek,message).upcase
encrypted_cryptogram
end
def hex_string_from_unpacked val, bytes
val.ljust(bytes * 2, "0")
end
end
boomedukpt FFFF9876543210E00008 "%B5452300551227189^HOGAN/PAUL^08043210000000725000000?"
(my ruby gem, the KSN and the plain text string)
2542353435323330303535313232373138395e484f47414e2f5041554c5e30383034333231303030303030303732353030303030303f000000000000000000000000000000000000
(my ruby gem doing a puts on the unpacked string after calling hex_string_from_unpacked)
C25C1D1197D31CAA87285D59A892047426D9182EC11353C0B82D407291CED53DA14FB107DC0AAB9974DB6E5943735BFFE7D72062708FB389E65A38C444432A6421B7F7EDD559AF11
(my ruby gem doing a puts on the encrypted string)
%B5452300551227189^HOGAN/PAUL^08043210000000725000000?
(my ruby gem doing a puts after calling decrypt on the dukpt gem)
Look at this: https://github.com/sgbj/Dukpt.NET, I was in a similar situation where i wondered how to implement dukpt on the terminal when the terminal has its own function calls which take the INIT and KSN to create the first key, so my only problem was to make sure the INIT key was generated the same way on the terminal as it is in the above mentioned repo's code, which was simple enough using ossl encryption library for 3des with ebc and applying the appropriate masks.
Right now, this is what I am doing:
1. SHA-1 a password like "pass123", use the first 32 characters of the hexadecimal decoding for the key
2. Encrypt with AES-256 with just whatever the default parameters are
^Is that secure enough?
I need my application to encrypt data with a password, and securely. There are too many different things that come up when I google this and some things that I don't understand about it too. I am asking this as a general question, not any specific coding language (though I'm planning on using this with Java and with iOS).
So now that I am trying to do this more properly, please follow what I have in mind:
Input is a password such as "pass123" and the data is
what I want to encrypt such as "The bank account is 038414838 and the pin is 5931"
Use PBKDF2 to derive a key from the password. Parameters:
1000 iterations
length of 256bits
Salt - this one confuses me because I am not sure where to get the salt from, do I just make one up? As in, all my encryptions would always use the salt "F" for example (since apparently salts are 8bits which is just one character)
Now I take this key, and do I hash it?? Should I use something like SHA-256? Is that secure? And what is HMAC? Should I use that?
Note: Do I need to perform both steps 2 and 3 or is just one or the other okay?
Okay now I have the 256-bit key to do the encryption with. So I perform the encryption using AES, but here's yet another confusing part (the parameters).
I'm not really sure what are the different "modes" to use, apparently there's like CBC and EBC and a bunch of others
I also am not sure about the "Initialization Vector," do I just make one up and always use that one?
And then what about other options, what is PKCS7Padding?
For your initial points:
Using hexadecimals clearly splits the key size in half. Basically, you are using AES-128 security wise. Not that that is bad, but you might also go for AES-128 and use 16 bytes.
SHA-1 is relatively safe for key derivation, but it shouldn't be used directly because of the existence/creation of rainbow tables. For this you need a function like PBKDF2 which uses an iteration count and salt.
As for the solution:
You should not encrypt PIN's if that can be avoided. Please make sure your passwords are safe enough, allow pass phrases.
Create a random number per password and save the salt (16 bytes) with the output of PBKDF2. The salt does not have to be secret, although you might want to include a system secret to add some extra security. The salt and password are hashed, so they may have any length to be compatible with PBKDF2.
No, you just save the secret generated by the PBKDF2, let the PBKDF2 generate more data when required.
Never use ECB (not EBC). Use CBC as minimum. Note that CBC encryption does not provide integrity checking (somebody might change the cipher text and you might never know it) or authenticity. For that, you might want to add an additional MAC, HMAC or use an encryption mode such as GCM. PKCS7Padding (identical to PKCS5Padding in most occurences) is a simple method of adding bogus data to get N * [blocksize] bytes, required by block wise encryption.
Don't forget to prepend a (random) IV to your cipher text in case you reuse your encryption keys. An IV is similar to a salt, but should be exactly [blocksize] bytes (16 for AES).