I am using Threefish of BouncyCastle directly, not via security API.
byte[] key = new byte[1024];
...
byte[] tweak = new byte[16];
...
ThreefishEngine threefishEngine = new ThreefishEngine(1024);
threefishEngine.init(true, new TweakableBlockCipherParameters(new KeyParameter(key), tweak));
...
threefishEngine.processBlock(...);
Interestingly, equal blocks will be encrypted in the same way. I have expected that the results of the previous block would affect the result of the next one (because key scheduling produces permanently different keys) and that same blocks will be encrypted differently.
In the source code of BouncyCastle, ThreefishEngine.java, tweak is used for encryption in the method encryptBlock, but tweak never changes. Also for encryption of the next block the previous is not used, neither original unencrypted block nor the encrypted block. In the ThreefishEngine there is no method to change the tweak after ThreefishEngine is initialized. Key scheduling in ThreefishEngine is done within single block only and has no effect for the encryption of the next block.
Here are the questions.
1. Is that correct, that equal blocks produce equal encrypted blocks? Is my expectation wrong, that equal blocks should be encrypted differently? Actually this is the essential difference between block and stream ciphers and otherwise that would be a stream cipher.
2. Is key scheduling done withing a block only (different keys applied to different parts of the block)? Is key scheduling the same for each block? That would explain why encrypted blocks are equal.
Related
I have searched and found examples of AES with the default 256 key size and find it worked already. But when I want to use 128 key size, there is little information.
I have extracted code from the aes test from CryptoJS:
var C = CryptoJS;
var plainText = '00112233445566778899aabbccddeeff';
var key = '000102030405060708090a0b0c0d0e0f';
var encryptedText = C.AES.encrypt(C.enc.Hex.parse(plainText), C.enc.Hex.parse(key), { mode: C.mode.ECB, padding: C.pad.NoPadding }).ciphertext.toString();
console.log(encryptedText);
var decryptedText = C.AES.decrypt(C.lib.CipherParams.create({ ciphertext: C.enc.Hex.parse(encryptedText) }), C.enc.Hex.parse(key), { mode: C.mode.ECB, padding: C.pad.NoPadding }).toString();
console.log(decryptedText);
This worked, but if I want to use a different plain text like 'Hello World' then it failed. Also what if I want to use a password like 'my-key-001'? As I have read that CryptoJS expect to use 256 key size if I pass a password.
Your help is much appreciated.
This worked, but if I want to use a different plain text like 'Hello World' then it failed.
You have used the noPadding and that is the issue. The example is multiple of 16-byte that causes no problem, however your next plaintext is not. You can use noPadding if
your message is an exact multiple of the block size, though still not recommended.
you want to pad the message yourself, probably that is you want to test a new padding scheme that we don't see in your code.
You should you padding like
padding: CryptoJS.pad.Pkcs7
As I have read that CryptoJS expect to use 256 key size if I pass a password.
CryptoJS supports AES-128, AES-192, and AES-256. According to your key size it will select the key variants. If you use a password it will generate a 256-bit size. That is %40 times slower than AES-128 since it requires 14 rounds. However use a good password, see below.
Also what if I want to use a password like 'my-key-001'?
A password with high entropy is important otherwise the attackers can be successful by testing passwords. The key generation cannot increase entropy. Therefore you need a good way to generate high entropy passwords like using diceware.
How to encrypt in AES using CryptoJS with key size of 128?
Just provide a 128-bit key.
Does AES-128 has 128-bit security
Any block cipher, not only AES, has vulnerable to multi-target attacks. In that case it is not providing 128-bit security. Therefore you should use 192 or 256-bit keys.
For a more detailed see this question Has AES-128 been fully broken?
mode: C.mode.ECB
The ECB mode of operations is not advised, it is insecure and it leaks pattern. You should use modern encryption modes like AES-GCM which provides you not confidentiality but also, integrity and authentication.
While using GCM mode, make sure that you never use the same IV/nonce again under the same key. AES-GCM uses CTR mode for encryption and under the same key if the IV/nonce repeated then crig-dragging is possible. Also, it can leak the authentication key.
I helped a small non-profit create a cool "anonymous" help line where their clientele can send in SMS questions to a phone number and organization can respond without knowing who the client is. It's proven to be a very helpful tool for the organization. I am in the process of re-writing it from scratch and am looking for advice on how to handle the anonymity aspect of it.
The way that v1 works is by simply encrypting / decrypting the client phone number and using that as the client id in the data model using 'aes-256-ctr'. Since the staff / leadership of the nonprofit do not have access to the encryption key, they can legitimately claim to not be able to have access to any p.i.i. not disclosed voluntary by their clients. When they issue a message response to the client, I simply need to decrypt the client id to get the phone that I need in order to dispatch the message.
v1 encrpytion strategy:
export const encrypt = (text) => {
const cipher = crypto.createCipher(algorithm, password);
let crypted = cipher.update(text, 'utf8', 'hex');
crypted += cipher.final('hex');
return crypted;
};
export const decrypt = (text) => {
const decipher = crypto.createDecipher(algorithm, password);
let dec = decipher.update(text, 'hex', 'utf8');
dec += decipher.final('utf8');
return dec;
};
Lately though, I've been getting console warnings because new node versions will output a warning when using createCipher instead of createCipheriv to utilize initialization vector. On my v2 rewrite I switched it over to createCipheriv, but that has the effect of "randomizing" the encryption output and breaking my ability to rely upon the encrypted phone number as a reliable, consistent client id. Being able to reliably group client messages is an important requirement to provide a conversational view in the UX.
How might I properly encrypt the identifying phone number such that only I can decrypt it at message send time and still have some kind of consistent internal ID by which to group messages? I don't HAVE to rely upon the encrypted phone number as the client id, but I can't think of any reliable alternative (via a lookup table of some kind). If possible I don't want to fall back on createCipher without the I.V. if its not a best practice.
I should note that this application is not dealing with any legally sensitive information or anything high stakes like that. I probably could get away with storing the unencrypted phone number in the database and just not exposing it to the front end and maintain anonymity, I just really don't like that idea if there is a sensible workaround.
Any ideas?
You should be generating a new random (or UUID-style) client id for each new client. The benefit of this is that you can keep using a secure encryption method and still identify and lookup users quickly.
If you want the phone number field to act as a unique lookup, simply take an HMAC of the phone number and use that for lookups (as well as storing encrypted) - it can't ever be decrypted and is guaranteed to be consistent across invocations for the same phone number under a given key.
EDIT: To migrate existing data, you should, for each record, take the client_id (currently the encrypted phone number), decrypt it, HMAC it, and store the result in a new column. Then, generate a new ID and set that to be the client_id. This allows lookup by actual client_id and by phone number.
We are developing an application that has to work with data that is enycrpted by LoraWan (https://www.lora-alliance.org)
We have already found the documentation of how they encrypt their data, and have been reading through it for the past few days (https://www.lora-alliance.org/sites/default/files/2018-04/lorawantm_specification_-v1.1.pdf) but currently still can't solve our problem.
We have to use AES 128-bit ECB decryption with zero-padding to decrypt the messages, but the problem is it's not working because the encrypted messages we are receiving are not long enough for AES 128 so the algorithm returns a "Data is not a complete block" exception on the last line.
An example key we receive is like this: D6740C0B8417FF1295D878B130784BC5 (not a real key). It is 32 characters long, so 32 bytes, but if treat it as hexadecimal, then it becomes 16 bytes long, which is what is needed for AES 128-bit. This is the code we use to convert the Hex from String:
public static string HextoString(string InputText)
{byte[] hex= Enumerable.Range(0, InputText.Length)
.Where(x => x % 2 == 0)
.Select(x => Convert.ToByte(InputText.Substring(x, 2), 16))
.ToArray();
return System.Text.Encoding.ASCII.GetString(hex);}
(A small thing to note for the above code is that we are not sure what Encoding to use, as we could not find it in the Lora documentation and they have not told us, but depending on this small setting we could be messing up our decryption (though we have tried all possible combinations, ascii, utf8, utf7, etc))
An example message we receive is: d3 73 4c which we are assuming is also in hexadecimal. This is only 6 bytes, and 3 bytes if we convert it from hexa to normal, compared to the 16 bytes we'd need minimum to match the key length.
This is the code for Aes 128 decrypt we are using:
private static string Aes128Decrypt(string cipherText, string key){
string decrypted = null;
var cipherPlainTextBytes = HexStringToByteArray(cipherText);
//var cipherPlainTextBytes = ForcedZeroPadding(HexStringToByteArray(cipherText));
var keyBytes = HexStringToByteArray(key);
using (var aes = new AesCryptoServiceProvider())
{
aes.KeySize = 128;
aes.Key = keyBytes;
aes.Mode = CipherMode.ECB;
aes.Padding = PaddingMode.Zeros;
ICryptoTransform decryptor = aes.CreateDecryptor(aes.Key, aes.IV);
using (MemoryStream ms = new MemoryStream(cipherPlainTextBytes, 0, cipherPlainTextBytes.Length))
{
using (CryptoStream cs = new CryptoStream(ms, decryptor, CryptoStreamMode.Read))
{
using (StreamReader sr = new StreamReader(cs))
{
decrypted = sr.ReadToEnd();
}
}
}
}
return decrypted;}
So obviously this is going to return "Data is an incomplete block" at sr.ReadToEnd().
As you can see from the example, in that one commented out line, we have also tried to "Pad" the text to the correct size with a full zero byte array of correct length (16 - cipherText), in which case the algorithm runs fine, but it returns complete gibberish and not the original text.
We already have tried all of the modes of operation and messed around with padding modes as well. They are not providing us with anything but a cipherText, and a key for that text. No Initialization vector either, so we are assuming we are supposed to be generating that every time (but for ECB it isn't even needed iirc)
What's more is, they are able to encrypt-decrypt their messages just fine. What is most puzzling about this is that I have been googling this for days now and I cannot find a SINGLE example on google where the CIPHERTEXT is shorter than the key during decryption.
Obviously I have found examples where the message they are Encrypting is shorter than what is needed, but that is what padding is for on the ENCRYPTION side (right?). So that when you then receive the padded message, you can tell the algorithm what padding mode was used to make it correct length, so then it can seperate the padding from the message. But in all of those cases the recieved message during decryption is of correct length.
So the question is - what are we doing wrong? is there some way to decrypt with ciphertexts that are shorter than the key? Or are they messing up somewhere by producing ciphers that are too short?
Thanks for any help.
In AES-ECB, the only valid ciphertext shorter than 16-byte is empty. That 16-byte limit is the block (not key) size of AES, which happens to match the key size for AES-128.
Therefore, the question's
An example message we receive is: d3 73 4c
does not show an ECB encrypted message (since a comment tells that's from a JSON, that can't be bytes that happen to show as hex). And that's way too short to be a FRMPayload (per this comment) for a Join-Accept, since the spec says of the later:
1625 The message is either 16 or 32 bytes long.
Could it be that whatever that JSON message contains is not a full FRMPayload, but a fragment of a packet, encoded as hexadecimal pair with space separator? As long as it is not figured out how to build a FRMPayload, there's not point in deciphering it.
Update: If that mystery message is always 3 bytes, and if it is always the same for a given key (or available a single time per key), then per Maarten Bodewes's comment it might be a Key Check Value. The KCV is often the first 3 bytes of the encryption of the all-zero value with the key per the raw block cipher (equivalently: per ECB). Herbert Hanewinkel's javascript AES can work fully offline (which is necessary to not expose the key), and be used to manually validate an hypothesis. It tells that for the 16-byte key given in the question, a KCV would be cd15e1 (or c076fc per the variant in the next section).
Also: it is used CreateDecryptor to craft a gizmo in charge of the ECB decryption. That's likely incorrect in the context of decryption of a LoraWan payload, which requires ECB encryption for decryption of some fields:
1626 Note: AES decrypt operation in ECB mode is used to encrypt the join-accept message so that the end-device can use an AES encrypt operation to decrypt the message. This way an end device only has to implement AES encrypt but not AES decrypt.
In the context of decryption of a LoraWan packets, you want to communicate with the AES engine using byte arrays, not strings. Strings have an encoding, when LoraWan ciphertext and corresponding plaintext does not. Others seems to have managed to coerce the nice .NET do-it-all crypto API to get a low-level job done.
In the HextoString code, I vaguely get that the intention and perhaps outcome is that hex becomes the originally hex input as a byte array (fully rid of hexadecimal and other encoding sin; in which case the variable hex should be renamed to something on the tune of pure_bytes). But then I'm at loss about System.Text.Encoding.ASCII.GetString(hex). I'd be surprised if it just created a byte string from a byte array, or turned the key back to hexadecimal for later feeding to HexStringToByteArray in Aes128Decrypt. Plus this makes me fear that any byte in [0x80..0xFF] might turn to 0x3F, which is not nice for key, ciphertext, and corresponding LoraWan payload. These have no character encoding when de-hexified.
My conclusion is that if HexStringToByteArray does what its name suggests, and given the current interface of Aes128Decrypt, HextoString should simply remove whitespace (or is unneeded if HexStringToByteArray removes whitespace on the fly). But my recommendation is to change the interface to use byte arrays, not strings (see previous section).
As an aside: creating an ICryptoTransform object from its key is supposed to be performed once for multiple uses of the object.
The code encrypt/decrypt function with openssl library, like following...
EVP_EncryptInit_ex( ctx, EVP_aes_256_cbc(), NULL, key, iv)
It can work, when the key length is not equal 256 bits(32 bytes).
The key length can be any. Why?
For example, it works fine, and no error received:
char key[]="012345678901234567890";
Why does EVP_CIPHER is EVP_aes_256_cbc() succeed when key length is not equal 256bits?
You seem to be asking why you can encrypt using EVP_aes_256_cbc when the key is smaller, like 128-bits.
If you supply an undersized key you are reading random bytes/garbage at the tail of the key bytes. The function is reading bytes, but you don't know what they are. You may get lucky on the local machine and be able to encrypt and decrypt. It will almost certainly fail to decrypt on a different machine.
Valgrind should alert you to the problem of reading uninaitalezed [key] memory. Asan should alert about a read in a guard zone.
I don't believe EVP_aes_256_cbc pads or expands. Like #Zaph said, always use the correct size. If you need to "stretch" a smaller key into a bigger one, then see HMAC-based Extract-and-Expand Key Derivation Function (HKDF), which extracts entropy then expands it.
As the title says, I am trying to decrypt DUKPT encrypted track data coming from a DUKPT enabled scanner.
I have the ANSI Standard (X9.24) for DUKPT and have successfully implemented the ability to generate the IPEK from the KSN and BDK. Furthermore, I have successfully implemented the ability to generate the Left and Right MAC Request and Response Keys by XORing the PIN Encryption Keys. Lastly, I am able to generate the EPB.
From here, I don't understand how to generate the MAC Request and Response from the L/R Keys that I have generated.
Lastly, once I get to that step, what comes next? When do I actually have the key that decrypts the track data sent by a DUKPT enabled device?
I am aware of the Thales Simulator and jPOS. My code is currently referencing the Thales Simulator to do all of its work. But, the file decryption process just isn't returning the expected data.
If anybody can offer some insight into decrypting track data, it would be much appreciated.
http://thalessim.codeplex.com/
http://jpos.org/
I spent too much time studying the horrible X9.24 spec and finally got both the encryption and decryption working with my vendor’s examples and marketing promptly decided to switch vendors. Since it is a standard, you would think that anybody’s implementation would be the same. I wish. Anyway, there are variations on how things are implemented. You have to study the fine print to make sure you are working things the same as your other side.
But that is not your question.
First if you need to decrypt a data track from a credit card, you are probably interested in producing a key that will decrypt the data based upon the original super secret Base Derivation Key. That has nothing to do with the MAC generation and is only mentioned in passing in that dreadful spec. You need to generate the IPEK for that key serial number and device ID and repeatedly apply the “Non-reversible Key Generation Process” from the spec if bits are set in the counter part of the full key serial number from the HSM.
That part of my code looks like this: (Sorry for the long listing in a posting.)
/*
* Bit "zero" set (this is a 21 bit register)(ANSI counts from the left)
* This will be used to test each bit of the encryption counter
* to decide when to find another key.
*/
testBit=0x00100000;
/*
* We have to "encrypt" the IPEK repeatedly to find the current key
* (See Section A.3). Each time we encrypt (generate a new key),
* we need to use the all prior bits to the left of the current bit.
* The Spec says we will have a maximum of ten bits set at any time
* so we should not have to generate more than ten keys to find the
* current encryption key.
*/
cumBits=0;
/*
* For each of the 21 possible key bits,
* if it is set, we need to OR that bit into the cumulative bit
* variable and set that as the KSN count and "encrypt" again.
* The encryption we are using the goofy ANSI Key Generation
* subroutine from page 50.
*/
for(int ii=0; ii<21; ii++)
{
if( (keyNumber&testBit) != 0)
{
char ksr[10];
char eightByte[8]={0};
cumBits |= testBit;
ksn.count=cumBits; /* all bits processed to date */
memcpy(ksr, &ksn,10); /* copy bit structure to char array*/
memcpy(crypt,&ksr[2],8); /* copy bytes 2 through 9 */
/*
* Generate the new Key overwriting the old.
* This will apply the "Non-reversible Key Generation Process"
* to the lower 64 bits of the KSN.
*/
keyGen(&key, &crypt, &key);
}
testBit>>=1;
}
Where
keyNumber is the current counter from the ksn
ksn is an 80 bit structure that contains the 80 bit Key Serial Number from the HSM
crypt is a 64 bit block of data I have it of type DES_cblock since I am using openSSL.
key is a 128 bit double DES_cblock structure.
The keyGen routine is almost verbatim from the “Non-reversible Key Generation Process” local subroutine on page 50 of the spec.
At the end of this, the key variable will contain the key that can be used for the decryption, almost. The dudes that wrote the spec added some “variant” behavior to the key to keep us on our toes. If the key is to be used for decrypting a data stream such as a credit card track, you will need to XOR bytes 5 and 13 with 0xFF and Triple DES encrypt the key with itself (ECB mode). My code looks like:
DOUBLE_KEY keyCopy;
char *p;
p=(char*)&key;
p[ 5]^=0xff;
p[13]^=0xff;
keyCopy=key;
des3(&keyCopy, (DES_cblock *)&key.left, &key.left);
des3(&keyCopy, (DES_cblock *)&key.right, &key.right);
If you are using this to decrypt a PIN block, you will need to XOR bytes 7 and 15 with 0xFF. (I am not 100% sure this should not be applied for the stream mode as well but my vendor is leaving it out.)
If it is a PIN block, it will be encrypted with 3-DES in ECB mode. If it is a data stream, it will be encrypted in CBC mode with a zero initialization vector.
(Did I mention I don’t much care for the spec?) It is interesting to note that the encryption side could be used in a non-hardware, tamper resistant security module if the server side (above) remembers and rejects keys that have been used previously. The technology is pretty neat. The ANSI spec leaves something to be desired but the technology is all right.
Good luck.
/Bob Bryan
For data encryption, the variant is 0000000000FF0000.0000000000FF0000 so you need to XOR bytes 5 and 13 instead of 7 and 15. In addition, you need an additional 3DES self-encryption step of each key parts (left and right).
Here is the relevant code in jPOS
https://github.com/jpos/jPOS/blob/master/jpos/src/main/java/org/jpos/security/jceadapter/JCESecurityModule.java#L1843-1856