In the (PK)ZIP specification at https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT, specifically in the Strong Encryption Specification (SES) section, there is a line on deriving a key from a password:
MasterSessionKey = DeriveKey(SHA1(Password))
What's DeriveKey?
(In WinZip's AES documentation at https://www.winzip.com/en/support/aes-encryption/, they use PBKDF2 with 1000 iterations. I don't see any similar explanation in APPNOTE)
PKWARE implemented a strong encryption in version 5, but did not provide the algorithm of encoding/decoding (Method For Strongly Encrypted .ZIP Files - Patent US 2020/0250329 A1). In this algorithm AES encryption was implemented as part of it. You can define this by strong encryption (bit 6) = yes in General Purpose Flag.
After that WinZip could not use this algo, so it invented another one. You can define this by strong encryption (bit 6) = no in General Purpose Flag and AesExtraFieldRecord with signature 0x990.
As you can see there're two ways to encrypt a zip file. All open source software use the second one. The first one is available only by PKWARE SecureZIP
You can find example of this alogirthm in (7zip) Strong.cpp:35. In java it should look like this:
public static byte[] getMasterKey(String password) {
byte[] data = password.getBytes(StandardCharsets.UTF_8);
byte[] sha1 = DigestUtils.sha1(data);
return DeriveKey(sha1);
}
private static byte[] DeriveKey(byte[] digest) {
byte[] buf = new byte[kDigestSize * 2]; // kDigestSize = 20
DeriveKey2(digest, (byte)0x36, buf, 0);
DeriveKey2(digest, (byte)0x5C, buf, kDigestSize);
return Arrays.copyOfRange(buf, 0, 32);
}
private static void DeriveKey2(byte[] digest, byte c, byte[] dest, int offs) {
byte[] buf = new byte[64];
Arrays.fill(buf, c);
for (int i = 0; i < kDigestSize; i++)
buf[i] ^= digest[i];
byte[] sha1 = DigestUtils.sha1(buf);
System.arraycopy(sha1, 0, dest, offs, sha1.length);
}
Demo:
String password = "JohnDoe";
byte[] masterKey = getMasterKey(password);
The next paragraph 'defines' it
7.2.5.3 The function names and parameter requirements will depend on
the choice of the cryptographic toolkit selected. Almost any
toolkit supporting the reference implementations for each
algorithm can be used. The RSA BSAFE(r), OpenSSL, and Microsoft
CryptoAPI libraries are all known to work well.
I guess it's up to you to decide which of the encryption algorithms you want to use and go from there
I want to decrypt fmp4 segment.
This segment was encrypt with HLS Apple Tools (https://developer.apple.com/documentation/http_live_streaming/about_apple_s_http_live_streaming_tools)
METHOD is AES-128
IV is 1d48fc5dee84b5a3e9a428f055e03c2e
I have a key and IV (you can got the key, and segment in google drive https://drive.google.com/drive/folders/1xF-C9EXFvT8qjI--sBB6QMPn8cNW7L-D?usp=sharing)
To decrypt I use Poco library.
This is my code:
Poco::Crypto::Cipher::ByteVec readKey(const std::string& uri) {
Poco::Crypto::Cipher::ByteVec key;
auto stream = Stream::makeStream(uri);
if (stream->open(uri, {})) {
key.resize(KEY_SIZE);
stream->read((char*)&key[0], KEY_SIZE);
}
return key;
}
std::vector<uint8_t> _key = readKey("./unit-tests/resources/cipher-stream/file.key");
std::string ivSrc = "1d48fc5dee84b5a3e9a428f055e03c2e";
Poco::Crypto::Cipher::ByteVec iv {ivSrc.begin(), ivSrc.end()};
Poco::Crypto::CipherKey key("aes-128-cbc", _key, iv);
Poco::Crypto::Cipher::Ptr cipher = Poco::Crypto::CipherFactory::defaultFactory().createCipher(key);
Poco::FileInputStream src("./unit-tests/resources/cipher-stream/fileSequence1.m4s");
Poco::FileOutputStream dst("./unit-tests/resources/cipher-stream/fileSequence1_dec.m4s");
Poco::Crypto::CryptoOutputStream decryptor(dst, cipher->createDecryptor());
Poco::StreamCopier::copyStream(src, decryptor);
// decryptor.close();
src.close();
dst.close();
Problem description:
After decryption I got distorted data. You can see this at the beginning of the file. Please see picture below. On the right side of the image file is distorted.
The correct data you can see on the left side.
You're using the wrong IV; that will lead to the first block (16 bytes) being corrupted. Your IV hex value is 1d48fc5dee84b5a3e9a428f055e03c2e, but you're interpreting that as ASCII. It's using the first 16 bytes of your string and ignoring the rest.
I haven't used Poco in a long time and don't remember if there's a hex parser handy, but that's what you need. Or write the IV directly in hex rather than as an ASCII string.
For public key encryption and diffie-hellman in libsodium, I typically make private keys simply by generating 32 random bytes with randombytes_buf and then derive the public key (when needed) using crypto_scalarmult_base.
Is there any benefit to using crypto_box_keypair to generate a keypair (other than syntax)? Or does this function basically do exactly that?
This is exactly what the crypto_box_keypair() function does.
The benefits of this function are clarity, and guarantee that the secret key is properly generated.
https://download.libsodium.org/doc/public-key_cryptography/public-key_signatures.html
for example:
unsigned char pk[crypto_sign_PUBLICKEYBYTES]; //Variable declarations
unsigned char sk[crypto_sign_SECRETKEYBYTES]; Variable declarations
crypto_sign_keypair(pk, sk);
NSData *privateKeyData = [NSData dataWithBytes:sk length:crypto_box_SECRETKEYBYTES];
NSData *publicKeyData = [NSData dataWithBytes:pk length:crypto_box_PUBLICKEYBYTES];
NSLog(#"%#",privateKeyData); // target publick key data and secret key data
NSLog(#"%#",publicKeyData);
//Other
NSLog(#"%s\n\n=====\n\n\n%s",pk,sk); //(nullable const void *)bytes
Byte *byte = (Byte *)[publicKeyData bytes];
NSLog(#"%s",byte);
I just start learning Bouncy Castle for AES encryption/decryption. I am using AES/CBC/PKCS7PADDING with 256-bit key.
BC can encrypt and decrypt text successfully, however after decryption I notice that there are always a few padding of null (0x00), which therefore fails my hash comparison. For example, suppose original input string is “1234567890”, the decrypted byte array is always:
{0x49,0x50,0x51,0x52,0x53,0x54,0x55,0x56,0x57,0x48,0x00,0x00,0x00,0x00,0x00,0x00}
Why the padding is not 0x06,0x06,0x06,0x06,0x06,0x06? And is there any way to deterministically tell the padding length (could be 0) after encryption so that I can get exactly the same string before encryption?
When you specify PKCS7, BC will add the padding to the data before encrypting, and remove it again when decrypting. PKCS7 with AES would always add at least 1 byte of padding, and will add enough data to make the input a multiple of the AES block size. When decrypting the padding is verified to be correct, and in the case of PKCS7 also serve as an indicator of how much of the last block of decrypted data is padding, and how much is real data.
If you try decrypting the encrypted and padded data without specifying PKCS7 in the decrypt step, the padding would still be in the decrypted data.
Edit:
To illustrate my point .. here is some Java code that encrypts "1234567890" with AES/CBC/PKCS7, and then decrypts it again both with and without the PKCS7 padding:
public class BCTest {
public static void doTest() throws Exception {
Security.addProvider(new BouncyCastleProvider());
byte[] clearData = "1234567890".getBytes();
SecretKey secretKey = new SecretKeySpec("0123456789ABCDEF".getBytes(), "AES");
AlgorithmParameterSpec IVspec = new IvParameterSpec("0123456789ABCDEF".getBytes());
// encrypt with PKCS7 padding
Cipher encrypterWithPad = Cipher.getInstance("AES/CBC/PKCS7PADDING", "BC");
encrypterWithPad.init(Cipher.ENCRYPT_MODE, secretKey, IVspec);
byte[] encryptedData = encrypterWithPad.doFinal(clearData);
System.out.println("Encryped data (" + encryptedData.length + " bytes): \t" + toHexString(encryptedData));
// decrypt with PKCS7 pad
Cipher decrypterWithPad = Cipher.getInstance("AES/CBC/PKCS7PADDING", "BC");
decrypterWithPad.init(Cipher.DECRYPT_MODE, secretKey, IVspec);
byte[] buffer1 = new byte[encryptedData.length];
int decryptLen1 = decrypterWithPad.doFinal(encryptedData, 0, encryptedData.length, buffer1);
System.out.println("Decrypted with Pad (" + decryptLen1 + " bytes): \t" + toHexString(buffer1));
// decrypt without PKCS7 pad
Cipher decrypterWithoutPad = Cipher.getInstance("AES/CBC/NOPADDING", "BC");
decrypterWithoutPad.init(Cipher.DECRYPT_MODE, secretKey, IVspec);
byte[] buffer2 = new byte[encryptedData.length];
int decryptLen2 = decrypterWithoutPad.doFinal(encryptedData, 0, encryptedData.length, buffer2);
System.out.println("Decrypted without Pad (" + decryptLen2 + " bytes):\t" + toHexString(buffer2));
}
private static String toHexString(byte[] bytes) {
return javax.xml.bind.DatatypeConverter.printHexBinary(bytes);
}
public static void main(String[] args) throws Exception {
BCTest.doTest();
}
}
Output:
Encryped data (16 bytes): 602CAE14358D0AC5C96E2D46D17E58E3
Decrypted with Pad (10 bytes): 31323334353637383930000000000000
Decrypted without Pad (16 bytes): 31323334353637383930060606060606
When decrypting with the padding option, the output have been striped of the padding - and the cipher indicates 10 bytes of decrypted data - the rest of the buffer is 0 filled. Decrypting without the padding option, results in the padding now being part of the decrypted data.
Edit2:
Now seeing the original code, confirms my hunch. The methode GetOutputSize don't return the output size of the decrypted string, but only the maximum needed space in an output buffer. The methode have the following documentation in the BC code:
/**
* return the size of the output buffer required for an update plus a
* doFinal with an input of len bytes.
*
* #param len the length of the input.
* #return the space required to accommodate a call to update and doFinal
* with len bytes of input.
*/
DoFinal returns the actual length of the decrypted data put in the buffer.
So in
byte[] plainTextBuffer = new byte[cipher.GetOutputSize(data.Length - IV_LENGTH)];
int length = cipher.DoFinal(data, iv.Length, data.Length - iv.Length, plainTextBuffer, 0);
The plainTextBuffer would be slightly larger than the actual decrypted data - the actual length of data would be in length.
i am using c# from bouncycastle. looks to me this might be a bug from bouncycastle, or at least bouncycastle c# implementation does not follow pkcs7 spec exactly.
my solution is to chop off the trailing bytes that are not included in the return length of DoFinal. still not very sure why there are padding of 0x00, which as said should not exist at all.
below is the code. i used AES/CBC/PKCS7PADDING for both encryption and decryption.
encryption --->
ICipherParameters keyParams = ParameterUtilities.CreateKeyParameter("AES", keyByte);
ICipherParameters aesIVKeyParam = new ParametersWithIV(keyParams, StringToByteArray(IV_STRING));
byte[] iv = ((ParametersWithIV) aesIVKeyParam).GetIV();
IBufferedCipher cipher = CipherUtilities.GetCipher("AES/CBC/PKCS7PADDING");
cipher.Init(true, aesIVKeyParam);
byte[] cipherText = new byte[iv.Length + cipher.GetOutputSize(data.Length)];
Array.Copy(iv, 0, cipherText, 0, iv.Length);
int length = cipher.DoFinal(data, 0, data.Length, cipherText, iv.Length);
decryption --->
ICipherParameters keyParams = ParameterUtilities.CreateKeyParameter("AES", keyByte);
byte[] iv = new byte[IV_LENGTH];
Array.Copy(data, 0, iv, 0, IV_LENGTH);
ICipherParameters aesIVKeyParam = new ParametersWithIV(keyParams, iv);
IBufferedCipher cipher = CipherUtilities.GetCipher("AES/CBC/PKCS7PADDING");
cipher.Init(false, aesIVKeyParam);
byte[] plainTextBuffer = new byte[cipher.GetOutputSize(data.Length - IV_LENGTH)];
int length = cipher.DoFinal(data, iv.Length, data.Length - iv.Length, plainTextBuffer, 0);
I want to store a password (no hash) on my disk. it's nothing sensitive but i just don't want it in plaintext on my disk.
what i tried till now is:
converting the string in binary and XOR it with the binary of a key.
bool ok = true;
QByteArray qbaPW("mypass");
long long intPW = qbaPW.toHex().toLongLong( &ok, 16 );
QString binPW = QString::number( intPW, 2);
but the thing is, that it only works with short passwords. if they are too long intPW gets too big for longlong. any ideas how can avoid that thing?
cheers
A QByteArray is like a char array[len] in C. You can access individual members and do whatever you please with them. For example:
QByteArray const key("mykey");
QByteArray password("password");
for (int ik = 0, ip = 0;
ip < password.length();
++ ip, ik = (ik+1 < key.length() ? ik+1 : 0)) {
password[ip] = password[ip] ^ key[ik];
}
Since this just XORs with the key, you repeat this procedure to decrypt the password. A good key will be generated randomly and will be longer than the longest password you envisage (say 64 characters).
Do note that this method is only reasonably safe if the users are explicitly informed not to reuse any other password in your application - otherwise you're essentially leaking passwords that are supposed to be secure.