I want to decrypt fmp4 segment.
This segment was encrypt with HLS Apple Tools (https://developer.apple.com/documentation/http_live_streaming/about_apple_s_http_live_streaming_tools)
METHOD is AES-128
IV is 1d48fc5dee84b5a3e9a428f055e03c2e
I have a key and IV (you can got the key, and segment in google drive https://drive.google.com/drive/folders/1xF-C9EXFvT8qjI--sBB6QMPn8cNW7L-D?usp=sharing)
To decrypt I use Poco library.
This is my code:
Poco::Crypto::Cipher::ByteVec readKey(const std::string& uri) {
Poco::Crypto::Cipher::ByteVec key;
auto stream = Stream::makeStream(uri);
if (stream->open(uri, {})) {
key.resize(KEY_SIZE);
stream->read((char*)&key[0], KEY_SIZE);
}
return key;
}
std::vector<uint8_t> _key = readKey("./unit-tests/resources/cipher-stream/file.key");
std::string ivSrc = "1d48fc5dee84b5a3e9a428f055e03c2e";
Poco::Crypto::Cipher::ByteVec iv {ivSrc.begin(), ivSrc.end()};
Poco::Crypto::CipherKey key("aes-128-cbc", _key, iv);
Poco::Crypto::Cipher::Ptr cipher = Poco::Crypto::CipherFactory::defaultFactory().createCipher(key);
Poco::FileInputStream src("./unit-tests/resources/cipher-stream/fileSequence1.m4s");
Poco::FileOutputStream dst("./unit-tests/resources/cipher-stream/fileSequence1_dec.m4s");
Poco::Crypto::CryptoOutputStream decryptor(dst, cipher->createDecryptor());
Poco::StreamCopier::copyStream(src, decryptor);
// decryptor.close();
src.close();
dst.close();
Problem description:
After decryption I got distorted data. You can see this at the beginning of the file. Please see picture below. On the right side of the image file is distorted.
The correct data you can see on the left side.
You're using the wrong IV; that will lead to the first block (16 bytes) being corrupted. Your IV hex value is 1d48fc5dee84b5a3e9a428f055e03c2e, but you're interpreting that as ASCII. It's using the first 16 bytes of your string and ignoring the rest.
I haven't used Poco in a long time and don't remember if there's a hex parser handy, but that's what you need. Or write the IV directly in hex rather than as an ASCII string.
Related
In the (PK)ZIP specification at https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT, specifically in the Strong Encryption Specification (SES) section, there is a line on deriving a key from a password:
MasterSessionKey = DeriveKey(SHA1(Password))
What's DeriveKey?
(In WinZip's AES documentation at https://www.winzip.com/en/support/aes-encryption/, they use PBKDF2 with 1000 iterations. I don't see any similar explanation in APPNOTE)
PKWARE implemented a strong encryption in version 5, but did not provide the algorithm of encoding/decoding (Method For Strongly Encrypted .ZIP Files - Patent US 2020/0250329 A1). In this algorithm AES encryption was implemented as part of it. You can define this by strong encryption (bit 6) = yes in General Purpose Flag.
After that WinZip could not use this algo, so it invented another one. You can define this by strong encryption (bit 6) = no in General Purpose Flag and AesExtraFieldRecord with signature 0x990.
As you can see there're two ways to encrypt a zip file. All open source software use the second one. The first one is available only by PKWARE SecureZIP
You can find example of this alogirthm in (7zip) Strong.cpp:35. In java it should look like this:
public static byte[] getMasterKey(String password) {
byte[] data = password.getBytes(StandardCharsets.UTF_8);
byte[] sha1 = DigestUtils.sha1(data);
return DeriveKey(sha1);
}
private static byte[] DeriveKey(byte[] digest) {
byte[] buf = new byte[kDigestSize * 2]; // kDigestSize = 20
DeriveKey2(digest, (byte)0x36, buf, 0);
DeriveKey2(digest, (byte)0x5C, buf, kDigestSize);
return Arrays.copyOfRange(buf, 0, 32);
}
private static void DeriveKey2(byte[] digest, byte c, byte[] dest, int offs) {
byte[] buf = new byte[64];
Arrays.fill(buf, c);
for (int i = 0; i < kDigestSize; i++)
buf[i] ^= digest[i];
byte[] sha1 = DigestUtils.sha1(buf);
System.arraycopy(sha1, 0, dest, offs, sha1.length);
}
Demo:
String password = "JohnDoe";
byte[] masterKey = getMasterKey(password);
The next paragraph 'defines' it
7.2.5.3 The function names and parameter requirements will depend on
the choice of the cryptographic toolkit selected. Almost any
toolkit supporting the reference implementations for each
algorithm can be used. The RSA BSAFE(r), OpenSSL, and Microsoft
CryptoAPI libraries are all known to work well.
I guess it's up to you to decide which of the encryption algorithms you want to use and go from there
I an working on ECIES and need to load peer public key.
Load EC Public key
I an using ECDH and need to load peer public key.
When I try to load public key from PEM file , seems no issue
Issue here:
EVP_PKEY * get_peer_key()
{
// base64 certificate data of alice_pub_key.pem
char *buffer= "MFYwEAYHKoZIzj0CAQYFK4EEAAoDQgAEjWrT7F97QrSqGrlIgPK8dphNBicNO6gDLfOIMjhF2MiLuuzd7L7BP+bLCuNtKKe/2dOkgPqgXv4BFWqgp6PZXQ=="`
// calculate buffer length
int l = strlen(buffer)
//create bio from buffer
BIO *in = BIO_new_mem_buf(buffer,l)
//gnerate ec key
EC_KEY *eckey = PEM_read_bio_EC_PUBKEY(in,NULL,NULL,NULL)` // ==> FAIL
//need to convert to EVP format
EVP_PKEY *peerKey = EVP_PKEY_new()
//assign ec key evp
if(EVP_PKEY_assign_EC_KEY(peerKey,eckey) != 1 )
printf("\n error hapened");
return peerKey;
}
Works fine:
EVP_PKEY * get_peer_key()
{
//Load PEM format file
char * infile = "alice_pub_key.pem";
//create bio
BIO *in = BIO_new(BIO_s_file());
//read bio file
BIO_read_filename(in , infile);
//create eckey
EC_KEY *eckey = PEM_read_bio_EC_PUBKEY(in,NULL,NULL,NULL); // ==> success
// create peer key
EVP_PKEY *peerKey = EVP_PKEY_new();
//assign public key
if(EVP_PKEY_assign_EC_KEY(peerKey,eckey) != 1 )
printf("\n error hapened");
return peerKey;
}
Can some one suggest whats going wrong while reading base64 data of pem file
There are two ways of solving this:
Creating a PEM using a header and footer line and line breaks (at the 64th character;
Base 64 decoding the text and then handling it by parsing the resulting ASN.1 / DER binary;
I'd prefer the latter, as I abhor adding lines and such, it is error prone at best, and string manipulations should be avoided where possible.
Note that this assumes that the base 64 contains a SubjectPublicKeyInfo structure which I've shown you earlier. Otherwise you may have to find out how to parse a X9.62 structure or just a point.
Background
I've been struggling with decrypting an apparently well-formed cipher text for about a day. Assume we've got the following hex-encoded cipher text which contains exactly 160 characters thereby having 80 bytes.
QString c = "1BFAC407AF0D440A2D6176C0B5D125AA96088490299AC18C74623C0EF1BB1372E554FC4150A8066220E943697BE2491D8AE13AA036B298425AC510A8A917D59EBB69708B9040AB3A84C63043EAD4AB07";
QString k = CryptoUtils::hexEncode("abc");
QString p = CryptoUtils::decrypt(c, k);
qDebug() << p;
Provided we're using AES 256, AFAIK, the key must be of length 32 bytes and cipher text of a length of multiple of 16 bytes, which all these consditions are met regarding my snippet code.
Please note that I'm using SHA256 feeding with a pass phrase to generate a 32 bytes key. So, this ensures that all keys are of length 32 bytes.
Full source codes of those function can be found on my repo on GitHub (at branch Part1).
My Question
When I want to run this code, my app crashes. Here's the exception:
terminate called after throwing an instance of 'CryptoPP::InvalidCiphertext'
what(): StreamTransformationFilter: invalid PKCS #7 block padding found
The program has unexpectedly finished.
I searched around about this problem and figured out it could be because of the trailing \0 once you encrypted the plain text. However, I couldn't just solve the problem. Please help me out, it's just driving me crazy.
Full source codes of those function can be found on my repo on GitHub
I'd make these changes at minimum:
QString CryptoUtils::encrypt(QString text, QString keyhex)
{
...
// name the variable, kill the memory leak
SHA256 sha256;
StringSource ss1(decodedKey, size, true, new HashFilter(sha256, new ArraySink(key, AES::MAX_KEYLENGTH)));
...
// name the variable
StringSource ss2(plain, true, new StreamTransformationFilter(Encryptor, new HexEncoder(new StringSink(encrypted))));
// verify embedded NULLs don't affect results
QString qs = QString::fromStdString(encrypted);
assert(qs.length() == encrypted.length());
}
And:
QString CryptoUtils::decrypt(QString text, QString keyhex)
{
// bad karma here...
string encrypted = text.toStdString();
assert(encrypted.length() == text.length());
...
// name the variable, kill the memory leak
SHA256 sha256;
StringSource ss1(decodedKey, size, true, new HashFilter(sha256, new ArraySink(key, AES::MAX_KEYLENGTH)));
...
// name the variable,
StringSource ss2(encrypted, true, new HexDecoder(new StreamTransformationFilter(Decryptor, new StringSink(plain))));
// verify embedded NULLs don't affect results
QString qs = QString::fromStdString(plain);
assert(qs.length() == plain.length());
}
The hexEncode function seems to misbehave:
QString CryptoUtils::hexEncode(QString text)
{
byte *bytearray = (byte *) text.toLatin1().data();
int length = text.toLatin1().length();
return hexEncode(bytearray, length);
}
Should be replaced with:
QString CryptoUtils::hexEncode(QString text)
{
byte *bytearray = (byte *) text.toStdString().data();
int length = text.length();
return hexEncode(bytearray, length);
}
For public key encryption and diffie-hellman in libsodium, I typically make private keys simply by generating 32 random bytes with randombytes_buf and then derive the public key (when needed) using crypto_scalarmult_base.
Is there any benefit to using crypto_box_keypair to generate a keypair (other than syntax)? Or does this function basically do exactly that?
This is exactly what the crypto_box_keypair() function does.
The benefits of this function are clarity, and guarantee that the secret key is properly generated.
https://download.libsodium.org/doc/public-key_cryptography/public-key_signatures.html
for example:
unsigned char pk[crypto_sign_PUBLICKEYBYTES]; //Variable declarations
unsigned char sk[crypto_sign_SECRETKEYBYTES]; Variable declarations
crypto_sign_keypair(pk, sk);
NSData *privateKeyData = [NSData dataWithBytes:sk length:crypto_box_SECRETKEYBYTES];
NSData *publicKeyData = [NSData dataWithBytes:pk length:crypto_box_PUBLICKEYBYTES];
NSLog(#"%#",privateKeyData); // target publick key data and secret key data
NSLog(#"%#",publicKeyData);
//Other
NSLog(#"%s\n\n=====\n\n\n%s",pk,sk); //(nullable const void *)bytes
Byte *byte = (Byte *)[publicKeyData bytes];
NSLog(#"%s",byte);
How do you decrypt aes 128 ctr encrypted file from the middle for http range support?
Here is the encrypted file:
https://www.dropbox.com/s/8e9qembud6n3z7i/encrypted.txt?dl=0
the key is base64 encrypted: E7VQWj3cv1JUi5pklirtDQ9SRJt1DhiqYgzPSpIiVP0
Mega docs: https://mega.co.nz/#doc
The IV is calculated by decrypting the key which gives an array:
Array
(
[0] => 330649690
[1] => 1037877074
[2] => 1418435172
[3] => 2519395597
[4] => 257049755
[5] => 1963858090
[6] => 1645006666
[7] => 2451723517
)
The IV is obtained by slicing the array at 4th offset with length of two And the last two elements of the array are filled with 0:
Array
(
[0] => 257049755
[1] => 1963858090
[2] => 0
[3] => 0
)
Then the key is XOR'd and made into a 128bit array which is then converted into string by the php function pack:
$key = array($key[0] ^ $key[4], $key[1] ^ $key[5], $key[2] ^ $key[6], $key[3] ^ $key[7]);
$key = base64_encode(a32_to_str($key));
$iv = base64_encode(a32_to_str($iv));
Then the file is decrypted using the normal php aes library. I am using mcrypt_generic for the decryption process.
The problem arises when I try to decrypt the file from 2nd byte or the 3rd or the middle.
It works fine if I decrypt it from the 1st byte.
Another thing I have noticed is, If I decrypt the file from 2nd byte, but before that, I decrypt a random string or just the digit 0, the decryption works from the 2nd byte then.
I suppose it has something to do with the IV block counter. I decrypt a random byte then continue decrypting the actual cipher so it works.
I need to start decrypting the file from the start, lets say from the 40mb offset to support live strem seeking.
But that would consume too much memory because I will have to decrypt 40mb of 0's before seeking can be done.
How can I move the IV counter value to 40mb offset ??
I read that IV is increased by +1 for each block for decryption. But since my IV is an array I have tried everything it does not work if I add 1 in it.
I've been at it for months with no fruit. Please help
Here is my previous question which helped understanding the process a bit: AES 128 bit CTR partial file decryption with PHP
Your initial research is indeed correct. In CTR mode, the IV (or nonce) is simply incremented by 1 after each encryption operation. (Encryption and decryption are the same operation in CTR mode, so you can substitute one word for the other as necessary.)
In other words, the state of a CTR mode cipher can be predicted in advance – just add the number of blocks already encrypted to the initial IV. In particular, the state does not depend on the plaintext in any way. AES has a block size of 16, so you would add the number of bytes encrypted divided by 16.
The IV can be considered a 128-bit integer stored in big endian. The cryptography API you use represents it as an array of four 32-bit integers. Simply add the number of blocks to the fourth integer before initializing the cipher. If you think you'll need to handle more than four billion blocks or so, you need to add handling for overflow to the third integer.
The slightly trickier part is initializing the cipher to a state where you have already encrypted a number of bytes that is not divisible by the block size. The solution is to first initialize the cipher to the number of bytes already encrypted divided by 16, rounded down, and then encrypting (the number of bytes already encrypted mod 16) dummy bytes. I believe this is in fact what you already suspected.
You're writing in PHP, but I'm posting a method from a Mega downloader program which I've written in Java in case it helps:
public Cipher getDownloadCipher(final long startPosition) throws Exception {
final Cipher cipher = Cipher.getInstance("AES/CTR/NoPadding");
final ByteBuffer buffer = ByteBuffer.allocate(16).put(nonce);
buffer.asLongBuffer().put(startPosition / 16);
cipher.init(Cipher.DECRYPT_MODE, new SecretKeySpec(key, "AES"), new IvParameterSpec(buffer.array()));
final int skip = (int) (startPosition % 16);
if (skip != 0) {
if (cipher.update(new byte[skip]).length != skip) {
//that should always work with a CTR mode cipher
throw new IOException("Failed to skip bytes from cipher");
}
}
return cipher;
}