Let's say we have HLS segments encrypted with AES-128 method. But segments data restricted with EXT-X-BYTERANGE. Could you please advice me how I can decrypt this?
Can I download this part of data with HTTP range request, and then decrypt this data with key and IV, or decryptor must be initialize from begin of the segment?
…
#EXT-X-KEY:METHOD=AES-128,URI="http://server/file.key",IV=0x1d48fc5dee84b5a3e9a428f055e03c2e
#EXTINF:6.000000,
#EXT-X-BYTERANGE:92580#822
//vidione.loc/storage/AEFIXUum/res-0/segment.m4s
#EXTINF:6.000000,
#EXT-X-BYTERANGE:124656#93402
//vidione.loc/storage/AEFIXUum/res-0/segment.m4s
…
Related
I've generated a random 256 bit symmetric key, in a file, to use for encrypting some data using the OpenSSL command line which I need to decrypt later programmatically using the OpenSSL library. I'm not having success, and I think the problem might be in the initialization vector I'm using (or not using).
I encrypt the data using this command:
/usr/bin/openssl enc -aes-256-cbc -salt -in input_filename -out output_filename -pass file:keyfile
I'm using the following call to initialize the decrypting of the data:
EVP_DecryptInit_ex(ctx, EVP_aes_256_cbc(), nullptr, keyfile.data(), nullptr))
keyfile is a vector<unsigned char> that holds the 32 bytes of the key. My question is regarding that last parameter. It's supposed to be an initialization vector to the cipher algorithm. I didn't specify an IV when encrypting, so some default must have been used.
Does passing nullptr for that parameter mean "use the default"? Is the default null, and nothing is added to the first cipher block?
I should mention that I'm able to decrypt from the command line without supplying an IV.
What is the default IV when encrypting with EVP_aes_256_cbc() [sic] cipher...
Does passing nullptr for that parameter mean "use the default"? Is the default null, and nothing is added to the first cipher block?
There is none. You have to supply it. For completeness, the IV should be non-predictable.
Non-Predictable is slightly different than both Unique and Random. For example, SSLv3 used to use the last block of ciphertext for the next block's IV. It was Unique, but it was neither Random nor Non-Predictable, and it made SSLv3 vulnerable to chosen plaintext attacks.
Other libraries do clever things like provide a null vector (a string of 0's). Their attackers thank them for it. Also see Why is using a Non-Random IV with CBC Mode a vulnerability? on Stack Overflow and Is AES in CBC mode secure if a known and/or fixed IV is used? on Crypto.SE.
/usr/bin/openssl enc -aes-256-cbc...
I should mention that I'm able to decrypt from the command line without supplying an IV.
OpenSSL uses an internal mashup/key derivation function which takes the password, and derives a key and iv. Its called EVP_BytesToKey, and you can read about it in the man pages. The man pages also say:
If the total key and IV length is less than the digest length and MD5 is used then the derivation algorithm is compatible with PKCS#5 v1.5 otherwise a non standard extension is used to derive the extra data.
There are plenty of examples of EVP_BytesToKey once you know what to look for. Openssl password to key is one in C. How to decrypt file in Java encrypted with openssl command using AES in one in Java.
EVP_DecryptInit_ex(ctx, EVP_aes_256_cbc(), nullptr, keyfile.data(), nullptr))
I didn't specify an IV when encrypting, so some default must have been used.
Check your return values. A call should have failed somewhere along the path. Maybe not at EVP_DecryptInit_ex, but surely before EVP_DecryptFinal.
If its not failing, then please file a bug report.
EVP_DecryptInit_ex is an interface to the AES decryption primitive. That is just one piece of what you need to decrypt the OpenSSL encryption format. The OpenSSL encryption format is not well documented, but you can work it backwards from the code and some of the docs. The key and IV computation is explained in the EVP_BytesToKey documentation:
The key and IV is derived by concatenating D_1, D_2, etc until enough
data is available for the key and IV. D_i is defined as:
D_i = HASH^count(D_(i-1) || data || salt)
where || denotes concatentaion, D_0 is empty, HASH is the digest
algorithm in use, HASH^1(data) is simply HASH(data), HASH^2(data) is
HASH(HASH(data)) and so on.
The initial bytes are used for the key and the subsequent bytes for the
IV.
"HASH" here is MD5. In practice, this means you compute hashes like this:
Hash0 = ''
Hash1 = MD5(Hash0 + Password + Salt)
Hash2 = MD5(Hash1 + Password + Salt)
Hash3 = MD5(Hash2 + Password + Salt)
...
Then you pull of the bytes you need for the key, and then pull the bytes you need for the IV. For AES-128 that means Hash1 is the key and Hash2 is the IV. For AES-256, the key is Hash1+Hash2 (concatenated, not added) and Hash3 is the IV.
You need to strip off the leading Salted___ header, then use the salt to compute the key and IV. Then you'll have the pieces to feed into EVP_DecryptInit_ex.
Since you're doing this in C++, though, you can probably just dig through the enc code and reuse it (after verifying its license is compatible with your use).
Note that the OpenSSL IV is randomly generated, since it's the output of a hashing process involving a random salt. The security of the first block doesn't depend on the IV being random per se; it just requires that a particular IV+Key pair never be repeated. The OpenSSL process ensures that as long as the random salt is never repeated.
It is possible that using MD5 this way entangles the key and IV in a way that leaks information, but I've never seen an analysis that claims that. If you have to use the OpenSSL format, I wouldn't have any hesitations over its IV generation. The big problems with the OpenSSL format is that it's fast to brute force (4 rounds of MD5 is not enough stretching) and it lacks any authentication.
Currently I am working on a ChipCard EMV device decryption. Down below is the related data I have after using the transaction (TLV format as Tag Length Value):
<DFDF54> --- It means KSN
0A
950003000005282005B4
<DFDF59>---- per instruction, it is called Encrypted Data Primative
82 ---- length of value in hex, when more than 255 degits, use 82
00D815F35E7846BF4F34E56D7A42E9D24A59CDDF8C3D565CD3D42A341D4AD84B0B7DBFC02DE72A57770D4F795FAB2CE3A1F253F22E0A8BA8E36FA3EA38EE8C95FEBA3767CDE0D3FBB6741A47BE6734046B8CBFB6044C6EE5F98C9DABCD47BC3FD371F777E7E1DCFA16EE5718FKLIOE51A749C7ECC736CB7780AC39DE062DAACC318219E9AAA26E3C2CE28B82C8D22178DA9CCAE6BBA20AC79AB985FF13611FE80E26C34D27E674C63CAC1933E3F9B1BE319A5D12D16561C334F931A5E619243AF398D9636B0A8DC2ED5C6D1C7C795C00D083C08953BC8679C60
I know BDK for this device is 0123456789ABCDEFFEDCBA9876543210. Per decryption instruction, it mentioned that DFDF59 contains the following tags:
FC<len>/* container for encrypted generic data */
F2<len>/*container for Batch Data*/
... /*Batch Data tags*/
F3<len>/*container for Reversal Data, if any*/
... /*Reversal Data tags*/
Per instruction, it mentioned "MAC variant of MSR DUKPT", where MAC stands for message authentication code, and "Parse the data through TLV format. For encrypted data tag, use TDES_Decrypt_CBC to decrypt it".
I tried to use 3DES DUKPT using KSN, BDK, and encrypted data DFDF59. It wouldn't work. Can anyone in decryption field give me some advice? Our vendor is very reluctant to share their knowledge ...
I have no idea how MAC is really playing a role here in decryption.... I thought MAC is just an integrity check.... I am using session key for 3DES DUKPT that was generated from KSN and BDK. this works for other decryptions in this device, but doesn't solve the DFDF59 (chip card EMV decryption).... That is why I start to wonder whether I am using the right session key or not.... Feel free to just throw ideas out there. Thank you!
If you look closely at DUKPT internals it generates a transaction key out of the current future keys and encryption counter. This 'transaction key' for a specific KSN has several variants (which effectively are just xor masks that you put on the transaction key to differentiate it for PIN, MAC req, MAC rsp, data encryption req and rsp usages). These variants mean that you use a different key to generate PIN and different key to encrypt data (so that you cannot ie. decrypt/attack PINblock when able to select data buffer arbitrarily). Using MAC variant means only that for the encryption operation you will be using a certain mask for the DUKPT transaction key.
I have Key Serial Number (KSN), Base Derivation Key (BDK), and encrypted string.
KSN = 9500030000044520002B
BDK = 0123456789ABCDEFFEDCBA9876543210
Encrypted string
23F87C010DCD08E0211D509F3310B1A63564D44134A512AA0740CA2A0FD81BF045AF70395C537774680B566548C2966DFD7F575CC756408A89BCF12A93B8873114FF6EFC69014EA0E0A4EBD392EF40A3F1E15012B3D613E18E4CFD4DE3AFCD771D8B2CE8AB54B1CB7671F24F8562262AAA603C45BF87DE33407234927D7CDA28C86CE29E05A9D03ED65EB3D5DDD3C15A61A79AB8CB7481828339A0B099EDC3BBE3A1C416A06E965FF3CAF2CC395E691AB86C325183EF3A245A3DDF53CBD6D6AEFD0769F560165E4B5C99EBA2584AD3EC
The decryption from the Device Manufacturer Menu is supposed to be:
Right answer
FC8200C6F28200C282027C008E1200000000000000004201410342035E031F005F24031912315F25039507019F0607A00000000310109F0702FFC09F0D0500000000009F0E0500000000009F0F0500000000009F100706010A039600009F2608738091EC178FF5709F2701409F36020017950508001410009B02E8009C01009F3303E0F8C89F34034203009F370450577DCF9F40057000B0B001DFDF70050000000000DFDF71050000000000DFDF7205000000000057134761739981010014D19122010123456789012F000000000000
My decryption turned out to be:
E4771F740C1B1D45DFA193A1DF73C80B3CF68F625DB4604A4C5C392B72BDC98236FAA09D32D674A45D7C3AFAA08E26DAD0B3ABC14662F9D386B9C2F7B992EC35BD7B765450C9E8E56D5D1CEAAE0641E5CB66D2102C58A6325C845E62BBB8994E0441AF1887CDD0C3C0AD2A8AD178A949C0944D46A04DA08DAE30FA571CE4C0EACE4AAFC503708AD0240584C80506F98F0C8D8E348F0A5B0E949EED814C8F1C2B9157B66455715685673D0F1B816AD4AEF7743EB9E339361A733F1EF37CA1DC5D468CA988DDD9E74E7CCC9F999FDBBCFE
and session key turned out to be:
7ED0EDD8A1961ABF7CFC995219B80FC57ED0EDD8A1961ABF
Can any decryption professional run a little 3DES-DUKPT and check it for me?
The instruction from the Device manufacturer claims that 3DES-DUKPT is formed, using Data Skye variant and (KSN)" for the encrypted value.
THank you!
I recommend using the BP-Tools freeware to check your results.
It contains a DUKPT calculator.
Ok, the original task is to track users among 2 "friendly" web-sites who are able to share users cookies (lets say, I have example.com and my friend has mysite.com and also he has a domain simple.example.com so he can set cookies on .example.com).
To track users activity we want to set unique cookie, this cookie should be unique and 32 bytes long (ascii). Quite simple from this point of view and can be implemented as such:
md5(microtime)
and that's it, but now we have new constraints:
we should be able to tell who exactly has set the cookie: exmaple.com engine or mysite.com engine
32 bytes longs is a must, still
we should be able to encrypt timestamp (when cookies was issued)
first and last character of the resulting cookie value should be different so we can do A/B testing basing on the cookie (so we could always say if last character of the cookie is "> K", show this users "feature A")
Given that the resulting string should always be 32 or less characters long and data should be encrypted and decrypted (not by users, of course) and the string should be unique for the users, it makes the task quite complex.
My thought and questions:
we should use symmetric key encryption (solves constraints 1 and 3), but it this case how do we ensure that resulting string is no longer than 32 chars (constraint 2)?
is there other solution on the problem given that amount of data we need to encrypt is: timestamp and microseconds (14 bytes), site-issuer flag (1 byte) = 15 bytes total
My first take was to pack data into binary string and than base64-encode it. The result would be 8-chars long base64-encoded string:
def encode():
base64( pack('Lv', timestamp, microseconds) )
Add site-issuer flag and chars at the beginning and the end:
def getCookie():
rand('a'...'Z') + encode() + issuerFlagChar() + rand('a'...'Z')
So, the result is 11 chars long and we meet constraint 2 easily.
But the problem is: this algorithm is not secure for sure, I'm not sure if the resulting string for millions of websites users is unique.
I wonder if I could use DES or AES for this purpose but I'm not sure that the resulting string will always meet constraint 2 (resulting string should be no longer than 32 ascii chars).
Is there symmetric key algorithms that ensure something like "if you encrypt N bytes with M-bytes key you will have resulting data length of Math.Ceil(N*2+1/M) bytes"? So the resulting length would be predictable?
Setting aside the fact that you should indeed consult a security consultant, the actual question you pose can easily be answered:
Is there symmetric key algorithms that ensure something like "if you encrypt N bytes with M-bytes key you will have resulting data length of Math.Ceil(N*2+1/M) bytes"? So the resulting length would be predictable?
Yes there are. And they are called Block Ciphers.
By definition, every block cipher has the property that the length of the ciphertext is equal to the length of the plain text. In practice most block ciphers (inclusing DES and AES) cheat a bit because they require the plaintext to be padded to the length of the block before they start encrypting.
In other words, given a plaintext of N bytes and a block size of B, the ciphertext will have a length of B*(Math.ceil(N/B)) bytes.
Note how I am talking about the block size, which is different from the key size. The key size is actually irrelevant in this case.
For example, AES uses a block size of 128 bits, or 16 bytes. This means that if your plain text is between 17 and 32 bytes long, AES will guarantee that your ciphertext is 32 bytes long. This is independent from the key size you choose, which can be one of 128, 192 or 256 bits (16, 24 or 32 bytes).
First of all, you need to know whether you want to encrypt or sign the data.
Encrypting will prevent users from seeing the data, but they are still able to modify it in some ways depending on the encryption type. For example, decrypting a modified ciphertext will simply give corrupted data, it won't fail.
Signing, on the other hand, will prevent users from modifying the data, that is, your code will be able to detect the data has been modified. A simple algorithm for this is HMAC.
I'll assume you want both. My solution below does both.
Your cookie must be 32 bytes long, which is 256 bits. We are going to use 128 bits for encrypted data and 128 bits for the HMAC.
For the data, I will encode the timestamp as a 64bit integer (more than enough even if you want to store it to microsecond precision). The site that issued the cookie can be stored as 1 bit if you have two sites, but I'll store it in a 32bit integer because we have plenty of space. Same for a tag you can use for a/b testing.
All the data is exactly 128 bits, 16 bytes. This is the exact size of an AES block. So, we will encrypt it with AES!
The other 16 bytes will be a MAC of the ciphertext (Encrypt then MAC). I used HMAC-SHA256, which has 256bits of output. We only have room for 128bits, so I have truncated it. In theory this makes it less secure, but in practice 128bit is big enough to make a brute-force attempt impossible.
Decrypting the cookie is the reverse process: calculate the HMAC of the given ciphertext and check it matches the given MAC. If so, proceeed to decrypt the ciphertext and unpack the data.
Here's the code:
from struct import pack, unpack
from Crypto.Cipher import AES
import hashlib
import hmac
AES_KEY = hashlib.sha256(b"secret key 1 asdfasdf").digest()
HMAC_KEY = hashlib.sha256(b"secret key 2 asdfasdf").digest()
# timestamp: 64bit unix timestamp
# site: 32bit integer, which site issued the cookie
# tag: 32bit integer, tag used for a/b testing.
def encrypt_cookie(timestamp, site, tag):
# Pack the data
data = pack('QII', timestamp, site, tag)
# Encrypt it
aes = AES.new(AES_KEY, AES.MODE_ECB, 'This is an IV456')
ciphertext = aes.encrypt(data)
# Do HMAC of the ciphertext
sig = hmac.new(HMAC_KEY, ciphertext, hashlib.sha256).digest()
sig = sig[:16] # Truncate to only first 16 bytes.
return ciphertext + sig
def decrypt_cookie(cookie):
# Do HMAC of the ciphertext
sig = hmac.new(HMAC_KEY, cookie[:16], hashlib.sha256).digest()
sig = sig[:16] # Truncate to only first 16 bytes.
# Check the HMAC is ok
if sig != cookie[16:]:
raise Exception("Cookie has been tampered with")
# Decrypt it
aes = AES.new(AES_KEY, AES.MODE_ECB, 'This is an IV456')
data = aes.decrypt(cookie[:16])
# unPack the data
timestamp, site, tag = unpack('QII', data)
return timestamp, site, tag
cookie = encrypt_cookie(1, 2, 3)
print(len(cookie)) # prints: 32
print(decrypt_cookie(cookie)) # prints: 1, 2, 3
# Change a single byte in the cookie, the last one
cookie = cookie[:31] + b'0'
print(decrypt_cookie(cookie)) # raises the exception
I'm curious to know why the cookie must be 32bytes though. Seems a weird requirement, and if you didn't have it, you'd be able to use many libraries that are designed to solve exactly this problem, such as Django signing if you're using Django.
I have an application to decrypt media packets.
it require me to provide Master key and salt key.
my SDP provide me (after negotiation ended) with
AES_CM_128_HMAC_SHA1_80 inline:Fu8vxnU4x1fcCzbhNrtDV0eq4RnaK4n2/jarOigZ
according to SDP rfc the string after the "inline:" is:
"concatenated master key and salt, base64 encoded"
when the master key is X bytes and the salt is Y bytes.
I am tyring :
byte[] masterAndSalt = Convert.FromBase64String("Fu8vxnU4x1fcCzbhNrtDV0eq4RnaK4n2/jarOigZ")
and then get the first x bytes to the master and the other Y for salt.
but my app says my keys are wrong, i don't understand - should i use some else than Convert.FromBase64String ?
OK, i got it right.
on AES_CM_128_HMAC_SHA1_80 cipher the Master key is 16 byte, and the salt is 14 byte long.
what should be done is to use Convert.FromBase64String on the key,
which produced a 30 byte long array, take the first 16 as master, and the last 14 as salt.
the decryption algorithm should produce the session key and salt from it (along with other info).