I'm writing a Go script that will decrypt some legacy data that is encrypted with EVP_aes_256_cbc and an RSA public key.
In C this would be something like:
key_size = EVP_OpenInit(&ctx, EVP_aes_256_cbc(), evp_key, eklen, iv, pkey);
//...
EVP_OpenUpdate(&ctx, destination, &len_out, buffer_in, buffer_size)
//...
EVP_OpenFinal(&ctx, destination+len_out, &len_out);
I have the evp_key and iv byte array equivalents in Go, but I must confess the order of how EVP works in OpenSSL eludes me (I'm fairly competent in C, but I can't get a grasp on the process by which this decryption happens from looking at the OpenSSL source.)
In Go, I can get this far:
pKey := //rsa.PrivateKey
eklen := 32
evpKey := "// hidden 32 byte array"
iv := "// hidden 16 byte array"
c, err := aes.NewCipher(iv)
cbc := cipher.NewCBCDecrypter(c, iv)
And here's where I get lost. I have an evpKey and the pKey, but I'm not sure how to decrypt the data from here. OpenSSL uses RSA_decrypt_old or something like that, but I'm unable to track down what that actually means.
Is there a Go equivalent or do I need to bust out the much-too-expensive cgo package and roll up my sleeves?
Update (Resolution):
For anyone looking to replicate the EVP behavior in Go or just wondering how EVP works exactly, the following is breakdown.
If you know the C (or Java or whatever OpenSSL implementation) was encrypting with something like:
// pseudo-code: don't copypasta and expect amazing
EVP_PKEY_assign_RSA(pkey, public_key);
EVP_CIPHER_CTX_init(&ctx);
EVP_SealInit(&ctx, EVP_aes_256_cbc(), &evp_key, &evp_key_len, iv, &pkey, 1);
EVP_SealUpdate(&ctx, buffer_out, &encrypt_len, (unsigned char*)buffer_in, len);
EVP_SealFinal(&ctx, buffer_out+encrypt_len, &encrypt_len);
The "Seal" actually just encrypts the key with the RSA public key.
In Go to decrypt something like that:
evpKeyBytes := "// the rsa.PublicKey encoded evpKey"
evpKey, err := rsa.DecryptPKCS1v15(rand.Reader, PrivateKeyRSA, evpKeyBytes)
c, err := aes.NewCipher(evpKey)
cbc := cipher.NewCBCDecrypter(c, iv)
decryptedDataBytes := make([]bytes, 2048) // some message size
cbc.CryptBlocks(decryptedDataBytes, encryptedDataBytes)
data = string(decryptedDataBytes)
// data should have the expected decrypted result.
NewCipher expects the key not the iv, and since you're passing it a 128bit iv it works as aes128cbc.
Related
I want to decrypt fmp4 segment.
This segment was encrypt with HLS Apple Tools (https://developer.apple.com/documentation/http_live_streaming/about_apple_s_http_live_streaming_tools)
METHOD is AES-128
IV is 1d48fc5dee84b5a3e9a428f055e03c2e
I have a key and IV (you can got the key, and segment in google drive https://drive.google.com/drive/folders/1xF-C9EXFvT8qjI--sBB6QMPn8cNW7L-D?usp=sharing)
To decrypt I use Poco library.
This is my code:
Poco::Crypto::Cipher::ByteVec readKey(const std::string& uri) {
Poco::Crypto::Cipher::ByteVec key;
auto stream = Stream::makeStream(uri);
if (stream->open(uri, {})) {
key.resize(KEY_SIZE);
stream->read((char*)&key[0], KEY_SIZE);
}
return key;
}
std::vector<uint8_t> _key = readKey("./unit-tests/resources/cipher-stream/file.key");
std::string ivSrc = "1d48fc5dee84b5a3e9a428f055e03c2e";
Poco::Crypto::Cipher::ByteVec iv {ivSrc.begin(), ivSrc.end()};
Poco::Crypto::CipherKey key("aes-128-cbc", _key, iv);
Poco::Crypto::Cipher::Ptr cipher = Poco::Crypto::CipherFactory::defaultFactory().createCipher(key);
Poco::FileInputStream src("./unit-tests/resources/cipher-stream/fileSequence1.m4s");
Poco::FileOutputStream dst("./unit-tests/resources/cipher-stream/fileSequence1_dec.m4s");
Poco::Crypto::CryptoOutputStream decryptor(dst, cipher->createDecryptor());
Poco::StreamCopier::copyStream(src, decryptor);
// decryptor.close();
src.close();
dst.close();
Problem description:
After decryption I got distorted data. You can see this at the beginning of the file. Please see picture below. On the right side of the image file is distorted.
The correct data you can see on the left side.
You're using the wrong IV; that will lead to the first block (16 bytes) being corrupted. Your IV hex value is 1d48fc5dee84b5a3e9a428f055e03c2e, but you're interpreting that as ASCII. It's using the first 16 bytes of your string and ignoring the rest.
I haven't used Poco in a long time and don't remember if there's a hex parser handy, but that's what you need. Or write the IV directly in hex rather than as an ASCII string.
In the code below (also at http://play.golang.org/p/77fRvrDa4A but takes "too long to process" in the browser there) the 124 byte version of the sourceText won't encrypt because: "message too long for RSA public key size" of 1024. It, and the longer 124 byte sourceText version, work with 2048 bit key size.
My question is how does one exactly calculate the key size in rsa.GenerateKey given the byte length of the source text? (A small paragraph size of text takes nearly 10 seconds at 4096 key size, and I don't know the length of the sourceText until runtime.)
There's a very brief discussion of this at https://stackoverflow.com/a/11750658/3691075, but it's not clear to me as I'm not a crypto guy.
My goal is to encrypt, store in a DB and decrypt about 300-byte long JSON strings. I control both the sending and the receiving end. Text is encrypted once, and decrypted many times. Any hints of strategy would be appreciated.
package main
import (
"crypto/md5"
"crypto/rand"
"crypto/rsa"
"fmt"
"hash"
"log"
"time"
)
func main() {
startingTime := time.Now()
var err error
var privateKey *rsa.PrivateKey
var publicKey *rsa.PublicKey
var sourceText, encryptedText, decryptedText, label []byte
// SHORT TEXT 92 bytes
sourceText = []byte(`{347,7,3,8,7,0,7,5,6,4,1,6,5,6,7,3,7,7,7,6,5,3,5,3,3,5,4,3,2,10,3,7,5,6,65,350914,760415,33}`)
fmt.Printf("\nsourceText byte length:\n%d\n", len(sourceText))
// LONGER TEXT 124 bytes
// sourceText = []byte(`{347,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,65,350914,760415,33}`)
// fmt.Printf("\nsourceText byte length:\n%d\n", len(sourceText))
if privateKey, err = rsa.GenerateKey(rand.Reader, 1024); err != nil {
log.Fatal(err)
}
// fmt.Printf("\nprivateKey:\n%s\n", privateKey)
privateKey.Precompute()
if err = privateKey.Validate(); err != nil {
log.Fatal(err)
}
publicKey = &privateKey.PublicKey
encryptedText = encrypt(publicKey, sourceText, label)
decryptedText = decrypt(privateKey, encryptedText, label)
fmt.Printf("\nsourceText: \n%s\n", string(sourceText))
fmt.Printf("\nencryptedText: \n%x\n", encryptedText)
fmt.Printf("\ndecryptedText: \n%s\n", decryptedText)
fmt.Printf("\nDone in %v.\n\n", time.Now().Sub(startingTime))
}
func encrypt(publicKey *rsa.PublicKey, sourceText, label []byte) (encryptedText []byte) {
var err error
var md5_hash hash.Hash
md5_hash = md5.New()
if encryptedText, err = rsa.EncryptOAEP(md5_hash, rand.Reader, publicKey, sourceText, label); err != nil {
log.Fatal(err)
}
return
}
func decrypt(privateKey *rsa.PrivateKey, encryptedText, label []byte) (decryptedText []byte) {
var err error
var md5_hash hash.Hash
md5_hash = md5.New()
if decryptedText, err = rsa.DecryptOAEP(md5_hash, rand.Reader, privateKey, encryptedText, label); err != nil {
log.Fatal(err)
}
return
}
One does not usually calculate the RSA key size based on payload. One simply needs to select one RSA key size based on a compromise between security (bigger is better) and performance (smaller is better). If that is done, use hybrid encryption in conjunction with AES or another symmetric cipher to actually encrypt the data.
If the payload doesn't exceed 300 bytes and you're using OAEP (at least 42 bytes of padding), then you can easily calculate the minimum key size:
(300 + 42) * 8 = 2736 bit
That's already a reasonable size key. It provides good security according to today's norms and is fairly fast. There is no need to apply a hybrid encryption scheme for this.
Now, you may notice that the key size isn't a power of 2. This is not a problem. You should however use a key size that is a multiple of 64 bit, because processors use 32-bit and 64-bit primitives to do the actual calculation, so you can increase the security without a performance penalty. The next such key size would be:
ceil((300 + 42) * 8 / 64.0) * 64 = 2752 bit
Here are some experimental results what some languages/frameworks accept (not performance-wise) as the key size:
Golang: multiple of 1 bit and >= 1001 (sic!) [used ideone.com]
PyCrypto: multiple of 256 bit and >= 1024 [local install]
C#: multiple of 16 bit and >= 512 [used ideone.com]
Groovy: multiple of 1 bit and >= 512 [local install]
Java: multiple of 1 bit and >= 512 [used ideone.com: Java & Java7]
PHP/OpenSSL Ext: multiple of 128 bit and >= 640 [used ideone.com]
Crypto++: multiple of 1 bit and >= 16 [local install with maximal validation toughness of 3]
Before you decide to use some kind of specific key size, you should check that all frameworks support that size. As you see, there are vastly varying results.
I tried to write some performance tests of key generation, encryption and decryption with different key sizes: 512, 513, 514, 516, 520, 528, 544, 576. Since I don't know any go, it would be hard to get the timing right. So I settled for Java and Crypto++. The Crypto++ code is probably very buggy, because key generation for 520-bit and 528-bit keys is up to seven orders of magnitude faster than for the other key sizes which is more or less constant for the small key size window.
In Java the key generation was pretty clear in that the generation of a 513-bit key was 2-3 times slower than for a 512-bit key. Other than that the results are nearly linear. The graph is normalized and the numbers of iterations is 1000 for the full keygen-enc-dec cycle.
The decryption makes a little dip at 544-bit which is a multiple of 32-bit. Since it was executed on a 32-bit debian, this might mean that indeed there are some performance improvements, but on the other hand the encryption was slower for that key size.
Since this benchmark wasn't done in Go, I won't give any advice on how small the overhead can be.
I have a problem with decryption when I try to decrypt the same byte slice again.
Example of code for clarification:
package main
import (
"fmt"
"crypto/cipher"
"crypto/des"
)
const (
// tripleKey is TripleDES key string (3x8 bytes)
tripleKey = "12345678asdfghjkzxcvbnmq"
)
var (
encrypter cipher.BlockMode
decrypter cipher.BlockMode
)
func init() {
// tripleDESChiper is chiper block based on tripleKey used for encryption/decryption
tripleDESChiper, err := des.NewTripleDESCipher([]byte(tripleKey))
if err != nil {
panic(err)
}
// iv is Initialization Vector used for encrypter/decrypter creation
ciphertext := []byte("0123456789qwerty")
iv := ciphertext[:des.BlockSize]
// create encrypter and decrypter
encrypter = cipher.NewCBCEncrypter(tripleDESChiper, iv)
decrypter = cipher.NewCBCDecrypter(tripleDESChiper, iv)
}
func main() {
message := "12345678qwertyuia12345678zxcvbnm,12345678poiuytr"
data := []byte(message)
hash := encrypt(data)
decoded1 := decrypt(hash)
decoded2 := decrypt(hash)
decoded3 := decrypt(hash)
decoded4 := decrypt(hash)
fmt.Printf("encrypted data : %x\n", data)
fmt.Printf("1 try of decryption result : %x\n", decoded1)
fmt.Printf("2 try of decryption result : %x\n", decoded2)
fmt.Printf("3 try of decryption result : %x\n", decoded3)
fmt.Printf("4 try of decryption result : %x\n", decoded4)
}
func encrypt(msg []byte) []byte {
encrypted := make([]byte, len(msg))
encrypter.CryptBlocks(encrypted, msg)
return encrypted
}
func decrypt(hash []byte) []byte {
decrypted := make([]byte, len(hash))
decrypter.CryptBlocks(decrypted, hash)
return decrypted
}
This code is also available and runnable
on the playground.
It gives the following result:
encrypted data : 313233343536373871776572747975696131323334353637387a786376626e6d2c3132333435363738706f6975797472
1 try of decryption result : 313233343536373871776572747975696131323334353637387a786376626e6d2c3132333435363738706f6975797472
2 try of decryption result : 5e66fa74456402c271776572747975696131323334353637387a786376626e6d2c3132333435363738706f6975797472
3 try of decryption result : 5e66fa74456402c271776572747975696131323334353637387a786376626e6d2c3132333435363738706f6975797472
4 try of decryption result : 5e66fa74456402c271776572747975696131323334353637387a786376626e6d2c3132333435363738706f6975797472
As you can see the first decryption works well and returns valid result,
but all other tries returns the wrong result.
The first 16 bytes of result is not as in source byte slice.
Can somebody describe what I am doing wrong?
Short version: don't reuse the decrypter object.
Longer version: You're using a cipher in CBC mode: when encrypting the data, the plaintext for block N is XOR-ed with the ciphertext for block N-1 (or the IV, on the first block). On decryption this is done in reverse.
This means that when you try and reuse your decrypter object you don't get the correct results because the state isn't correct - it is decrypting the blocks as if they were subsequent blocks in your message. A peculiarity of CBC is that an incorrect IV will only affect the first decrypted block.
I've writing a client-side app in Go that needs to interact with a C program on the server-side. The client does an AES CFB encrypt and the server decrypts. Unfortunately the server-side has a bug with reusing an initialization vector. It tries to do 3 decrypt operations based on:-
key1, iv
key2, iv
key3, iv
Due to this issue the iv is actually modified between decrypt operations. My problem now is how to reproduce this behaviour on the client side using Go.
By inserting a Println into the encrypt function below, I can see the cfb struct which, I think, contains the modified IV for the next block but because it's a stream interface, I'm not sure how to extract it into a byte slice. Any suggestions?
Thanks
package main
import (
"fmt"
"encoding/hex"
"crypto/cipher"
"crypto/aes"
)
func encrypt_aes_cfb(plain, key, iv []byte) (encrypted []byte) {
block, err := aes.NewCipher(key)
if err != nil {
panic(err)
}
encrypted = make([]byte, len(plain))
stream := cipher.NewCFBEncrypter(block, iv)
stream.XORKeyStream(encrypted, plain)
fmt.Println(stream)
return
}
func main() {
plain := []byte("Hello world...16Hello world...32")
key := make([]byte, 32)
iv := make([]byte, 16)
enc := encrypt_aes_cfb(plain, key, iv)
fmt.Println("Key: ", hex.EncodeToString(key))
fmt.Println("IV: ", hex.EncodeToString(iv))
fmt.Println("Enc: ", hex.EncodeToString(enc))
}
Going down the path you're hinting at is a bit ugly, and prone to break when the implementation changes.
You can get the IV from the stream by:
s := reflect.Indirect(reflect.ValueOf(stream))
lastIV := s.FieldByName("next").Bytes()
But, there's an easier way! Concatenate the plain text inputs, so that the stream for the second starts with the IV from the end of the first (and so on).
Playground Example
combined := append(plain, plain2...)
encCombined := encrypt_aes_cfb(combined, key, iv)
enc := encCombined[:len(plain)]
enc2 := encCombined[len(plain):]
Trying to emulate an algorithm in Go that is basically AES ECB Mode encryption.
Here's what I have so far
func Decrypt(data []byte) []byte {
cipher, err := aes.NewCipher([]byte(KEY))
if err == nil {
cipher.Decrypt(data, PKCS5Pad(data))
return data
}
return nil
}
I also have a PKCS5Padding algorithm, which is tested and working, which pads the data first. I cant find any information on how to switch the encryption mode in the Go AES package (it's definitely not in the docs).
I have this code in another language, which is how I know this algorithm isn't working quite correctly.
EDIT: Here is the method as I have interpreted from on the issue page
func AESECB(ciphertext []byte) []byte {
cipher, _ := aes.NewCipher([]byte(KEY))
fmt.Println("AESing the data")
bs := 16
if len(ciphertext)%bs != 0 {
panic("Need a multiple of the blocksize")
}
plaintext := make([]byte, len(ciphertext))
for len(plaintext) > 0 {
cipher.Decrypt(plaintext, ciphertext)
plaintext = plaintext[bs:]
ciphertext = ciphertext[bs:]
}
return plaintext
}
This is actually not returning any data, maybe I screwed something up when changing it from encripting to decripting
Electronic codebook ("ECB") is a very straightforward mode of operation. The data to be encrypted is divided into byte blocks, all having the same size. For each block, a cipher is applied, in this case AES, generating the encrypted block.
The code snippet below decrypts AES-128 data in ECB (note that the block size is 16 bytes):
package main
import (
"crypto/aes"
)
func DecryptAes128Ecb(data, key []byte) []byte {
cipher, _ := aes.NewCipher([]byte(key))
decrypted := make([]byte, len(data))
size := 16
for bs, be := 0, size; bs < len(data); bs, be = bs+size, be+size {
cipher.Decrypt(decrypted[bs:be], data[bs:be])
}
return decrypted
}
As mentioned by #OneOfOne, ECB is insecure and very easy to detect, as repeated blocks will always encrypt to the same encrypted blocks. This Crypto SE answer gives a very good explanation why.
Why? We left ECB out intentionally: it's insecure, and if needed it's
trivial to implement.
https://github.com/golang/go/issues/5597
I used your code so I feel the need to show you how I fixed it.
I am doing the cryptopals challenges for this problem in Go.
I'll walk you through the mistake since the code is mostly correct.
for len(plaintext) > 0 {
cipher.Decrypt(plaintext, ciphertext)
plaintext = plaintext[bs:]
ciphertext = ciphertext[bs:]
}
The loop does decrypt the data but does not put it anywhere. It simply shifts the two arrays along producing no output.
i := 0
plaintext := make([]byte, len(ciphertext))
finalplaintext := make([]byte, len(ciphertext))
for len(ciphertext) > 0 {
cipher.Decrypt(plaintext, ciphertext)
ciphertext = ciphertext[bs:]
decryptedBlock := plaintext[:bs]
for index, element := range decryptedBlock {
finalplaintext[(i*bs)+index] = element
}
i++
plaintext = plaintext[bs:]
}
return finalplaintext[:len(finalplaintext)-5]
What this new improvement does is store the decrypted data into a new []byte called finalplaintext. If you return that you get the data.
It's important to do it this way since the Decrypt function only works one block size at a time.
I return a slice because I suspect it's padded. I am new to cryptography and Go so anyone feel free to correct/revise this.
Ideally you want to implement the crypto/cipher#BlockMode interface. Since an official one doesn't exist, I used crypto/cipher#NewCBCEncrypter as a starting point:
package ecb
import "crypto/cipher"
type ecbEncrypter struct { cipher.Block }
func newECBEncrypter(b cipher.Block) cipher.BlockMode {
return ecbEncrypter{b}
}
func (x ecbEncrypter) BlockSize() int {
return x.Block.BlockSize()
}
func (x ecbEncrypter) CryptBlocks(dst, src []byte) {
size := x.BlockSize()
if len(src) % size != 0 {
panic("crypto/cipher: input not full blocks")
}
if len(dst) < len(src) {
panic("crypto/cipher: output smaller than input")
}
for len(src) > 0 {
x.Encrypt(dst, src)
src, dst = src[size:], dst[size:]
}
}
I was confused by a couple of things.
First i needed a aes-256 version of the above algorithm, but apparently the aes.Blocksize (which is 16) won't change when the given key has length 32. So it is enough to give a key of length 32 to make the algorithm aes-256
Second, the decrypted value still contains padding and the padding value changes depending on the length of the encrypted string. E.g. when there are 5 padding characters the padding character itself will be 5.
Here is my function which returns a string:
func DecryptAes256Ecb(hexString string, key string) string {
data, _ := hex.DecodeString(hexString)
cipher, _ := aes.NewCipher([]byte(key))
decrypted := make([]byte, len(data))
size := 16
for bs, be := 0, size; bs < len(data); bs, be = bs+size, be+size {
cipher.Decrypt(decrypted[bs:be], data[bs:be])
}
// remove the padding. The last character in the byte array is the number of padding chars
paddingSize := int(decrypted[len(decrypted)-1])
return string(decrypted[0 : len(decrypted)-paddingSize])
}