I am trying to implement a SOAP client that's supposed to receive an message (XML) with MTOM/XOP attachment. The message body and attachment are encrypted using separate keys. The encryption algorithm used is AES128-CBC. The body is base64 encoded and decrypts fine, however the attachment is binary and has odd lengths (530 bytes, 527 bytes, etc).
My question is, is it possible that the server is messing something up in encryption? Or is it possible to produce AES128 output that's not multiple of 16?
AES in CBC mode will always produce output that is a multiple of 16 bytes.
If AES is used in CTR (Counter) mode) it produces output with the same length as the input.
AES is a block cipher and works with 16-byte blocks that is 16-bytes of data at a time. For CBC mode the input and output must be a multiple of 16-bytes. CTR mode does not have this requirement.
The way around this is to add padding bytes to the input prior to encryption and remove them after decryption. This can be done automatically by specifying a padding type, generally PKCS#7 padding.
Related
I have a requirement for one of my projects in which I am expecting a few of the incoming fields encrypted as AES-256 when sent to us by upstream. The incoming file is comma delimited. Is there a possibility that the AES encrypted fields may contain "," throwing off the values to different fields? What about if it is pipe delimited or some other delimiter?
Also, how what should be the datatype of these encrypted fields in order to read these encrypted fields using an ETL tool?
Thanks in Advance
AES as a block cipher is a family of permutations selected by the key. The output is expected to be random ( more precisely we believe that AES is a Pseudo-Random-Permutation)
AES ( like any block cipher) outputs binary data, usually as a byte array and bytes can take any value between 0 and 256 with equal probability.
You are not alone;
Transmitting binary data can create problems, especially in protocols that are designed to deal with textual data. To avoid it altogether, we don't transmit binary data. Many of the programming errors related to encryption on Stack Overflow are due to sending binary data over text-based protocols. Most of the time this works, but occasionally it fails and the coders wonder about the problem. The binary data corrupts the network protocol.
Therefore hex, base64, or similar encodings are necessary to mitigate this. Base64 is not totally URL safe and one can make it URL safe with a little work.
And note that has nothing to do with security; it is about visibility and interoperability.
I'm writing a python script to encrypt/decrypt strings with RSA. I have no problems with the algorithm itself, but I don't understand how to use it correctly.
I mean, there is no point in encrypting every each symbol in a string separately. Because same symbols will give us same ciphers (like Caesar cipher). So I think I should divide the whole message into blocks of same length.
But it makes it difficult to decrypt the message. Because after you encrypt the blocks, the length of each block could change. So when decrypting, you don't know where a certaing blocks starts and where it ends.
For example, when I encrypt "stronger" with RSA I get:
5716225862
I divided the original message into 4 blocks of 4 symbols. But after encrypting I get a message of 10 symbols. And that's the problem.
Hope, you understand what I mean. Sorry for my bad English.
Simply said, RSA is not for directly encrypting plain text, it is used for encrypting a symmetric key (AES, for instance), and this is with this symmetric key that you will encrypt (and further decrypt) your plain text.
Since the plain text may have any size, it is encrypted using AES with a stream cipher (for instance AES-256-GCM) or a block cipher (for instance AES-256-CBC).
With AES, same symbols will not give same ciphers, since you have to choose a new random IV (initialization vector) each time you encrypt your plain text.
So, you need to use a 2-steps encryption scheme: use RSA to encrypt a symmetric key, and use this symmetric key to encrypt your plain text.
i was wondering exactly how block cipher worked, i need to end up with 10 or less blocks after its done and i really haven't found anything online that can tell me how many blocks it will make and if i can limit it to 10 or less?
If you know of any resources like websites that can help me with this it would be greatly appreciated!
A Block Cypher such as AES encrypts data a block at a time appending the block of encrypted data to the output. Block sized vary and AES uses a 16-byte block. (RSA is not a block cypher)
Since data is frequently not a multiple go the block size data is added to the end of the data, this is called padding and the most commonly used padding is PKCS#7 née PKCS#5. (As an option see CTR mode below)
Additionally there are modes such as ECB, CBC, CTR and others. ECB (Electric Code Book) is mostly a null mode in that it does nothing. CBC (Cypher Block Chaining) is a mode that includes some data from the previous block and also had an IV so that encryption the same data does not produce the same encrypted data. CTR (Counter) is a streaming mode that does not require padding but has other requirements such as a Nonce.
My device doesn't support full 3DES (EDE). How can I emulate one using standard DES? Encryption mode is CBC.
You start by picking three independent DES keys which are not related to each other in any way.
You will want to put DES into ECB mode, not CBC mode. You also need to ensure that each encryption and decryption operation is done only on 64-bit blocks and nothing more or less. Padding schemes and the likes will cause a vulnerability in the implementation and will lead to the discovery of the block content via brute force faster than a brute force against each key.
Using the first key, encrypt your plaintext. Using the second key, decrypt that value. Using the third key, encrypt the value for your full block. It looks like this:
Encrypt(k3, Decrypt(k2, Encrypt(k1, plaintext)))
Decryption is the other way around and looks like this:
Decrypt(k1, Encrypt(k2, Decrypt(k3, ciphertext)))
When you encrypt your blocks with 3DES you then need to apply your mode of operation like CBC or CTR and apply padding if needed.
Be careful.
Block mode encryption
What you do is that you split the key of size 128 bit (16 byte) or 168 bit (24 byte) in two or three pieces respectively. So for a 16 byte key K you would have two keys Ka and Kb, and for a 24 byte key you would have Ka, Kb and Kc. DES ABC keys have an effective strength of about 112 bits, DES ABA keys have an effective strength of about 80 bits.
To encrypt a single block of 8 bytes (the block size of both DES and 3DES) you would perform the following cryptographic operation: Cn = E(Ka, D(Kb, E(Kc, Mn))) where Mn is the n'th block of the plain text message and Cn the n'th block of the cipher text. If you don't have a Kc then you may use Ka (DES ABC key vs DES ABA key).
For this you need a single block DES encrypt, which is identical to a single block encrypt in ECB mode, or a single block encrypt with CBC and an IV consisting of 8 bytes valued 00h.
CBC
So that's the block encryption sorted, now you need some kind of encryption mode and padding mode. I'll explain CBC mode encryption here, ECB should not be used for encryption non-random data.
With CBC mode encryption you XOR a vector to the plain text. The vector is normally just the output of the last DESede encrypted block. As you don't have any preceding cipher text, you need to create the first vector yourself using random data. This vector is called the initialization vector or IV. See wikipedia for a clear picture.
Padding
Block cipher modes only allow full blocks of plain text to be encrypted. So you would need some kind of padding scheme. Although there are many padding modes, PKCS#5 padding is used most of the time. You should pad the plain text like this: pad with bytes valued 0Xh, where X is the number of padding bytes required to create a full block. X should be between 1 and 8: in other words, PKCS#5 padding is always used; this makes it possible to distinquish the padding bytes from the plain text.
If you use padding in an online protocol then you need to protect against padding oracle attacks. In this case it is highly recommended to use some form of integrity checks, e.g. by adding a HMAC over the cipher text using a separate key.
3DES is just DES used three times on the plaintext:
ciphertext = E_K3(D_K2(E_K1(plaintext)))
plaintext = D_K1(E_K2(D_K3(ciphertext)))
E_Kn = Encryption with Key number n.
D_Kn = Decryption with Key number n.
So you can easily "emulate" 3DES with just DES.
In CBC mode you'll need an IV to start with and then XOR the next plaintext block with the previous ciphertext block. If your device doesn't support CBC then this too is easily "emulated".
I want to encrypt small serialized data structures (~256 bytes) so I can pass them around (especially in URLs) safely. My current approach is to use a symmetric block cipher, and then to base 64 encode, then URL encode the cipher text. This yields an encoded cipher text that is (unsurprisingly) quite a bit longer than the original data structure. The length of these encoded ciphers is a bit of a usability problem; ideally I'd like the cipher text to be around the same length as the input text.
Is there a block cipher that can be configured to constrain the values of the output bytes to be in the URL-safe range? I assume there would be a security trade-off involved if there is.
For a given key K, a cipher has to produce a different ciphertext for each plaintext. If your message space is 256 bytes, the cipher has to be able to produce at least 256^256 different messages. This will require at least 256 bytes, and any reduction in the size of the output alphabet requires longer messages.
As you've seen, you can do some encoding afterward to avoid certain output symbols, at the cost of increased length. Furthermore, you would pay the same cost if the encoding were part of the encryption algorithm proper. That's why this isn't a feature of any encryption algorithm.
As others have mentioned, the only real answer is to reduce the size of the data you are encrypting so that you need to encode less data. (Either that or don't put the data in url's in the first place e.g. store the data in a database and put a unique id in the url). So compress > encrypt > encode.
If your data structure is 256 bytes long encrypting it with a block cipher of 8 bytes increases it up to 8 bytes (depending of the concrete input length).
Therefore before applying base64 you have up to 264 bytes which are increased by the base64 encoding up to 352 bytes.
Therefore as you can see the most overhead is created by the base64 encoding. There are some slightly more effective encodings available like base91 - but they are very uncommon.
If size matters I would recommend to compress the data before encrypting it.
URL encoding will not significantly expand a base64 encoded string, since 62 of the 64 characters do not need to be modified. However, you can use modified base64 encoding to do a little better. This encoding uses the '-' and '_' characters in place of the '+' and '/' characters to yield a slight efficiency improvement.
The cipher itself is not causing any significant data expansion. It will pad the data to be a multiple of the block length, but that is insignificant in your case. You might try compressing the input prior to encryption. 256 bytes is not much but you might see some improvement.