The goal
We’ve been using EasyByte’s Cryptocx library to encrypt documents using AES256. The library is 32-bit and is preventing us from building a 64-bit version of our application. EasyByte went out of business and there’s no 64-bit version, source code or support for the library.
We’d like to use Chilkat’s 64-bit ActiveX component to be able to decrypt the documents directly. Migrating all our customer’s documents by decrypting and re-encrypting them would be a huge cost.
I’m trying to infer the mode of operation (ECB, CBC…) so that I can send Chilkat the right parameters. The Cryptocx method we’re using only accepts the source and destination files and a string password:
http://www.easybyte.com/support/cryptocx/aesfileencrypt.html
What I tried
I don’t have a strong background in cryptography, but this is what I tried.
A 64-byte input file (4 blocks in AES) filled with As (hex 41). A 32-character string for the password, also filled with As. By calling the previous method to encrypt several times (same file, same password) and using a hex editor I can observe that files have five 16-byte blocks in the encrypted file.
The first four blocks are identical and the fifth differs. Is this the initialization vector being stored in the file? If it is and it’s different in all the files, shouldn’t the first four blocks also be different?
Assuming it was CBC (since the Wikipedia article claims it’s the most used) I tried removing the last 16 bytes from the file (using a hex editor) and using them as the IV to decrypt the file.
But I didn’t get the expected clear text.
Some code
This is the code I tried, in VB.net:
Dim chilkat As ChilkatCrypt2 = New ChilkatCrypt2
chilkat.UnlockComponent(licenseKey)
chilkat.CryptAlgorithm = "aes"
chilkat.KeyLength = 256
chilkat.CipherMode = "cbc"
chilkat.SetEncodedKey("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", "ascii")
chilkat.SetEncodedIV("36cd36dae64316e80d50871e3b26b4bb", "hex")
Dim result = chilkat.CkDecryptFile(pathToEncryptedFile, pathToDecryptedFile)
If result = 0 Then
Console.WriteLine(chilkat.LastErrorText)
Console.ReadLine()
End If
I would appreciate some direction. Can the right parameters even be inferred from the output without knowing the implementation details?
Thanks a lot.
Related
For
`BDK = "0123456789ABCDEFFEDCBA9876543210"` `KSN = "FFFF9876543210E00008"`
The ciphertext generated was below
"C25C1D1197D31CAA87285D59A892047426D9182EC11353C051ADD6D0F072A6CB3436560B3071FC1FD11D9F7E74886742D9BEE0CFD1EA1064C213BB55278B2F12"`
which I found here. I know this cipher-text is based on BDK and KSN but how this 128 length cipher text was generated? What are steps involved in it or algorithm used for this? Could someone explain in simple steps. I found it hard to understand the documents I got while googled.
Regarding DUKPT , there are some explanations given on Wiki. If that doesn't suffice you, here goes some brief explanation.
Quoting http://www.maravis.com/library/derived-unique-key-per-transaction-dukpt/
What is DUKPT?
Derived Unique Key Per Transaction (DUKPT) is a key management scheme. It uses one time encryption keys that are derived from a secret master key that is shared by the entity (or device) that encrypts and the entity (or device) that decrypts the data.
Why DUKPT?
Any encryption algorithm is only as secure as its keys. The strongest algorithm is useless if the keys used to encrypt the data with the algorithm are not secure. This is like locking your door with the biggest and strongest lock, but if you hid the key under the doormat, the lock itself is useless. When we talk about encryption, we also need to keep in mind that the data has to be decrypted at the other end.
Typically, the weakest link in any encryption scheme is the sharing of the keys between the encrypting and decrypting parties. DUKPT is an attempt to ensure that both the parties can encrypt and decrypt data without having to pass the encryption/decryption keys around.
The Cryptographic Best Practices document that VISA has published also recommends the use of DUKPT for PCI DSS compliance.
How DUKPT Works
DUKPT uses one time keys that are generated for every transaction and then discarded. The advantage is that if one of these keys is compromised, only one transaction will be compromised. With DUKPT, the originating (say, a Pin Entry Device or PED) and the receiving (processor, gateway, etc) parties share a key. This key is not actually used for encryption. Instead, another one time key that is derived from this master key is used for encrypting and decrypting the data. It is important to note that the master key should not be recoverable from the derived one time key.
To decrypt data, the receiving end has to know which master key was used to generate the one time key. This means that the receiving end has to store and keep track of a master key for each device. This can be a lot of work for someone that supports a lot of devices. A better way is required to deal with this.
This is how it works in real-life: The receiver has a master key called the Base Derivation Key (BDK). The BDK is supposed to be secret and will never be shared with anyone. This key is used to generate keys called the Initial Pin Encryption Key (IPEK). From this a set of keys called Future Keys is generated and the IPEK discarded. Each of the Future keys is embedded into a PED by the device manufacturer, with whom these are shared. This additional derivation step means that the receiver does not have to keep track of each and every key that goes into the PEDs. They can be re-generated when required.
The receiver shares the Future keys with the PED manufacturer, who embeds one key into each PED. If one of these keys is compromised, the PED can be rekeyed with a new Future key that is derived from the BDK, since the BDK is still safe.
Encryption and Decryption
When data needs to be sent from the PED to the receiver, the Future key within that device is used to generate a one time key and then this key is used with an encryption algorithm to encrypt the data. This data is then sent to the receiver along with the Key Serial Number (KSN) which consists of the Device ID and the device transaction counter.
Based on the KSN, the receiver then generates the IPEK and from that generates the Future Key that was used by the device and then the actual key that was used to encrypt the data. With this key, the receiver will be able to decrypt the data.
Source
First, let me quote the complete sourcecode you linked and of which you provided only 3 lines...
require 'bundler/setup'
require 'test/unit'
require 'dukpt'
class DUKPT::DecrypterTest < Test::Unit::TestCase
def test_decrypt_track_data
bdk = "0123456789ABCDEFFEDCBA9876543210"
ksn = "FFFF9876543210E00008"
ciphertext = "C25C1D1197D31CAA87285D59A892047426D9182EC11353C051ADD6D0F072A6CB3436560B3071FC1FD11D9F7E74886742D9BEE0CFD1EA1064C213BB55278B2F12"
plaintext = "%B5452300551227189^HOGAN/PAUL ^08043210000000725000000?\x00\x00\x00\x00"
decrypter = DUKPT::Decrypter.new(bdk, "cbc")
assert_equal plaintext, decrypter.decrypt(ciphertext, ksn)
end
end
Now, you're asking is how the "ciphertext" was created...
Well, first thing we know is that it is based on "plaintext", which is used in the code to verify if decryption works.
The plaintext is 0-padded - which fits the encryption that is being tested by verifying decryption with this DecrypterTest TestCase.
Let's look at the encoding code then...
I found the related encryption code at https://github.com/Shopify/dukpt/blob/master/lib/dukpt/encryption.rb.
As the DecrypterTEst uses "cbc", it becomes apparent that the encrypting uses:
#cipher_type_des = "des-cbc"
#cipher_type_tdes = "des-ede-cbc"
A bit more down that encryption code, the following solves our quest for an answer:
ciphertext = des_encrypt(...
Which shows we're indeed looking at the result of a DES encryption.
Now, DES has a block size of 64 bits. That's (64/8=) 8 bytes binary, or - as the "ciphertext" is a hex-encoded text representation of the bytes - 16 chars hex.
The "ciphertext" is 128 hex chars long, which means it holds (128 hex chars/16 hex chars=) 8 DES blocks with each 64 bits of encrypted information.
Wrapping all this up in a simple answer:
When looking at "ciphertext", you are looking at (8 blocks of) DES encrypted data, which is being represented using a human-readable, hexadecimal (2 hex chars = 1 byte) notation instead of the original binary bytes that DES encryption would produce.
As for the steps involved in "recreating" the ciphertext, I tend to tell you to simply use the relevant parts of the ruby project where you based your question upon. Simply have to look at the sourcecode. The file at "https://github.com/Shopify/dukpt/blob/master/lib/dukpt/encryption.rb" pretty much explains it all and I'm pretty sure all functionality you need can be found at the project's GitHub repository. Alternatively, you can try to recreate it yourself - using the preferred programming language of your choice. You only need to handle 2 things: DES encryption/decryption and bin-to-hex/hex-to-bin translation.
Since this is one of the first topics that come up regarding this I figured I'd share how I was able to encode the ciphertext. This is the first time I've worked with Ruby and it was specifically to work with DUKPT
First I had to get the ipek and pek (same as in the decrypt) method. Then unpack the plaintext string. Convert the unpacked string to a 72 byte array (again, forgive me if my terminology is incorrect).
I noticed in the dukpt gem author example he used the following plain text string
"%B5452300551227189^HOGAN/PAUL ^08043210000000725000000?\x00\x00\x00\x00"
I feel this string is incorrect as there shouldn't be a space after the name (AFAIK).. so it should be
"%B5452300551227189^HOGAN/PAUL^08043210000000725000000?\x00\x00\x00\x00"
All in all, this is the solution I ended up on that can encrypt a string and then decrypt it using DUKPT
class Encrypt
include DUKPT::Encryption
attr_reader :bdk
def initialize(bdk, mode=nil)
#bdk = bdk
self.cipher_mode = mode.nil? ? 'cbc' : mode
end
def encrypt(plaintext, ksn)
ipek = derive_IPEK(bdk, ksn)
pek = derive_PEK(ipek, ksn)
message = plaintext.unpack("H*").first
message = hex_string_from_unpacked(message, 72)
encrypted_cryptogram = triple_des_encrypt(pek,message).upcase
encrypted_cryptogram
end
def hex_string_from_unpacked val, bytes
val.ljust(bytes * 2, "0")
end
end
boomedukpt FFFF9876543210E00008 "%B5452300551227189^HOGAN/PAUL^08043210000000725000000?"
(my ruby gem, the KSN and the plain text string)
2542353435323330303535313232373138395e484f47414e2f5041554c5e30383034333231303030303030303732353030303030303f000000000000000000000000000000000000
(my ruby gem doing a puts on the unpacked string after calling hex_string_from_unpacked)
C25C1D1197D31CAA87285D59A892047426D9182EC11353C0B82D407291CED53DA14FB107DC0AAB9974DB6E5943735BFFE7D72062708FB389E65A38C444432A6421B7F7EDD559AF11
(my ruby gem doing a puts on the encrypted string)
%B5452300551227189^HOGAN/PAUL^08043210000000725000000?
(my ruby gem doing a puts after calling decrypt on the dukpt gem)
Look at this: https://github.com/sgbj/Dukpt.NET, I was in a similar situation where i wondered how to implement dukpt on the terminal when the terminal has its own function calls which take the INIT and KSN to create the first key, so my only problem was to make sure the INIT key was generated the same way on the terminal as it is in the above mentioned repo's code, which was simple enough using ossl encryption library for 3des with ebc and applying the appropriate masks.
I am Implementing the AES128 bit encryption/Decryption in iOS application for sending/receiving data from .net server, I almost done but during unit testing I got some issue in encryption string, some encrypted string are not similar as on .net server, Can say 98 percent strings are correct at both side but issue comes in 2 percent strings , when I match the both side encrypted string then found at iOS end generated string is little short and .net end it is long string. One more thing i found the iOS string is the substring of .net string. When i tried to decrypt the iOS generated encrypted string, it is not decrypted showing null but when I try to decrypt the .net server generated encrypted string (it was larger than the iOS) I am able to se the decrypted string.
Using the same KEY(16 character long at server and iOS end).
could you please suggest the solution or where I am wrong .
Thanks a lot to all.
Original string: "custId=10&mode=1"
KEY= "PasswordPassword"
at iOS encrypted string:
r51TbJpBLYDkcPC+Ei6Rmg==
at .net encrpted string:
r51TbJpBLYDkcPC+Ei6RmtY2fuzv3RsHzsXt/RpFxAs=
padding for encryption = kCCOptionPKCS7Padding;
I followed this tutorial.
http://automagical.rationalmind.net/2009/02/12/aes-interoperability-between-net-and-iphone/
A similar question found on CryptoSE
My Version TL;DR
Essentially .net and iOS both have different implementations, and since the guide you are following is from 2009 I would expect that it is rather out of date by now given there have been at least 1 major revision bump in each of the platforms since then.
Original Answer Gives the following answer:
I can immediately think of four reasons:
They're both not using AES256. I see in the Obj-C document a direct statement that they are using AES256 (unless you deliberately change it), I don't see any statement in the Visual Basic document that says what key size they're using (unless that's what they mean by "Block Bits").
Different keys. AES256 takes a key of 256 bits; there's no standard method to take a five character string and convert that into a 256 bit value. Now, there are a lot of possible methods; there's no particular assurance that they both use the same one.
Different modes of operation. The AES block cipher takes 128-bit values, and translates that into 128-bit values. However, not all our messages can fit into 128 bits, and in addition, sometimes there are other things we'd like to do other than message encryption. A Mode of Operation is a method that takes a block cipher, and uses it as a tool to perform some more generally useful function (such as encrypting a much longer message). There are a number of standard modes of operations, the Obj-C document states that it is using CBC mode; the Visual Basic document has scary sounding words which might be a garbled explination of CBC mode.
IVs. Some modes of operation (such as CBC mode) have the encryptor select an "Initialization Vector" randomly; that can be translated along with the encrypted message (because the decryptor will need that value). One of the things that this Initialization Vector does if you encrypt the message a second time, the second ciphertext will not resemble the first ciphertext at all; that way, someone listening will not be able to deduce that you've just repeated a message. The Obj-C document specifically says that it will pick a random IV (unless to tell give it one yourself).
As you can see, there are a bunch of reasons why the two ciphertexts may be different. One thing you can try: hand the ciphertext from one to the other, and ask them to decrypt it; if they can, you can be pretty sure that both sides are doing basically the same thing.
As you can see, there are a bunch of reasons why the two ciphertexts may be different. One thing you can try: hand the ciphertext from one to the other, and ask them to decrypt it; if they can, you can be pretty sure that both sides are doing basically the same thing.
Goal (General)
My ultimate (long term) goal is to write an importer for a binary file into another application
Question Background
I am interested in two fields within a binary file format. One is
encrypted, and the other is compressed and possibly also encrypted
(See how I arrived at this conclusion here).
I have a viewer program (I'll call it viewer.exe) which can open these files for viewing. I'm hoping this can offer up some clues.
I will (soon) have a correlated deciphered output to compare and have values to search for.
This is the most relevant stackoverflow Q/A I have found
Question Specific
What is the best strategy given the resources I have to identify the algorithm being used?
Current Ideas
I realize that without the key, identifying the algo from just data is practically impossible
Having a file and a viewer.exe, I must have the key somewhere. Whether it's public, private, symmetric etc...that would be nice to figure out.
I would like to disassemble the viewer.exe using OllyDbg with the findcrypt plugin as a first step. I'm just not proficient enough in this kind of thing to accomplish it yet.
Resources
full example file
extracted binary from the field I am interested in
decrypted data In this zip archive there is a binary list of floats representing x,y,z (model2.vertices) and a binary list of integers (model2.faces). I have also included an "stl" file which you can view with many free programs but because of the weird way the data is stored in STL's, this is not what we expect to come out of the original file.
Progress
1. I disassembled the program with Olly, then did the only thing I know how to do at this poing and "searched for all referenced text" after pausing the porgram right before it imports of of the files. Then I searched for words stings like "crypt, hash, AES, encrypt, SHA, etc etc." I came up with a bunch of things, most notably "Blowfish64" which seems to go nicely with the fact that mydata occasionally is 4 bytes too long (and since it is guranteed to be mod 12 = 0) this to me looks like padding for 64 bit block size (odd amounts of vertices result in non mod 8 amounts of bytes). I also found error messages like...
“Invalid data size, (Size-4) mod 8 must be 0"
After reading Igor's response below, here is the output from signsrch. I've updated this image with green dot's which cause no problems when replaced by int3, red if the program can't start, and orange if it fails when loading a file of interest. No dot means I haven't tested it yet.
Accessory Info
Im using windows 7 64 bit
viewer.exe is win32 x86 application
The data is base64 encoded as well as encrypted
The deciphered data is groups of 12 bytes representing 3 floats (x,y,z coordinates)
I have OllyDb v1.1 with the findcrypt plugin but my useage is limited to following along with this guys youtube videos
Many encryption algorithms use very specific constants to initialize the encryption state. You can check if the binary has them with a program like signsrch. If you get any plausible hits, open the file in IDA and search for the constants (Alt-B (binary search) would help here), then follow cross-references to try and identify the key(s) used.
You can't differentiate good encryption (AES with XTS mode for example) from random data. It's not possible. Try using ent to compare /dev/urandom data and TrueCrypt volumes. There's no way to distinguish them from each other.
Edit: Re-reading your question. The best way to determine which symmetric algorithm, hash and mode is being used (when you have a decryption key) is to try them all. Brute-force the possible combinations and have some test to determine if you do successfully decrypt. This is how TrueCrypt mounts a volume. It does not know the algo beforehand so it tries all the possibilities and tests that the first few bytes decrypt to TRUE.
I found a "lua aes" solution on the web a while ago. And have some concern about its safety.
It states that:
-- Do not use for real encryption, because the password is easily viewable while encrypting.
It says this at its "file encryption test" script.
My questions are:
Why is that, how is it any different from encrypting a string and writing it to a file?
How could it be viewable while encryption? Is it viewable after encryption too?
Basically, Is it safe to use or not?
Is there anyone who can confirm this who has used it? I mailed the original developer but the email address was invalid.
Should I be using it at all?
I assume there are two reasons why that recommendation was made:
Strings are immutable in Lua, so there is no way to overwrite a string with different data
once it's created.
In Lua, objects are garbage collected. The garbage collector runs only at certain points in
the program, and the application has no way of telling when the garbage collector will run after there are no more references to the object. Until then, the password string will remain in memory by point 1.
See Java's case, which is similar to Lua:
Why is char[] preferred over String for passwords?
As you can see there, using char arrays instead of strings is a better way to store passwords, since arrays are mutable and can be reinitialized to zero when done.
The closest Lua equivalent to a char array is a table filled with numbers. Here the password is stored as a table, rather than a string, where each element in the table consists of the integer representation of each character. For example, "pass" becomes {0x70,0x61,0x73,0x73}. After the table containing the password is used to encrypt or decrypt, it is filled with zeros before it's unreachable by the program and eventually gets garbage collected.
According to your comment, I may have misunderstood. Maybe the "file encryption test" stores the password in plain text along with the encrypted file, allowing anyone with access to the file, even attackers, the ability to trivially decrypt it. The points above still apply, though. This is still only a guess, however; I can't know exactly what you mean unless you provide a link to the encryption library you mention.
I've taken a look at the AES library and the concern about the password being "easily viewable" occurs because the user types the password in plain text, through the command line or terminal, in order to start the Lua program, even though the output of the program contains only cipher text. A slightly more secure way of providing the password would be not to show the input (as is done in sudo) or to mask the input with dots or stars (as is done in many Web pages).
Either that or the points given above are perhaps the only logical explanation.
You may also try out alternate methods, like LuaCrypto, which is a binding to OpenSSL and is able to encrypt data using the AES standard.
I am trying to create end to end SMS encryption application, but don't want to use standard encryption algorithm.The idea is to convert the text of message to completely different meaningful text so that over the network it doesn't seem encrypted. I am assuming messages only in English language.
In implementation part, I have first compressed the message using huffman encoding which gives me compressed stream of bits. Now for encryption I don't have any idea. Is it possible to build a dictionary of some random text or what other way can be used for getting such encryption? Decryption at other receiving end is also required.
Are you asking for a a codebook? If you have access to an English language dictionary of 65536 entries for example, you can take every 16-bits of your message as an index into this table to get a word. Good luck converting this into a real cryptosystem.