I'm learning in detail how PGP system works but there are somethings that are not said everywhere I tried to look about it ;
According to this diagram :
https://upload.wikimedia.org/wikipedia/commons/4/4d/PGP_diagram.svg
When we are encrypting, we use the Data and a Random key then we have the protected data ( the one with locket ).
Here's the first problem, how are these data crypted ? Which algorithm has been used ?
My second problem is at the last encryption ;
Locket data + Locket key = Encrypted message
Same thing here, how ? What did they used ?
Also, I read that somewhere, they are hashing the whole data so you can't change it or it will break everything, but when do they do that ?
Thanks in advance !
So your questions are:
How are the data and random key encrypted and which algorithm was used.
What algorithm was used to get an encrypted message from locket data + locket key.
The message digest algorithm now used in PGP (Version 5.0 and later) is
called SHA, which stands for Secure Hash Algorithm, designed by the NSA
for the National Institute of Standards and Technology (NIST). SHA is a
160-bit hash algorithm.
That should answer both your questions. Also checkout this pdf
SHA256 is actaully used in bitcoin, funny how things are related. I remember there used to be free PGP downloads to play around with but sadly PGP is a symantec product now, progres....
Related
I have a tcl/tk based tool, which uses network password for authentication. Issue is that, it is saving password in the logs/history. So objective is to encrypt the password.
I tried to use aes package. But at the very beginning aes::init asks for keydata and initialization vector (16 byte). So how to generate IV and keydata. Is is some Random number? I am a novice in encryption algorithms.
If you have the password in the logs/history, why not fix the bug of logging/storing it in the first place?
Otherwise there are distinct things you might want:
A password hashing scheme like PBKDF2, bcrypt, argon2 etc. to store a password in a safe way and compare some user input to it. This is typically the case when you need to implement some kind of authentication with passwords on the server side.
A password encryption and protection scheme like AES. You need a password to authenticate to some service automatically, and it requires some form of cleartext password.
You have some secret data and need to securly store it to in non cleartext form.
If you have case 1, don't use the aespackage, it is the wrong tool for the job. If you have case 2, the aes package might help you, but you just exchanged the problem of keeping the password secret with the other problem of keeping the key secret (not a huge win). So the only viable case where aes is an option might be 3.
Lets assume you need to store some secret data in a reversible way, e.g. case 3 from above.
AES has a few possible modes of operation, common ones you might see are ECB, CBC, OFB, GCM, CTR. The Tcllib package just supports ECB and CBC, and only CBC (which is the default) is really an option to use.
Visit Wikipedia for an example why you should never use ECB mode.
Now back to your actual question:
Initialization Vector (IV)
This is a random value you pick for each encryption, it is not secret, you can just publish it together with the encrypted data. Picking a random IV helps to make two encrypted blocks differ, even if you use the same key and cleartext.
Secret Key
This is also a random value, but you must keep it secret, as it can be used for encryption and decryption. You often have the same key for multiple encryptions.
Where to get good randomness?
If you are on Linux, BSD or other unixoid systems just read bytes from /dev/urandom or use a wrapper for getrandom(). Do NOT use Tcls expr {rand()} or similar pseudorandom number generators (PRNG). On Windows TWAPI and the CryptGenRandom function would be the best idea, but sadly there is no Tcl high level wrapper included.
Is that enough?
Depends. If you just want to hide a bit of plaintext from cursory looks, maybe. If you have attackers manipulating your data or actively trying to hack your system, less so. Plain AES-CBC has a lot of things you can do wrong, and even experts did wrong (read about SSL/TLS 1.0 problems with AES-CBC).
Final words: If you are a novice in encryption algorithms, be sure you understand what you want and need to protect, there are a lot of pitfalls.
If I read the Tcler's Wiki page on aes, I see that I encrypt by doing this:
package require aes
set plaintext "Some super-secret bytes!"
set key "abcd1234dcba4321"; # 16 bytes
set encrypted [aes::aes -dir encrypt -key $key $plaintext]
and I decrypt by doing:
# Assuming the code above was run...
set decrypted [aes::aes -dir decrypt -key $key $encrypted]
Note that the decrypted text has NUL (zero) bytes added on the end (8 of them in this example) because the encryption algorithm always works on blocks of 16 bytes, and if you're working with non-ASCII text then encoding convertto and encoding convertfrom might be necessary.
You don't need to use aes::init directly unless you are doing large-scale streaming encryption. Your use case doesn't sound like it needs that sort of thing. (The key data is your “secret”, and the initialisation vector is something standardised that usually you don't need to set.)
For
`BDK = "0123456789ABCDEFFEDCBA9876543210"` `KSN = "FFFF9876543210E00008"`
The ciphertext generated was below
"C25C1D1197D31CAA87285D59A892047426D9182EC11353C051ADD6D0F072A6CB3436560B3071FC1FD11D9F7E74886742D9BEE0CFD1EA1064C213BB55278B2F12"`
which I found here. I know this cipher-text is based on BDK and KSN but how this 128 length cipher text was generated? What are steps involved in it or algorithm used for this? Could someone explain in simple steps. I found it hard to understand the documents I got while googled.
Regarding DUKPT , there are some explanations given on Wiki. If that doesn't suffice you, here goes some brief explanation.
Quoting http://www.maravis.com/library/derived-unique-key-per-transaction-dukpt/
What is DUKPT?
Derived Unique Key Per Transaction (DUKPT) is a key management scheme. It uses one time encryption keys that are derived from a secret master key that is shared by the entity (or device) that encrypts and the entity (or device) that decrypts the data.
Why DUKPT?
Any encryption algorithm is only as secure as its keys. The strongest algorithm is useless if the keys used to encrypt the data with the algorithm are not secure. This is like locking your door with the biggest and strongest lock, but if you hid the key under the doormat, the lock itself is useless. When we talk about encryption, we also need to keep in mind that the data has to be decrypted at the other end.
Typically, the weakest link in any encryption scheme is the sharing of the keys between the encrypting and decrypting parties. DUKPT is an attempt to ensure that both the parties can encrypt and decrypt data without having to pass the encryption/decryption keys around.
The Cryptographic Best Practices document that VISA has published also recommends the use of DUKPT for PCI DSS compliance.
How DUKPT Works
DUKPT uses one time keys that are generated for every transaction and then discarded. The advantage is that if one of these keys is compromised, only one transaction will be compromised. With DUKPT, the originating (say, a Pin Entry Device or PED) and the receiving (processor, gateway, etc) parties share a key. This key is not actually used for encryption. Instead, another one time key that is derived from this master key is used for encrypting and decrypting the data. It is important to note that the master key should not be recoverable from the derived one time key.
To decrypt data, the receiving end has to know which master key was used to generate the one time key. This means that the receiving end has to store and keep track of a master key for each device. This can be a lot of work for someone that supports a lot of devices. A better way is required to deal with this.
This is how it works in real-life: The receiver has a master key called the Base Derivation Key (BDK). The BDK is supposed to be secret and will never be shared with anyone. This key is used to generate keys called the Initial Pin Encryption Key (IPEK). From this a set of keys called Future Keys is generated and the IPEK discarded. Each of the Future keys is embedded into a PED by the device manufacturer, with whom these are shared. This additional derivation step means that the receiver does not have to keep track of each and every key that goes into the PEDs. They can be re-generated when required.
The receiver shares the Future keys with the PED manufacturer, who embeds one key into each PED. If one of these keys is compromised, the PED can be rekeyed with a new Future key that is derived from the BDK, since the BDK is still safe.
Encryption and Decryption
When data needs to be sent from the PED to the receiver, the Future key within that device is used to generate a one time key and then this key is used with an encryption algorithm to encrypt the data. This data is then sent to the receiver along with the Key Serial Number (KSN) which consists of the Device ID and the device transaction counter.
Based on the KSN, the receiver then generates the IPEK and from that generates the Future Key that was used by the device and then the actual key that was used to encrypt the data. With this key, the receiver will be able to decrypt the data.
Source
First, let me quote the complete sourcecode you linked and of which you provided only 3 lines...
require 'bundler/setup'
require 'test/unit'
require 'dukpt'
class DUKPT::DecrypterTest < Test::Unit::TestCase
def test_decrypt_track_data
bdk = "0123456789ABCDEFFEDCBA9876543210"
ksn = "FFFF9876543210E00008"
ciphertext = "C25C1D1197D31CAA87285D59A892047426D9182EC11353C051ADD6D0F072A6CB3436560B3071FC1FD11D9F7E74886742D9BEE0CFD1EA1064C213BB55278B2F12"
plaintext = "%B5452300551227189^HOGAN/PAUL ^08043210000000725000000?\x00\x00\x00\x00"
decrypter = DUKPT::Decrypter.new(bdk, "cbc")
assert_equal plaintext, decrypter.decrypt(ciphertext, ksn)
end
end
Now, you're asking is how the "ciphertext" was created...
Well, first thing we know is that it is based on "plaintext", which is used in the code to verify if decryption works.
The plaintext is 0-padded - which fits the encryption that is being tested by verifying decryption with this DecrypterTest TestCase.
Let's look at the encoding code then...
I found the related encryption code at https://github.com/Shopify/dukpt/blob/master/lib/dukpt/encryption.rb.
As the DecrypterTEst uses "cbc", it becomes apparent that the encrypting uses:
#cipher_type_des = "des-cbc"
#cipher_type_tdes = "des-ede-cbc"
A bit more down that encryption code, the following solves our quest for an answer:
ciphertext = des_encrypt(...
Which shows we're indeed looking at the result of a DES encryption.
Now, DES has a block size of 64 bits. That's (64/8=) 8 bytes binary, or - as the "ciphertext" is a hex-encoded text representation of the bytes - 16 chars hex.
The "ciphertext" is 128 hex chars long, which means it holds (128 hex chars/16 hex chars=) 8 DES blocks with each 64 bits of encrypted information.
Wrapping all this up in a simple answer:
When looking at "ciphertext", you are looking at (8 blocks of) DES encrypted data, which is being represented using a human-readable, hexadecimal (2 hex chars = 1 byte) notation instead of the original binary bytes that DES encryption would produce.
As for the steps involved in "recreating" the ciphertext, I tend to tell you to simply use the relevant parts of the ruby project where you based your question upon. Simply have to look at the sourcecode. The file at "https://github.com/Shopify/dukpt/blob/master/lib/dukpt/encryption.rb" pretty much explains it all and I'm pretty sure all functionality you need can be found at the project's GitHub repository. Alternatively, you can try to recreate it yourself - using the preferred programming language of your choice. You only need to handle 2 things: DES encryption/decryption and bin-to-hex/hex-to-bin translation.
Since this is one of the first topics that come up regarding this I figured I'd share how I was able to encode the ciphertext. This is the first time I've worked with Ruby and it was specifically to work with DUKPT
First I had to get the ipek and pek (same as in the decrypt) method. Then unpack the plaintext string. Convert the unpacked string to a 72 byte array (again, forgive me if my terminology is incorrect).
I noticed in the dukpt gem author example he used the following plain text string
"%B5452300551227189^HOGAN/PAUL ^08043210000000725000000?\x00\x00\x00\x00"
I feel this string is incorrect as there shouldn't be a space after the name (AFAIK).. so it should be
"%B5452300551227189^HOGAN/PAUL^08043210000000725000000?\x00\x00\x00\x00"
All in all, this is the solution I ended up on that can encrypt a string and then decrypt it using DUKPT
class Encrypt
include DUKPT::Encryption
attr_reader :bdk
def initialize(bdk, mode=nil)
#bdk = bdk
self.cipher_mode = mode.nil? ? 'cbc' : mode
end
def encrypt(plaintext, ksn)
ipek = derive_IPEK(bdk, ksn)
pek = derive_PEK(ipek, ksn)
message = plaintext.unpack("H*").first
message = hex_string_from_unpacked(message, 72)
encrypted_cryptogram = triple_des_encrypt(pek,message).upcase
encrypted_cryptogram
end
def hex_string_from_unpacked val, bytes
val.ljust(bytes * 2, "0")
end
end
boomedukpt FFFF9876543210E00008 "%B5452300551227189^HOGAN/PAUL^08043210000000725000000?"
(my ruby gem, the KSN and the plain text string)
2542353435323330303535313232373138395e484f47414e2f5041554c5e30383034333231303030303030303732353030303030303f000000000000000000000000000000000000
(my ruby gem doing a puts on the unpacked string after calling hex_string_from_unpacked)
C25C1D1197D31CAA87285D59A892047426D9182EC11353C0B82D407291CED53DA14FB107DC0AAB9974DB6E5943735BFFE7D72062708FB389E65A38C444432A6421B7F7EDD559AF11
(my ruby gem doing a puts on the encrypted string)
%B5452300551227189^HOGAN/PAUL^08043210000000725000000?
(my ruby gem doing a puts after calling decrypt on the dukpt gem)
Look at this: https://github.com/sgbj/Dukpt.NET, I was in a similar situation where i wondered how to implement dukpt on the terminal when the terminal has its own function calls which take the INIT and KSN to create the first key, so my only problem was to make sure the INIT key was generated the same way on the terminal as it is in the above mentioned repo's code, which was simple enough using ossl encryption library for 3des with ebc and applying the appropriate masks.
When talking about asymmetric encryption it is often said that due to some reasons you should not encrypt your entire message asymmetrically, e.g. performance considerations.
The usually suggested workflow is:
Create a random key for symmetric encryption
Encrypt the message using this random key
Encrypt the random key using asymmetric encryption
Send the encrypted message and the encrypted key to the recipient
So far, so good.
Just two questions:
How do I send the encrypted message and the encrypted key in combination? Is there a standard for that? Or am I completely free on how to do this? (I'd prefer a standard if there is one).
Is there any best practice on how to create the random key? To be more specific: Is there a specific function in OpenSSL that should be used for that?
This was answered in comments. I would better recommend to check OpenPGP - it is somehow easier to understand, CMS has a huge ASN.1 structures overload.
Random key just should be completely random byte string, generated from good random source.
I'm some newbie of data encryption. I goggling data encryption and mostly found md5 and SHA algorithms.Which technology do you consider to be the best for data storage/security and why?
MD5 and SHA are hash functions, they create fingerprint - fixed-length representation - from the bunch of data. For example, they are extensively used as a way to check consistency of your iso image downloads for many open-source products, but this means you can use them to create fingerprints from any selection of bytes. So they do not encrypt.
If you want to encrypt, you should check for encryption algorithms. The most feasible now I believe is AES (Advanced encryption Standard) if you look for symmetric encryption algorithms (eg. where key for encryption and decryption is the same or easily computed one from another) or RSA if you look for asymmetric (where you have 2 keys - public and private, and compute private key from public is hard task).
If you are about to create digital signatures, you may want to check things like DSA (digital signature algorithm) and ECDSA (DSA over elliptic curves.). Note that asymmetric algorithms work over numbers with extended precision - like 512, 1024, 2048 bits and so on. You need special libraries that can handle such numbers. If you use C++ I can recommend you trying Crypto++. Find something similar for other languages.
I hope this was useful for you.
If your data is password and you want to store it some where then Use MD5 or SHA Hash.
There are two advantages of these.
A hash can not be decrypted to recover old value so your password will never ever will be cracked even if you provide MD5/SHA hash of password to some one :)
A hash of particular string will be always same so you can compare passwords based on Hash.
AES :
Symmetric algorithm so faster
Use with PKCS5Padding and CBC mode
Always store IV
Symmetric so you need same key while you are decrypting encrypted text so you can not at all share keys.
RSA
PKI Infrastructure to exchange Keys
Slow
There are other algorithms also Like DES(Not So Secure), 3DES(Often called Tripple DES- Not enough secure compared to DES)
It is not advisable to store email addresses in databases in plain text, so I would like to find out the best algorithm to do this. Options are:
(From the documentation)
CFMX_COMPAT: the algorithm used in ColdFusion MX and prior releases. This algorithm is the least secure option (default).
AES: the Advanced Encryption Standard specified by the National Institute of Standards and Technology (NIST) FIPS-197.
BLOWFISH: the Blowfish algorithm defined by Bruce Schneier.
DES: the Data Encryption Standard algorithm defined by NIST FIPS-46-3.
DESEDE: the "Triple DES" algorithm defined by NIST FIPS-46-3.
Another questions is where should the key be stored? In the database or in the source code? Will it be encrypted or not? If it will be encrypted, then the question raises of how the key that will encrypt the key be stored.
Should it be stored in the source code, will sourceless distribution be good?
I would use AES. it's the fastest of those listed and the strongest.
As for where to store the key, that is the $64,000 question. You should not put it in the DB (At least not in the same DB as the data it is being used to encipher) or in your source code.
Key management is a beast of a topic. NIST has hundreds of pages of documentation on ways to do it.
http://csrc.nist.gov/groups/ST/toolkit/key_management.html
Key Management involves proper generaton, exchange, storage, rotation, and destruction of keys. You should not use the same key forever (a very common mistake) nor store it improperly.
You should take a look at the NIST guidelines and determine a strategy that works for you and adequately protects your data based on its sensitivity.
Use AES or DESEDE - they're strong and in my experience have a lot of wide compatibility should you need to port this information for some reason.
As for the key, this isn't REAL critical data. Typically you would create a compsite key out of a unique piece of information for that data (like the userId) and a private key (salt) such as a constant in the code base:
Somewhere in your global settings / constants :
<cfset myCodeBaseKey = "NateIsAwesome">
Then when you're ready to encrypt:
<cfset myKey = hash(myCodeBaseKey & user.userId, "SHA")>
P.s. it works better if you use that exact salt phrase I hear. :P~