I'm a bit confused on encryption file formats.
Let's say I want to encrypt a file with AES-256. I run the file through the encryption algorithm and I now have a stream of encrypted bytes.
I obviously can write that stream of bytes to a file, but any third-party encryption application is not going to understand it since it's not expecting just a raw stream of encrypted bytes.
Into what file formats can I write that so that other encryption tools can understand it?
The ones I know of (I think) are:
PKCS#7
ASN.1
DER
PEM
PKCS#8
but I'm not sure how they all relate to each other.
Apparently the AESCrypt utility also has a format, which appears to be its own proprietary format:
http://www.aescrypt.com/aes_file_format.html
Is there a cheatsheet anywhere on this stuff? I've been googling and found bits and pieces, but never felt like I had the complete picture.
PKCS#8 is not an encrypted-file format, it's a format for private keys.
ASN.1 and DER are rules for translating a structured message into binary. They are not, in and of themselves, a file format, although they're used to define and describe file formats.
PKCS#7 is closely related to PEM, and they're both formats for public-key encrypted files. They are defined in terms of base-64 encapsulated DER encoded ASN.1 messages. They are the basis of the S/MIME format for secure internet mail. (see RFC3851)
In parallel with S/MIME is the OpenPGP file format, also mainly designed for public-key encrypted files. (See RFC4880)
In both S/MIME and OpenPGP formats, there is a block which contains symmetric-key encrypted data. It is possible to create valid S/MIME or OpenPGP files containing only this block. In this way, the S/MIME (a.k.a. PKCS#7) and OpenPGP formats can be used for symmetric-key encryption also.
AES is an encryption algorithm, not a file format.
As you point out, there are lots of knobs and levers on the algorithm - key strength is one. AES-256 just means, the AES algorithm with 256-bit key. But there are lots of other knobs. Mode, for one. AES has a number of modes: CBC, ECB, OFB, CFB, CTR, and others. Another is the IV, which applies to some modes. Padding is another. Usually these knobs are exposed in the AES api for whatever framework you're using.
In most cases AES is combined with other crypto technology - for example password-based key derivation (PBKDF2) is often used to generate keys or IVs. MAC's are often used to verify the integrity of the encrypted data.
Different tools use AES to encrypt, and if they want their data to be readable, they publish the list of knobs they use, and how they are set, as well as how any related crypto technology might be used.
When creating a file format, you'll need to store or publish those kinds of things, if you want your file to be readable by other applications.
You might want to look into Crypt4GH which was standardized at the end of 2019.
Crypt4GH, a new standard file container format from the Global
Alliance for Genomics and Health (GA4GH), allows genomic data to
remain secure throughout their lifetime, from initial sequencing to
sharing with professionals at external organizations.
From what I can see it is similar - in terms of crypto - to NaCl's crypto_box, but with the advantage of formalizing a file format on disk.
JSON Web Encryption RFC 7516 is an IETF standard that can do what you are looking for, it can handle AES in addition to other crypto algorithms.
JSON Web Encryption (JWE) represents encrypted content using JSON-
based data structures [RFC7159]. The JWE cryptographic mechanisms
encrypt and provide integrity protection for an arbitrary sequence of
octets.
There implementing of JWE in multiple languages for example in Java you can use nimbus
Related
I have a keystring which allows customer to have additional features.
Obviously I would like the software to check that this string is valid, and not modified.
Is the following idea feasible:
get the key string as encrypted value, and encode it in Base64
(my encrypted string is around 100 characters, for my purpose)
calculate the checksum (MD5) of course using a private salt.
weave the checksum into the encrypted data
In principle :
xxxxCxxxxxxCxxxxxxxxCxxxxxxxxxxCxxxxxxxxxxxxxCxxx
the places to weave into the encrypted data could be determined by first cher of the encrypted, creating up to 16 different patterns.
On checking the code validity I simply "unweave" the checksum, test if it's correct, and thereby know if the data has been modified.
Is my line of thoughts correct ?
The cryptographic feature you're thinking of is called "authentication," and there are many well-established approaches. You should strongly avoid inventing your own, particularly using a long-outdated hash like MD5. When an encryption system is authenticated, it can detect changes to the ciphertext.
Your best approach is to use an authenticated cipher mode, such as AES-GCM. Used correctly, that combines encryption an authentication in a single operation. While decrypting an authenticated scheme, the decryption will fail if the cipher text has been modified.
If you don't have access to AES-GCM, the next option is AES-CBC+HMAC, which uses the more ubiquitous AES-CBC with a random IV, and appends a type of encrypted hash (called an HMAC) to the end of the message to authenticate it. In order to authenticate, you need to remove the HMAC, use it to validate that the cipher text is unmodified, and then proceed to decrypt normally. This scheme is generally called "encrypt then MAC."
The implementation details will depend on your language and frameworks.
This is a Paw question. Is there a Paw encryption add-on that supports AES working in the commonly used modes and lengths?
I have a request that includes encrypted fields. These fields are AES encrypted. The key is 128 bits long. The encryption mode is CBC. The encryption is performed by the mobile app. The key is inside the mobile app.
At the moment, I capture the requests in Burp, Import them into Paw, and then I have the valid request.
I know the plaintext. I know the key. I know the algorithm. I know the Initialization Vector. There must be a better way.
I could not find anything on: https://paw.cloud/extensions/
There are two new DynamicValues that should allow you to do what you wish to do:
AES Decrypt
AES Encrypt
They are based on an npm port of the CryptoJS library and should expose most of the API for decryption/encryption in AES.
Here's a couple screenshot to show you what it looks like:
Since the DynamicValues are quite recent, there may be a few issues with them. Don't hesitate to report them.
What is the most secure hash algorithm to use in ColdFusion 9 (non-Enterprise)?
According to the CF documentation, these are the options:
MD5: (default) Generates a 32-character, hexadecimal string, using the
MD5 algorithm (The algorithm used in ColdFusion MX and prior
releases).
SHA: Generates a 40-character string using the Secure Hash Standard
SHA-1 algorithm specified by Nation Institute of Standards and
Technology (NIST) FIPS-180-2.
SHA-256: Generates a 44-character string using the SHA-256 algorithm
specified by FIPS-180-2.
SHA-384: Generates a 64-character string using the SHA-384 algorithm
specified by FIPS-180-2.
SHA-512: Generates an 128-character string using the SHA-1 algorithm
specified by FIPS-180-2.
But in this article, it says not to use MD5 or SHA-1
I am also a little skeptical about the cf documentation. It says encoding "SHA-512" uses SHA-1, but the description of "SHA-512" for the Enterprise version is "The 512-bit secure hash algorithm defined by FIPS 180-2 and FIPS 198." And the output of SHA-512 is larger than SHA-384. Sorry, I am having a hard time getting my head around all these different encoding principles.
Hashes are not secure by themselves, anything that can be hashed can be broken. So in the security world you might think, ok I need to run the hash multiple times to obscure it more, but that doesn't secure the information, it just means someone has to repeat that same process and iterate over the hash multiple times. If they know the hash algorithm you used and assume they do, it's not secure. Sha-256 should be good enough for hashing information unless you are trying to secure the information. Hashes should never ever be used by themselves to secure information. Just because it isn't human readable does not make it secure.
If you want to secure something use coldfusion a encrypt functions and make sure you use a decent algorithm, like AES because the default in coldfusion is not secure. Then you need to use some entropic data from the information you're securing to ensure you have a unique encryption key that would be hard for someone to guess or find. Do not hard code a single key in your code, this will make it easy for someone to find and utilize a pattern in all of your encryptions.
Use something like bcrypt or scyrpt for storing passwords. I know they are more work to put into use and require java integration in coldfusion but they are much more secure ways of storing information. Remember that even with bcrypt or scrypt the information can be compromised given enough time and someone willing to put the effort into decrypting it. Be paranoid when securing information.
I want to do searching on encrypted data. Which means that there is the need to have the same ciphertext every time I encrypt the same plaintext. I.e. think of a list of encrypted names and I want to find all "Kevin"'s in it. I would now encrypt "Kevin" and search the database for the encrypted text. All hits will be "Kevin"'s — but still only the one who has the password knows.
Now my question: What about security if I use the same salt and IV (to get the effect described above) all the time? Is the encryption still secure? Or is there any other method to do searching on encrypted data?
If you want to do a deterministic encryption then you should use an encryption mode
that has been designed for deterministic encryption (and not modify an encryption mode designed for something else).
One possibility is the SIV encryption mode described in
RFC 5297.
(Of course, deterministic encryption has its drawbacks, but discussing this is not part of this question.)
I have a csv file that required to be encrypted. I want to ask that is there any difference between encrypt the content and encrypt the file? Or they are the same? Because our client doesn't clearly specify what they want?
Thanks,
Encryption will always produce binary result, which can be "armored" (base64-encoded and wrapped) then in order to get the text. All implementations conformant to OpenPGP standard handle armored data by decoding it to binary automatically, so it doesn't really matter what mode you specify. One special case is when you plan to paste the result to some other text data (such as text document), then armoring is mandatory.
Usually when they say PGP encryption, they are talking about commercial programs that encrypt a file or files into one gpg file with their public, which they can decrypt with their private key.
The other way is encrypt it yourself line-by-line in code (there are libraries that do that using file streams; very easy).
Your client one way or another has to tell you how that expect to decrypt the data later on, which will tell you how you encrypt in the first place.