I'm in the process of designing an encryption algorithm. The algorithm is symmetric (single key).
How do you measure an algorithms strength in terms of bits? Is the key length the strength of the algorithm?
EDIT:
Lesson 1: Don't design an encryption algorithm, AES and others are
designed and standardized by academics for a reason
Lesson 2: An encryption algorithms strength is not measured in bits, key sizes are. An algorithm's strength is determined by its design. In general, an algorithm using a larger key size is harder to brute-force, and thus stronger.
First of all, is this for anything serious? If it is, stop right now. Don't do it. Designing algorithms is one of the hardest things in the world. Unless you have years and years of experience breaking ciphers, you will not design anything remotely secure.
AES and RSA serve two very different purposes. The difference is more than just signing. RSA is a public key algorithm. We use it for encryption, key exchange, digital signatures. AES is a symmetric block cipher. We use it for bulk encryption. RSA is very slow. AES is very fast. Most modern cryptosystems use a hybrid approach of using RSA for key exchange, and then AES for the bulk encryption.
Typically when we say "128-bit strength", we mean the size of the key. This is incredibly deceptive though, in that there is much more to the strength of an algorithm than the size of it's key. In other words, just because you have a million bit key, it means nothing.
The strength of an algorithm, is defined both in terms of it's key size, as well as it's resistance to cryptanalytic attacks. We say an algorithm is broken if there exists an attack better than brute force.
So, with AES and a 128-bit key, AES is considered "secure" if there is no attack that less than 2^128 work. If there is, we consider it "broken" (in an academic sense). Some of these attacks (for your searching) include differential cryptanalysis, linear cryptanalysis, and related key attacks.
How we brute force an algorithm also depends on it's type. A symmetric block cipher like AES is brute forced by trying every possible key. For RSA though, the size of the key is the size of the modulus. We don't break that by trying every possible key, but rather factoring. So the strength of RSA then is dependent on the current state of number theory. Thus, the size of the key doesn't always tell you it's actual strength. RSA-128 is horribly insecure. Typically RSA key sizes are 1024-bits+.
DES with a 56-bit key is stronger than pretty much EVERY amateur cipher ever designed.
If you are interested in designing algorithms, you should start by breaking other peoples. Bruce Schenier has a self-study course in cryptanalysis that can get you started: http://www.schneier.com/paper-self-study.html
FEAL is one of the most broken ciphers of all time. It makes for a great starting place of learning block cipher cryptanalysis. The source code is available, and there are countless published papers on it, so you can always "look up the answer" if you get stuck.
You can compare key lengths for the same algorithm. Between algorithms it does not make too much sense.
If the algorithm is any good (and it would be very hard to prove that for something homegrown), then it gets more secure with a longer key size. Adding one bit should (again, if the algorithm is good) double the effort it takes to brute-force it (because there are now twice as many possible keys).
The more important point, though, is that this only works for "good" algorithms. If your algorithm is broken (i.e. it can be decrypted without trying all the keys because of some design flaws in it), then making the key longer probably does not help much.
If you tell me you have invented an algorithm with a 1024-bit key, I have no way to judge if that is better or worse than a published 256-bit algorithm (I'd err on the safe side and assume worse).
If you have two algorithms in your competition, telling the judge the key size is not helping them to decide which one is better.
Oh man, this is a really difficult problem. One is for sure - key length shows nothing about encryption algorithm strength.
I can only think of two measures of encryption algorithm strength:
Show your algorithm to professional cryptanalyst. Algorithm strength will be proportional to the time cryptanalyst has taken to break your encryption.
Strong encryption algorithms makes encrypted data look pretty much random. So - measure randomness of your encrypted data. Algorithm strength should be proportional to encrypted data randomness degree. Warning - this criteria is just for playing arround, doesn't shows real encryption scheme strength !
So real measure is first, but with second you can play around for fun.
Assuming the algorithm is sound and that it uses the entire key range...
Raise the number of unique byte values for each key byte to the power of the number of bytes.
So if you are using only ASCII characters A-Z,a-z,0-9, that's 62 unique values - a 10 byte key using these values is 62^10. If you are using all 256 values, 0x00 - 0xFF, a 10 byte key is 256^10 (or 10 * 8 bits per byte = 2 ^ 80).
"Bits of security" is defined by NIST (National Institute of Standards and Technology), in:
NIST SP 800-57 Part 1, section 5.6.1 "Comparable Algorithm Strengths".
Various revisions of SP 800-57 Part 1 from NIST:
http://csrc.nist.gov/publications/PubsSPs.html#800-57-part1
Current version:
http://csrc.nist.gov/publications/nistpubs/800-57/sp800-57_part1_rev3_general.pdf
The "strength" is defined as "the amount of work needed to “break the algorithms”", and 5.6.1 goes on to describe that criterion at some length.
Table 2, in the same section, lays out the "bits of security" achieved by different key sizes of various algorithms, including AES, RSA, and ECC.
Rigorously determining the relative strength of a novel algorithm will require serious work.
My quick and dirty definition is "the number of bits that AES would require to have the same average cracking time". You can use any measure you like for time, like operations, wall time, whatever. If yours takes as long to crack as a theoretical 40-bit AES message would (2^88 less time than 128-bit AES), then it's 40 bits strong, regardless of whether you used 64,000 bit keys.
That's being honest, and honestly is hard to find in the crypto world, of course. For hilarity, compare it to plain RSA keys instead.
Obviously it's in no way hard and fast, and it goes down every time someone finds a better crack, but that's the nature of an arbitrary "strength-in-terms-of-bits" measure. Strength-in-terms-of-operations is a much more concrete measure.
Related
I have to implement a simple hashing algorithm.
Input data:
Value (16-bit integer).
Key (any length).
Output data:
6-bit hash (number 0-63).
Requirements:
It should be practically impossible to predict hash value if you only have the input value but not the key. More specific: if I known hash(x) for x < M, it should be hard to predict hash(M) without knowing the key.
Possible solutions:
Keep full mapping as a key. So the key has length 2^16*6 bits. It's too long for my case.
Linear code. Key is a generator matrix. It's length is 16*6. But it's easy to find generator matrix using several known hash values.
Are there any other possibilities?
A HMAC seems to be what you want. So a possibility for you could be to use a SHA-based HMAC and just use a substring of the resulting hash. This should be relatively safe, since the bits of a cryptographic hash should be as independent and unpredictable as possible.
Depending on your environment, this could however take too much processing time, so you might have to chose a simpler hashing scheme to construct your HMAC.
Original Answer the discussion in the comments is based on:
Since you can forget cryptographic properties anyway (it is trivial to find collisions via bruteforce attacks on a 5-bit hash) you might as well use something like CRC or Hamming Codes and get error-detection for free
Mensi' suggestion to use truncated HMAC is a good one, but if you do happen to be on a highly constrained system and want something faster or simpler, you could take any block cipher, encrypt your 16-bit value (padded to a full block) with it and truncate the result to 6 bits.
Unlike HMAC, which computes a pseudorandom function, a block cipher is a pseudorandom permutation — every input maps to a different output. However, when you throw away all but six bits of the block cipher's output, what remains will look very much like a pseudorandom function. There will be a very tiny bias against repeated outputs, but (assuming that the block cipher's block size is much larger than 6 bits, which it should be) it'll be so small as to be all but undetectable.
A good block cipher choice for very low-end systems might be TEA or its successors XTEA and XXTEA. While there are some known attacks on these ciphers, they all require much more extensive access to the cipher than should be possible in your application.
I understand how standard random number generators work. But when working with crytpography, the random numbers really have to be random.
I know there are instruments that read cosmic white noise to help generate secure hashes, but your standard PC doesn't have this.
How does a cryptographically secure random number generator get its values with no repeatable patterns?
A cryptographically secure number random generator, as you might use for generating encryption keys, works by gathering entropy - that is, unpredictable input - from a source which other people can't observe.
For instance, /dev/random(4) on Linux collects information from the variation in timing of hardware interrupts from sources such as hard disks returning data, keypresses and incoming network packets. This approach is secure provided that the kernel does not overestimate how much entropy it has collected. A few years back the estimations of entropy from the various different sources were all reduced, making them far more conservative. Here's an explanation of how Linux estimates entropy.
None of the above is particularly high-throughput. /dev/random(4) probably is secure, but it maintains that security by refusing to give out data once it can't be sure that that data is securely random. If you want to, for example, generate a lot of cryptographic keys and nonces then you'll probably want to resort to hardware random number generators.
Often hardware RNGs are designed about sampling from the difference between a pair of oscillators that are running at close to the same speed, but whose rates are varied slightly according to thermal noise. If I remember rightly, the random number generator that's used for the UK's premium bond lottery, ERNIE, works this way.
Alternate schemes include sampling the noise on a CCD (see lavaRND), radioactive decay (see hotbits) or atmospheric noise (see random.org, or just plug an AM radio tuned somewhere other than a station into your sound card). Or you can directly ask the computer's user to bang on their keyboard like a deranged chimpanzee for a minute, whatever floats your boat.
As andras pointed out, I only thought to talk about some of the most common entropy gathering schemes. Thomas Pornin's answer and Johannes Rössel's answer both do good jobs of explaining how one can go about mangling gathered entropy in order to hand bits of it out again.
For cryptographic purposes, what is needed is that the stream shall be "computationally indistinguishable from uniformly random bits". "Computationally" means that it needs not be truly random, only that it appears so to anybody without access to God's own computer.
In practice, this means that the system must first gather a sequence of n truly random bits. n shall be large enough to thwart exhaustive search, i.e. it shall be infeasible to try all 2^n combinations of n bits. This is achieved, with regards to today's technology, as long as n is greater than 90-or-so, but cryptographers just love powers of two, so it is customary to use n = 128.
These n random bits are obtained by gathering "physical events" which should be unpredictable, as far as physics are concerned. Usually, timing is used: the CPU has a cycle counter which is updated several billions times per second, and some events occur with an inevitable amount of jitter (incoming network packets, mouse movements, key strokes...). The system encodes these events and then "compresses" them by applying a cryptographically secure hash function such as SHA-256 (output is then truncated to yield our n bits). What matters here is that the encoding of the physical events has enough entropy: roughly speaking, that the said events could have collectively assumed at least 2^n combinations. The hash function, by its definition, should make a good job at concentrating that entropy into a n-bit string.
Once we have n bits, we use a PRNG (Pseudo-Random Number Generator) to crank out as many bits as necessary. A PRNG is said to be cryptographically secure if, assuming that it operates over a wide enough unknown n-bit key, its output is computationally indistinguishable from uniformly random bits. In the 90's, a popular choice was RC4, which is very simple to implement, and quite fast. However, it turned out to have measurable biases, i.e. it was not as indistinguishable as was initially wished for. The eSTREAM Project consisted in gathering newer designs for PRNG (actually stream ciphers, because most stream ciphers consist in a PRNG, which output is XORed with the data to encrypt), documenting them, and promoting analysis by cryptographers. The eSTREAM Portfolio contains seven PRNG designs which were deemed secure enough (i.e. they resisted analysis and cryptographers tend to have a good understanding of why they resisted). Among them, four are "optimized for software". The good news is that while these new PRNG seem to be much more secure than RC4, they are also noticeably faster (we are talking about hundreds of megabytes per second, here). Three of them are "free for any use" and source code is provided.
From a design point of view, PRNG reuse much of the elements of block ciphers. The same concepts of avalanche and diffusion of bits into a wide internal state are used. Alternatively, a decent PRNG can be built from a block cipher: simply use the n-bit sequence as key into a block cipher, and encrypt successive values of a counter (expressed as a m-bit sequence, if the block cipher uses m-bit blocks). This produces a pseudo-random stream of bits which is computationally indistinguishable from random, as long as the block cipher is secure, and the produced stream is no longer than m*2^(m/2) bits (for m = 128, this means about 300 billions of gigabytes, so that's big enough for most purposes). That kind of usage is known as counter mode (CTR).
Usually, a block cipher in CTR mode is not as fast as a dedicated stream cipher (the point of the stream cipher is that, by forfeiting the flexibility of a block cipher, better performance is expected). However, if you happen to have one of the most recent CPU from Intel with the AES-NI instructions (which are basically an AES implementation in hardware, integrated in the CPU), then AES with CTR mode will yield unbeatable speed (several gigabytes per second).
First of all, the point of a cryptographically secure PRNG is not to generate entirely unpredictable sequences. As you noted, the absence of something that generates large volumes of (more or less) true randomness1 makes that impossible.
So you resort to something which is only hard to predict. “Hard” meaning here that it takes unfeasibly long by which time whatever it was necessary for would be obsolete anyway. There are a number of mathematical algorithms that play a part in this—you can get a glimpse if you take some well-known CSPRNGs and look at how they work.
The most common variants to build such a PRNG are:
Using a stream cipher, which already outputs a (supposedly secure) pseudo-random bit stream.
Using a block cipher in counter mode
Hash functions on a counter are also sometimes used. Wikipedia has more on this.
General requirements are just that it's unfeasible to determine the original initialization vector from a generator's bit stream and that the next bit cannot be easily predicted.
As for initialization, most CSPRNGs use various sources available on the system, ranging from truly random things like line noise, interrupts or other events in the system to other things like certain memory locations, &c. The initialization vector is preferrably really random and not dependent on a mathematical algorithm. This initialization was broken for some time in Debian's implementation of OpenSSL which led to severe security problems.
1 Which has its problems too and one has to be careful in eliminating bias as things such as thermal noise has different characteristics depending on the temperature—you almost always have bias and need to eliminate it. And that's not a trivial task in itself.
In order for a random number generator to be considered cryptographically secure, in needs to be secure against attack by an adversary who knows the algorithm and a (large) number of previously generated bits. What this means is that someone with that information can't reconstruct any of the hidden internal state of the generator and give predictions of what the next bits produced will be with better than 50% accuracy.
Normal pseudo-random number generators are generally not cryptographically secure, as reconstructing the internal state from previously output bits is generaly trivial (often, the entire internal state is just the last N bits produced directly). Any random number generator without good statistical properties is also not cryptographically secure, as its output is at least party predictable even without knowing the internal state.
So, as to how they work, any good crypto system can be used as a cryptographically secure random number generator -- use the crypto system to encrypt the output of a 'normal' random number generator. Since an adversary can't reconstruct the plaintext output of the normal random number generator, he can't attack it directly. This is a somewhat circular definition an begs the question of how you key the crypto system to keep it secure, which is a whole other problem.
Each generator will use its own seeding strategy, but here's a bit from the Windows API documentation on CryptGenRandom
With Microsoft CSPs, CryptGenRandom uses the same random number
generator used by other security components. This allows numerous
processes to contribute to a system-wide seed. CryptoAPI stores an
intermediate random seed with every user. To form the seed for the
random number generator, a calling application supplies bits it might
have—for instance, mouse or keyboard timing input—that are then
combined with both the stored seed and various system data and user
data such as the process ID and thread ID, the system clock, the
system time, the system counter, memory status, free disk clusters,
the hashed user environment block. This result is used to seed the
pseudorandom number generator (PRNG).
In Windows Vista with Service Pack 1 (SP1) and later, an
implementation of the AES counter-mode based PRNG specified in NIST
Special Publication 800-90 is used. In Windows Vista, Windows Storage
Server 2003, and Windows XP, the PRNG specified in Federal Information
Processing Standard (FIPS) 186-2 is used. If an application has access
to a good random source, it can fill the pbBuffer buffer with some
random data before calling CryptGenRandom. The CSP then uses this data
to further randomize its internal seed. It is acceptable to omit the
step of initializing the pbBuffer buffer before calling
CryptGenRandom.
How is the encryption algorithm's security dependent on factoring large numbers?
For example, I've read on some math-programming forums that by using the Quadratic Sieve or the General Number Field Sieve, one can factor a 256 bit number with relative ease on commercially available hardware.
How does this translate to being able to break the security of algorithms such as RSA, AES, etc? Is being able to factor numbers the length of the key enough?
Is there anyone knowledgeable in cryptography and encryption algorithms who could shed a little light on it?
RSA, the cryptoalgorithm, relies on number theory, specifically the multiplication of two large primes and the fact this is difficult to factor, to differentiate between public and private keys.
Here's a question on Yahoo answers where someone has given some detail: http://answers.yahoo.com/question/index?qid=20070125183948AALJ40l
It relies on a few facts:
n=p.q is easy to calculate but hard to reverse
Fermat's little theorem: http://en.wikipedia.org/wiki/Fermat%27s_little_theorem
Various results of number theory.
It is not factoring large numbers that is difficult, it is factoring two large numbers whose only factors are themselves large primes, because finding those primes is difficult.
A quick search through my bookmarks gives me this: the mathematical guts of rsa encryption if you're interested in how it works. Also, some explanation here too - just re-read my num-theory notes to be clear.
n = p*q gives you a large number given p,q prime.
phi(n) = (p-1)(q-1). This is an extension of Fermat's little theorem More on why we need this and why it works on my blog here: http://vennard.org.uk/weblog/2010/02/a-full-explanation-of-the-rsa-algorithm/
Which means, if we choose a number E coprime (no common prime factors) to (p-1)(q-1) we can find Es inverse mod phi(n).
Which we do, we find DE = 1(p-1)(q-1) or rather we solve using euclid's greatest common divisor algorithm, extended version.
Now, given all of the above, if we take T^E (pq) we get C. However, if we take (T^E)^D (pq) we get T back again.
AES isn't the same - it isn't public key cryptography. AES takes one key and uses that in both directions, encryption and decryption. The process is basically very difficult to undo, rather like a hash but designed to be reversible. It however, does not rely on factoring large numbers into primes for its security; it relies entirely on the strength of the key and the inability to deduce either the key from the algorithm or the key given known plaintext and the algorithm.
Wikipedia has a good article on AES for high level with a good link that shows you how it works - see here and here. I particularly like the latter link.
How is the an encryption algorithm's security dependent on factoring large numbers?
The missing phrase is "public-key", as in "How is the public key encryption algorithm's security..."
In modern cryptography there two major categories of ciphers, symmetric (secret key) and public-key (which uses a public/private key pair).
Within each category, you will find the key sizes relatively close. For public-key systems like RSA and DH/DSA, both used in OpenPGP e-mail encryption, common key sizes are 1024-bit and larger these days (early 2010). This has to do with the mathematical requirements of suitable keys used to encryption and decrypt messages. For RSA, in short, it is many time easier to generate a factor of two random large prime numbers and do multiplication with them, compared to factoring of very large number that has no small factors. As you've discovered the factoring of very large numbers is the "problem" or approach needed to break RSA via brute force.
Diffie-Hellman / Digital Signature Algorithm (DH/DSA) are based on a different mathematical problem, calculating discrete logarithms.
Due to properties of the public and private key pairs, the search space is limited to factors of large primes numbers, which becomes incredibly sparse, so it makes sense to try to be far more intelligent then simply trying to factor every very large number.
Whereas with symmetric ciphers like AES, RC6, RC4, Twofish, DES and Triple-DES, these algorithms use a random key of a given bit length. Any non-trivial (i.e. 0x000...000 may be a poor key choice) random key is suitable. So these systems, if there is no attack against the algorithm itself, you can simply search brute force through the key space (i.e. try all 2^256 possible keys) to decrypt a message without the secret key. Since any key is suitable, the density of keys is 2^256.
I'm ignoring Quantum Computing (theoretic and practical), mainly because a) I can't give a solid answer, and b) it represents a large paradigm shift that turns much applied mathematics and computer science of computational complexity potentially on its head, that basic understanding is still a moving target. Oh, and most of my enemies don't have a quantum computer yet. :)
I hope that explains the general difference between the two types of crypto systems, such as RSA and AES.
Sidebar: Cryptography is a rich and complex topic, where the basics may be simple enough to understand, and even write a naive ("textbook") implementation, the complex subtleties of a secure implementation makes it best for programmers who are not cryptography experts to use high level crypto-systems including using well-known standard protocols to improve your chances that the cryptography of a system is not the exploitable flaw in a system.
AES is much different, AES creates a SPN, Substitution Permutation Network. It generates s-boxes (substitution boxes) based on polynomial functions generated at encryption time. It runs it through 10-14 rounds of byte-level substitution and bit-level permuting, the bit length of the key determining the number of rounds and the round keys.
RSA is based on factors of large prime numbers which are extremely hard to do computationally, but quite easy to initially encrypt.
RSA is broken by factoring. Actually, RSA is two algorithms, one for (asymmetric) encryption and one for digital signatures; both use the same primitive. In RSA, there is a public value (the modulus, often noted n) which is a product of two (or more) distinct prime factors. Factoring n reveals the private key. Factoring becomes harder when the size of n increases. The current record (published earlier this year) is for a 768-bit integer; it took four years of big computing and hard work by very smart people. The same people openly admit that they have little clue of how they could try the same stunt on a 1024-bit integer (there is a part of the best known factorization algorithm which requires an awful lot of fast RAM, and for a 1024-bit integer that would require a ludicrously huge machine). Current recommendations of key length for RSA are 1024 bits for short term, 2048 bits for long term security. Note that computational cost of RSA increases with key size as well, so we do not want to use really big keys without a good reason. A basic PC will produce about 1000 RSA signatures per second (and per core) with a 1024-bit key, and eight times less with a 2048-bit key. This is still quite good.
There are other asymmetric encryption algorithms, and digital signature algorithms. Somewhat related to RSA is the Rabin-Williams encryption algorithm; factoring also breaks it. Then there are algorithms based on discrete logarithm (in the multiplicative group of numbers modulo a big prime): Diffie-Hellman (key exchange), DSA (signature), El Gamal (encryption)... for these algorithms, factorization is no direct threat; but they rely on the same parts of number theory, and the best known algorithm for discrete logarithm is very similar to the best known algorithm for factorization (and has the same name: GNFS -- as General Number Field Sieve). So it is suspected that an advance in factorization would result from an advance in number theory which would be likely to shed some light on discrete logarithm as well.
The discrete logarithm algorithms can be applied to other groups, the most popular being elliptic curves. Elliptic curves are not impacted by factorization. If factorization became easy, thus scraping RSA and indirectly jeopardizing DSA and Diffie-Hellman, then we would switch to ECDH and ECDSA; standards and implementations exist and are deployed.
"Symmetric cryptography", i.e. hash functions (MD5, SHA-256...), authentication code (HMAC, CBC-MAC...), symmetric encryption (AES, 3DES...), random number generation (RC4...) and related activities, are totally unaffected by factorization. For these algorithms, keys are mere bunches of bits, without any special structure; there is nothing to factor.
Which of them are preferred in which circumstances?
I'd like to see the list of evaluation crtieria for the various modes, and maybe a discussion of the applicability of each criterion.
For example,
I think one of the criteria is "size of the code" for encryption and decryption, which is important for micro-code embedded systems, like 802.11 network adapters. IF the code required to implement CBC is much smaller than that required for CTR (I don't know this is true, it's just an example), then I could understand why the mode with the smaller code would be preferred. But if I am writing an app that runs on a server, and the AES library I am using implements both CBC and CTR anyway, then this criterion is irrelevant.
See what I mean by "list of evaluation criteria and applicability of each criterion" ??
This isn't really programming related but it is algorithm related.
Please consider long and hard if you can't get around implementing your own cryptography
The ugly truth of the matter is that if you are asking this question you will probably not be able to design and implement a secure system.
Let me illustrate my point: Imagine you are building a web application and you need to store some session data. You could assign each user a session ID and store the session data on the server in a hash map mapping session ID to session data. But then you have to deal with this pesky state on the server and if at some point you need more than one server things will get messy. So instead you have the idea to store the session data in a cookie on the client side. You will encrypt it of course so the user cannot read and manipulate the data. So what mode should you use? Coming here you read the top answer (sorry for singling you out myforwik). The first one covered - ECB - is not for you, you want to encrypt more than one block, the next one - CBC - sounds good and you don't need the parallelism of CTR, you don't need random access, so no XTS and patents are a PITA, so no OCB. Using your crypto library you realize that you need some padding because you can only encrypt multiples of the block size. You choose PKCS7 because it was defined in some serious cryptography standards. After reading somewhere that CBC is provably secure if used with a random IV and a secure block cipher, you rest at ease even though you are storing your sensitive data on the client side.
Years later after your service has indeed grown to significant size, an IT security specialist contacts you in a responsible disclosure. She's telling you that she can decrypt all your cookies using a padding oracle attack, because your code produces an error page if the padding is somehow broken.
This is not a hypothetical scenario: Microsoft had this exact flaw in ASP.NET until a few years ago.
The problem is there are a lot of pitfalls regarding cryptography and it is extremely easy to build a system that looks secure for the layman but is trivial to break for a knowledgeable attacker.
What to do if you need to encrypt data
For live connections use TLS (be sure to check the hostname of the certificate and the issuer chain). If you can't use TLS, look for the highest level API your system has to offer for your task and be sure you understand the guarantees it offers and more important what it does not guarantee. For the example above a framework like Play offers client side storage facilities, it does not invalidate the stored data after some time, though, and if you changed the client side state, an attacker can restore a previous state without you noticing.
If there is no high level abstraction available use a high level crypto library. A prominent example is NaCl and a portable implementation with many language bindings is Sodium. Using such a library you do not have to care about encryption modes etc. but you have to be even more careful about the usage details than with a higher level abstraction, like never using a nonce twice. For custom protocol building (say you want something like TLS, but not over TCP or UDP) there are frameworks like Noise and associated implementations that do most of the heavy lifting for you, but their flexibility also means there is a lot of room for error, if you don't understand in depth what all the components do.
If for some reason you cannot use a high level crypto library, for example because you need to interact with existing system in a specific way, there is no way around educating yourself thoroughly. I recommend reading Cryptography Engineering by Ferguson, Kohno and Schneier. Please don't fool yourself into believing you can build a secure system without the necessary background. Cryptography is extremely subtle and it's nigh impossible to test the security of a system.
Comparison of the modes
Encryption only:
Modes that require padding:
Like in the example, padding can generally be dangerous because it opens up the possibility of padding oracle attacks. The easiest defense is to authenticate every message before decryption. See below.
ECB encrypts each block of data independently and the same plaintext block will result in the same ciphertext block. Take a look at the ECB encrypted Tux image on the ECB Wikipedia page to see why this is a serious problem. I don't know of any use case where ECB would be acceptable.
CBC has an IV and thus needs randomness every time a message is encrypted, changing a part of the message requires re-encrypting everything after the change, transmission errors in one ciphertext block completely destroy the plaintext and change the decryption of the next block, decryption can be parallelized / encryption can't, the plaintext is malleable to a certain degree - this can be a problem.
Stream cipher modes: These modes generate a pseudo random stream of data that may or may not depend the plaintext. Similarly to stream ciphers generally, the generated pseudo random stream is XORed with the plaintext to generate the ciphertext. As you can use as many bits of the random stream as you like you don't need padding at all. Disadvantage of this simplicity is that the encryption is completely malleable, meaning that the decryption can be changed by an attacker in any way he likes as for a plaintext p1, a ciphertext c1 and a pseudo random stream r and attacker can choose a difference d such that the decryption of a ciphertext c2=c1⊕d is p2 = p1⊕d, as p2 = c2⊕r = (c1 ⊕ d) ⊕ r = d ⊕ (c1 ⊕ r). Also the same pseudo random stream must never be used twice as for two ciphertexts c1=p1⊕r and c2=p2⊕r, an attacker can compute the xor of the two plaintexts as c1⊕c2=p1⊕r⊕p2⊕r=p1⊕p2. That also means that changing the message requires complete reencryption, if the original message could have been obtained by an attacker. All of the following steam cipher modes only need the encryption operation of the block cipher, so depending on the cipher this might save some (silicon or machine code) space in extremely constricted environments.
CTR is simple, it creates a pseudo random stream that is independent of the plaintext, different pseudo random streams are obtained by counting up from different nonces/IVs which are multiplied by a maximum message length so that overlap is prevented, using nonces message encryption is possible without per message randomness, decryption and encryption are completed parallelizable, transmission errors only effect the wrong bits and nothing more
OFB also creates a pseudo random stream independent of the plaintext, different pseudo random streams are obtained by starting with a different nonce or random IV for every message, neither encryption nor decryption is parallelizable, as with CTR using nonces message encryption is possible without per message randomness, as with CTR transmission errors only effect the wrong bits and nothing more
CFB's pseudo random stream depends on the plaintext, a different nonce or random IV is needed for every message, like with CTR and OFB using nonces message encryption is possible without per message randomness, decryption is parallelizable / encryption is not, transmission errors completely destroy the following block, but only effect the wrong bits in the current block
Disk encryption modes: These modes are specialized to encrypt data below the file system abstraction. For efficiency reasons changing some data on the disc must only require the rewrite of at most one disc block (512 bytes or 4kib). They are out of scope of this answer as they have vastly different usage scenarios than the other. Don't use them for anything except block level disc encryption. Some members: XEX, XTS, LRW.
Authenticated encryption:
To prevent padding oracle attacks and changes to the ciphertext, one can compute a message authentication code (MAC) on the ciphertext and only decrypt it if it has not been tampered with. This is called encrypt-then-mac and should be preferred to any other order. Except for very few use cases authenticity is as important as confidentiality (the latter of which is the aim of encryption). Authenticated encryption schemes (with associated data (AEAD)) combine the two part process of encryption and authentication into one block cipher mode that also produces an authentication tag in the process. In most cases this results in speed improvement.
CCM is a simple combination of CTR mode and a CBC-MAC. Using two block cipher encryptions per block it is very slow.
OCB is faster but encumbered by patents. For free (as in freedom) or non-military software the patent holder has granted a free license, though.
GCM is a very fast but arguably complex combination of CTR mode and GHASH, a MAC over the Galois field with 2^128 elements. Its wide use in important network standards like TLS 1.2 is reflected by a special instruction Intel has introduced to speed up the calculation of GHASH.
Recommendation:
Considering the importance of authentication I would recommend the following two block cipher modes for most use cases (except for disk encryption purposes): If the data is authenticated by an asymmetric signature use CBC, otherwise use GCM.
ECB should not be used if encrypting more than one block of data with the same key.
CBC, OFB and CFB are similar, however OFB/CFB is better because you only need encryption and not decryption, which can save code space.
CTR is used if you want good parallelization (ie. speed), instead of CBC/OFB/CFB.
XTS mode is the most common if you are encoding a random accessible data (like a hard disk or RAM).
OCB is by far the best mode, as it allows encryption and authentication in a single pass. However there are patents on it in USA.
The only thing you really have to know is that ECB is not to be used unless you are only encrypting 1 block. XTS should be used if you are encrypting randomly accessed data and not a stream.
You should ALWAYS use unique IV's every time you encrypt, and they should be random. If you cannot guarantee they are random, use OCB as it only requires a nonce, not an IV, and there is a distinct difference. A nonce does not drop security if people can guess the next one, an IV can cause this problem.
A formal analysis has been done by Phil Rogaway in 2011, here. Section 1.6 gives a summary that I transcribe here, adding my own emphasis in bold (if you are impatient, then his recommendation is use CTR mode, but I suggest that you read my paragraphs about message integrity versus encryption below).
Note that most of these require the IV to be random, which means non-predictable and therefore should be generated with cryptographic security. However, some require only a "nonce", which does not demand that property but instead only requires that it is not re-used. Therefore designs that rely on a nonce are less error prone than designs that do not (and believe me, I have seen many cases where CBC is not implemented with proper IV selection). So you will see that I have added bold when Rogaway says something like "confidentiality is not achieved when the IV is a nonce", it means that if you choose your IV cryptographically secure (unpredictable), then no problem. But if you do not, then you are losing the good security properties. Never re-use an IV for any of these modes.
Also, it is important to understand the difference between message integrity and encryption. Encryption hides data, but an attacker might be able to modify the encrypted data, and the results can potentially be accepted by your software if you do not check message integrity. While the developer will say "but the modified data will come back as garbage after decryption", a good security engineer will find the probability that the garbage causes adverse behaviour in the software, and then he will turn that analysis into a real attack. I have seen many cases where encryption was used but message integrity was really needed more than the encryption. Understand what you need.
I should say that although GCM has both encryption and message integrity, it is a very fragile design: if you re-use an IV, you are screwed -- the attacker can recover your key. Other designs are less fragile, so I personally am afraid to recommend GCM based upon the amount of poor encryption code that I have seen in practice.
If you need both, message integrity and encryption, you can combine two algorithms: usually we see CBC with HMAC, but no reason to tie yourself to CBC. The important thing to know is encrypt first, then MAC the encrypted content, not the other way around. Also, the IV needs to be part of the MAC calculation.
I am not aware of IP issues.
Now to the good stuff from Professor Rogaway:
Block ciphers modes, encryption but not message integrity
ECB: A blockcipher, the mode enciphers messages that are a multiple of n bits by separately enciphering each n-bit piece. The security properties are weak, the method leaking equality of blocks across both block positions and time. Of considerable legacy value, and of value as a building block for other schemes, but the mode does not achieve any generally desirable security goal in its own right and must be used with considerable caution; ECB should not be regarded as a “general-purpose” confidentiality mode.
CBC: An IV-based encryption scheme, the mode is secure as a probabilistic encryption scheme, achieving indistinguishability from random bits, assuming a random IV. Confidentiality is not achieved if the IV is merely a nonce, nor if it is a nonce enciphered under the same key used by the scheme, as the standard incorrectly suggests to do. Ciphertexts are highly malleable. No chosen ciphertext attack (CCA) security. Confidentiality is forfeit in the presence of a correct-padding oracle for many padding methods. Encryption inefficient from being inherently serial. Widely used, the mode’s privacy-only security properties result in frequent misuse. Can be used as a building block for CBC-MAC algorithms. I can identify no important advantages over CTR mode.
CFB: An IV-based encryption scheme, the mode is secure as a probabilistic encryption scheme, achieving indistinguishability from random bits, assuming a random IV. Confidentiality is not achieved if the IV is predictable, nor if it is made by a nonce enciphered under the same key used by the scheme, as the standard incorrectly suggests to do. Ciphertexts are malleable. No CCA-security. Encryption inefficient from being inherently serial. Scheme depends on a parameter s, 1 ≤ s ≤ n, typically s = 1 or s = 8. Inefficient for needing one blockcipher call to process only s bits . The mode achieves an interesting “self-synchronization” property; insertion or deletion of any number of s-bit characters into the ciphertext only temporarily disrupts correct decryption.
OFB: An IV-based encryption scheme, the mode is secure as a probabilistic encryption scheme, achieving indistinguishability from random bits, assuming a random IV. Confidentiality is not achieved if the IV is a nonce, although a fixed sequence of IVs (eg, a counter) does work fine. Ciphertexts are highly malleable. No CCA security. Encryption and decryption inefficient from being inherently serial. Natively encrypts strings of any bit length (no padding needed). I can identify no important advantages over CTR mode.
CTR: An IV-based encryption scheme, the mode achieves indistinguishability from random bits assuming a nonce IV. As a secure nonce-based scheme, the mode can also be used as a probabilistic encryption scheme, with a random IV. Complete failure of privacy if a nonce gets reused on encryption or decryption. The parallelizability of the mode often makes it faster, in some settings much faster, than other confidentiality modes. An important building block for authenticated-encryption schemes. Overall, usually the best and most modern way to achieve privacy-only encryption.
XTS: An IV-based encryption scheme, the mode works by applying a tweakable blockcipher (secure as a strong-PRP) to each n-bit chunk. For messages with lengths not divisible by n, the last two blocks are treated specially. The only allowed use of the mode is for encrypting data on a block-structured storage device. The narrow width of the underlying PRP and the poor treatment of fractional final blocks are problems. More efficient but less desirable than a (wide-block) PRP-secure blockcipher would be.
MACs (message integrity but not encryption)
ALG1–6: A collection of MACs, all of them based on the CBC-MAC. Too many schemes. Some are provably secure as VIL PRFs, some as FIL PRFs, and some have no provable security. Some of the schemes admit damaging attacks. Some of the modes are dated. Key-separation is inadequately attended to for the modes that have it. Should not be adopted en masse, but selectively choosing the “best” schemes is possible. It would also be fine to adopt none of these modes, in favor of CMAC. Some of the ISO 9797-1 MACs are widely standardized and used, especially in banking. A revised version of the standard (ISO/IEC FDIS 9797-1:2010) will soon be released [93].
CMAC: A MAC based on the CBC-MAC, the mode is provably secure (up to the birthday bound) as a (VIL) PRF (assuming the underlying blockcipher is a good PRP). Essentially minimal overhead for a CBCMAC-based scheme. Inherently serial nature a problem in some application domains, and use with a 64-bit blockcipher would necessitate occasional re-keying. Cleaner than the ISO 9797-1 collection of MACs.
HMAC: A MAC based on a cryptographic hash function rather than a blockcipher (although most cryptographic hash functions are themselves based on blockciphers). Mechanism enjoys strong provable-security bounds, albeit not from preferred assumptions. Multiple closely-related variants in the literature complicate gaining an understanding of what is known. No damaging attacks have ever been suggested. Widely standardized and used.
GMAC: A nonce-based MAC that is a special case of GCM. Inherits many of the good and bad characteristics of GCM. But nonce-requirement is unnecessary for a MAC, and here it buys little benefit. Practical attacks if tags are truncated to ≤ 64 bits and extent of decryption is not monitored and curtailed. Complete failure on nonce-reuse. Use is implicit anyway if GCM is adopted. Not recommended for separate standardization.
authenticated encryption (both encryption and message integrity)
CCM: A nonce-based AEAD scheme that combines CTR mode encryption and the raw
CBC-MAC. Inherently serial, limiting speed in some contexts. Provably secure, with good bounds, assuming the underlying blockcipher is a good PRP. Ungainly construction that demonstrably does the job. Simpler to implement than GCM. Can be used as a nonce-based MAC. Widely standardized and used.
GCM: A nonce-based AEAD scheme that combines CTR mode encryption and a GF(2128)-based universal hash function. Good efficiency characteristics for some implementation environments. Good provably-secure results assuming minimal tag truncation. Attacks and poor provable-security bounds in the presence of substantial tag truncation. Can be used as a nonce-based MAC, which is then called GMAC. Questionable choice to allow nonces other than 96-bits. Recommend restricting nonces to 96-bits and tags to at least 96 bits. Widely standardized and used.
Anything but ECB.
If using CTR, it is imperative that you use a different IV for each message, otherwise you end up with the attacker being able to take two ciphertexts and deriving a combined unencrypted plaintext. The reason is that CTR mode essentially turns a block cipher into a stream cipher, and the first rule of stream ciphers is to never use the same Key+IV twice.
There really isn't much difference in how difficult the modes are to implement. Some modes only require the block cipher to operate in the encrypting direction. However, most block ciphers, including AES, don't take much more code to implement decryption.
For all cipher modes, it is important to use different IVs for each message if your messages could be identical in the first several bytes, and you don't want an attacker knowing this.
Have you start by reading the information on this on Wikipedia - Block cipher modes of operation? Then follow the reference link on Wikipedia to NIST: Recommendation for Block Cipher Modes of Operation.
You might want to chose based on what is widely available. I had the same question and here are the results of my limited research.
Hardware limitations
STM32L (low energy ARM cores) from ST Micro support ECB, CBC,CTR GCM
CC2541 (Bluetooth Low Energy) from TI supports ECB, CBC, CFB, OFB, CTR, and CBC-MAC
Open source limitations
Original rijndael-api source - ECB, CBC, CFB1
OpenSSL - command line CBC, CFB, CFB1, CFB8, ECB, OFB
OpenSSL - C/C++ API CBC, CFB, CFB1, CFB8, ECB, OFB and CTR
EFAES lib [1] - ECB, CBC, PCBC, OFB, CFB, CRT ([sic] CTR mispelled)
OpenAES [2] - ECB, CBC
[1] http://www.codeproject.com/Articles/57478/A-Fast-and-Easy-to-Use-AES-Library
[2] https://openaes.googlecode.com/files/OpenAES-0.8.0.zip
There are new timing vulnerabilities in the CBC mode of operation.
https://learn.microsoft.com/en-us/dotnet/standard/security/vulnerabilities-cbc-mode
Generally the sole existence of a chaining mode already reduces the theoretically security as chaining widens the attack surface and also make certain kind of attacks more feasible. On the other hand, without chaining, you can at most encrypt 16 bytes (128 bits) securely, as that's the block size of AES (also of AES-192 and AES-256) and if your input data exceeds that block size, what else would you do than using chaining? Just encrypting the data block by block? That would be ECB and ECB has the worst security to begin with. Anything is more secure than ECB.
Most security analyses recommend that you always use either CBC or CTR, unless you can name any reason why you cannot use one of these two modes. And out of these two modes, they recommend CBC if security is your main concern and CTR if speed is your main concern. That's because CTR is slightly less secure than CBC because it has a higher likeliness of IV (initialization vector) collision, since the presence of the CTR counter reduces the IV value space, and attackers can change some ciphertext bits to damage exactly the same bits in plaintext (which can be an issue if the attacker knows exact bit positions in the data). On the other hand, CTR can be fully parallelized (encryption and decryption) and requires no data padding to a multiple of the block size.
That said, they still claim CFB and OFB to be secure, however slightly less secure and they have no real advantages to begin with. CFB shares the same weaknesses as CBC and on top of that isn't protected against replay attacks. OFB shares the same weaknesses as CTR but cannot be parallelized at all. So CFB is like CBC without padding but less secure and OFB is like CTR but without its speed benefits and a wider attack surface.
There is only one special case where you may want to use OFB and that's if you need to decrypt data in realtime (e.g. a stream of incoming data) on hardware that is actually too weak for doing so, yet you will know the decryption key way ahead of time. As in that case, you can pre-calculate all the XOR blocks in advance and store them somewhere and when the real data arrives, the entire decryption is just XOR'ing the incoming data with the stored XOR blocks and that requires very little computational power. That's the one thing you can do with OFB that you cannot do with any other chaining.
For performance analysis, see this paper.
For a detailed evaluation, including security, see this paper.
I know one aspect: Although CBC gives better security by changing the IV for each block, it's not applicable to randomly accessed encrypted content (like an encrypted hard disk).
So, use CBC (and the other sequential modes) for sequential streams and ECB for random access.
First off, I would like to ask if any of you know of an encryption algorithm that uses a key to encrypt the data, but no key to decrypt the data. This seems highly unlikely, if not impossible to me, so sorry if it's a stupid question.
My final question is, say you have access to the plain text data before it is encrypted, the key used to encrypt the plain text data, and the resulting encrypted data, would figuring out which algorithm used to encrypt the data be feasible?
First off, I would like to ask
if any of you know of an encryption
algorithm that uses a key to encrypt
the data, but no key to decrypt the
data.
No. There are algorithms that use a different key to decrypt than to encrypt, but a keyless method would rely on secrecy of the algorithm, generally regarded as a poor idea.
My final question is, say you have
access to the plain text data before
it is encrypted, the key used to
encrypt the plain text data, and the
resulting encrypted data, would
figuring out which algorithm used to
encrypt the data be feasible?
Most likely yes, especially given the key. A good crypto algorithm relies on the secrecy of the key, and the key alone. See kerckhoff's principle.
Also if a common algorithm is used it would be a simple matter of trial and error, and besides cryptotext often is accompanied by metadata which tells you algorithm details.
edit: as per comments, you may be thinking of digital signature (which requires a secret only on the sender side), a hash algorithm (which requires no key but isn't encryption), or a zero-knowledge proof (which can prove knowledge of a secret without revealing it).
Abstractly, we can think of the encryption system this way:
-------------------
plaintext ---> | algorithm & key | ---> ciphertext
-------------------
The system must guarantee the following:
decrypt(encrypt(plaintext, algorithm, key), algorithm, key) = plaintext
First off, I would like to ask
if any of you know of an encryption
algorithm that uses a key to encrypt
the data, but no key to decrypt the
data.
Yes, in such a system the key is redundant; all the "secrecy" lies in the algorithm.
My final question
is, say you have access to the plain
text data before it is encrypted, the
key used to encrypt the plain text
data, and the resulting encrypted
data, would figuring out which
algorithm used to encrypt the data be
feasible?
In practice, you'll probably have a small space of algorithms, so a simple brute-force search is feasible. However, there may be more than one algorithm that fits the given information. Consider the following example:
We define the following encryption and decryption operations, where plaintext, ciphertext, algorithm, and key are real numbers (assume algorithm is nonzero):
encrypt(plaintext, algorithm, key) = algorithm x (plaintext + key) = ciphertext
decrypt(ciphertext, algorithm, key) = ciphertext/algorithm - key = plaintext
Now, suppose that plaintext + key = 0. We have ciphertext = 0 for any choice of algorithm. Hence, we cannot deduce the algorithm used.
First off, I would like to ask if any of you know of an encryption algorithm that uses a key to encrypt the data, but no key to decrypt the data.
What are you getting at? It's trivial to come up with a pair of functions that fits the letter of the specification, but without knowing the intent it's hard to give a more helpful answer.
say you have access to the plain text data before it is encrypted, the key used to encrypt the plain text data, and the resulting encrypted data, would figuring out which algorithm used to encrypt the data be feasible?
If the algorithm is any good the output will be indistinguishable from random noise, so there is no analytic solution to this. As a practical matter, there are only so many trusted algorithms in wide use. Trying each one in turn would be quick, but would be complicated by the fact that an implementation has some freedom with regard to things like byte order (little-endian vs big-endian), key derivation (if you had a pass-phrase instead of the actual cryptographic key itself), encryption modes and padding.
As frankodwyer points out, this situation is not part of usual threat models. This would work in your favor, as it makes it more likely that the algorithm is a well-known one.
The best you could do without a known key in the decoder would be to add a bit of obscurity. For example, if the first step of the decode algorythm is to strip out everything except for every tenth character, then your encode key may be used to seed some random garbage for nine out of every ten characters. Thus, with different keys you could achieve different encoded results which would be decoded to the same message, with no key necessary for the decoder.
However, this does not add much real security and should not be solely relied on to protect crucial data. I'm just thinking of a case where it would be possible to do so yes I suppose it could - if you were just trying to prove a point or add one more level of security.
I don't believe that there is such an algorithm that would use a key to encrypt, but not to decrypt. (Silly answers like a 26 character Caesar cipher aside...)
To your second question, yes; it just depends on how much time you're willing to spend on it. In theoretical cryptography it is assumed that the algorithm can always be determined. Whether that be through theft of the algorithm or a physical machine, or as in your case having a plain text and cipher text pair.