I have a few questions about CRC:
How can I tell, for a given CRC polynom and a n-bits data, what is the largest number of bits in error that is guaranteed to be detected?
Is it true that ALWAYS - the bigger the polynom degree, the more errors that can be detected from that CRC?
Thanks!
I will assume that "How many errors can it detect" is the largest number of bits in error that is always guaranteed to be detected.
You would need to search for the minimum weight codeword of length n for that polynomial, also referred to as its Hamming distance. The number of bit errors that that CRC is guaranteed to detect is one less than that minimum weight. There is no alternative to what essentially amounts to a brute-force search. See Table 1 in this paper by Koopman for some results. As an example, for messages 2974 or fewer bits in length, the standard CRC-32 has a Hamming distance of no less than 5, so any set of 4 bit errors would be detected.
Not necessarily. Polynomials can vary widely in quality. Given the performance of a polynomial on a particular message length, you can always find a longer polynomial that has worse performance. However a well-chosen longer polynomial should in general perform better than a well-chosen shorter polynomial. However even there, for long message lengths you may find that both have a Hamming distance of two.
It's not a question of 'how many'. It's a question of what proportion and what kind. 'How many' depends on those things and on the length of the input.
Yes.
Related
I was studying error detection in computer networks and I came to know about the following methods -
Single Bit parity Check
2d parity Check
Checksum
Cyclic Redundancy Check
But after studying only a bit (lmao pun), I came across cases where they fail.
The methods fail when -
Single Bit parity Check - If an even number of bits has been inverted.
2d parity Check - If an even number of bits are inverted in the same position.
Checksum - addition of 0 to a frame does not change the result, and sequence is not maintained.
(for e.g. in the data - 10101010 11110000 11001100 10111001 if we add 0 to any of the
four frames here)
CRC - A CRC of n-bit for g(x) = (x+l)*p(x) can detect:
All burst errors of length less than or equal to n.
All burst errors affecting an odd number of bits.
All burst errors of length equal to n + 1 with probability (2^(n-1) − l)/2^n − 1
All burst errors of length greater than n + 1 with probability (2^(n-1) − l)/2^n
[the CRC-32 polynomial will detect all burst errors of length greater than 33 with
probability (2^32 − l)/2^32; This is equivalent to a 99.99999998% accuracy rate]
Copied from here - https://stackoverflow.com/a/65718709/16778741
As we can see these methods fail because of some very obvious shortcomings.
So my question is - why were these still allowed and not rectified and what do we use these days?
Its like the people who made them forgot to cross check
It is a tradeoff between effort and risk. The more redundant bits are added, the smaller the risk of undetected error.
Extra bits mean additional memory or network bandwidth consumption. It depends on the application, which additional effort is justified.
Complicated checksums add some computational overhead as well.
Modern checksum or hash functions can drive the remaining risk to very small ranges tolerable for the vast majority of applications.
Only 0.00000002% of burst errors will be missed. But what is not stated is the likelihood of these burst errors occurring. That number is dependent on the network implementation. In most cases the likelihood of a undetectable burst error will be very close to zero or zero for an ideal network.
Multiplying almost zero with almost zero is really close to zero.
Undetected errors in CRCs is more of academic interest than practical reality.
I want to generate some numbers, which should attempt to share as few common bit patterns as possible, such that collisions happen at minimal amount. Until now its "simple" hashing with a given amount of output bits. However, there is another 'constraint'. I want to minimize the risk that, if you take one number and change it by toggling a small amount of bits, you end up with another number you've just generated. Note: I don't want it to be impossible or something, I want to minimize the risk!
How to calculate the probability for a list with n numbers, where each number has m bits? And, of course, what would be a suitable method to generate those numbers? Any good articles about this?
To answer this question precisely, you need to say what exactly you mean by "collision", and what you mean by "generate". If you just want the strings to be far apart from each other in hamming distance, you could hope to make an optimal, deterministic set of such strings. It is true that random strings will have this property with high probability, so you could use random strings instead.
When you say
Note: I don't want it to be impossible or something, I want to minimize the risk!
this sounds like an XY problem. If some outcome is the "bad thing" then why do you want it to be possible, but just low probability? Shouldn't you want it not to happen at all?
In short I think you should look up the term "error correcting code". The codewords of any good error correcting code, with any parameters that you feel like, will have the minimal risk of collision in the presence of random noise, for that number of code words of that length, and they can typically be generated very easily using matrix multiplication.
I used AMD's two-stage reduction example to compute the sum of all numbers from 0 to 65 536 using floating point precision. Unfortunately, the result is not correct. However, when I modify my code, so that I compute the sum of 65 536 smaller numbers (for example 1), the result is correct.
I couldn't find any error in the code. Is it possible that I am getting wrong results, because of the float type? If this is the case, what is the best approach to solve the issue?
This is a "side effect" of summing floating point numbers using finite precision CPU's or GPU's. The accuracy depends the algorithm and the order the values are summed. The theory and practice behind is explained in Nicholas J, Higham's paper
The Accuracy of Floating Point Summation
http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=7AECC0D6458288CD6E4488AD63A33D5D?doi=10.1.1.43.3535&rep=rep1&type=pdf
The fix is to use a smarter algorithm like the Kahan Summation Algorithm
https://en.wikipedia.org/wiki/Kahan_summation_algorithm
And the Higham paper has some alternatives too.
This problem illustrates the nature of benchmarking, the first rule of the benchmark is to get the
right answer, using realistic data!
There is probably no error in the coding of your kernel or host application. The issue is with the single-precision floating point.
The correct sum is: 65537 * 32768 = 2147516416, and it takes 31 bits to represent it in binary (10000000000000001000000000000000). 32-bit floats can only hold integers accurately up to 2^24.
"Any integer with absolute value less than [2^24] can be exactly represented in the single precision format"
"Floating Point" article, wikipedia
This is why you are getting the correct sum when it is less than or equal to 2^24. If you are doing a complete sum using single-precision, you will eventually lose accuracy no matter which device you are executing the kernel on. There are a few things you can do to get the correct answer:
use double instead of float if your platform supports it
use int or unsigned int
sum a smaller set of numbers eg: 0+1+2+...+4095+4096 = (2^23 + 2^11)
Read more about single precision here.
What is the difference between 'accurate' and 'precise' ?
If there is a difference, can you give an example of
a number that is accurate but not precise
a number that is precise but not accurate
a number that is both accurate and precise
Thanks!
Precision refers to how much information is conveyed by a number (in terms of number of digits) whereas accuracy is a measure of "correctness".
Let's take the π approximation 22/7, for our purposes, 3.142857143.
For your specific questions:
a number that is accurate but not precise: 3.14. That's certainly accurate in terms of closeness, given the precision available. There is no other number with three significant digits that is closer to the target (both 3.13 and 3.15 are further away from the real value).
a number that is precise but not accurate: 99999.12345678901234567890. That's much more precise since it conveys more information. Unfortunately its accuracy is way off since it's nowhere near the target value.
a number that is both accurate and precise: 3.142857143. You can get more precise (by tacking zeros on the end) but no more accurate.
Of course, that's if the target number is actually 3.142857143. If it's 22/7, then you can get more accurate and precise, since 3.142857143 * 7 = 22.000000001. The actual decimal number for that fraction is an infinitely repeating one (in base 10):
3 . 142857 142857 142857 142857 142857 ...
and so on, so you can keep adding precision and accuracy in that representation by continuing to repeat that group of six digits. Or, you can maximise both by just using 22/7.
One way to think of it is this:
A number that is "precise" has a lot of digits. But it might not be very correct.
A number that is "accurate" is correct, but may not have a lot of digits.
Examples:
3.14 is an "accurate" approximation to Pi. But it is not very precise.
3.13198408654198 is a very "precise" approximation to Pi, but it is not accurate,
3.14159265358979 is both accurate and precise.
So precision gives a lot of information. But says nothing about how correct it is.
Accuracy says how correct the information is, but says nothing about how much information there is.
Assume the exact time right now is 13:01:03.1234
Accurate but not precise - it's 13:00 +/- 0:05
Precise but not accurate - it's 13:15:01.1425
Accurate and precise - it's 13:01:03.1234
The standard example I always heard involved a dart board:
accurate but not precise: lots of darts scattered evenly all over the dart board
precise but not accurate: lots of darts concentrated in one spot of the dart board, that is not the bull's eye
both: lots of darts concentrated in the bull's eye
Accuracy is about getting the right answer. Precision is about repeatedly getting the same answer.
Precision and accuracy are defined by significant digits. Accuracy is defined by the number of significant digits while precision is identified by the location of the last significant digit. For instance the number 1234 is more accurate than 0.123 because 1234 had more significant digits. The number 0.123 is more precise because the 3 (last significant figure) is in the thousandths place. Both types of digits typically only relevant because they are the results of a measurement. For instance, you can have a decimal number that's exact such as 0.123 such as 123/1000 as defined, thus the discussion of precision has no real meaning because 0.123 was given or defined;however, if you were to measure something and come up with that value, then 0.123 indicates the precision of the tool used to measure it.
The real confusion occurs when combining these numbers such as adding, subtracting, multiply and dividing. For example, when adding two numbers that are the result of a measurement, the answer can only be as precise as the least precise number. Think of it as a chain is only as strong as its weakest link.
Accuracy are very often confused with precision but they are much different.
Accuracy is degree to which the measured value agrees with true value.
Example-Our objective is to make rod of 25mm And we are able to make it of 25 mm then it is accurate.
Precision is the repeatability of the measuring process.
Example-Our objective is to make 10 rods of 25mm and we make all rods of 24mm then we are precise as we make all rods of same size,but it is not accurate as true value is 25 mm.
If I want to send a d-bit packet and add another r bits for error correction code (d>r)
how many errors I can find and correct at most?
You have 2^d different kinds of packets of length d bits you want to send. Adding your r bits to them makes them into codewords of length d+r, so now you have 2^d possible codewords you could send. The receiver could get 2^(d+r) different received words(codewords with possible errors). The question then becomes, how do you map those 2^(d+r) received words to the 2^d codewords?
This comes down to the minimum distance of the code. That is, for each pair of codewords, find the number of bits where they differ, then take the smallest of those values.
Let's say you had a minimum distance of 3. You received a word and you notice that it isn't one of the codewords. That is, there's an error. So, for the lack of a better decoding algorithm, you flip the first bit, and see if its a codeword. If it isn't you flip it back and flip the next one. Eventually, you get a codeword. Since all codewords differ in 3 positions, you know this codeword is the "closest" to the received word, since you would have to flip 2 bits in the received word to get to another codeword. If you didn't get a codeword from flipping just one bit at a time, you can't figure out where the errors are, since there are multiple codewords you could get to by flipping two bits, but you know there are at least two errors.
This leads to the general principle that for a minimum distance md, you can detect md-1 errors and correct floor((md-1)/2) errors. Calculating the minimum distance depends on the details of how you generate the codewords, otherwise known as the code. There are various bounds you can use to figure out an upper limit on md based on d and (d+r).
Paul mentioned the Hamming Code, which is a good example. It achieves the Hamming bound. For the (7,4) Hamming code, you have 4 bit messages and 7 bit codewords, and you achieve a minimum distance of 3. Obviously*, you are never going to get a minimum distance greater than the number of bits you are adding so this is the very best you can do. Don't get too used to this though. The Hamming code is one of the few examples of a non-trivial perfect code, and most of those have a minimum distance that is less than the number of bits you add.
*It's not really obvious, but I'm pretty sure it's true for non-trivial error correcting codes. Adding one parity bit gets you a minimum distance of two, allowing you to detect an error. The code consisting of {000,111} gets you a minimum distance of 3 by adding just 2 bits, but it's trivial.
You should probably read the wikipedia page on this:
http://en.wikipedia.org/wiki/Error_detection_and_correction
It sounds like you specifically want a Hamming Code:
http://en.wikipedia.org/wiki/Hamming_code#General_algorithm
Using that scheme, you can look up some example values from the linked table.