I'm studying information theory but one thing I can't seem to work out.
I know that given a linear code C and a generator matrix M I can work out all the possible codewords of C.
However I do not understand:
What a parity check matrix is http://en.wikipedia.org/wiki/Parity-check_matrix
How to make a parity check matrix from a generator matrix
I'd really appreciate any pointers!
Thanks!
I think your link explains it fairly well, but I'll try to simplify further.
Let x be your message, a k-element row vector. Let G be your generator matrix, an k-by-n binary matrix where n > k. Let y be your n-element transmitted codeword where y = xG. Let z be your n-element received codeword.
Hopefully, z = y. But when transmitting y across a noisy channel, it is possible for y to become corrupted, e.g., z != y.
An (n-k)-by-n parity matrix H is applied to the received codeword z to check if z is valid. The vector w = zH' can detect up to a certain number of bit errors in z.
In coding theory, a parity-check matrix of a linear block code C is a generator matrix of the dual code. As such, a codeword c is in C if and only if the matrix-vector product Hc=0.
The rows of a parity check matrix are parity checks on the codewords of a code. That is, they show how linear combinations of certain digits of each codeword equal zero. For example, the parity check matrix
specifies that for each codeword, digits 1 and 2 should sum to zero (according to the second row) and digits 3 and 4 should sum to zero.
LDPC i believe uses parity check matrix. more generally error control/correction algorithms
Related
I am trying to understand how likely a Cyclic Redundancy Check is to fail, given a particular divisor P(x). I am specifically interested in failures resulting from an odd number of bit flips in my message, an example to follow.
Some prerequisite info:
CRC is a very commonly used way to detect errors in computer networks.
P(x), G(x), R(x), and T(x) are all polynomials under binary field arithmetic (i.e., all coefficients are mod2: 0 or 1).'
P(x) is the polynomial that we are given and that we will divide by.
E(x) is an error pattern. It is XORed with T(x) to get T'(x). G(x) is the message that we want to send.
R(x) is the remainder of G(x)/P(x) or just G(x)modP(x).
T(x) is our sent data, (x^k)G(x)+R(x), where k is the degree of P(x).
T'(x) is our received data but potentially with errors.
When T'(x) is received, if T'(x)modP(x)=0 then it is said to be error-free. It may not actually be error-free.
Proof:
Assume an odd number of errors has x + 1 as a factor.
Then E(x) = (x + 1)T(x).
Evaluate E(x) for x = 1 â E(x) = E(1) = 1 since there are odd number of terms.
But (x + 1)T(x) = (1 + 1)T(1) = 0.
Therefore, E(x) cannot have x + 1 as a factor.
Example:
Say my P(x)=x^7 +1=10000001\
Let G(x)=x^7+x^6+x^3+x+1=11001011\
So, T(x)=110010111001010\
When E(x)=011111011111111\
T'(x)=E(x)XORT(x)=101101100110101\
T'(x) modulus P(x)=0, a failure.\
I simulated the results on a particular message(T(x)), namely 11001011, and found CRC to fail 42 of the 16384 possible odd parity bit flips that I attempted. Failure means that T'(x)modP(x)=0.
I expected odd parity bit errors to be caught based on the above proof.
Is the proof wrong, or am I doing my example calculation wrong?
What I really want to know is, given P(x)=x^7 +1, what are the offs that any general message with an odd number of bit flips will be erroneous but not be caught as being erroneous?
Sorry, this is so long-winded but I just want to make sure everything is super clear.
Can someone please explain what this part of CRC codes from Tannenbaum computer networks means!
If there has been a single-bit error, E(x) = x^i , where i determines which bit is
in error. If G(x) contains two or more terms, it will never divide into E(x), so all
single-bit errors will be detected.
And
If there have been two isolated single-bit errors, E(x) = x^i + x^j , where i > j.
Alternatively, this can be written as E(x) = x^j (x^(i â j) + 1). If we assume that G(x)
is not divisible by x, a sufficient condition for all double errors to be detected is
that G(x) does not divide x ^k + 1 for any k up to the maximum value of i â j (i.e.,
up to the maximum frame length). Simple, low-degree polynomials that give pro-
tection to long frames are known. For example, x ^15 + x ^14 + 1 will not divide
x ^k + 1 for any value of k below 32,768.
Please post in simple terms so I can understand it a bit more!. EXAMPLEs are appreciated. Thanks in advance!
A message is a sequence of bits. You can convert any sequence of bits into a polynomial by just making each bit the coefficient of 1, x, x2, etc. starting with the first bit. So 100101 becomes 1+x3+x5.
You can make these polynomials useful by considering their coefficients to be members of the simplest finite field, GF(2), which consists only of the elements 0 and 1. There addition is the exclusive-or operation and multiplication is the and operation.
Now you can do all the things you did with polynomials in high school, but where the coefficients are over GF(2). So 1+x added to x+x2 becomes 1+x2. 1+x times 1+x becomes 1+x2. (Work it out.)
Cyclic Redundancy Checks (CRCs) are derived from this approach to binary message arithmetic, where a message converted to a polynomial is divided by a special constant polynomial whose degree is the number of bits in the CRC. Then the coefficients of the remainder of that polynomial division is the CRC of that message.
Read Ross William's CRC tutorial for more. (Real CRCs are not just that remainder, but you'll see.)
I have number of strings (n strings) and I am computing edit distance between strings in a way that I take first one and compare it to the (n-1) remaining strings, second one and compare it to (n-2) remaining, ..., comparing until I ran out of the strings.
Why would an average edit distance be computed as sum of all the edit distances between all the strings divided by the number of comparisons squared. This squaring is confusing me.
Thanks,
Jannine
I assume you have somewhere an answer that seems to come with a squared factor -which I'll take as n^2, where n is the number of strings (not the number of distinct comparisons, which is n*(n-1)/2, as +flaschenpost points to ). It would be easier to give you a more precise answer if you'd exactly quote what that answer is.
From what I understand of your question, it isn't, at least it's not the usual sample average. It is, however, a valid estimator of central tendency with the caveat that it is a biased estimator.
See https://en.wikipedia.org/wiki/Bias_of_an_estimator.
Let's define the sample average, which I will denote as X', by
X' = \sum^m_i X_i/N
IF N=m, we get the standard average. In your case, this is the number of distinct pairs which is m=n*(n-1)/2. Let's call this average Xo.
Then if N=n*n, it is
X' = (n-1)/(2*n) Xo
Xo is an unbiased estimator of the population mean \mu. Therefore, X' is biased by a factor f=(n-1)/(2*n). For n very large this bias tends to 1/2.
That said, it could be that the answer you see has a sum that runs not just over distinct pairs. The normalization would then change, of course. For instance, we could extend that sum to all pairs without changing the average value: The correct normalization would then be N = n*(n-1); the value of the average would still be Xo though as the number of summands has double as well.
Those things are getting easier to understand if done by hand with pen and paper for a small example.
If you have the 7 Strings named a,b,c,d,e,f,g, then the simplest version would
Compare a to b, a to c, ... , a to g (this are 6)
Compare b to a, b to c, ... , b to g (this are 6)
. . .
Compare g to a, g to b, ... , g to f (this are 6)
So you have 7*6 or n*(n-1) values, so you divide by nearly 7^2. This is where the square comes from. Maybe you even compare a to a, which should bring a distance of 0 and increase the values to 7*7 or n*n. But I would count it a bit as cheating for the average distance.
You could double the speed of the algorithm, just changing it a small bit
Compare a to b, a to c, ... , a to g (this are 6)
Compare b to c, ... , b to g (this are 5)
Compare c to d, ... , b to g (this are 4)
. . .
Compare f to g (this is 1)
That is following good ol' Gauss 7*6/2, or n*(n-1)/2.
So in Essence: Try doing a simple example on paper and then count your distance values.
Since Average is still and very simply the same as ever:
sum(values) / count(values)
I am new user in R. Could you please tell me or introduce some refrences which describe the tol argument in calculating a QR decomposition in R?
For example what is the difference of this two lines:
qr(A, tol=1e-07) #Doesn't work
qr(A, tol=1e-20) #Works
Why do I get my desired resullt with such a small value of tol, but not with the bigger value?
The tol argument controls whether qr will return a value or not for a column depending on whether the column has been judged to be linearly dependent. I would think that reducing the tol value below 1e-16 would be defeating the purpose of that check. (That's pretty much the pragmatic definition of zero in double precision math.)
First look at qr.default and then find the FORTRAN code:
http://svn.r-project.org/R/trunk/src/appl/dqrdc2.f
This is the comment from the FORTRAN routine that describes the logic:
c cycle the columns from l to p left-to-right until one
c with non-negligible norm is located. a column is considered
c to have become negligible if its norm has fallen below
c tol times its original norm. the check for l .le. k
c avoids infinite cycling.
I'm basically looking for a summation function that will compute multinomials given the number of variables and a degree.
Example
2 Variables; 2 Degrees:
x^2+y^2+x*y+x+y+1
Thanks.
See Knuth The Art of Computer Programming, Vol. 4, Fascicle 3 for a comprehensive answer.
Short answer: it's enough to generate all multinomial expressions in n variables with degree exactly d. Then, for your problem, you can either put together the answers with degrees âĪd, or add a dummy variable "1".
The problem of generating all expressions with degree exactly d is thus simply one of generating all ordered partitions (i.e., all nonnegative integer solutions to x1 + ... + xn = d), and this can be done with a simple backtracking algorithm. ("Depth-first search")
Given N variables, and a maximum degree of D, you have an array of D slots to fill with all possible combinations of variables.
[_, _, ..., _, _]
You are allowed to fill the slots with any of the N variables any number <= D times total. Since multiplication is commutative, it suffices to not care about ordering of variables. As such, this problem is reduced to generating (1) partitions of an integer and (2) subsets of a set.
I hope this is at least a start to your solution.
This also seems to be a Dynamic programming variant of the 0-1 Knapsack problem. Here we would be interested in all possible leaves of the decision tree.