For solving rectangular linear systems Ax=b, where A is mxn and n>m, Matlab performs first a QR factorization then solves a triangular linear system by substitution. For example:
if A is a 4x6 matrix:
A =
0.8147 0.9058 0.1270 0.9134 0.6324 0.0975
0.2785 0.5469 0.9575 0.9649 0.1576 0.9706
0.9572 0.4854 0.8003 0.1419 0.4218 0.9157
0.7922 0.9595 0.6557 0.0357 0.8491 0.9340
and
b=
-0.9661
0.1590
-0.0391
-0.2491
to solve Ax=b we simply do:
x=A\b;
and the solution is:
x =
-0.4284
-0.6475
0
-0.1153
0
0.7662
The underlying steps in this computation are:
QR decomposition of A, then Ax=bwould be equivalent to Q*R*x=b.
As inv(Q)=Q'and R is upper triangular, solving the system would return to solving R*x=b1 where b1=Q'*b.
As R is upper triangular of size 6x4, we have to perform a back-substitution. How does Matlab perform the back substitution knowing that R is not square?
I'm porting this computation to C and have succesfully done it until R*x=b1, I'm lost with the back substitution and I want to find the same result as Matlab.
Edit:
For the above matrix A:
Solving Ax=bis equivalent to solving Rx=b1where:
R =
-1.5117 -1.3991 -1.0952 -0.7786 -1.0819 -1.3007
0 -0.5641 -0.2197 -0.6538 -0.2920 -0.2481
0 0 -0.8692 -0.2077 0.1422 -0.9295
0 0 0 -0.8426 0.2182 0.2125
and
b1 =
-0.9661
0.1590
-0.0391
-0.2491
This system is solved by back-substitution as R is upper triangular, this is straightforward in case of square triangular matrices (i.e m=n), However, R is not square. How does Matlab perform the back-substitution to find x?
So I'm not 100% sure what MATLAB does, but I can tell you the following:
For your underdetermined matrix A (nxm), we can take A' = QR. We also see that A = R'Q'. Note that now, R has n nonzero rows. We can then solve the system by recognizing R'Q'x=b, then Q'x=inv(R')*b. The RHS can be solved by back-substitution, we'll call it y. So Q'x=y. But Q is self-Hermitian, so x=Qy. And there you have the least squares solution.
Doing this in MATLAB we get:
>> [Q,R] = qr(A')
Q =
-0.4918 0.2143 -0.6131 -0.5675 -0.1086 0.0503
-0.5468 0.0638 0.0596 0.5238 -0.5922 -0.2614
-0.0767 -0.6389 -0.1919 -0.0733 0.2275 -0.7014
-0.5514 -0.2397 0.6743 -0.3550 0.1568 0.1821
-0.3818 0.2094 -0.1834 0.4750 0.7368 0.0904
-0.0589 -0.6637 -0.3090 0.2158 -0.1351 0.6291
R =
-1.6565 -1.1588 -1.0907 -1.3634
0 -1.3597 -0.8286 -0.6385
0 0 -0.9760 -0.9745
0 0 0 0.5972
0 0 0 0
0 0 0 0
>> Q*(R'\b)
ans =
-0.4256
-0.3057
0.3568
-0.2745
-0.2823
0.4249
>>
I used backslash above for R'\b, but it's clear you can back-substitute in your own routine.
We can verify this by computing the solution using the pseudo-inverse of A:
>> A'*inv(A*A')*b
ans =
-0.4256
-0.3057
0.3568
-0.2745
-0.2823
0.4249
You can easily check that this is a valid solution of course. This should be readily implementable in C.
Hope this helps.
How MATLAB implements in details the under-determined case only Mathworks know! But a way you can do it is as follow.
A*x = b
A' = Q*R
Now if you look at R you will have something of the form
R = [R1; zeros]
where R1 is square.
you can express the solution as:
x = Q * [inv(R1')*b; zeros]
as you indicates that you already solved the problem of inverting a square matrix that should be easy :)
Related
On this website it states for the calculation of "d" you must use the formula e.d = 1 Mod Phi. I don't quite understand as I believed 1 Mod anything will result in 1 (or 0 in case 1 Mod 1). This simplifies the formula to e.d = 1 which clearly is not correct with the given test values :
e = 17 , Phi = 3120 , d = 2753.
I would appreciate some clarification on how the RSA calculates the decryption key.
Wikipedia says this is a Modular multiplicative inverse. The formula should be read as e * d equals 1, when the result is expressed modulo Phi, or (e*d) % Phi == 1 in pseudocode.
Right now I use the package FFTW in order to get some Fourier Transforms I am interested in. However, I'm wondering if there is already a package of FFT that can do the transformation in a vector space which is of the form kron(C2, Rn), where C2 means a 2x2 system and Rn represents the "spatial" subspace in which one is interested in getting the Fourier Transform. In other words, does it exist a routine that implements:
kron(Id2x2, FFT)[kron(C2, Rn)] = kron(C2, FFT(Rn))
Of course the real problem I am interested is in the "two particle case" where the vector space (Hilbert space) is kron(kron(C2, Rn),kron(C2, Rn)), so in this case the routine would need an operator like kron(kron(Id2x2, FFT), kron(Id2x2, FFT)).
Note 1: I haven't tried to do the problem taking partial traces, but in my case this option simply may not work because the states are sparse, i.e. it might be ineficient.
Note 2: Note that (unless I'm mistaken) for kron(C2, Rn) one could do "twice" the fft (one in each sector of C2). However this might also be ineficient for large vector spaces.
Here's an example of what I think you are asking. res is computed by FFT from mat = kron(C2, Rn), and this is (as you say) a wasteful way of doing kron(C2, fft(Rn)) since it the FFT along the k dimension is re-done for each of the 2×2 other dimensions. But the point, presumably, is to do this for "entangled" states in the product space -- a generic likemat = rand(8,2) cannot be decomposed into factors kron(likeC2, likeRn).
(If instead you are really only interested in "un-entangled" product states, then you should probably just work with their components. Combining with kron will then always be wasteful. The package Kronecker.jl may help for some things, but I don't think it knows about fft.)
This uses my package to handle kron-like operations; you could just write out the necessary reshapes yourself, too.
julia> C2 = [1 2; 3 4]; Rn = [1,10,0,0];
julia> mat = kron(C2,Rn)
8×2 Matrix{Int64}:
1 2
10 20
0 0
0 0
3 4
30 40
0 0
0 0
julia> using TensorCast, FFTW
# notation: kron is a reshape of a tensor product, to combine i & k
julia> kron(C2,Rn) == #cast out[(k,i),j] := C2[i,j] * Rn[k]
true
# reshape mat to put the index from Rn in its own dimension:
julia> #cast tri[k,i,j] := mat[(k,i),j] (i in 1:2);
julia> summary(tri)
"4×2×2 Array{Int64, 3}"
# then fft(tri, 1) is the FFT along only that, reshape back:
julia> #cast res[(ktil,i),j] := fft(tri, 1)[ktil,i,j]
8×2 Matrix{ComplexF64}:
11.0+0.0im 22.0+0.0im
1.0-10.0im 2.0-20.0im
-9.0+0.0im -18.0+0.0im
1.0+10.0im 2.0+20.0im
33.0+0.0im 44.0+0.0im
3.0-30.0im 4.0-40.0im
-27.0+0.0im -36.0+0.0im
3.0+30.0im 4.0+40.0im
julia> res ≈ kron(C2, fft(Rn))
true
julia> res ≈ fft(mat, 1)
false
julia> fft(Rn)
4-element Vector{ComplexF64}:
11.0 + 0.0im
1.0 - 10.0im
-9.0 + 0.0im
1.0 + 10.0im
# if fft() understood the dims keyword, it could be tidier:
julia> _fft(x; dims) = fft(x, dims);
julia> #cast _res[(k,i),j] := _fft(k) mat[(k,i),j] (i in 1:2);
julia> _res ≈ res
true
x=[-4 -2; 3 1] [x y] + [-t -2t-1]
x(0)=[3 -5]
As I know firstly, when the matrix is denoted by A, we must compute exp(At) by diagonalizing A: if A=PDP^(-1) for a diagonal D then exp(At) = P exp(Dt) P^(-1) where exp(Dt) is a diagonal matrix with (exp(Dt))_{ii} = exp(D_(ii) t)...
How can I write The Maple code? ( Sorry, I write in here since I didn't find maple.stackexchange)
restart: with(LinearAlgebra):
A := Matrix(2,2,[-4,-2,3,1]);
....
I know I need to use the extended euclidean algorithm, but I'm not sure exactly what calculations I need to do. I have huge numbers. Thanks
Well, d is chosen such that d * e == 1 modulo (p-1)(q-1), so you could use the Euclidean algorithm for that (finding the modular multiplicative inverse).
If you are not interested in understanding the algorithm, you can just call BigInteger#modInverse directly.
d = e.modInverse(p_1.multiply(q_1))
Given that, p=11, q=7, e =17, n=77, φ (n) = 60 and d=?
First substitute values from the formula:-
ed mod φ (n) =1
17 d mod 60 = 1
The next step: – take the totient of n, which is 60 to your left hand side and [e] to your right hand side.
60 = 17
3rd step: – ask how many times 17 goes to 60. That is 3.5….. Ignore the remainder and take 3.
60 = 3(17)
Step 4: – now you need to balance this equation 60 = 3(17) such that left hand side equals to right hand side. How?
60 = 3(17) + 9 <== if you multiply 3 by 17 you get 51 then plus 9, that is 60. Which means both sides are now equal.
Step 5: – Now take 17 to your left hand side and 9 to your right hand side.
17 = 9
Step 6:- ask how many times 9 goes to 17. That is 1.8…….
17 = 1(9)
Step 7:- Step 4: – now you need to balance this 17 = 1(9)
17 = 1(9) + 8 <== if you multiply 1 by 9 you get 9 then plus 8, that is 17. Which means both sides are now equal.
Step 8:- again take 9 to your left hand side and 8 to your right hand side.
9 = 1(8)
9 = 1(8) + 1 <== once you reached +1 to balance your equation, you may stop and start doing back substitution.
Step A:-Last equation in step 8 which is 9 = 1(8) + 1 can be written as follows:
1.= 9 – 1(8)
Step B:-We know what is (8) by simple saying 8 = 17 – 1(9) from step 7. Now we can re-write step A as:-
1=9 -1(17 – 1(9)) <== here since 9=1(9) we can re-write as:-
1=1(9)-1(17) +1(9) <== group similar terms. In this case you add 1(9) with 1(9) – that is 2(9).
1=2(9)-1(17)
Step C: – We know what is (9) by simple saying 9 = 60 – 3(17) from step 4. Now we can re-write step B as:-
1=2(60-3(17) -1(17)
1=2(60)-6(17) -1(17) <== group similar terms. In this case you add 6(17) with 1(17) – that is 7(17).
1=2(60)-7(17) <== at this stage we can stop, nothing more to substitute, therefore take the value next 17. That is 7. Subtract it with the totient.
60-7=d
Then therefore the value of d= 53.
I just want to augment the Sidudozo's answer and clarify some important points.
First of all, what should we pass to Extended Euclidean Algorthim to compute d ?
Remember that ed mod φ(n) = 1 and cgd(e, φ(n)) = 1.
Knowing that the Extended Euclidean Algorthim is based on the formula cgd(a,b) = as + bt, hence cgd(e, φ(n)) = es + φ(n)t = 1, where d should be equal to s + φ(n) in order to satisfy the
ed mod φ(n) = 1 condition.
So, given the e=17 and φ(n)=60 (borrowed from the Sidudozo's answer), we substitute the corresponding values in the formula mentioned above:
cgd(e, φ(n)) = es + φ(n)t = 1 ⇔ 17s + 60t = 1.
At the end of the Sidudozo's answer we obtain s = -7. Thus d = s + φ(n) ⇔ d = -7 + 60 ⇒ d = 53.
Let's verify the results. The condition was ed mod φ(n) = 1.
Look 17 * 53 mod 60 = 1. Correct!
The approved answer by Thilo is incorrect as it uses Euler's totient function instead of Carmichael's totient function to find d. While the original method of RSA key generation uses Euler's function, d is typically derived using Carmichael's function instead for reasons I won't get into. The math needed to find the private exponent d given p q and e without any fancy notation would be as follows:
d = e^-1*mod(((p-1)/GCD(p-1,q-1))(q-1))
Why is this? Because d is defined in the relationship
de = 1*mod(λ(n))
Where λ(n) is Carmichael's function which is
λ(n)=lcm(p-1,q-1)
Which can be expanded to
λ(n)=((p-1)/GCD(p-1,q-1))(q-1)
So inserting this into the original expression that defines d we get
de = 1*mod(((p-1)/GCD(p-1,q-1))(q-1))
And just rearrange that to the final formula
d = e^-1*mod(((p-1)/GCD(p-1,q-1))(q-1))
More related information can be found here.
Here's the code for it, in python:
def inverse(a, n):
t, newt = 0, 1
r, newr = n, a
while newr:
quotient = r // newr # floor division
t, newt = newt, t - quotient * newt
r, newr = newr, r - quotient * newr
if r > 1:
return None # there's no solution
if t < 0:
t = t + n
return t
inverse(17, 60) # returns 53
adapted from pseudocode found in wiki: https://en.wikipedia.org/wiki/Extended_Euclidean_algorithm#Pseudocode
Simply use this formula,
d = (1+K(phi))/e. (Very useful when e and phi are small numbers)
Lets say, e = 3 and phi = 40
we assume K = 0, 1, 2... until your d value is not a decimal
assume K = 0, then
d = (1+0(40))/3 = 0. (if it is a decimal increase the K value, don't bother finding the exact value of the decimal)
assume K = 2, then
d = (1+2(40)/3) = 81/3 = 27
d = 27.
Assuming K will become exponentially easy with practice.
Taken the values p=7, q=11 and e=17.
then the value of n=p*q=77 and f(n)=(p-1)(q-1)=60.
Therefore, our public key pair is,(e,n)=(7,77)
Now for calvulating the value of d we have the constraint,
e*d == 1 mod (f(n)), [here "==" represents the **congruent symbol**].
17*d == 1 mod 60
(17*53)*d == 53 mod 60, [7*53=901, which gives modulus value 1]
1*d == 53 mod 60
hence,this gives the value of d=53.
Therefore our private key pair will be, (d,n)=(53,77).
Hope this help. Thank you!
I'm trying to figure out how to implement RSA crypto from scratch (just for the intellectual exercise), and i'm stuck on this point:
For encryption, c = me mod n
Now, e is normally 65537. m and n are 1024-bit integers (eg 128-byte arrays). This is obviously too big for standard methods. How would you implement this?
I've been reading a bit about exponentiation here but it just isn't clicking for me:
Wikipedia-Exponentiation by squaring
This Chapter (see section 14.85)
Thanks.
edit: Also found this - is this more what i should be looking at? Wikipedia- Modular Exponentiation
Exponentiation by squaring:
Let's take an example. You want to find 1723. Note that 23 is 10111 in binary. Let's try to build it up from left to right.
// a exponent in binary
a = 17 //17^1 1
a = a * a //17^2 10
a = a * a //17^4 100
a = a * 17 //17^5 101
a = a * a //17^10 1010
a = a * 17 //17^11 1011
a = a * a //17^22 10110
a = a * 17 //17^23 10111
When you square, you double the exponent (shift left by 1 bit). When you multiply by m, you add 1 to the exponent.
If you want to reduce modulo n, you can do it after each multiplication (rather than leaving it to the end, which would make the numbers get very large).
65537 is 10000000000000001 in binary which makes all of this pretty easy. It's basically
a = m
repeat 16 times:
a = a * a
a = a mod n
a = a * m
a = a mod n
where of course a, n and m are "big integers". a needs to be at least 2048 bits as it can get as large as (n-1)2.
For an efficient algorithm you need to combine the exponentiation by squaring with repeated application of mod after each step.
For odd e this holds:
me mod n = m ⋅ me-1 mod n
For even e:
me mod n = (me/2 mod n)2 mod n
With m1 = m as a base case this defines a recursive way to do efficient modular exponentiation.
But even with an algorithm like this, because m and n will be very large, you will still need to use a type/library that can handle integers of such sizes.
result = 1
while e>0:
if (e & 1) != 0:
result = result * m
result = result mod n
m = m*m
m = m mod n
e = e>>1
return result
This checks bits in the exponent starting with the least significant bit. Each time we move up a bit it corresponds to doubling the power of m - hence we shift e and square m. The result only gets the power of m multiplied in if the exponent has a 1 bit in that position. All multiplications need to be reduced mod n.
As an example, consider m^13. 11 = 1101 in binary. so this is the same as m^8 * m^4 * m. Notice the powers 8,4,(not 2),1 which is the same as the bits 1101. And then recall that m^8 = (m^4)^2 and m^4 = (m^2)^2.
If g(x) = x mod 2^k is faster to calculate for your bignum library than f(x) = x mod N for N not divisible by 2, then consider using Montgomery multiplication. When used with modular exponentiation, it avoids having to calculate modulo N at each step, you just need to do the "Montgomeryization" / "un-Montgomeryization" at the beginning and end.