Related
I have spent a lot of time to learn about implementing/visualizing dynamic programming problems using iteration but I find it very hard to understand, I can implement the same using recursion with memoization but it is slow when compared to iteration.
Can someone explain the same by a example of a hard problem or by using some basic concepts. Like the matrix chain multiplication, longest palindromic sub sequence and others. I can understand the recursion process and then memoize the overlapping sub problems for efficiency but I can't understand how to do the same using iteration.
Thanks!
Dynamic programming is all about solving the sub-problems in order to solve the bigger one. The difference between the recursive approach and the iterative approach is that the former is top-down, and the latter is bottom-up. In other words, using recursion, you start from the big problem you are trying to solve and chop it down to a bit smaller sub-problems, on which you repeat the process until you reach the sub-problem so small you can solve. This has an advantage that you only have to solve the sub-problems that are absolutely needed and using memoization to remember the results as you go. The bottom-up approach first solves all the sub-problems, using tabulation to remember the results. If we are not doing extra work of solving the sub-problems that are not needed, this is a better approach.
For a simpler example, let's look at the Fibonacci sequence. Say we'd like to compute F(101). When doing it recursively, we will start with our big problem - F(101). For that, we notice that we need to compute F(99) and F(100). Then, for F(99) we need F(97) and F(98). We continue until we reach the smallest solvable sub-problem, which is F(1), and memoize the results. When doing it iteratively, we start from the smallest sub-problem, F(1) and continue all the way up, keeping the results in a table (so essentially it's just a simple for loop from 1 to 101 in this case).
Let's take a look at the matrix chain multiplication problem, which you requested. We'll start with a naive recursive implementation, then recursive DP, and finally iterative DP. It's going to be implemented in a C/C++ soup, but you should be able to follow along even if you are not very familiar with them.
/* Solve the problem recursively (naive)
p - matrix dimensions
n - size of p
i..j - state (sub-problem): range of parenthesis */
int solve_rn(int p[], int n, int i, int j) {
// A matrix multiplied by itself needs no operations
if (i == j) return 0;
// A minimal solution for this sub-problem, we
// initialize it with the maximal possible value
int min = std::numeric_limits<int>::max();
// Recursively solve all the sub-problems
for (int k = i; k < j; ++k) {
int tmp = solve_rn(p, n, i, k) + solve_rn(p, n, k + 1, j) + p[i - 1] * p[k] * p[j];
if (tmp < min) min = tmp;
}
// Return solution for this sub-problem
return min;
}
To compute the result, we starts with the big problem:
solve_rn(p, n, 1, n - 1)
The key of DP is to remember all the solutions to the sub-problems instead of forgetting them, so we don't need to recompute them. It's trivial to make a few adjustments to the above code in order to achieve that:
/* Solve the problem recursively (DP)
p - matrix dimensions
n - size of p
i..j - state (sub-problem): range of parenthesis */
int solve_r(int p[], int n, int i, int j) {
/* We need to remember the results for state i..j.
This can be done in a matrix, which we call dp,
such that dp[i][j] is the best solution for the
state i..j. We initialize everything to 0 first.
static keyword here is just a C/C++ thing for keeping
the matrix between function calls, you can also either
make it global or pass it as a parameter each time.
MAXN is here too because the array size when doing it like
this has to be a constant in C/C++. I set it to 100 here.
But you can do it some other way if you don't like it. */
static int dp[MAXN][MAXN] = {{0}};
/* A matrix multiplied by itself has 0 operations, so we
can just return 0. Also, if we already computed the result
for this state, just return that. */
if (i == j) return 0;
else if (dp[i][j] != 0) return dp[i][j];
// A minimal solution for this sub-problem, we
// initialize it with the maximal possible value
dp[i][j] = std::numeric_limits<int>::max();
// Recursively solve all the sub-problems
for (int k = i; k < j; ++k) {
int tmp = solve_r(p, n, i, k) + solve_r(p, n, k + 1, j) + p[i - 1] * p[k] * p[j];
if (tmp < dp[i][j]) dp[i][j] = tmp;
}
// Return solution for this sub-problem
return dp[i][j];;
}
We start with the big problem as well:
solve_r(p, n, 1, n - 1)
Iterative solution is only to, well, iterate all the states, instead of starting from the top:
/* Solve the problem iteratively
p - matrix dimensions
n - size of p
We don't need to pass state, because we iterate the states. */
int solve_i(int p[], int n) {
// But we do need our table, just like before
static int dp[MAXN][MAXN];
// Multiplying a matrix by itself needs no operations
for (int i = 1; i < n; ++i)
dp[i][i] = 0;
// L represents the length of the chain. We go from smallest, to
// biggest. Made L capital to distinguish letter l from number 1
for (int L = 2; L < n; ++L) {
// This double loop goes through all the states in the current
// chain length.
for (int i = 1; i <= n - L + 1; ++i) {
int j = i + L - 1;
dp[i][j] = std::numeric_limits<int>::max();
for (int k = i; k <= j - 1; ++k) {
int tmp = dp[i][k] + dp[k+1][j] + p[i-1] * p[k] * p[j];
if (tmp < dp[i][j])
dp[i][j] = tmp;
}
}
}
// Return the result of the biggest problem
return dp[1][n-1];
}
To compute the result, just call it:
solve_i(p, n)
Explanation of the loop counters in the last example:
Let's say we need to optimize the multiplication of 4 matrices: A B C D. We are doing an iterative approach, so we will first compute the chains with the length of two: (A B) C D, A (B C) D, and A B (C D). And then chains of three: (A B C) D, and A (B C D). That is what L, i and j are for.
L represents the chain length, it goes from 2 to n - 1 (n is 4 in this case, so that is 3).
i and j represent the starting and ending position of the chain. In case L = 2, i goes from 1 to 3, and j goes from 2 to 4:
(A B) C D A (B C) D A B (C D)
^ ^ ^ ^ ^ ^
i j i j i j
In case L = 3, i goes from 1 to 2, and j goes from 3 to 4:
(A B C) D A (B C D)
^ ^ ^ ^
i j i j
So generally, i goes from 1 to n - L + 1, and j is i + L - 1.
Now, let's continue with the algorithm assuming that we are at the step where we have (A B C) D. We now need to take into account the sub-problems (which are already calculated): ((A B) C) D and (A (B C)) D. That is what k is for. It goes through all the positions between i and j and computes the sub problems.
I hope I helped.
The problem with recursion is the high number of stack frames that need to be pushed/popped. This can quickly become the bottle-neck.
The Fibonacci Series can be calculated with iterative DP or recursion with memoization. If we calculate F(100) in DP all we need is an array of length 100 e.g. int[100] and that's the guts of our used memory. We calculate all entries of the array pre-filling f[0] and f[1] as they are defined to be 1. and each value just depends on the previous two.
If we use a recursive solution we start at fib(100) and work down. Every method call from 100 down to 0 is pushed onto the stack, AND checked if it's memoized. These operations add up and iteration doesn't suffer from either of these. In iteration (bottom-up) we already know all of the previous answers are valid. The bigger impact is probably the stack frames; and given a larger input you may get a StackOverflowException for what was otherwise trivial with an iterative DP approach.
Is the big(O) of the following recursive code simply O(n choose k)?
int nchoosek(int n, int k) {
if (n == k) return 1;
if (k == 0) return 1;
return nchoosek(n-1, k) + nchoosek(n-1, k-1);
}
I'm not sure if the notation is correct but I guess you can write the recurrence for this function like
T(n) = T(n-1, k) + T(n-1, k-1) + O(1)
Since you have two possible paths we just have to analyse the worst case of each and choose the slowest.
Worst case of T(n-1, k)
Given that 0<k<n and k is as far as possible from n then we have
T(n-1, k) = T(n-1) = O(n)
Worst case of T(n-1, k-1)
We need 0<k<n and k should be as close to n as possible. Then
T(n-1, k-1) = T(n-1) = O(n)
Therefore T(n, k) = 2*O(n) + O(1) = O(n)
Another way of seeing this is by reducing the problem to other known problems, for example you can solve the same problem by using the definition of the choose function in terms of the Factorial function:
nCk = n!/k!(n-k)!
The running time of the Factorial is O(n) even in the recursive case.
nCk requires calculating the Factorial three times:
n! => O(n)
k! => O(k) = O(n)
(n-k)! => O(n-k) = O(n)
Then the multiplication and division are both constant time O(1), hence the running time is:
T(n) = 3*O(n) + 2*O(1) = O(n)
what is the fastest method to calculate this, i saw some people using matrices and when i searched on the internet, they talked about eigen values and eigen vectors (no idea about this stuff)...there was a question which reduced to a recursive equation
f(n) = (2*f(n-1)) + 2 , and f(1) = 1,
n could be upto 10^9....
i already tried using DP, storing upto 1000000 values and using the common fast exponentiation method, it all timed out
im generally weak in these modulo questions, which require computing large values
f(n) = (2*f(n-1)) + 2 with f(1)=1
is equivalent to
(f(n)+2) = 2 * (f(n-1)+2)
= ...
= 2^(n-1) * (f(1)+2) = 3 * 2^(n-1)
so that finally
f(n) = 3 * 2^(n-1) - 2
where you can then apply fast modular power methods.
Modular exponentiation by the square-and-multiply method:
function powerMod(b, e, m)
x := 1
while e > 0
if e%2 == 1
x, e := (x*b)%m, e-1
else b, e := (b*b)%m, e//2
return x
C code for calculating 2^n
const int mod = 1e9+7;
//Here base is assumed to be 2
int cal_pow(int x){
int res;
if (x == 0) res=1;
else if (x == 1) res=2;
else {
res = cal_pow(x/2);
if (x % 2 == 0)
res = (res * res) % mod;
else
res = (((res*res) % mod) * 2) % mod;
}
return res;
}
I have a RSA private key with modulus m, public exponent e and private exponent d, but the program I am using needs the modulus's prime factors p and q.
Is it possible to use e and d to get p and q?
Yes -- once you know the modulus N, and public/private exponents d and e, it is not too difficult to obtain p and q such that N=pq.
This paper by Dan Boneh describes an algorithm for doing so. It relies
on the fact that, by definition,
de = 1 mod phi(N).
For any randomly chosen "witness"
in (2,N), there is about a 50% chance of being able to use it to find a nontrivial
square root of 1 mod N (call it x). Then gcd(x-1,N) gives one of the factors.
You can use the open source tool I have developed in 2009 that converts RSA keys between the SFM format (n,e,d) and CRT format (p,q,dp,dq,u), and the other way around. It is on SourceForge : http://rsaconverter.sourceforge.net/
The algorithm I implemented is based on ideas presented by Dan Boneh, as described by the previous answer.
I hope this will be useful.
Mounir IDRASSI - IDRIX
I posted a response on the crypto stack exchange answering the same question here. It uses the same approach as outlined in Boneh's paper, but does a lot more explanation as to how it actually works. I also try to assume a minimal amount of prior knowledge.
Hope this helps!
I put in the effort to dig through Boneh's paper. The "algorithm" for deriving (p, q) from (n, d) is buried at the end of §1.1, coded in maths jargon, and left as an exercise for the reader to render out of his (rather terse) proof that it's efficient to do so.
Let 〈N, e〉 be an RSA public key. Given the private key d, one can efficiently factor the modulus N = pq.
Proof. Compute k = de − 1. By definition of d and e we know that k is a multiple of φ(N). Since φ(N) is even, k = 2tr with r odd and t ≥ 1. We have gk = 1 for every g ∈ ℤN×, and therefore gk/2 is a square root of unity modulo N. By the Chinese Remainder Theorem, 1 has four square roots modulo N = pq. Two of these square roots are ±1. The other two are ±x where x satisfies x = 1 mod p and x = −1 mod q. Using either one of these last two square roots, the factorization of N is revealed by computing gcd(x − 1, N). A straightforward argument shows that if g is chosen at random from ℤN× then with probability at least 1/2 (over the choice of g) one of the elements in the sequence gk/2, gk/4, …, gk/2t mod N is a square root of unity that reveals the factorization of N. All elements in the sequence can be efficiently computed in time O(n3) where n = log2(N).
Obviously, this is pretty close to meaningless for anyone who doesn't know what $Z_N^\ast$ is, and has a pretty nonlinear structure that takes a good deal of time to twist into a linear algorithm.
So here is the worked solution:
from random import randrange
from math import gcd
def ned_to_pqe(secret_key):
"""
https://crypto.stanford.edu/~dabo/papers/RSA-survey.pdf#:~:text=Given%20d%2C,reveals%20the%20factorization%20of%20N%2E
"""
n, e, d = secret_key
k = d * e - 1
t = bit_scan1(k)
trivial_sqrt1 = {1, n - 1}
while True:
g = randrange(2, n - 1)
for j in range(1, t + 1):
x = pow(g, k >> j, n)
if pow(x, 2, n) == 1:
if x in trivial_sqrt1: continue
p = gcd(x - 1, n)
q = n // p
if q > p: p, q = q, p
return p, q, e
def pqe_to_ned(secret_key):
p, q, e = secret_key
n = p * q
l = (p - 1) * (q - 1)
d = pow(e, -1, l)
return n, e, d
def bit_scan1(i):
"""
https://gmpy2.readthedocs.io/en/latest/mpz.html#mpz.bit_scan1
"""
# https://stackoverflow.com/a/63552117/1874170
return (i & -i).bit_length() - 1
def test():
secret_key = (
# https://en.wikipedia.org/wiki/RSA_numbers#RSA-100
# Should take upwards of an hour to factor on a consumer desktop ca. 2022
1522605027922533360535618378132637429718068114961380688657908494580122963258952897654000350692006139,
65537,
1435319569480661473883310243084583371347212233430112391255270984679722445287591616684593449660400673
)
if secret_key != pqe_to_ned(ned_to_pqe(secret_key)):
raise ValueError
if __name__ == '__main__':
test()
print("Self-test OK")
Live demo (JS):
function ned_to_pqe({n, e, d}) {
// https://crypto.stanford.edu/~dabo/papers/RSA-survey.pdf#:~:text=Given%20d%2C,reveals%20the%20factorization%20of%20N%2E
let k = d * e - 1n;
let t = scan1(k);
let trivial_sqrt1 = new Set([1n, n - 1n]);
while (true) {
let g = insecure_randrange(2n, n - 1n);
for ( let j = t ; j > 0 ; --j ) {
let x = bn_powMod(g, k >> j, n);
if (bn_powMod(x, 2n, n) === 1n) {
if (trivial_sqrt1.has(x)) continue;
let p = gcd(x - 1n, n), q = n/p;
if (q > p) [p, q] = [q, p];
return {p, q, e};
}
}
}
}
function pqe_to_ned({p, q, e}) {
let n = p * q;
let l = (p - 1n) * (q - 1n);
let d = bn_modInv(e, l);
return {n, e, d};
}
function bn_powMod(x, e, m) {
// h/t https://umaranis.com/2018/07/12/calculate-modular-exponentiation-powermod-in-javascript-ap-n/
if (m === 1n) return 0n;
let y = 1n;
x = x % m;
while (e > 0n) {
if (e % 2n === 1n) //odd number
y = (y * x) % m;
e = e >> 1n; //divide by 2
x = (x * x) % m;
}
return y;
}
function bn_modInv(x, m) {
// TOY IMPLEMENTATION
// DO NOT USE IN GENERAL-PURPOSE CODE
// h/t https://rosettacode.org/wiki/Modular_inverse#C
let m0 = m, t, q;
let x0 = 0n, y = 1n;
if (m === 1n) return 1n;
while (x > 1n) {
q = x / m;
t = m;
m = x % m;
x = t;
t = x0;
x0 = y - q * x0;
y = t;
}
if (y < 0n) y += m0;
return y;
}
function gcd(a, b) {
// h/t https://stackoverflow.com/a/17445304/1874170
while (b) {
[a, b] = [b, a % b];
}
return a;
}
function scan1(i) {
// https://gmplib.org/manual/Integer-Logic-and-Bit-Fiddling#mpz_scan1
let k = 0n;
if ( i !== 0n ) {
while( (i & 1n) === 0n ) {
i >>= 1n;
k += 1n;
}
}
return k;
}
function insecure_randrange(a, b) {
// h/t https://arxiv.org/abs/1304.1916
let numerator = 0n;
let denominator = 1n;
let n = (b - a);
while (true) {
numerator <<= 1n;
denominator <<= 1n;
numerator |= BigInt(Math.random()>1/2);
if (denominator >= n) {
if (numerator < n)
return a + numerator;
numerator -= n;
denominator -= n;
}
}
}
<form action="javascript:" onsubmit="(({target:form,submitter:{value:action}})=>{eval(action)(form)})(event)">
<p>
<label for="p">p=</label><input name="p" value="37975227936943673922808872755445627854565536638199" /><br />
<label for="q">q=</label><input name="q" value="40094690950920881030683735292761468389214899724061" /><br />
<label for="n">n=</label><input name="n" /><br />
<label for="e">e=</label><input name="e" placeholder="65537" /><br />
<label for="d">d=</label><input name="d" /><br />
</p>
<p>
<button type="submit" value="pqe2nd">Get (n,d) from (p,q,e)</button><br />
<button type="submit" value="delpq">Forget (p,q)</button><br />
<button type="submit" value="ned2pq">Get (p,q) from (n,e,d)</button>
</form>
<script>
function pqe2nd({elements}) {
if (!elements['e'].value) elements['e'].value = elements['e'].placeholder;
let p = BigInt(elements['p'].value||undefined);
let q = BigInt(elements['q'].value||undefined);
let e = BigInt(elements['e'].value||undefined);
let {n, d} = pqe_to_ned({p,q,e});
elements['n'].value = n.toString();
elements['d'].value = d.toString();
}
function ned2pq({elements}) {
if (!elements['e'].value) elements['e'].value = elements['e'].placeholder;
let n = BigInt(elements['n'].value||undefined);
let e = BigInt(elements['e'].value||undefined);
let d = BigInt(elements['d'].value||undefined);
let {p, q} = ned_to_pqe({n,e,d});
elements['p'].value = p.toString();
elements['q'].value = q.toString();
}
function delpq({elements}) {
elements['p'].value = null;
elements['q'].value = null;
}
</script>
To answer the question as-stated in the title: factoring N entails finding N. But you cannot, in the general case, derive N from (e, d). Therefore, you cannot, in the general case, derive the factors of N from (e, d); QED.
finding n from (e, d) is computationally feasible with fair probability, or even certainty, for a small but observable fraction of RSA keys of practical interest
If you want to try to do so anyway, you'll need to be able to factorize e * d - 1 (if I understand the above-linked answer correctly):
from itertools import permutations
def ed_to_pq(e, d):
# NOT ALWAYS POSSIBLE -- the number e*d-1 must be small enough to factorize
# h/t https://crypto.stackexchange.com/a/81620/8287
factors = factorize(e * d - 1)
factors.sort()
# Unimplemented optimization:
# if two factors are larger than (p * q).bit_length()//4
# and the greater of (p, q) is not many times bigger than the lesser,
# then you can safely assume that the large factors belong to (p-1) and (q-1)
# and thereby reduce the number of iterations in the following loops
# Unimplemented optimization:
# permutations are overkill for this partitioning scheme;
# a clever mathematician could come up with something more efficient
# Unimplemented optimization:
# prune permutations based on "sanity" factor of logarithm knapsacking
l = len(factors)
for arrangement in permutations(factors):
for l_pm1 in range(1, l - 1):
for l_qm1 in range(1, l_pm1):
pm1 = prod(arrangement[:l_pm1])
qm1 = prod(arrangement[l_pm1:l_pm1+l_qm1])
try:
if pow(e, -1, pm1 * qm1) == d:
return (pm1 + 1, qm1 + 1)
except Exception:
pass
from functools import reduce
from operator import mul
def prod(l):
return reduce(mul, l)
I wrote a code segment to determine the longest path in a graph. Following is the code. But I don't know how to get the computational complexity in it because of the recursive method in the middle. Since finding the longest path is an NP complete problem I assume it's something like O(n!) or O(2^n), but how can I actually determine it?
public static int longestPath(int A) {
int k;
int dist2=0;
int max=0;
visited[A] = true;
for (k = 1; k <= V; ++k) {
if(!visited[k]){
dist2= length[A][k]+longestPath(k);
if(dist2>max){
max=dist2;
}
}
}
visited[A]=false;
return(max);
}
Your recurrence relation is T(n, m) = mT(n, m-1) + O(n), where n denotes number of nodes and m denotes number of unvisited nodes (because you call longestPath m times, and there is a loop which executes the visited test n times). The base case is T(n, 0) = O(n) (just the visited test).
Solve this and I believe you get T(n, n) is O(n * n!).
EDIT
Working:
T(n, n) = nT(n, n-1) + O(n)
= n((n-1)T(n, n-2) + O(n)) + O(n) = ...
= n(n-1)...1T(n, 0) + O(n)(1 + n + n(n-1) + ... + n(n-1)...2)
= O(n)(1 + n + n(n-1) + ... + n!)
= O(n)O(n!) (see http://oeis.org/A000522)
= O(n*n!)