I want to show that the recursion of quicksort run on best time time on n log n.
i got this recursion formula
M(0) = 1
M(1) = 1
M(n) = min (0 <= k <= n-1) {M(K) + M(n - k - 1)} + n
show that M(n) >= 1/2 (n + 1) lg(n + 1)
what i have got so far:
By induction hyposes
M(n) <= min {M(k) + M(n - k - 1} + n
focusing on the inner expresison i got:
1/2(k + 1)lg(k + 1) + 1/2(n - k)lg(n - k)
1/2lg(k + 1)^(k + 1) + 1/2lg(n - k)^(n - k)
1/2(lg(k + 1)^(k + 1) + lg(n - k)^(n - k)
1/2(lg((k + 1)^(k + 1) . (n - k)^(n - k))
But i think im doing something wrong. i think the "k" should be gonne but i cant see how this equation would cancel out all the "k". So, probably, im doing something wrong
You indeed want to get rid of k. To do this, you want to find the lower bound on the minimum of M(k) + M(n - k - 1). In general it can be arbitrarily tricky, but in this case the standard approach works: take derivative by k.
((k+1) ln(k+1) + (n-k) ln(n-k))' =
ln(k+1) + (k+1)/(k+1) - ln(n-k) - (n-k)/(n-k) =
ln((k+1) / (n-k))
We want the derivative to be 0, so
ln((k+1) / (n-k)) = 0 <=>
(k+1) / (n-k) = 1 <=>
k + 1 = n - k <=>
k = (n-1) / 2
You can check that it's indeed a local minimum.
Therefore, the best lower bound on M(k) + M(n - k - 1) (which we can get from the inductive hypothesis) is reached for k=(n-1)/2. Now you can just substitute this value instead of k, and n will be your only remaining variable.
Related
I'm trying to find the big O of T(n) = 2T(n/2) + 1. I figured out that it is O(n) with the Master Theorem but I'm trying to solve it with recursion and getting stuck.
My solving process is
T(n) = 2T(n/2) + 1 <= c * n
(we know that T(n/2) <= c * n/2)
2T(n/2) + 1 <= 2 * c * n/2 +1 <= c * n
Now I get that 1 <= 0.
I thought about saying that
T(n/2) <= c/2 * n/2
But is it right to do so? I don't know if I've seen it before so I'm pretty stuck.
I'm not sure what you mean by "solve it with recursion". What you can do is to unroll the equation.
So first you can think of n as a power of 2: n = 2^k.
Then you can rewrite your recurrence equation as T(2^k) = 2T(2^(k-1)) + 1.
Now it is easy to unroll this:
T(2^k) = 2 T(2^(k-1)) + 1
= 2 (T(2^(k-2) + 1) + 1
= 2 (2 (T(2^(k-3) + 1) + 1) + 1
= 2 (2 (2 (T(2^(k-4) + 1) + 1) + 1) + 1
= ...
This goes down to k = 0 and hits the base case T(1) = a. In most cases one uses a = 0 or a = 1 for exercises, but it could be anything.
If we now get rid of the parentheses the equation looks like this:
T(n) = 2^k * a + 2^k + 2^(k-1) + 2^(k-2) + ... + 2^1 + 2^0
From the beginning we know that 2^k = n and we know 2^k + ... + 2 + 1 = 2^(k+1) -1. See this as a binary number consisting of only 1s e.g. 111 = 1000 - 1.
So this simplifies to
T(n) = 2^k * a + 2^(k+1) - 1
= 2^k * a + 2 * 2^k - 1
= n * a + 2 * n - 1
= n * (2 + a) - 1
And now one can see that T(n) is in O(n) as long as a is a constant.
I have a Computer Science Midterm tomorrow and I need help determining the complexity of a particular recursive function as below, which is much complicated than the stuffs I've already worked on: it has two variables
T(n) = 3 + mT(n-m)
In simpler cases where m is a constant, the formula can be easily obtained by writing unpacking the relation; however, in this case, unpacking doesn't make the life easier as follows (let's say T(0) = c):
T(n) = 3 + mT(n-m)
T(n-1) = 3 + mT(n-m-1)
T(n-2) = 3 + mT(n-m-2)
...
Obviously, there's no straightforward elimination according to these inequalities. So, I'm wondering whether or not I should use another technique for such cases.
Don't worry about m - this is just a constant parameter. However you're unrolling your recursion incorrectly. Each step of unrolling involves three operations:
Taking value of T with argument value, which is m less
Multiplying it by m
Adding constant 3
So, it will look like this:
T(n) = m * T(n - m) + 3 = (Step 1)
= m * (m * T(n - 2*m) + 3) + 3 = (Step 2)
= m * (m * (m * T(n - 3*m) + 3) + 3) + 3 = ... (Step 3)
and so on. Unrolling T(n) up to step k will be given by following formula:
T(n) = m^k * T(n - k*m) + 3 * (1 + m + m^2 + m^3 + ... + m^(k-1))
Now you set n - k*m = 0 to use the initial condition T(0) and get:
k = n / m
Now you need to use a formula for the sum of geometric progression - and finally you'll get a closed formula for the T(n) (I'm leaving that final step to you).
Question 5 on Determining complexity for recursive functions (Big O notation) is:
int recursiveFun(int n)
{
for(i=0; i<n; i+=2)
// Do something.
if (n <= 0)
return 1;
else
return 1 + recursiveFun(n-5);
}
To highlight my question, I'll change the recursive parameter from n-5 to n-2:
int recursiveFun(int n)
{
for(i=0; i<n; i+=2)
// Do something.
if (n <= 0)
return 1;
else
return 1 + recursiveFun(n-2);
}
I understand the loop runs in n/2 since a standard loop runs in n and we're iterating half the number of times.
But isn't the same also happening for the recursive call? For each recursive call, n is decremented by 2. If n is 10, call stack is:
recursiveFun(8)
recursiveFun(6)
recursiveFun(4)
recursiveFun(2)
recursiveFun(0)
...which is 5 calls (i.e. 10/2 or n/2). Yet the answer provided by Michael_19 states it runs in n-5 or, in my example, n-2. Clearly n/2 is not the same as n-2. Where have I gone wrong and why is recursion different from iteration when analyzing for Big-O?
Common way to analyze big-O of a recursive algorithm is to find a recursive formula that "counts" the number of operation done by the algorithm. It is usually denoted as T(n).
In your example: the time complexity of this code can be described with the formula:
T(n) = C*n/2 + T(n-2)
^ ^
assuming "do something is constant Recursive call
Since it's pretty obvious it will be in O(n^2), let's show Omega(n^2) using induction:
Induction Hypothesis:
T(k) >= C/8 *k^2 for 0 <= k < n
And indeed:
T(n) = C*n/2 + T(n-2) >= (i.h.) C*n/2 + C*(n-2)^2 / 8
= C* n/2 + C/8(n^2 - 4n + 2) =
= C/8 (4n + n^2 - 4n + 2) =
= C/8 *(n^2 + 2)
And indeed:
T(n) >= C/8 * (n^2 + 2) > C/8 * n^2
Thus, T(n) is in big-Omega(n^2).
Showing big-O is done similarly:
Hypothesis: T(k) <= C*k^2 for all 2 <= k < n
T(n) = C*n/2 + T(n-2) <= (i.h.) C*n/2 + C*(n^2 - 4n + 4)
= C* (2n + n^2 - 4n + 4) = C (n^2 -2n + 4)
For all n >= 2, -2n + 4 <= 0, so for any n>=2:
T(n) <= C (n^2 - 2n + 4) <= C^n^2
And the hypothesis is correct - and by definition of big-O, T(n) is in O(n^2).
Since we have shown T(n) is both in O(n^2) and Omega(n^2), it is also in Theta(n^2)
Analyzing recursion is different from analyzing iteration because:
n (and other local variable) change each time, and it might be hard to catch this behavior.
Things get way more complex when there are multiple recursive calls.
Please verify my logic to see if what I'm attempting is valid or shady.
W(n) = W(n/2) + nlg(n)
W(1) = 1
n =2^k
By trying the pattern
line 1 : W (2^k) = W(2^k-1) + nlgn
line 2 : = W(2^k-2) + nlgn + nlgn
...
line i : = W(2^k-i) + i*nlgn
and then solve the rest for i.
I just want to make sure it's cool if I substitute in 2k in one place (on line 1) but not in the other for the n lg n.
I find by subbing in 2^k for 2^k lg(2^k) gets really greasy.
Any thoughts are welcome (specifically if I should be subbing in 2^k, and if I should then how would you suggest the solution)
It's fine to switch back and forth between n and 2k as needed because you're assuming that n = 2k. However, that doesn't mean that what you have above is correct. Remember that as n decreases, the value of n log n keeps decreasing as well, so it isn't the case that the statement
line i = W(2k-i) + i * n lg n
is true.
Let's use the iteration method one more time:
W(n) = W(n / 2) + n log n
= (W(n / 4) + (n/2) log (n/2)) + n log n
= W(n / 4) + (n/2) (log n - 1) + n log n
= W(n / 4) + n log n / 2 - n / 2 + n log n
= W(n / 4) + (1 + 1/2) n log n - n / 2
= (W(n / 8) + (n / 4) log(n/4)) + (1 + 1/2) n log n - n / 2
= W(n / 8) + (n / 4) (log n - 2) + (1 + 1/2) n log n - n / 2
= W(n / 8) + n log n / 4 - n / 2 + (1 + 1/2) log n - n / 2
= W(n / 8) + (1 + 1/2 + 1/4) n log n - n
= (W(n / 16) + (n / 8) log(n/8)) + (1 + 1/2 + 1/4) n log n - n
= W(n / 16) + (n / 8) (log n - 3)) + (1 + 1/2 + 1/4) n log n - n
= W(n / 16) + n log n / 8 - 3n / 8 + (1 + 1/2 + 1/4) n log n - n
= W(n / 16) + (1 + 1/2 + 1/4 + 1/8) n log n - n - 3/8n
We can look at this to see if we spot a pattern. First, notice that the n log n term has a coefficient of (1 + 1/2 + 1/4 + 1/8 + ...) as we keep expanding outward. This series converges to 2, so when the iteration stops that term will be between n log n and 2n log n. Next, look at the coefficient of the -n term. If you look closely, you'll notice that this coefficient is -1 times
(1 / 2 + 2 / 4 + 3 / 8 + 4 / 16 + 5 / 32 + ... )
This is the sum of i / 2i, which converges to 2. Therefore, if we iterate this process, we'll find at each step that the value is Θ(n log n) - Θ(n), so the overall recurrence solves to Θ(n log n).
Hope this helps!
I am having trouble understanding the concept of recurrences. Given you have T(n) = 2T(n/2) +1 how do you calculate the complexity of this relationship? I know in mergesort, the relationship is T(n) = 2T(n/2) + cn and you can see that you have a tree with depth log2^n and cn work at each level. But I am unsure how to proceed given a generic function. Any tutorials available that can clearly explain this?
The solution to your recurrence is T(n) ∈ Θ(n).
Let's expand the formula:
T(n) = 2*T(n/2) + 1. (Given)
T(n/2) = 2*T(n/4) + 1. (Replace n with n/2)
T(n/4) = 2*T(n/8) + 1. (Replace n with n/4)
T(n) = 2*(2*T(n/4) + 1) + 1 = 4*T(n/4) + 2 + 1. (Substitute)
T(n) = 2*(2*(2*T(n/8) + 1) + 1) + 1 = 8*T(n/8) + 4 + 2 + 1. (Substitute)
And do some observations and analysis:
We can see a pattern emerge: T(n) = 2k * T(n/2k) + (2k − 1).
Now, let k = log2 n. Then n = 2k.
Substituting, we get: T(n) = n * T(n/n) + (n − 1) = n * T(1) + n − 1.
For at least one n, we need to give T(n) a concrete value. So we suppose T(1) = 1.
Therefore, T(n) = n * 1 + n − 1 = 2*n − 1, which is in Θ(n).
Resources:
https://www.cs.duke.edu/courses/spring05/cps100/notes/slides07-4up.pdf
http://www.cs.duke.edu/~ola/ap/recurrence.html
However, for routine work, the normal way to solve these recurrences is to use the Master theorem.