I am new to the concept of Big Theta ( Θ )run-time complexity,
I have the following recurrence relations to analyze,
T(n) = 2T(n/3) + 5n2 and I got Θ(2)
T(n) = T(n/4) + n4 and I got Θ(n4)
Please verify my answers.
Your answers are correct.
You can solve these kind of problems by applying Master Theorem.
The Link is to Master Theorem,
http://en.wikipedia.org/wiki/Master_theorem#Generic_form
If T(n) = a T(n/b) + f(n) where a >= 1 and b > 1
We need to consider case 3 of Master Theorem,
Case 3: if f(n) = Θ(nc) where c > logba
Then T(n) = Θ(nc)
First recurrence
T(n) = 2T(n/3) + 5n2
a = 2, b = 3 and f(n) = 5 n2
There for, f(n) = Θ(nc), where c = 2.
Now c > logba since 2 > log32.
Thus T(n) = Θ(n2) as mentioned by you.
Second Recurrence
T(n) = T(n/4) + n4
a = 1, b = 4 and f(n) = n4
There for, f(n) = Θ(nc), where c = 4.
Now c > logba since 4 > log41.
Thus T(n) = Θ(n4) as mentioned by you.
These are both correct. Because the second term of each recurrence equation is of a much higher order than the first, it will dominate the first term (in layman's terms).
Related
Here I got a recursive function and I want to solve that ( finding the time complexity ) with Substitution Method (mathematical induction) .
𝑇(𝑛) = 2𝑇(𝑛/2) +1
In the question mentioned that our guess should be Ω(log n) . Actually I used mathematical induction to prove that the T(n) = O(n) then because that n = Ω(log n) so the T(n) is too .but I was not successful to prove it correctly .
I have seen all the answers for this function that asked before but they were not solved with Substitution Method . could you help me to prove that in this way .
Your strategy is correct. We prove by induction that T(n) = Theta(n).
We assume w.l.o.g. the initial condition T(n) = 1.
Induction hypothesis: T(n) = 2n - 1.
Base case: T(1) = 1 = 2 * 1 - 1.
Inductive case:
T(n) = 2T(n/2) + 1
= 2 (2n/2 - 1) + 1 (inductive hypothesis)
= 2n - 1 (QED)
Hence T(n) = Theta(n), and from there it follows e.g. that T(n) = Omega(log n) etc.
The running time for this recurrence relation is O(nlogn). Since I am new to algorithm how would I show that mathematically?
T(n) = 2⋅T(n/2) + O(n)
T(n) = 2 ( 2⋅T(n/4) + O(n) ) + O(n) // since T(n/2) = 2⋅T(n/4) + O(n)
So far I can see that if I suppose n to be a power of 2 like n = 2m, then may be I can show that, but I am not getting the clear picture. Can anyone help me?
If you use the master theorem, you get the result you expected.
If you want to proof this "by hand", you can see this easily by supposing n = 2m is a power of 2 (as you already said). This leads you to
T(n) = 2⋅T(n/2) + O(n)
= 2⋅(2⋅T(n/4) + O(n/2)) + O(n)
= 4⋅T(n/4) + 2⋅O(n/2) + O(n)
= 4⋅(2⋅T(n/8) + O(n/4)) + 2⋅O(n/2) + O(n)
= Σk=1,...,m 2k⋅O(n/2k)
= Σk=1,...,m O(n)
= m⋅O(n)
Since m = log₂(n), you can write this as O(n log n).
At the end it doesn't matter if n is a power of 2 or not.
To see this, you can think about this: You have an input of n (which is not a power of 2) and you add more elements to the input until it contains n' = 2m Elements with m ∈ ℕ and log(n) ≤ m ≤ log(n) + 1, i.e. n' is the smalest power of 2 that is greater than n. Obviously T(n) ≤ T(n') holds and we know T(n') is in
O(n'⋅log(n')) = O(c⋅n⋅log(c⋅n)) = O(n⋅log(n) + n⋅log(c)) = O(n⋅log(n))
where c is a constant between 1 and 2.
You can do the same with the greatest power of 2 that is smaller than n. This gives leads you to T(n) ≥ T(n'') and we know T(n'') is in
O(n''⋅log(n'')) = O(c⋅n⋅log(c⋅n)) = O(n⋅log(n))
where c is a constant between 1/2 and 1.
In total you get, that the complexity of T(n) is bounded by the complexitys of T(n'') and T(n') wich are both O(n⋅log(n))and so T(n) is also in O(n⋅log(n)), even if it is not a power of 2.
In our Data Structures class we are learning how to solve recurrence relations in 1 variable. Unfortunately some things seem to come "out of the blue".
For example, some exercises already tell you how to substitute the variable n:
Compute T(n) for n = 2^k
T(n) = a for n =< 2
T(n) = 8T(n/2) + bn^2 (a and b are > 0)
But some exercises just give you the T(n) without providing a replacement for the variable n:
T(n) = 1 n =<1
T(n) = 2T(n/4) + sqrt(n)
I used the iterative method and arrived to the right answer: sqrt(n) + (1/2) * sqrt(n) * Log(n).
But when the professor explained she started by saying: "Let n = 4^k", which is what I mean by "out of the blue". Using that fact the answer is simpler to obtain.
But how is the student supposed to come up with that?
This is another example:
T(n) = 1 n =<1
T(n) = 2T( (n-1)/2 ) + n
Here I started again with the iterative method but I can't reach a definitive answer, it looks more complex that way.
After 3 iterative steps I arrived to this:
T(n) = 4T( (n-2)/4 ) + 2n - 1
T(n) = 8T( (n-3)/8 ) + 3n - 3
T(n) = 16T( (n-4)/16 ) + 4n - 6
I am inclined to say T(i) = 2^i * T( (n-i)/2^i ) + i*n - ? This last part I can't figure out, maybe I made a mistake.
However in the answer she provides she starts again with another substitution: Let n = (2^k) -1. I don’t see where this comes from - why would I do this? What is the logic behind that?
In all of these cases, these substitutions are reasonable because they rewrite the recurrence as one of the form S(k) = aS(k - 1) + f(k). These recurrences are often easier to solve than other recurrences because they define S(k) purely in terms of S(k - 1).
Let's do some examples to see how this works. Consider this recurrence:
T(n) = 1 (if n ≤ 1)
T(n) = 2T(n/4) + sqrt(n) (otherwise)
Here, the size of the problem shrinks by a factor of four on each iteration. Therefore, if the input is a perfect power of four, then the input will shrink from size 4k to 4k-1, from 4k-1 to 4k-2, etc. until the recursion bottoms out. If we make this substitution and let S(k) = T(4k), then we get hat
S(0) = 1
S(k) = 2S(k - 1) + 2k
This is now a recurrence relation where S(k) is defined in terms of S(k - 1), which can make the recurrence easier to solve.
Let's look at your original recurrence:
T(n) = a (for n ≤ 2)
T(n) = 8T(n/2) + bn2
Notice that the recursive step divides n by two. If n is a perfect power of two, then the recursive step considers the power of two that comes right before n. Letting S(k) = T(2k) gives
S(k) = a (for k ≤ 1)
S(k) = 8S(k - 1) + b22k
Notice how that S(k) is defined in terms of S(k - 1), which is a much easier recurrence to solve. The choice of powers of two was "natural" here because it made the recursive step talk purely about the previous value of S and not some arbitrarily smaller value of S.
Now, look at the last recurrence:
T(n) = 1 (n ≤ 1)
T(n) = 2T( (n-1)/2 ) + n
We'd like to make some substitution k = f(n) such that T(f(n)) = 2T(f(n) - 1) + n. The question is how to do that.
With some trial and error, we get that setting f(n) = 2n - 1 fits the bill, since
(f(n) - 1) / 2 = ((2n - 1) - 1) / 2 = (2n - 2) / 2 = 2n-1 - 1 = f(n) - 1
Therefore, letting k = 2n - 1 and setting S(k) = T(2n - 1), we get
S(n) = 1 (if n ≤ 1)
S(n) = 2S(n - 1) + 2n - 1
Hope this helps!
How would I go about solving this kind of recurrence using the Master Theorem?
T(n) = 4T(n/2) + n2 + logn
I have no idea how to go about doing this, but I'm pretty sure it is possible to solve it using Master Theorem. Do I have to ignore one of the terms? Any help is appreciated, thanks.
The Master Theorem works for functions that can be written as
T(n) = aT(n / b) + f(n)
Here, you have that a = 4, b = 2, and f(n) = n2 + log n. Notice that we're grouping "n2 + log n" together as the f(n) term, rather than treating it as two separate terms.
Now that we've done that, we can apply the Master Theorem directly. Notice that logb a = log2 4 = 2 and that f(n) = Θ(n2), so by the Master Theorem this solves to Θ(n2 log n). The reason this works is that n2 + log n = Θ(n2), and the Master Theorem only cares about the asymptotic complexity of f(n). In fact, any of these recurrences can be solved the same way:
T(n) = 4T(n / 2) + n2 + 137n + 42
T(n) = 4T(n / 2) + 5n2 + 42n log n + 42n + 5 log n + 106
T(n) = 4T(n / 2) + 0.5n2 + n log137 n + n log n + n2 / log n + 5
Hope this helps!
On my midterm I had the problem:
T(n) = 8T(n/2) + n^3
and I am supposed to find its big theta notation using either the masters or alternative method. So what I did was
a = 8, b = 2 k = 3
log28 = 3 = k
therefore, T(n) is big theta n3. I got 1/3 points so I must be wrong. What did I do wrong?
T(n) = aT(n/b) + f(n)
You applied the version when f(n) = O(n^(log_b(a) - e)) for some e > 0.
This is important, you need this to be true for some e > 0.
For f(n) = n^3, b = 2 and a = 8,
n^3 = O(n^(3-e)) is not true for any e > 0.
So your picked the wrong version of the Master theorem.
You need to apply a different version of Master theorem:
if f(n) = Theta ((log n)^k * n^log_b(a)) for some k >= 0,
then
T(n) = Theta((log n)^(k+1) * n^log_b(a))
In your problem, you can apply this case, and that gives T(n) = Theta(n^3 log n).
An alternative way to solve your problem would be:
T(n) = 8 T(n/2) + n^3.
Let g(n) = T(n)/n^3.
Then
n^3 *g(n) = 8 * (n/2)^3 * g(n/2)+ n^3
i.e g(n) = g(n/2) + 1.
This implies g(n) = Theta(logn) and so T(n) = Theta(n^3 logn).