I am trying to understand Big-O notation through a book I have and it is covering Big-O by using functions although I am a bit confused. The book says that O(g(n)) where g(n) is the upper bound of f(n). So I understand that means that g(n) gives the max rate of growth for f(n) at larger values of n.
and that there exists an n_0 where the rate of growth of cg(n) (where c is some constant) and f(n) have the same rate of growth.
But what I am confused is on these examples on finding Big O in mathmatical functions.
This book says findthe upper bound for f(n) = n^4 +100n^2 + 50
they then state that n^4 +100n^2 + 50 <= 2n^4 (unsure why the 2n^4)
then they some how find n_0 =11 and c = 2, I understand why the big O is O(n^4) but I am just confused about the rest.
This is all discouraging as I don't understand but I feel like this is an important topic that I must understand.
If any one is curious the book is Data Structures and Algorithms Made Easy by Narasimha Karumanchi
Not sure if this post belongs here or in the math board.
Preparations
First, lets state, loosely, the definition of f being in O(g(n)) (note: O(g(n)) is a set of functions, so to be picky, we say that f is in O(...), rather than f(n) being in O(...)).
If a function f(n) is in O(g(n)), then c · g(n) is an upper bound on
f(n), for some constant c such that f(n) is always ≤ c · g(n),
for large enough n (i.e. , n ≥ n0 for some constant n0).
Hence, to show that f(n) is in O(g(n)), we need to find a set of constants (c, n0) that fulfils
f(n) < c · g(n), for all n ≥ n0, (+)
but this set is not unique. I.e., the problem of finding the constants (c, n0) such that (+) holds is degenerate. In fact, if any such pair of constants exists, there will exist an infinite amount of different such pairs.
Showing that f ∈ O(n^4)
Now, lets proceed and look at the example that confused you
Find an upper asymptotic bound for the function
f(n) = n^4 + 100n^2 + 50 (*)
One straight-forward approach is to express the lower-order terms in (*) in terms of the higher order terms, specifically, w.r.t. bounds (... < ...).
Hence, we see if we can find a lower bound on n such that the following holds
100n^2 + 50 ≤ n^4, for all n ≥ ???, (i)
We can easily find when equality holds in (i) by solving the equation
m = n^2, m > 0
m^2 - 100m - 50 = 0
(m - 50)^2 - 50^2 - 50 = 0
(m - 50)^2 = 2550
m = 50 ± sqrt(2550) = { m > 0, single root } ≈ 100.5
=> n ≈ { n > 0 } ≈ 10.025
Hence, (i) holds for n ≳ 10.025, bu we'd much rather present this bound on n with a neat integer value, hence rounding up to 11:
100n^2 + 50 ≤ n^4, for all n ≥ 11, (ii)
From (ii) it's apparent that the following holds
f(n) = n^4 + 100n^2 + 50 ≤ n^4 + n^4 = 2 · n^4, for all n ≥ 11, (iii)
And this relation is exactly (+) with c = 2, n0 = 11 and g(n) = n^4, and hence we've shown that f ∈ O(n^4). Note again, however, that the choice of constants c and n0 is one of convenience, that is not unique. Since we've shown that (+) holds for on set of constants (c,n0), we can show that it holds for an infinite amount of different such choices of constants (e.g., it naturally holds for c=10 and n0=20, ..., and so on).
Related
The question is:
Show or disprove that for two functions f,g, if f is not in O(g) then g is in O(f).
My counterexample:
Let f(n) be f(n) = n^2 : if n is even
or n^4 : if n is odd
Let g(n) be g(n) = n^3
This is an example for f not in O(g) and g not in O(f).
Is my example wrong? If so, why?
Do you have any other examples?
Your counterexample works. A proof might look like this:
Suppose f were O(g). Then there is a positive constant c and an n0 such that for n >= n0, f(n) <= c * g(n). Let n' be an odd integer greater than or equal to n0. Then we have n'^4 <= c * n'^3. Dividing both sides by n'^3 gives n' <= c. However, this cannot be true for all n' > n0; so there are even n greater than n0 for which the condition does not hold, a contradiction.
The proof the other way around is similar, except you divide both sides by n'^2.
I think the kind of counterexample you identified is a good one for this; a function that oscillates by an asymptotically increasing amount and a function that goes somewhere in the middle.
I want to implement a simproc capable of rewriting the argument of sin into a linear combination x + k * pi + k' * pi / 2 (where ideally k' = 0 or k' = 1) and then apply existing lemmas about additions of arguments in sines.
The steps could be as follows:
Pattern match the goal to extract the argument of sin(expr):
fun dest_sine t =
case t of
(#{term "(sin):: real ⇒ real"} $ t') => t'
| _ => raise TERM ("dest_sine", [t]) ;
Prove that for some x, k, k': expr = x + k*pi + k' * pi/2.
Use existing lemmas to rewrite to a simpler trigonometric function:
fun rewriter x k k' =
if (k mod 2 = 0 andalso k' = 0) then #{term "sin"} $ x
else if (k mod 2 = 0 andalso k' = 1) then #{term "cos"} $ x
else if (k mod 2 = 1 andalso k' = 0) then #{term "-sin"} $ x
else #{term "-cos"} $ x
I'm stuck at step two. The idea is to use algebra simplifications to obtain the x,k,k' where the theorem holds. I believe schematic goals should do this but I haven't ever used them.
My thoughts
Could I rather assume that the expression is of this form and let the simplifier find it so that the simproc can be triggered?
If I first start assuming the linear form x + k*pi + k' * pi/2 then:
Extract x,k,k' from this combination.
Apply rewriter and obtain the corresponding term to be rewritten two.
Apply in a sequence: rules dealing with + pi/2, rules dealing with + 2 pi
I would start easy and ignore the pi / 2 part for now.
You probably want to build a simproc that matches on anything of the form sin x. Then you want to write a conversion that takes that term x (which is assumed to be a sum of several terms) and brings it into the form a + of_int b * p.
A conversion is essentially a function of type cterm → thm which takes a cterm ct and returns a theorem of the form ct ≡ …, i.e. it's a form of deterministic rewriting (a conversion can also fail by throwing a CTERM exception, by convention). There are a lot of combinators for building and using these in Pure/conv.ML.
This is probably a bit fiddly. You essentially have to descend through the term and, for each atom (i.e. anything not of the form _ + _) you have to figure out whether it can be brought into the form of_int … * pi (e.g. again by writing a conversion that does this transformation – to make it easy you can omit this part so that your procedure only works if the terms are already in that form) and then group all the terms of the form of_int … * pi to the right and all the terms not of that form to the left using associativity and commutativity.
I would suggest this:
Define a function SIN_SIMPROC_ATOM x n = x + of_int n * pi
Write a conversion sin_atom_conv that rewrites of_int n * pi to SIN_SIMPROC_ATOM 0 n and everything else into SIN_SIMPROC_ATOM x 0
Write a conversion that descends through +, applies sin_atom_conv to every atom, and then applies some kind of combination rule like SIN_SIMPROC_ATOM x1 n1 + SIN_SIMPROC_ATOM x2 n2 = SIN_SIMPROC_ATOM (x1 + x2) (n1 + n2)
In the end, you have rewritten your entire form to the form sin (SIN_SIMPROC_ATOM x n), and then you can apply some suitable rule to that.
It's not quite clear to me how to best handle the parity of n. You could rewrite sin (SIN_SIMPROC_ATOM x n) = (-1) ^ nat ¦n¦ * sin x but I'm not sure if that's what the user really wants in most cases. It might make more sense to only do that if you can deduce the parity of n statically (e.g. by using the simplifier) and then directly simplify to sin x or -sin x.
The situation becomes even more complicated if you want to include halves of π. You can of course extend SIN_SIMPROC_ATOM by a second term for halves of π (and one for doubles of π as well to make it more uniform). Or you could ad all of them together so that you just have a single integer n that describes your multiples of π/2, and k multiples of π simply contribute 2k to that term. And then you have to figure out what n mod 4 is – possibly again with the simplifier or with some clever static method.
I'm trying to find time complexity (big O) of a recursive formula.
I tried to find a solution, you may see the formula and my solution below:
Like Brenner said, your last assumption is false. Here is why: Let's take the definition of O(n) from the Wikipedia page (using n instead of x):
f(n) = O(n) if and only if there exist constants c, n0 s.t. |f(n)| <= c |g(n)|, for alln >= n0.
We want to check if O(2^n^2) = O(2^n). Clearly, 2^n^2 is in O(2^n^2), so let's pick f(n) = 2^n^2 and check if this is in O(2^n). Put this into the above formula:
exists c, n0: 2^n^2 <= c * 2^n for all n >= n0
Let's see if we can find suitable constant values n0 and c for which the above is true, or if we can derive a contradiction to proof that it is not true:
Take the log on both sides:
log(2^n^2) <= log(c * 2 ^ n)
Simplify:
2 ^n log(2) <= log(c) + n * log(2)
Divide by log(2):
n^2 <= log(c)/log(2) * n
It's easy to see know that there is no c, n0 for which the above is true for all n >= n0, thus O(2^n^2) = O(n^2) is not a valid assumption.
The last assumption you've specified with the question mark is false! Do not make such assumptions.
The rest of the manipulations you've supplied seem to be correct. But they actually bring you nowhere.
You should have finished this exercise in the middle of your draft:
T(n) = O(T(1)^(3^log2(n)))
And that's it. That's the solution!
You could actually claim that
3^log2(n) == n^log2(3) ==~ n^1.585
and then you get:
T(n) = O(T(1)^(n^1.585))
which is somewhat similar to the manipulations you've made in the second part of the draft.
So you can also leave it like this. But you cannot mess with the exponent. Changing the value of the exponent changes the big-O classification.
Can you prove using reflexivity that f(n) equals big Theta(f(n))? It seems straight forward when thinking about it because f(n) is bounded above and below by itself. But how will I write this down? And does this apply to big Omega and big O
I believe what you are intending to ask is (w.r.t. #emory:s answer) is something along the lines:
"For some function f(n), is it true that f ∈ ϴ(f(n))?"
If you emanate from the formal definition of Big-ϴ notation, it is quite apparent that this holds.
f ∈ ϴ(g(n))
⇨ For some positive constants c1, c2, and n0, the following holds:
c1 · |g(n)| ≤ |f(n)| ≤ c2 · |g(n)|, for all n ≥ n0 (+)
Let f(n) be some arbitrary real-valued function. Set g(n) = f(n) and choose, e.g., c1=0.5, c2=2, and n0 = 1. Then, naturally, (+) holds:
0.5 · |f(n)| ≤ |f(n)| ≤ 2 · |f(n)|, for all n ≥ 1
Hence, f ∈ ϴ(f(n)) holds.
No we can not because it is not true. ϴ(f(n)) is a set. f(n) is a member of that set. f(n)+1 is also a member of that set.
I read in a data structures book complexity hierarchy diagram that n is greater than 2log n. But cannot understand how and why. On using simple examples in power of 2 as n, I get values equal to n.
It is not mentioned in book , but I am assuming it to base 2 ( as context is DS complexity)
a) Is O(n) > O(pow(2,logn))?
b) Is O(pow(2,log n)) better than O(n)?
Notice that 2logb n = 2log2 n / log2 b = n(1 / log2 b). If log2 b ≥ 1 (that is, b ≥ 2), then this entire expression is strictly less than n and is therefore O(n). If log2 b < 1 (that is, b < 2), then this expression is of the form n1 + ε and therefore not O(n). Therefore, it boils down to what the log base is. If b ≥ 2, then the expression is O(n). If b < 2, then the expression is ω(n).
Hope this helps!
There is a constant factor in there somewhere, but it's not in the right place to make O(n) equal to O(pow(2,log n)), assuming log means the natural logarithm.
n = 2 ** log2(n) // by definition of log2, the base-2 logarithm
= 2 ** (log(n)/log(2)) // standard conversion of logs from one base to another
n ** log(2) = 2 ** log(n) // raise both sides of that to the log(2) power
Since log(2) < 1, O(n ** log(2)) < O(n ** 1). Sure, there is only a constant ratio between the exponents, but the fact remains that they are different exponents. O(n ** 3) is greater than O(n ** 2) for the same reason: even though 3 is bigger than 2 by only a constant factor, it is bigger and the Orders are different.
We therefore have
O(n) = O(n ** 1) > O(n ** log(2)) = O(2 ** log(n))
Just like in the book.