Big oh notation says that all g(n) are an element c.f(n), O(g(n)) for some constant c.
I have always wondered and never really understood why we need this arbitrary constant to multiply with the bounding function f(n) to get our bounds?
Also how does one decide what number this constant should be?
The constant itself doesn't characterize the limiting behavior of the f(n) compared to g(n).
It is used for the mathematical definition, which enforces the existence of a constant M such that
If such a constant exists then you can state that f(x) is an O(g(x)), and this is the usual notation when analyzing algorithms, you just don't care about which is the constant but just the complexity of operations itself. The constant is able make that disequation correct by ensuring that M|g(x)| is an upper bound of f(x).
How to find that constant depends on f(x) and g(x) and it is the mathematical point that must be proved to ensure that f(x) has a g(x) big-o so there's not a general rule. Look at this example.
Consider function
f(n) = 4 * n
Doesn't it make sense to call this function O(n) since it grows "as fast" as g(n) = n.
But without constant in definition of O you can't find n0 such as that for all n > n0, f(n) <= n. That's why you need constant, and indeed from condition,
4 * n <= c * n for all n > n0
you can get n0 == 0, c == 4.
Related
I am trying to find all roots [f(x) = 0] in a function. My current solution only works if they are spaced out enough and don't interfer with each other. (e.g. it works for x^2 - 2)
bool numberIsCloseToZero(num number){
return (num.parse(number.abs().toStringAsFixed(1)) == 0.0) ? true : false;
}
List<num> calculateRoots(String function){
num eval = 0.0;
List<num> roots = [];
for (num x = -10; x < 10; x += 0.1){
eval = calculateYOfX(function, x);
if (numberIsCloseToZero(num.parse(eval.toStringAsFixed(2)))){
roots.add(x);
}
}
return roots;
}
Obviously, this is due to my rounding. (e.g. the surrounding values of the root of x^2 are too close to zero, so it assumes they are roots as well). Do you think I should go through actually solving the equation instead of "brute forcing" the roots?
Thanks
If you can find analytical solution - use it. It is possible for low-degree polynomial equations (like mentioned x^2 - 2).
In general case - you definitely have to learn numerical methods - in this case, root finding.
Start with bisection method or Newthon's method. They allow to get more and more exact position of root at every step.
You'll need to put some restrictions on what is an allowable function, otherwise you have no hope.
For example without any restrictions you've no guarantee that there are is only a finite number of values (consider f(x)=sin(x) ), or even a finite number of values in a given interval (consider f(x)=x sin(1/x) ). Or even an infinity of connected zeros ( f(x) = max(0,x) )
And these cases are not even considered particularly pathological mathematical functions.
If you're willing to go down the path of requiring your function to be non-zero almost-everywhere, smooth, continuous and with bounded first and second derivatives then I think you may be able to come up with a relatively simple algorithm that guarantees you get all zeros in a given finite region.
(I'd look for a subdivision based algorithm which recursively splits the region and determines strict bounds on each interval.)
We can derive an example algorithm for when the derivative is bounded by a known constant i.e. |f'(x)| < D. Note that if we evaluate f at some point p then for any other point p+d we can show that f(p) - |d| D < f(p+d) < f(p) + |d| D.
Using this we can consider root finding in an interval [A,B] - which we can write as [p-d, p+d] where p=(A+B)/2, d=(B-A)/2. Sample f at the mid-point to get f(p). The minimum value f could take on the interval is f(p) - d D and the maximum value is f(p) + d D. We can only have a root in this interval if f(p)-d D <= 0 <= f(p) +d D which is equivalent to |f(p)| < d D.
If there can be no root in [A,B] we're done, otherwise we repeat on the two halves [A,p] and [p,B]. (some care needs to be taken in the case f(p)=0 )
I want to know, why this is O(n2) for 1+2+3+...+n?
For example, 1+2+3+4 = 4·(4+1)/2 = 10 but 42=16, so how come it's O(n2)?
In Big-O notation you forget about constant factors.
In your example S(n) = 1+2+...+n = n·(n+1)/2 is in O(n2) since you can find a constant number c with
S(n) < c · n2 for all n > n0
(just choose c = 1).
Notice: Big-O notation is an upper bound, i.e. S(n) grows not faster than n2.
Notice also, that S(n) also grows obviously not faster than n3 so it is also in O(n3).
Some additional:
You can also proof the other way around that n2 is in O(S(n)).
n2 < c·S(n) = c·n·(n+1)/2 holds for any c ≥ 2 for all n
So n2 is in O(S(n)). This means both functions grow asymtoticly equal. You can wirt this as S(n) is in Θ(n2).
When computing O(n) n(n-1)/2 is the same as n^2 since it has the highest complexity
1+2+3+...+n Sum of this equation is n(n+1)/2
Means it is n^2+n/2 therefore O(n^2).
Complexity,Big O will look at the biggest factor in the equation because logically it is the one that takes most time.( think of it in a computer program)
An algorithm runs in polynomial time if it's runtime is O(nk) for some k. However, I've also seen polynomial time defined as time nO(1).
I have some questions about this:
Why is nO(1) polynomial time? What happened to k?
If nO(1) is polynomial time, then 3n2 should be nO(1). But where did the 3 go? How does that work?
Thanks!
When you have an expression like "the runtime is O(n)" or "the runtime is O(n2)," the O(n) and O(n2) terms aren't actual functions. Instead, they're placeholders for some other function with some property. For example, take this statement:
The runtime of the algorithm is O(n)
This statement really means
There is some function f(n) where the runtime of the algorithm is f(n) and f(n) = O(n)
For example, if a function's actual runtime is 137n + 42, the statement "the runtime of the algorithm is O(n)" is true because there is some function (namely, f(n) = 137n + 42) where the runtime of the algorithm is f(n) and f(n) = O(n).
Given this, let's think about what the statement "the runtime of the algorithm is nO(1)" means. This statement is equivalent to
There is some function f(n) where the runtime of the algorithm is nf(n) and f(n) = O(1)
Now that we've gotten the terminology clearer, what exactly does this mean? Intuitively, a function is O(1) if it's eventually bounded from above by some constant. Therefore, any function f(n) that's O(1) must satisfy f(n) ≤ k once n gets sufficiently large. Therefore, at least intuitively, nO(1) means "n raised to some power that's at most k," which sounds like the definition of a polynomial function.
Of course, there's that pesky issue of constant factors. The function 137n3 is definitely O(n3), but it has a huge constant term in front. On the other hand, if we have a function of the form nO(1), there isn't a constant term in front of the n3. How do we handle this?
This is where we can get cute with the math. In the case of 137n3, note that when n > 1, we have
137n3 = nlogn137 n3 = n3 + logn 137
Notice that this is n raised to the power of logn 137. Although it might look like the function logn 137 grows as n grows larger, it actually has the opposite behavior: it decreases as n grows. The reason for this is that we can use the change of base formula to rewrite logn 137 as
logn 137 = log 137 / log n
Which clearly decreases in the long term when log n decreases. Therefore, the expression 3 + logn137 ends up being bounded from above by some constant, so it's O(1).
Using this technique, it's possible to convert O(nk) to nO(1) by choosing the exponent of n to be k plus the log base n of the constant factor in front of the nk term that comes up in the big-O notation. Similarly, we can convert back from nO(1) to O(nk) by choosing k to be any constant that upper-bounds the function hidden by the O(1) term in the exponent of n.
Hope this helps!
The purpose of the following code is to convert a polynomial from coefficient representation into value representation by dividing it into its odd and even powers and then recursing on the smaller polynomials.
function FFT(A, w)
Input: Coefficient representation of a polynomials A(x) of degree ≤ n-1, where n
is a power of 2w, an nth root of unity.
Output: Value representation A(w^0),...,A(w^(n-1))
if w = 1; return A(1)
express A(x) in the form A_e(x^2) and xA_o(x^2) /*where A_e are the even powers and A_o
the odd.*/
call FFT(A_e,w^2) to evaluate A_e at even of powers of w
call FFT(A_o,w^2) to evaluate A_o at even powers of w
for j = 0 to n-1;
compute A(w^j) = A_e(w^(2j))+w^j(A_o(w^(2j)))
return A(w^0),...,A(w^(n-1))
What is the for loop being used for?
Why is the pseudocode only adding the smaller polynomials, doesn't it need to subtract them too? (to calculate A(-x)). Isn't that what the algorithm completely based on? Adding and subtracting the smaller polynomials to reduce the points in half?*
Why are powers of "w" being evaluated as opposed to "x"?
I am not a too sure if this belongs here, since the question is quite mathematical. If you feel this question is off-topic, I would appreciate it if you moved it to a site where you felt this question would be more appropriate, rather that just closing it.
*Psuedocode was gotten from Algorithms by S. Dasgupta. Page 71.
The loop is for recursion.
No need to add for negative x; the FFT transforms from time to frequency space.
For binary search tree type of data structures, I see the Big O notation is typically noted as O(logn). With a lowercase 'l' in log, does this imply log base e (n) as described by the natural logarithm? Sorry for the simple question but I've always had trouble distinguishing between the different implied logarithms.
Once expressed in big-O() notation, both are correct. However, during the derivation of the O() polynomial, in the case of binary search, only log2 is correct. I assume this distinction was the intuitive inspiration for your question to begin with.
Also, as a matter of my opinion, writing O(log2 N) is better for your example, because it better communicates the derivation of the algorithm's run-time.
In big-O() notation, constant factors are removed. Converting from one logarithm base to another involves multiplying by a constant factor.
So O(log N) is equivalent to O(log2 N) due to a constant factor.
However, if you can easily typeset log2 N in your answer, doing so is more pedagogical. In the case of binary tree searching, you are correct that log2 N is introduced during the derivation of the big-O() runtime.
Before expressing the result as big-O() notation, the difference is very important. When deriving the polynomial to be communicated via big-O notation, it would be incorrect for this example to use a logarithm other than log2 N, prior to applying the O()-notation. As soon as the polynomial is used to communicate a worst-case runtime via big-O() notation, it doesn't matter what logarithm is used.
Big O notation is not affected by logarithmic base, because all logarithms in different bases are related by a constant factor, O(ln n) is equivalent to O(log n).
Both are correct. Think about this
log2(n)=log(n)/log(2)=O(log(n))
log10(n)=log(n)/log(10)=O(log(n))
logE(n)=log(n)/log(E)=O(log(n))
It doesn't really matter what base it is, since big-O notation is usually written showing only the asymptotically highest order of n, so constant coefficients will drop away. Since a different logarithm base is equivalent to a constant coefficient, it is superfluous.
That said, I would probably assume log base 2.
Yes, when talking about big-O notation, the base does not matter. However, computationally when faced with a real search problem it does matter.
When developing an intuition about tree structures, it's helpful to understand that a binary search tree can be searched in O(n log n) time because that is the height of the tree - that is, in a binary tree with n nodes, the tree depth is O(n log n) (base 2). If each node has three children, the tree can still be searched in O(n log n) time, but with a base 3 logarithm. Computationally, the number of children each node has can have a big impact on performance (see for example: link text)
Enjoy!
Paul
First you must understand what it means for a function f(n) to be O( g(n) ).
The formal definition is: *A function f(n) is said to be O(g(n)) iff |f(n)| <= C * |g(n)| whenever n > k, where C and k are constants.*
so let f(n) = log base a of n, where a > 1 and g(n) = log base b of n, where b > 1
NOTE: This means the values a and b could be any value greater than 1, for example a=100 and b = 3
Now we get the following: log base a of n is said to be O(log base b of n) iff |log base a of n| <= C * |log base b of n| whenever n > k
Choose k=0, and C= log base a of b.
Now our equation looks like the following: |log base a of n| <= log base a of b * |log base b of n| whenever n > 0
Notice the right hand side, we can manipulate the equation: = log base a of b * |log base b of n| = |log base b of n| * log base a of b = |log base a of b^(log base b of n)| = |log base a of n|
Now our equation looks like the following: |log base a of n| <= |log base a of n| whenever n > 0
The equation is always true no matter what the values n,b, or a are, other than their restrictions a,b>1 and n>0.
So log base a of n is O(log base b of n) and since a,b doesn't matter we can simply omit them.
You can see a YouTube video on it here: https://www.youtube.com/watch?v=MY-VCrQCaVw
You can read an article on it here: https://medium.com/#randerson112358/omitting-bases-in-logs-in-big-o-a619a46740ca
Technically the base doesn't matter, but you can generally think of it as base-2.