First of all - yes, I know that there are a ton kind of similar questions about this but I still don't get it.
So the Big-O Notation of this code is O(2^n)
public static int Fibonacci(int n)
{
if (n <= 1)
return n;
else
return Fibonacci(n - 1) + Fibonacci(n - 2);
}
And even though if I run it using let's say 6 , function is getting called 25 times as you can see in this picture:
Fibonacci
Shouldn't it be 64 because of O(2^6) = 64 ?
Few issues with the logic here:
Big O notation is only giving upper asymptotic bound, not a tight bound, that's what big Theta is for.
Fibonacci is actually Theta(phi^n), if you are looking for tighter bound (where phi is the "golden ration", phi ~= 1.618.
When talking about asymptotic notation, there isn't much point in talking about low numbers, and neglecting the constants - since they are omitted for the asymptotic complexity (This is not the case here though).
In here, the issue is fibonacci is indeed O(2^n), but that bound is not tight, so the actual number of calls is going to be lower than the estimated one (for sufficiently large n, onwards).
Related
I read recently in Steven Skiena "Algorithms design manual book" that the harmonic Sum is of O(lg n). I understand how this is possible mathematically, but when i saw the recursive solution to it, which is the following:
public static double harmonic(int n) {
if(n == 1) {
return 1.0;
} else {
return (1.0 / n) + harmonic(n - 1);
}
}
I can not see how this is O(lg n), I see it as O(n). is this code above really lg(n) if so how ? if not can someone refer me to an example where this is the case ?
I believe the confusion here is that big-O notation can be used not only for running times, but for the growth of functions in general.
As kaya3 comments, the return value of the function above is O(lg n), which means it grows asymptotically as fast as lg n as you increase n. This is quite likely what the book is claiming. It is a mathematical property of the harmonic series.
The running time of the function, however, takes O(n) time. This is what you are claiming, and it is perfectly true (assuming constant-time arithmetics).
If you are still confused about how big-O notation can be used for purposes other than the running time of algorithms, I recommend you check its definition.
When describing a loose bound, can I pick any value for the proper notation that is not close at all to the actual value for the asymptotic notation? If my function is n^2 + n, could I say a loose upper bound is O(n^3), and could I say that a loose lower bound is Ω(1)? Would such statements be valid? When do I know that my loose bound is valid or not?
From the here Difference between Big-O and Little-O Notation talking about Big-O vs little-o, it would be fine to say your n^2+n is O(n^3) and this is a loose upper bound. However most people would describe it as O(n^2) as this is more accurate and tells you that the algorithm is actually faster than O(n^3)
I think "loose bound" might come up more when something isn't completely known. For instance the naive recursive Fibonacci function:
def fibo(n):
if n == 1 or n == 2:
return 1
return fibo(n-1) + fibo(n-2)
You could say that this is O(2^n) because the function makes 2 recursive calls so it's doubling at each step. However, one side of this "tree" (if you were to expand out the function calls it looks sort of like a tree) will bottom out to a base case faster than the other because it's n-1 vs n-2. This means the "tree" is lopsided and so it will actually be faster than O(2^n). Hence O(2^n) is a "loose upper bound."
Finding a tighter bound is left as an exercise to the reader ;)
To simplify a big O expression
We omit all constants
We ignore lower powers of n
For example:
O(n + 5) = O(n)
O(n² + 6n + 7) = O(n²)
O(6n1/3 + n1/2 + 7) = O(n1/2)
Am I right in these examples?
1. We omit all constants
Well, strictly speaking, you don't omit all constants, only the outermost multiplicaive constant. That means O(cf(n)) = O(f(n)). Additive constants are fine too, since
f(n) < f(n)+c < 2f(n) starting with some n, therefore O(f(n)+c) = O(f(n)).
But you don't omit constants inside composite functions. Might be done sometimes (O(log(cn)) or even O(log(n^c)) for instance), but not in general. Consider for example 2^2n, it might be tempting to drop the 2 and put this in O(2^n), which is wrong.
2. We ignore lower powers of n
True, but remember, you don't always work with polynomial functions. You can generally ignore any added asymptotically lower functions. Say you have f(n) and g(n), when g(n) = O(f(n)), then O(f(n) + g(n)) = O(f(n)).
You cannot do this with multiplication.
You're almost right. The second rule should be that you ignore all but the term with the largest limit as n goes towards infinity. That's important if you have terms that are not powers of n, like logs or other mathematical functions.
It's also worth being aware that big O notation sometimes covers up important other details. An algorithm that is O(n log n) will have better performance than one that is O(n^2), but only if the input is large enough for those most terms to dominate the running time. It may be that for the sizes inputs you actually have to deal with in a specific application, the O(n^2) algorithm actually performs better!
Find the big-O running time for each of these functions:
T(n) = T(n - 2) + n²
Our Answers: n², n³
T(n) = 3T(n/2) + n
Our Answers: O(n log n), O(nlog₂3)
T(n) = 2T(n/3) + n
Our Answers: O(n log base 3 of n), O(n)
T(n) = 2T(n/2) + n^3
Our Answers: O(n³ log₂n), O(n³)
So we're having trouble deciding on the right answers for each of the questions.
We all got different results and would like an outside opinion on what the running time would be.
Thanks in advance.
A bit of clarification:
The functions in the questions appear to be running time functions as hinted by their T() name and their n parameter. A more subtle hint is the fact that they are all recursive and recursive functions are, alas, a common occurrence when one produces a function to describe the running time of an algorithm (even when the algorithm itself isn't formally using recursion). Indeed, recursive formulas are a rather inconvenient form and that is why we use the Big O notation to better summarize the behavior of an algorithm.
A running time function is a parametrized mathematical expression which allows computing a [sometimes approximate] relative value for the running time of an algorithm, given specific value(s) for the parameter(s). As is the case here, running time functions typically have a single parameter, often named n, and corresponding to the total number of items the algorithm is expected to work on/with (for e.g. with a search algorithm it could be the total number of records in a database, with a sort algorithm it could be the number of entries in the unsorted list and for a path finding algorithm, the number of nodes in the graph....). In some cases a running time function may have multiple arguments, for example, the performance of an algorithm performing some transformation on a graph may be bound to both the total number of nodes and the total number of vertices or the average number of connections between two nodes, etc.
The task at hand (for what appears to be homework, hence my partial answer), is therefore to find a Big O expression that qualifies the upper bound limit of each of running time functions, whatever the underlying algorithm they may correspond to. The task is not that of finding and qualifying an algorithm to produce the results of the functions (this second possibility is also a very common type of exercise in Algorithm classes of a CS cursus but is apparently not what is required here.)
The problem is therefore more one of mathematics than of Computer Science per se. Basically one needs to find the limit (or an approximation thereof) of each of these functions as n approaches infinity.
This note from Prof. Jeff Erikson at University of Illinois Urbana Champaign provides a good intro to solving recurrences.
Although there are a few shortcuts to solving recurrences, particularly if one has with a good command of calculus, a generic approach is to guess the answer and then to prove it by induction. Tools like Excel, a few snippets in a programming languages such as Python or also MATLAB or Sage can be useful to produce tables of the first few hundred values (or beyond) along with values such as n^2, n^3, n! as well as ratios of the terms of the function; these tables often provide enough insight into the function to find the closed form of the function.
A few hints regarding the answers listed in the question:
Function a)
O(n^2) is for sure wrong:
a quick inspection of the first few values in the sequence show that n^2 is increasingly much smaller than T(n)
O(n^3) on the other hand appears to be systematically bigger than T(n) as n grows towards big numbers. A closer look shows that O(n^3) is effectively the order of the Big O notation for this function, but that O(n^3 / 6) is a more precise notation which systematically exceed the value of T(n) [for bigger values of n, and/or as n tends towards infinity] but only by a minute fraction compared with the coarser n^3 estimate.
One can confirm that O(n^3 / 6) is it, by induction:
T(n) = T(n-2) + n^2 // (1) by definition
T(n) = n^3 / 6 // (2) our "guess"
T(n) = ((n - 2)^3 / 6) + n^2 // by substitution of T(n-2) by the (2) expression
= (n^3 - 2n^2 -4n^2 -8n + 4n - 8) / 6 + 6n^2 / 6
= (n^3 - 4n -8) / 6
= n^3/6 - 2n/3 - 4/3
~= n^3/6 // as n grows towards infinity, the 2n/3 and 4/3 factors
// become relatively insignificant, leaving us with the
// (n^3 / 6) limit expression, QED
i tried to get this work by using Newton's method as described here: wiki using the following code, but the problem is that it is giving accurate result only upto 16 decimal places. I tried to increase the number of iterations still the result is the same. I started with initial guess of 1. So how can i improve the accuracy of answer(upto 100 or more decimal places) ?
Thanks.
Code:
double x0,x1;
#define n 2
double f(double x0)
{
return ((x0*x0)-n);
}
double firstDerv(double x0)
{
return 2.0*x0;
}
int main()
{
x0 = n/2.0;
int i;
for(i=0;i<40000;i++)
{
x1=x0-(f(x0)/((firstDerv(x0))));
x0=x1;
}
printf("%.100lf\n",x1);
return 0;
}
To get around the problem of limited precision floating points, you can also use Newton's method to
find in each iteration a rational (a/b, with a and b integers) that is a better approximation of sqr(2).
If x=a/b is the value returned from you last iteration, then Newton's method states that the new estimate y=c/d is:
y = x/2 + 1/x = a/2b + b/a = (a^2+2b^2)(2ab)
so:
c= a^2 + 2b^2
d= 2ab
Precision doubles each iteration. You are still limited in the precision you can reach, because nominator and denominator quickly increase, but perhaps finding an implementation of large integers (or concocting one yourself) is easier than finding an implementation of arbitrary precision floating points. Also, if you are really interested in decimals, then this answer won't help you. It does give you a very precise estimate of sqr(2).
Just some iterates of a/b for the algorithm:
1/1 , 3/2 , 17/12 , 577/408 , 665857/470832.
665857/470832 approximates sqr(2) with an error of 1.59e-12. Error will remain to be of the order 1/a^2, so implementing a and b as longs will give you precision of 1e-37 -ish.
The floating point numbers on current machines are IEEE754 and have a limited precision (of about 15 digits).
If you want much more precision, you'll need bignums which are (slowly) provided by software libraries like GMP
You could also code your program in languages and implementations using bignums.
You simply can't do it with that approach; doubles don't have enough bits to get 100 places of precision. Consider using a library for arbitrary-precision such as GMP.
maybe this is because floating point numbers are approximated by computers through an m*10^e form. since m and e consist of a finite number of digits you cannot approximate all numbers with absolute accuracy.
think of 1/3 which is 0.333333333333333...