I wonder if the technique of divide and conquer always divide a problem into subproblems of same type? By same type, I mean one can implement it using a function with recursion. Can divide and conquer always be implemented by recursion?
Thanks!
"Always" is a scary word, but I can't think of a divide-and-conquer situation in which you couldn't use recursion. It is by definition that divide-and-conquer creates subproblems of the same form as the initial problem - these subproblems are continually broken down until some base case is reached, and the number of divisions correlates with the size of the input. Recursion is a natural choice for this kind of problem.
See the Wikipedia article for more good information.
A Divide-and-conquer algorithm is by definition one that can be solved by recursion. So the answer is yes.
Usually, yes! Merge sort is an example of the same. Here is an animated version of the same.
Yes. In divide and conquer algorithmic technique we divide the given bigger problem into smaller sub-problems. These smaller sub-problems must be similar to the bigger problem except that these are smaller in size.
For example, the problem of sorting an array of size N is no different from the problem of sorting an array of size N/2. Except that the latter problem size is smaller than that of former one.
If the smaller sub-problem is not similar to the bigger one, then the divide and conquer technique can not be used to solve the bigger problem. In other words, a given problem can be solved using divide and conquer technique only iff the given bigger problem can be divided into smaller sub problems which are similar to the bigger problem.
Examining merge sort algorithm will be enough for this question. After understanding implementation of merge sort algorithm with divide and conquer (also recursion) you will see how difficult it would be making it without recursion.
Actually the most important thing here is the complexity of the algorithm which is expressed with big-oh notation and nlogn for merge sort.
For mergesort exapmle there is another version which is called bottom-up merge sort. It is simple and non-recursive version of it.
it is about 10% slower than recursive, top-down mergesort on typical systems. You can refer to following link for more information. It is explained well in 3rd lecture.
https://www.coursera.org/learn/introduction-to-algorithms#
Recursion is a programming method where you define a function in terms of itself. The function generally calls itself with slightly modified parameters (in order to converge).
Divide the problem into two or more smaller subproblems.
Conquer the subproblems by solving them (recursively).
Combine the solutions to the subproblems into the solutions for the original problem.
Yes All Divide and Conquer always be implemented using recursion .
A typical Divide and Conquer algorithm solves a problem using following three steps.
Divide: Break the given problem into sub-problems of same type.
Conquer: Recursively solve these sub-problems
Combine: Appropriately combine the answers
Following are some standard algorithms that are Divide and Conquer algorithms.
1) Binary search,
2) Quick Sort,
3) Merge Sort,
4) Strassen’s Algorithm
Imagine P is a problem with size of n and S is the solution. In this case, if P is large enough to be divided into sub problem, for example P1, P2, P3, P4, ... , Pk; let say k sub problems and also there would be k solutions for each of k sub problems, like S1, S2, S3, ... , Sk; Now if we combine each solutions of sub problem together we can get the S result. In divide and conquer strategy what ever is the main problem all sube problems must be same. For example if P is sort then the P1, P2 and Pn must be sort too. So this is how it is recursive in nature. So, divide and conqure will be recursive.
Related
If I have a program with two simple loops and nothing else, giving me O(N+N). With time complexities Big O Notation are we allowed to simplify O(N+N) as O(2N)? If not, do how do both differ in terms of time complexity. I apologize for the simple question but just got into studying these concepts and my study guide keeps using O(N+N) instead of O(2N), which is confusing me.
Big O notation is a very well known concept, and so there are tons of material available online. One Google search away. Not really sure why would you ask here.
In particular, Wikipedia has a very long article: https://en.wikipedia.org/wiki/Big_O_notation
And there is a section on how sums work with this notation: https://en.wikipedia.org/wiki/Big_O_notation#Sum
One common misunderstanding is to assume that = in the notation actually is an equality operation. While, in fact, it is a set element operation, more commonly written as ∈.
Technically, you should be writing f(x) ∈ O(N). Writing f(x) = O(N) is wrong, as O(N) is really a set of functions. It is a set of all the functions that grow as fast as the linear function.
A sum of two functions that grow as fast as a linear function also grows as fast as a linear function.
O(f) is a set of functions that grow as fast as f.
Both O(N+N), and O(2N) are the same sets as O(N).
So, O(N+N) = O(2N) = O(N) and here, this = is really the set equality operation.
Though I have studied and able am able to understand some programs in recursion, I am still not able to intuitively obtain a solution using recursion as I do easily using Iteration. Is there any course or track available in order to build an intuition for recursion? How can one master the concept of recursion?
if you want to gain a thorough understanding of how recursion works, I highly recommend that you start with understanding mathematical induction, as the two are very closely related, if not arguably identical.
Recursion is a way of breaking down seemingly complicated problems into smaller bits. Consider the trivial example of the factorial function.
def factorial(n):
if n < 2:
return 1
return n * factorial(n - 1)
To calculate factorial(100), for example, all you need is to calculate factorial(99) and multiply 100. This follows from the familiar definition of the factorial.
Here are some tips for coming up with a recursive solution:
Assume you know the result returned by the immediately preceding recursive call (e.g. in calculating factorial(100), assume you already know the value of factorial(99). How do you go from there?)
Consider the base case (i.e. when should the recursion come to a halt?)
The first bullet point might seem rather abstract, but all it means is this: a large portion of the work has already been done. How do you go from there to complete the task? In the case of the factorial, factorial(99) constituted this large portion of work. In many cases, you will find that identifying this portion of work simply amounts to examining the argument to the function (e.g. n in factorial), and assuming that you already have the answer to func(n - 1).
Here's another example for concreteness. Let's say we want to reverse a string without using in-built functions. In using recursion, we might assume that string[:-1], or the substring until the very last character, has already been reversed. Then, all that is needed is to put the last remaining character in the front. Using this inspiration, we might come up with the following recursive solution:
def my_reverse(string):
if not string: # base case: empty string
return string # return empty string, nothing to reverse
return string[-1] + my_reverse(string[:-1])
With all of this said, recursion is built on mathematical induction, and these two are inseparable ideas. In fact, one can easily prove that recursive algorithms work using induction. I highly recommend that you checkout this lecture.
First off, apologies if there is a better way to format math equations, I could not find anything, but alas, the expressions are pretty short.
As part of an assigned problem I have to produce some code in C that will evaluate x^n/n! for an arbitrary x, and n = { 1-10 , 50, 100}
I can always brute force it with a large number library, but I am wondering if someone with better math skills then mine can suggest a better algorithm than something with a O(n!)...
I understand that I can split the numerator to x^(n/2)x^(n/2) for even values of n, and xx^(n-1/2)*x^(n-1/2) for odd values of n. And that I can further change that into a logarithm base x of n/2.
But I am stuck for multiple reasons:
1 - I do not think that computationally any of these changes actually make a lot of difference since they are not really helping me reduce the large number multiplications I have to perform, or their overall number.
2 - Even as I think of n! as 1*2*3*...*(n-1)*n, I still cannot rationalize a good way to simplify the overall equation.
3 - I have looked at Karatsuba's algorithm for multiplications, and although it is a possibility, it seems a bit complex for an intro to programming problem.
So I am wondering if you guys can think of any middle ground. I prefer explanations to straight answers if you have the time :)
Cheers,
My advice is to compute all the terms of the summation (put them in an array), and then sum them up in reverse order (i.e., smallest to largest) -- that reduces rounding error a little bit.
Note that you can compute the k-th term from the preceding one by multiplying by x/k -- you do not need to ever compute x^n or n! directly (this is important).
I am looking for a simple method to assign a number to a mathematical expression, say between 0 and 1, that conveys how simplified that expression is (being 1 as fully simplified). For example:
eval('x+1') should return 1.
eval('1+x+1+x+x-5') should returns some value less than 1, because it is far from being simple (i.e., it can be further simplified).
The parameter of eval() could be either a string or an abstract syntax tree (AST).
A simple idea that occurred to me was to count the number of operators (?)
EDIT: Let simplified be equivalent to how close a system is to the solution of a problem. E.g., given an algebra problem (i.e. limit, derivative, integral, etc), it should assign a number to tell how close it is to the solution.
The closest metaphor I can come up with it how a maths professor would look at an incomplete problem and mentally assess it in order to tell how close the student is to the solution. Like in a math exam, were the student didn't finished a problem worth 20 points, but the professor assigns 8 out of 20. Why would he come up with 8/20, and can we program such thing?
I'm going to break a stack-overflow rule and post this as an answer instead of a comment, because not only I'm pretty sure the answer is you can't (at least, not the way you imagine), but also because I believe it can be educational up to a certain degree.
Let's assume that a criteria of simplicity can be established (akin to a normal form). It seems to me that you are very close to trying to solve an analogous to entscheidungsproblem or the halting problem. I doubt that in a complex rule system required for typical algebra, you can find a method that gives a correct and definitive answer to the number of steps of a series of term reductions (ipso facto an arbitrary-length computation) without actually performing it. Such answer would imply knowing in advance if such computation could terminate, and so contradict the fact that automatic theorem proving is, for any sufficiently powerful logic capable of representing arithmetic, an undecidable problem.
In the given example, the teacher is actually either performing that computation mentally (going step by step, applying his own sequence of rules), or gives an estimation based on his experience. But, there's no generic algorithm that guarantees his sequence of steps are the simplest possible, nor that his resulting expression is the simplest one (except for trivial expressions), and hence any quantification of "distance" to a solution is meaningless.
Wouldn't all this be true, your problem would be simple: you know the number of steps, you know how many steps you've taken so far, you divide the latter by the former ;-)
Now, returning to the criteria of simplicity, I also advice you to take a look on Hilbert's 24th problem, that specifically looked for a "Criteria of simplicity, or proof of the greatest simplicity of certain proofs.", and the slightly related proof compression. If you are philosophically inclined to further understand these subjects, I would suggest reading the classic Gödel, Escher, Bach.
Further notes: To understand why, consider a well-known mathematical artefact called the Mandelbrot fractal set. Each pixel color is calculated by determining if the solution to the equation z(n+1) = z(n)^2 + c for any specific c is bounded, that is, "a complex number c is part of the Mandelbrot set if, when starting with z(0) = 0 and applying the iteration repeatedly, the absolute value of z(n) remains bounded however large n gets." Despite the equation being extremely simple (you know, square a number and sum a constant), there's absolutely no way to know if it will remain bounded or not without actually performing an infinite number of iterations or until a cycle is found (disregarding complex heuristics). In this sense, every fractal out there is a rough approximation that typically usages an escape time algorithm as an heuristic to provide an educated guess whether the solution will be bounded or not.
"Given an array of n integers, return an array of their factorials."
Instead of the straight forward way of iterating through the array and finding factorial for each, I was thinking of a memoized approach, where I store previously calculated factorials and use them in subsequent ones.
For example: 7! can be calculated much fasted if the result 6! is stored somewhere. However, I noticed the run time of both algorithms is still O(n). (I might be wrong) Does that imply that we're not speeding up the process here? If so, does that mean that memoization is not useful in problems with non-tree recursion? (In Fibonacci, we effectively prune the recursion tree by memoizing the previously found values, in the case of factorial, we don't really have the tree, more like a recursion ladder)
Any comments appreciated.
However, I noticed the run time of both algorithms is still O(n).
O-Notation is hiding the most important characteristic here. It only considers the length of the array and not the size of the numbers which is much much more important when dealing with factorials.
You should implement memoization with a hashtable. If you do then you will get O(1) for each entry asymptotically.
In the case without memoization, the time complexity should be O(n^2) since you would need (i-1) multiplications to calculate factorial(i) without memoization.