Functions which do not contain any delay assignments are synthesizable, hence all synthesized functions are combinational in nature. Will the function still remain synthesizable if we have a recursive call?
Taking an example of Factorial function (automatic).
This functions is present in a state machine. Every time the state is activated, the function would return the value to a register.
Will this function synthesize?
Yes, recursive function can be synthesizable, but only if the depth of recursion can be determined at compile time. That usually means the arguments to each top-level call to the function are constants. Like any other loop, a synthesis tool needs to have a fixed number of iterations to know exactly how much hardware logic needs to be allocated.
Related
By IO I am loosely referring to the type that is typically used to wrap impure functions (IO.delay) to be invoked at a later time (my understanding after reading Grokking Functional Programming. I am aware it could also wrap values - IO.pure)
On the other hand, in the Value of Values presentation, Rich Hickey states that (besides being immutable) values are comparable. In particular, a function is equal to another function if they both return the same value given the same arguments no matter when they are called. That is, pure functions are values. A corollary being that a pure function is equal to itself.
However, an impure function, which given the same arguments may return different values depending on when you call it, is not even equal to itself according to the definition highlighted above. Hence, I conclude impure functions are not values as they are not even equal to themselves.
How could instances of the IO type which wrap impure functions be values? The wrapped impure function is not a value. How could the wrapper be a value? According to the premise made above, an instance of the IO type which wraps an impure function is not even equal to itself!
In F# if I write
let p = printfn "something"
it will evaluate the expression once. Any subsequent references to p will evaluate to unit.
From a theoretical perspective, the definition of a function, this makes sense. A function should only return the same result for the same input.
However if I want the side-effect to occur (i.e. the output to the screen), then I need the pass an argument to p. Typically this argument is the unit value.
let p () = printfn "something"
But why will F# evaluate the function each time, when the argument is the same each time the function is applied? Surely the same reasoning should apply as in the first case? The input to the function p doesn't change therefore there is no need to evaluate it more than once.
The action printfn is, strictly speaking, not a function. It is, particularly, not a pure function, because it's not referentially transparent. This is possible because F# isn't a strictly functional language (it's a 'functional first' language). It makes no explicit distinction between pure functions and impure actions.
The return value of printfn "something" is () (unit), which means that p is bound to the unit value (). The fact that something is printed on the screen is a side effect of evaluating the expression.
F# is an eagerly evaluated language. That's why you see something printed on the screen as a side effect of binding printfn "something" to p. Once the expression is evaluated, p is only bound to () - the value.
F# doesn't memoize function calls, so when you change p to a function, it'll evaluate the function every time you call it with (). Since all functions can be impure, the compiler can't tell whether or not memoization would be appropriate, so it doesn't do that.
Other languages do this in different ways. Haskell, for example, is lazily evaluated, and also explicitly distinguishes between pure functions and impure actions, so it can apply different optimization in cases like these.
To expand on the answer given in the comments, the first p is an immutable value, while the second p is a function. If you refer to an immutable value multiple times, then (obviously) its value doesn't change over time. But if you invoke a function multiple times, it executes each time, even if the arguments are the same each time.
Note that this is true even for pure functional languages, such as Haskell. If you want to avoid this execution cost, there's a specific technique called memoization that can be used to return cached results when the same inputs occur again. However, memoization has its own costs, and I'm not aware of any mainstream functional language that automatically memoizes all function calls.
I've been recently learning about functional languages and how many don't include for loops. While I don't personally view recursion as more difficult than a for loop (and often easier to reason out) I realized that many examples of recursion aren't tail recursive and therefor cannot use simple tail recursion optimization in order to avoid stack overflows. According to this question, all iterative loops can be translated into recursion, and those iterative loops can be transformed into tail recursion, so it confuses me when the answers on a question like this suggest that you have to explicitly manage the translation of your recursion into tail recursion yourself if you want to avoid stack overflows. It seems like it should be possible for a compiler to do all the translation from either recursion to tail recursion, or from recursion straight to an iterative loop with out stack overflows.
Are functional compilers able to avoid stack overflows in more general recursive cases? Are you really forced to transform your recursive code in order to avoid stack overflows yourself? If they aren't able to perform general recursive stack-safe compilation, why aren't they?
Any recursive function can be converted into a tail recursive one.
For instance, consider the transition function of a Turing machine, that
is the mapping from a configuration to the next one. To simulate the
turing machine you just need to iterate the transition function until
you reach a final state, that is easily expressed in tail recursive
form. Similarly, a compiler typically translates a recursive program into
an iterative one simply adding a stack of activation records.
You can also give a translation into tail recursive form using continuation
passing style (CPS). To make a classical example, consider the fibonacci
function.
This can be expressed in CPS style in the following way, where the second
parameter is the continuation (essentially, a callback function):
def fibc(n, cont):
if n <= 1:
return cont(n)
return fibc(n - 1, lambda a: fibc(n - 2, lambda b: cont(a + b)))
Again, you are simulating the recursion stack using a dynamic data structure:
in this case, lambda abstractions.
The use of dynamic structures (lists, stacks, functions, etc.) in all previous
examples is essential. That is to say, that in order to simulate a generic
recursive function iteratively, you cannot avoid dynamic memory allocation,
and hence you cannot avoid stack overflow, in general.
So, memory consumption is not only related to the iterative/recursive
nature of the program. On the other side, if you prevent dynamic memory
allocation, your
programs are essentially finite state machines, with limited computational
capabilities (more interesting would be to parametrise memory according to
the dimension of inputs).
In general, in the same way as you cannot predict termination, you cannot
predict an unbound memory consumption of your program: working with
a Turing complete language, at compile time
you cannot avoid divergence, and you cannot avoid stack overflow.
Tail Call Optimization:
The natural way to do arguments and calls is to sort out the cleaning up when exiting or when returning.
For tail calls to work you need to alter it so that the tail call inherits the current frame. Thus instead of making a new frame it massages the frame so that the next call returns to the current functions caller instead of this function, which really only cleans up and returns if it's a tail call.
Thus TCO is all about cleaning up before the last call.
Continuation Passing Style - make tail calls out of everything
A compiler can change the code such that it only does primitive operations and pass it to continuations. Thus the stack usage gets moved onto the heap since the computation to be continued is made a function.
An example is:
function hypotenuse(k1, k2) {
return sqrt(add(square(k1), square(k2)))
}
becomes
function hypotenuse(k, k1, k2) {
(function (sk1) {
(function (sk2) {
(function (ar) {
k(sqrt(ar));
}(add(sk1,sk2));
}(square(k2));
}(square(k1));
}
Notice every function has exactly one call now and the order of evaluation is set.
According to this question, all iterative loops can be translated into recursion
"Translated" might be a bit of a stretch. The proof that for every iterative loop there is an equivalent recursive program is trivial if you understand Turing completeness: since a Turing machine can be implemented using strictly iterative structures and strictly recursive structures, every program that can be expressed in an iterative language can be expressed in a recursive language, and vice-versa. This means that for every iterative loop there is an equivalent recursive construct (and the other way around). However, that doesn't mean we have some automated way of transforming one into the other.
and those iterative loops can be transformed into tail recursion
Tail recursion can perhaps be easily transformed into an iterative loop, and the other way around. But not all recursion is tail recursion. Here's an example. Suppose we have some binary tree. It consists of nodes. Each node can have a left and a right child and a value. If a node has no children, then isLeaf returns true for it. We'll assume there's some function max that returns the maximum of two values, and if one of the values is null it returns the other one. Now we want to define a function that finds the maximum value among all the leaf nodes. Here it is in some pseudo-code I cooked up.
findmax(node) {
if (node == null) {
return null
}
if (node.isLeaf) {
return node.value
} else {
return max(findmax(node.left), findmax(node.right))
}
}
There's two recursive calls in the max function, so we can't optimize for tail recursion. We need the results of both before we can supply them to the max function and determine the result of the call for the current node.
Now, there may be a way of getting the same result, using recursion and only a single tail-recursive call. It is functionally equivalent, but it is a different algorithm. Compilers can do a lot of transformations to create a functionally equivalent program with lots of optimizations, but they're not quite clever enough to create functionally equivalent algorithms.
Even the transformation of a function that only calls itself recursively once into a tail-recursive version would be far from trivial. Such an adaptation usually employs some argument passed into the recursive invocation that is used as an "accumulator" for the current results.
Look at the next naive implementation for calculating a factorial of a number (e.g. fact(5) = 5*4*3*2*1):
fact(number) {
if (number == 1) {
return 1
} else {
return number * fact(number - 1)
}
}
It's not tail-recursive. But it can be made so in this way:
fact(number, acc) {
if (number == 1) {
return acc
} else {
return fact(number - 1, number * acc)
}
}
// Helper function
fact(number) {
return fact(number, 1)
}
This requires an interpretation of what is being done. Recognizing the case for stuff like this is easy enough, but what if you call a function instead of a multiplication? How will the compiler know that for the initial call the accumulator must be 1 and not, say, 0? How do you translate this program?
recsub(number) {
if (number == 1) {
return 1
} else {
return number - recsub(number - 1)
}
}
This is as of yet outside the scope of the sort of compiler we have now, and may in fact always be.
Maybe it would be interesting to ask this on the computer science Stack Exchange to see if they know of some papers or proofs that investigate this more in-depth.
this is a purely academic question:
I have recently been working with languages that use tail recursion optimization. For practice I wrote two recursive implementations of sum functions in R, one of them being tail recursive. I quickly realized there is no tail recursion optimization in R. I can live with that.
However, I also noticed a different level of allowed depth when using the local helper function for the tail recursion.
Here is the code:
## Recursive
sum <- function (i, end, fun){
if (i>=end) 0
else fun(i) + sum(i+1, end, fun)
}
## Tail recursive
sum_tail <- function (i, end, fun){
sum_helper<- function(i, acc){
if (i>=end) acc
else sum_helper(i+1, acc+fun(i))
}
sum_helper(i, 0)
}
## Simple example
harmonic <- function(k){
return(1/(k))
}
print(sum(1, 1200, harmonic)) # <- This works fine, but is close to the limit
# print(sum_tail(1, 1200, harmonic)) <- This will crash
print(sum_tail(1, 996, harmonic)) # <- This is the deepest allowed
I am fairly intrigued. Can someone explain this behavior or point me towards a document explaining how the allowed recursion depth is calculated?
I'm not sure of R's internal implementation of the call stack, but it's pretty obvious from here that there is a maximum stack depth. (Many languages have this for various reasons, mostly related to memory and detecting infinite recursion.) You can set it with options(), and the default setting seems to depends on the platform -- on my machine, I can do print(sum_tail(1, 996, harmonic)) without difficulty.
Sidebar: you really shouldn't name your naive implementation sum() because you wind up shadowing a builtin. I know you're just playing with recursion here, but you should also generally avoid doing your own implementation of sum() -- it's not provided just as a convenience function but also because it's non trivial to implement a numerically correct version of sum() with floating point.
In your naive implementation, the call to fun() returns before the recursive call -- this means that each recursive call increases the depth of the call stack by exactly 1. In the other case, you've got an additional function call that's waiting to be evaluated. For more details, you should look into how R handles closures and how lazy / eager evaluation in R is handled. If I recall correctly, R uses environments (roughly, R's notion of scope, and deeply related to closures) to wrap arguments in certain situations and delay their evaluation, thus effectively using lazy evaluation. There's a lot of information on R internals available online, see here for a quick overview of argument evaluation. I'm not sure how accurate I am on the details, but it seems that the arguments to the tail-call are themselves getting placed on the call stack, thus increasing the depth of the call-stack by more than 1.
Sidebar the Second: I don't recall well enough how R implements this, and I know placing helper functions in the body is common practice, but placing a helper function definition in the recursive call could lead to each recursive call defining the helper function anew. This could interact in various ways with the way environments and closures are handled, but I'm not sure.
The functions traceback() and trace() could be useful in exploring the call behavior if you're curious about more of the details.
I wrote a function that shall perform foldl similar to Haskell in Maxima,
foldl(f,ac,li):=block([con:[],acc:ac],/*print("List=",li,ac),*/
if (is(li#[])) then
(acc:apply(f,cons(acc,[first(li)])),
acc:foldl(f,acc,rest(li))),acc)$
And it works fine in folding the list from left side and evaluating along the way hence preventing any accumulation of long unevaluated expression in buffer.
The problem I am facing in running this with,
foldl(lambda([x,y],x+y),0,makelist(i,i,1,97));
Error in PROGN [or a callee]: Bind stack overflow.
But if I run it upto 96, it generates result appropriately.
I don't understand why is this simple addition causing problem as I don't have any infinte loop or memory hungry task going on.
Well, foldl is defined as a recursive function, and it will call itself as many times as there are elements in the list. So whether it works depends on the Lisp implementation-specific limit for the function call stack. For GCL it seems the limit is relatively small. For other Lisp implementations, the limit is greater. But the only way to make it work for all sizes of the list is to write it iteratively.
There are built-in functions similar to foldl -- see lreduce, rreduce, xreduce, and tree_reduce.