How can I get this function to be tail-recursive? - recursion

I'm still trying to implement 2-3 finger trees and I made good progress (repository). While doing some benchmarks I found out that my quite basic toList results in a StackOverflowException when the tree ist quite large. At first I saw an easy fix and made it tail-recursive.
Unfortunately, it turned out that toList wasn't the culprit but viewr was:
/// Return both the right-most element and the remaining tree (lazily).
let rec viewr<'a> : FingerTree<'a> -> View<'a> = function
| Empty -> Nil
| Single x -> View(x, lazyval Empty)
| Deep(prefix, deeper, One x) ->
let rest = lazy (
match viewr deeper.Value with
| Nil ->
prefix |> Digit.promote
| View (node, lazyRest) ->
let suffix = node |> Node.toList |> Digit.ofList
Deep(prefix, lazyRest, suffix)
)
View(x, rest)
| Deep(prefix, deeper, Digit.SplitLast(shorter, x)) ->
View(x, lazy Deep(prefix, deeper, shorter))
| _ -> failwith Messages.patternMatchImpossible
Looking for the only recursive call it is obvious that this is is not tail-recursive. Somehow I hoped this problem wouldn't exist because that call is wrapped in a Lazy which IMHO is similar to a continuation.
I heard and read of continuations but so far never (had to) use(d) them. I guess here I really need to. I've been staring at the code for quite some time, putting function parameters here and there, calling them other places… I'm totally lost!
How can this be done?
Update: The calling code looks like this:
/// Convert a tree to a list (left to right).
let toList tree =
let rec toList acc tree =
match viewr tree with
| Nil -> acc
| View(head, Lazy tail) -> tail |> toList (head::acc)
toList [] tree
Update 2: The code that caused the crash is this one.
let tree = seq {1..200000} |> ConcatDeque.ofSeq
let back = tree |> ConcatDeque.toList
The tree get built fine, I checked and it is only 12 levels deep. It's the call in line 2 that triggered the overflow.
Update 3: kvb was right, that pipe issue I ran into before has something to do with this. Re-testing the cross product of debug/release and with/without pipe it worked in all but one case: debug mode with the pipe operator crashed. The behavior was the same for 32 vs. 64 bit.
I'm quite sure that I was running release mode when posting the question but today it's working. Maybe there was some other factor… Sorry about that.
Although the crash is solved, I'm leaving the question open out of theoretical interest. After all, we're here to learn, aren't we?
So let me adapt the question:
From looking at the code, viewr is definitely not tail-recursive. Why doesn't it always blow up and how would one rewrite it using continuations?

Calling viewr never results in an immediate recursive call to viewr (the recursive call is protected by lazy and is not forced within the remainder of the call to viewr), so there's no need to make it tail recursive to prevent the stack from growing without bound. That is, a call to viewr creates a new stack frame which is then immediately popped when viewr's work is done; the caller can then force the lazy value resulting in a new stack frame for the nested viewr call, which is then immediately popped again, etc., so repeating this process doesn't result in a stack overflow.

Related

Avoid stackoverflow when recursively dismantling a string

I'm working on a solution for a problem for the advent of code 2018 (spoiler alert) where I need a function which takes a string (or a char list) and removes every pair of chars when they react. The exercise describes two chars, or "elements" in a "polymer", reacting when they are the same letter but only differ in case; so starting out with AbBc would leave you with Ac. Keep in mind that after a reaction two chars could end up next to each other, where they weren't before, and cause a new reaction.
I thought I could solve this by using a recursive function which only deals with the first two chars and recursively calls itself, but since the input string is quite large, this causes a stackoverflow exception:
let rec react polymer =
match polymer with
| [] -> []
| [x] -> [x]
| head::tail ->
let left = head
let right = List.head tail
let rest = List.tail tail
// 'reacts' takes two chars and
// returns 'true' when they react
match reacts left right with
// when reacts we go further with
// the rest as these two chars are
// obliterated
| true -> react rest
// no reaction means the left char
// remains intact and the right one
// could react with the first char
// of the rest
| false -> [left] # react tail
Then, just trying to solve the exercise to have a right answer to unit test against, I tried to do it imperatively, but that got messy real quick and now I'm kinda stuck. I'm teaching myself f# so any pointers are welcome. Can anyone solve this in a functional manner?
You can avoid stack overflow by rewriting your function to use tail recursion, which just means that the recursive call should be the last operation to execute.
When you do [left] # react tail you first make a recursive call, and then append [left] to the result of that. That means it has to keep the current function context, called a stack frame, around while it executes the recursive call, and if that recurses as well the stack frames add up until you get a stack overflow. But if there's no more work to be done in the current function context, the stack frame can be released (or reused), hence no stack overflow.
You can make it tail recursive by adding another function argument, conventionally called acc since it "accumulates" values. Instead of adding left to the return value of the recursive call we add it to the accumulator and pass that along. Then when we exhaust the input, we return the accumulator instead of the empty list.
I've also taken the liberty of the append, [left] # ..., as a cons, left::..., since the latter is much more efficient than the former. I've also moved left, right and rest to the pattern, since that's much neater and safer. You should generally avoid using List.head and List.tail since they fail on empty lists and are bugs just waiting to happen.
let rec react acc polymer =
match polymer with
| [] -> acc
| [x] -> x::acc
| left::right::rest ->
match reacts left right with
| true -> react acc rest
| false -> react (left::acc) (right::rest)
You could also use a guard instead of nested matches (which should really have been an if anyway):
let rec react acc polymer =
match polymer with
| [] ->
acc
| [x] ->
x::acc
| left::right::rest when reacts left right ->
react acc rest
| left::rest ->
react (left::acc) rest

Recursion in F# Example

I've started learning F# and following in the footsteps of example problems I've written my own statement. It's simple enough, but I'm getting an error that doesn't exist in similar recursion examples.
My function replace takes a list of integers, an swapVal integer and a newVal integer. It then recurses through the list and changes any 'swapVal' to 'newVal'.
let original = [1;3;1;4;1;6;1;9]
let rec replace list origVal newVal =
match list with //look at list
| [] -> [] //when ls empty, return empty list
| head :: tail when head = origVal -> newVal :: replace tail origVal newVal
//when list is a head attached to a tail, if head = origVal,
//call replace on tail and appead the newVal
|_ :: tail -> replace tail origVal newVal
//head not equal to original value, call replace tail and return result
Calling replace original 1 5 I'm getting the following error Script.fsx(144,9): error FS0039: The value or constructor 'original' is not defined. Searching online like here hasn't turned up any solutions. Even O'Reilly's programming F# says that it could be a scoping problem, but there's no way that scope is my error.
I feel like it could be that f# is typing my arguments incorrectly, but I don't know enough about f# to know how it types. I would cast the arguments to make sure, but I read that it's not possible.
Does anyone see any immediate errors?

Storing user input information in functional programming (Erlang) using only immutable variables

As a beginner in Erlang, I am working my way through the Programming Erlang book (2nd ed). I have a very hard time grasping how to store and periodically update external information (such as intermittent user input) using the principles of functional programming exclusively.
To take my present example, I am now in the beginning of the concurrent programming section (Chapter 12) where the book talks about the area server. Below is my variant of it.
As an exercise, I am trying to add to this module a way to store all the requests the user makes. But despite having a bit of experience with recursive programming the lack of mutable variables, in the sense of imperative languages, seems to be crippling in this particular instance.
I have tried looking up a few related resources on SE sites such as mutable state in functional programming and
immutability in fp but it doesn't really answer my question in a practical way. I know that what I am trying to accomplish can be done by use of the ETS (or even a database), or by using the process-memory of a new process which receives and maintains the history within itself.
But what I would really like to understand (and the point of this question) is if this can be accomplished using generic functional programming principles without having to use Erlang-specific tools. The commented out lines in the code segment indicate what I am naively expecting the first steps to look like.
-module(geometry_server4).
-export([start/0, client/2, loop/0]).
start() ->
spawn(geometry_server4, loop, []).
client(Pid_server, Geom_tuple) ->
Pid_server ! {self(), Geom_tuple},
%ok = storerequests(Geom_tuple),
receive
{area, Pid_server, Area} -> io:format("Client: Area of ~p is ~p~n", [Geom_tuple, Area]);
{error, Error} -> io:format("~p~n", [Error])
end.
%storerequests(Geom_tuple) -> addtolist(Geom_tuple, get_history()).
%
%addtolist(Item, History) ->
% [Item | History].
%get_history() -> ???
loop() ->
receive
{Client, {rectangle, S1, S2}} ->
Area = S1 * S2,
Client ! {area, self(), Area},
loop();
{Client, {square, S}} ->
Area = S * S,
Client ! {area, self(), Area},
loop();
{Client, _} ->
Client ! {error, "invalid parameters"},
loop()
end.
Based on the book, this toy server gets called in the terminal as:
1> c(geometry_server4).
2> P = geometry_server4:start().
3> geometry_server4:client(P, {square, 3}).
But what I would really like to understand (and the point of this
question) is if this can be accomplished using generic functional
programming principles without having to use Erlang-specific tools.
Yes, it can. You can use a loop variable to store what's known as the state.
First, a couple of preliminary points:
Don't post code with line numbers. You want someone to be able to copy your code and paste it in their text editor and be able to run the code.
In erlang, by convention you use camel case for variable names, such as ServerPid.
For your own sanity, don't use module names that are more than two letters long.
Consider putting all your server code in one portion of the file, and all the client code in another portion of the file. Your client code is in the middle of the server code.
-module(my).
%%-export([setup/1]).
-compile(export_all).
%%-include_lib("eunit/include/eunit.hrl").
%%
start() ->
spawn(my, loop, [[]]).
loop(History) ->
receive
{Client, {rectangle, S1, S2}=Tuple} ->
Area = S1 * S2,
Client ! {area, self(), Area},
loop([Tuple|History]); %Add Tuple to the history
{Client, {square, S}=Tuple} ->
Area = S * S,
Client ! {area, self(), Area},
loop([Tuple|History]);
{Client, history} ->
Client ! {history, self(), History},
loop([history|History]);
{Client, Other} ->
Client ! {error, self(), "invalid parameters"},
loop([{error, Other}|History])
end.
client(ServerPid, Req) ->
ServerPid ! {self(), Req},
receive
Reply -> io:format("~p~n", [Reply])
end.
test() ->
ServerPid = start(),
Requests = [
{rectangle, 2, 3},
{square, 4},
history,
"hello",
history
],
send_requests(Requests, ServerPid).
send_requests([], _) ->
done;
send_requests([Req|Reqs], ServerPid) ->
client(ServerPid, Req),
send_requests(Reqs, ServerPid).
In the shell:
1> c(my).
{ok,my}
2> my:test().
{area,<0.64.0>,6}
{area,<0.64.0>,16}
{history,<0.64.0>,[{square,4},{rectangle,2,3}]}
{error,<0.64.0>,"invalid parameters"}
{history,<0.64.0>,[{error,"hello"},history,{square,4},{rectangle,2,3}]}
done
3>

How does one specify a forward reference in Erlang?

I have been plaing around with Erlang, and decided to try to make a directory lister. After hacking some code together I hit as road block, as the second commented line shows an error message. Literally it can't see the iterate function. I've done a bit of research here and on google. I have tried exporting the functions as well. There is something here that I am not thinking about correctly. Can someone point me in the correct direction?
-module(iterate_dir).
% exporting iterate/1 does not make it visible.
-export([start/0, iterate/1, show_files/2]).
show_files([], _) ->
ok;
show_files([Head|Tail], Path) ->
FullPath = [Path] ++ [Head],
case filelib:is_dir(FullPath) of
% function iteratate/1 undefined
true -> io:format("Dir ~s\n", [FullPath]), iteratate(FullPath);
false-> io:format("File ~s\n", [FullPath])
end,
show_files(Tail, Path).
iterate(Directory) ->
case file:list_dir(Directory) of
{ok, Files} -> show_files(Files, Directory);
{error, Reason} -> io:format("Error ~s~n", [Reason])
end.
start() ->
io:format("Running~n"),
iterate("c:\\"),
io:format("Complete~n").
The function is called "iterate", you are calling it as "iteratate"
notice the extra "at" in the middle at the call site (and comment)

Erlang Hash Tree

I'm working on a p2p app that uses hash trees.
I am writing the hash tree construction functions (publ/4 and publ_top/4) but I can't see how to fix publ_top/4.
I try to build a tree with publ/1:
nivd:publ("file.txt").
prints hashes...
** exception error: no match of right hand side value [67324168]
in function nivd:publ_top/4
in call from nivd:publ/1
The code in question is here:
http://github.com/AndreasBWagner/nivoa/blob/886c624c116c33cc821b15d371d1090d3658f961/nivd.erl
Where do you think the problem is?
Thank You,
Andreas
Looking at your code I can see one issue that would generate that particular exception error
publ_top(_,[],Accumulated,Level) ->
%% Go through the accumulated list of hashes from the prior level
publ_top(string:len(Accumulated),Accumulated,[],Level+1);
publ_top(FullLevelLen,RestofLevel,Accumulated,Level) ->
case FullLevelLen =:= 1 of
false -> [F,S|T]=RestofLevel,
io:format("~w---~w~n",[F,S]),
publ_top(FullLevelLen,T,lists:append(Accumulated,[erlang:phash2(string:concat([F],[S]))]),Level);
true -> done
end.
In the first function declaration you match against the empty list. In the second declaration you match against a list of length (at least) 2 ([F,S|T]). What happens when FullLevelLen is different from 1 and RestOfLevel is a list of length 1? (Hint: You'll get the above error).
The error would be easier to spot if you would pattern match on the function arguments, perhaps something like:
publ_top(_,[],Accumulated,Level) ->
%% Go through the accumulated list of hashes from the prior level
publ_top(string:len(Accumulated),Accumulated,[],Level+1);
publ_top(1, _, _, _) ->
done;
publ_top(_, [F,S|T], Accumulated, Level) ->
io:format("~w---~w~n",[F,S]),
publ_top(FullLevelLen,T,lists:append(Accumulated,[erlang:phash2(string:concat([F],[S]))]),Level);
%% Missing case:
% publ_top(_, [H], Accumulated, Level) ->
% ...

Resources