Avoid stackoverflow when recursively dismantling a string - recursion

I'm working on a solution for a problem for the advent of code 2018 (spoiler alert) where I need a function which takes a string (or a char list) and removes every pair of chars when they react. The exercise describes two chars, or "elements" in a "polymer", reacting when they are the same letter but only differ in case; so starting out with AbBc would leave you with Ac. Keep in mind that after a reaction two chars could end up next to each other, where they weren't before, and cause a new reaction.
I thought I could solve this by using a recursive function which only deals with the first two chars and recursively calls itself, but since the input string is quite large, this causes a stackoverflow exception:
let rec react polymer =
match polymer with
| [] -> []
| [x] -> [x]
| head::tail ->
let left = head
let right = List.head tail
let rest = List.tail tail
// 'reacts' takes two chars and
// returns 'true' when they react
match reacts left right with
// when reacts we go further with
// the rest as these two chars are
// obliterated
| true -> react rest
// no reaction means the left char
// remains intact and the right one
// could react with the first char
// of the rest
| false -> [left] # react tail
Then, just trying to solve the exercise to have a right answer to unit test against, I tried to do it imperatively, but that got messy real quick and now I'm kinda stuck. I'm teaching myself f# so any pointers are welcome. Can anyone solve this in a functional manner?

You can avoid stack overflow by rewriting your function to use tail recursion, which just means that the recursive call should be the last operation to execute.
When you do [left] # react tail you first make a recursive call, and then append [left] to the result of that. That means it has to keep the current function context, called a stack frame, around while it executes the recursive call, and if that recurses as well the stack frames add up until you get a stack overflow. But if there's no more work to be done in the current function context, the stack frame can be released (or reused), hence no stack overflow.
You can make it tail recursive by adding another function argument, conventionally called acc since it "accumulates" values. Instead of adding left to the return value of the recursive call we add it to the accumulator and pass that along. Then when we exhaust the input, we return the accumulator instead of the empty list.
I've also taken the liberty of the append, [left] # ..., as a cons, left::..., since the latter is much more efficient than the former. I've also moved left, right and rest to the pattern, since that's much neater and safer. You should generally avoid using List.head and List.tail since they fail on empty lists and are bugs just waiting to happen.
let rec react acc polymer =
match polymer with
| [] -> acc
| [x] -> x::acc
| left::right::rest ->
match reacts left right with
| true -> react acc rest
| false -> react (left::acc) (right::rest)
You could also use a guard instead of nested matches (which should really have been an if anyway):
let rec react acc polymer =
match polymer with
| [] ->
acc
| [x] ->
x::acc
| left::right::rest when reacts left right ->
react acc rest
| left::rest ->
react (left::acc) rest

Related

Recursion in F# Example

I've started learning F# and following in the footsteps of example problems I've written my own statement. It's simple enough, but I'm getting an error that doesn't exist in similar recursion examples.
My function replace takes a list of integers, an swapVal integer and a newVal integer. It then recurses through the list and changes any 'swapVal' to 'newVal'.
let original = [1;3;1;4;1;6;1;9]
let rec replace list origVal newVal =
match list with //look at list
| [] -> [] //when ls empty, return empty list
| head :: tail when head = origVal -> newVal :: replace tail origVal newVal
//when list is a head attached to a tail, if head = origVal,
//call replace on tail and appead the newVal
|_ :: tail -> replace tail origVal newVal
//head not equal to original value, call replace tail and return result
Calling replace original 1 5 I'm getting the following error Script.fsx(144,9): error FS0039: The value or constructor 'original' is not defined. Searching online like here hasn't turned up any solutions. Even O'Reilly's programming F# says that it could be a scoping problem, but there's no way that scope is my error.
I feel like it could be that f# is typing my arguments incorrectly, but I don't know enough about f# to know how it types. I would cast the arguments to make sure, but I read that it's not possible.
Does anyone see any immediate errors?

How can I get this function to be tail-recursive?

I'm still trying to implement 2-3 finger trees and I made good progress (repository). While doing some benchmarks I found out that my quite basic toList results in a StackOverflowException when the tree ist quite large. At first I saw an easy fix and made it tail-recursive.
Unfortunately, it turned out that toList wasn't the culprit but viewr was:
/// Return both the right-most element and the remaining tree (lazily).
let rec viewr<'a> : FingerTree<'a> -> View<'a> = function
| Empty -> Nil
| Single x -> View(x, lazyval Empty)
| Deep(prefix, deeper, One x) ->
let rest = lazy (
match viewr deeper.Value with
| Nil ->
prefix |> Digit.promote
| View (node, lazyRest) ->
let suffix = node |> Node.toList |> Digit.ofList
Deep(prefix, lazyRest, suffix)
)
View(x, rest)
| Deep(prefix, deeper, Digit.SplitLast(shorter, x)) ->
View(x, lazy Deep(prefix, deeper, shorter))
| _ -> failwith Messages.patternMatchImpossible
Looking for the only recursive call it is obvious that this is is not tail-recursive. Somehow I hoped this problem wouldn't exist because that call is wrapped in a Lazy which IMHO is similar to a continuation.
I heard and read of continuations but so far never (had to) use(d) them. I guess here I really need to. I've been staring at the code for quite some time, putting function parameters here and there, calling them other places… I'm totally lost!
How can this be done?
Update: The calling code looks like this:
/// Convert a tree to a list (left to right).
let toList tree =
let rec toList acc tree =
match viewr tree with
| Nil -> acc
| View(head, Lazy tail) -> tail |> toList (head::acc)
toList [] tree
Update 2: The code that caused the crash is this one.
let tree = seq {1..200000} |> ConcatDeque.ofSeq
let back = tree |> ConcatDeque.toList
The tree get built fine, I checked and it is only 12 levels deep. It's the call in line 2 that triggered the overflow.
Update 3: kvb was right, that pipe issue I ran into before has something to do with this. Re-testing the cross product of debug/release and with/without pipe it worked in all but one case: debug mode with the pipe operator crashed. The behavior was the same for 32 vs. 64 bit.
I'm quite sure that I was running release mode when posting the question but today it's working. Maybe there was some other factor… Sorry about that.
Although the crash is solved, I'm leaving the question open out of theoretical interest. After all, we're here to learn, aren't we?
So let me adapt the question:
From looking at the code, viewr is definitely not tail-recursive. Why doesn't it always blow up and how would one rewrite it using continuations?
Calling viewr never results in an immediate recursive call to viewr (the recursive call is protected by lazy and is not forced within the remainder of the call to viewr), so there's no need to make it tail recursive to prevent the stack from growing without bound. That is, a call to viewr creates a new stack frame which is then immediately popped when viewr's work is done; the caller can then force the lazy value resulting in a new stack frame for the nested viewr call, which is then immediately popped again, etc., so repeating this process doesn't result in a stack overflow.

Tail Recursions in erlang

I'm learning Erlang from the very basic and have a problem with a tail recursive function. I want my function to receive a list and return a new list where element = element + 1. For example, if I send [1,2,3,4,5] as an argument, it must return [2,3,4,5,6]. The problem is that when I send that exact arguments, it returns [[[[[[]|2]|3]|4]|5]|6].
My code is this:
-module(test).
-export([test/0]).
test()->
List = [1,2,3,4,5],
sum_list_2(List).
sum_list_2(List)->
sum_list_2(List,[]).
sum_list_2([Head|Tail], Result)->
sum_list_2(Tail,[Result|Head +1]);
sum_list_2([], Result)->
Result.
However, if I change my function to this:
sum_list_2([Head|Tail], Result)->
sum_list_2(Tail,[Head +1|Result]);
sum_list_2([], Result)->
Result.
It outputs [6,5,4,3,2] which is OK. Why the function doesn't work the other way around([Result|Head+1] outputing [2,3,4,5,6])?
PS: I know this particular problem is solved with list comprehensions, but I want to do it with recursion.
For this kind of manipulation you should use list comprehension:
1> L = [1,2,3,4,5,6].
[1,2,3,4,5,6]
2> [X+1 || X <- L].
[2,3,4,5,6,7]
it is the fastest and most idiomatic way to do it.
A remark on your fist version: [Result|Head +1] builds an improper list. the construction is always [Head|Tail] where Tail is a list. You could use Result ++ [Head+1] but this would perform a copy of the Result list at each recursive call.
You can also look at the code of lists:map/2 which is not tail recursive, but it seems that actual optimization of the compiler work well in this case:
inc([H|T]) -> [H+1|inc(T)];
inc([]) -> [].
[edit]
The internal and hidden representation of a list looks like a chained list. Each element contains a term and a reference to the tail. So adding an element on top of the head does not need to modify the existing list, but adding something at the end needs to mutate the last element (the reference to the empty list is replaced by a reference to the new sublist). As variables are not mutable, it needs to make a modified copy of the last element which in turn needs to mutate the previous element of the list and so on. As far as I know, the optimizations of the compiler do not make the decision to mutate variable (deduction from the the documentation).
The function that produces the result in reverse order is a natural consequence of you adding the newly incremented element to the front of the Result list. This isn't uncommon, and the recommended "fix" is to simply list:reverse/1 the output before returning it.
Whilst in this case you could simply use the ++ operator instead of the [H|T] "cons" operator to join your results the other way around, giving you the desired output in the correct order:
sum_list_2([Head|Tail], Result)->
sum_list_2(Tail, Result ++ [Head + 1]);
doing so isn't recommended because the ++ operator always copies it's (increasingly large) left hand operand, causing the algorithm to operate in O(n^2) time instead of the [Head + 1 | Tail] version's O(n) time.

F# recursive function arguments and stack overflow

I am somewhat new to F#, and I came across some strange behaviour when I was working with some recursive functions. I have two different versions of it below:
Version 1:
This causes a stack overflow, though it seems that it shouldn't (at least to my noob eyes)
let rec iMake acc =
match acc with
| 10 -> 100
| _ -> iMake acc+1
Version2:
This one works as I would expect it to.
let rec iMake acc =
match acc with
| 10 -> 100
| _ -> iMake (acc+1)
The only difference is that version 2 puts the acc+1 expression into parenthesis. So my question is, why does the first version not work, but the second one does? Does this mean that I should put all of my function arguments into parenthesis to avoid this type of stuff in the future ?
Function call has higher precedence than binary operator +. So the first function actually works like:
let rec iMake acc =
match acc with
| 10 -> 100
| _ -> (iMake acc)+1

Erlang Hash Tree

I'm working on a p2p app that uses hash trees.
I am writing the hash tree construction functions (publ/4 and publ_top/4) but I can't see how to fix publ_top/4.
I try to build a tree with publ/1:
nivd:publ("file.txt").
prints hashes...
** exception error: no match of right hand side value [67324168]
in function nivd:publ_top/4
in call from nivd:publ/1
The code in question is here:
http://github.com/AndreasBWagner/nivoa/blob/886c624c116c33cc821b15d371d1090d3658f961/nivd.erl
Where do you think the problem is?
Thank You,
Andreas
Looking at your code I can see one issue that would generate that particular exception error
publ_top(_,[],Accumulated,Level) ->
%% Go through the accumulated list of hashes from the prior level
publ_top(string:len(Accumulated),Accumulated,[],Level+1);
publ_top(FullLevelLen,RestofLevel,Accumulated,Level) ->
case FullLevelLen =:= 1 of
false -> [F,S|T]=RestofLevel,
io:format("~w---~w~n",[F,S]),
publ_top(FullLevelLen,T,lists:append(Accumulated,[erlang:phash2(string:concat([F],[S]))]),Level);
true -> done
end.
In the first function declaration you match against the empty list. In the second declaration you match against a list of length (at least) 2 ([F,S|T]). What happens when FullLevelLen is different from 1 and RestOfLevel is a list of length 1? (Hint: You'll get the above error).
The error would be easier to spot if you would pattern match on the function arguments, perhaps something like:
publ_top(_,[],Accumulated,Level) ->
%% Go through the accumulated list of hashes from the prior level
publ_top(string:len(Accumulated),Accumulated,[],Level+1);
publ_top(1, _, _, _) ->
done;
publ_top(_, [F,S|T], Accumulated, Level) ->
io:format("~w---~w~n",[F,S]),
publ_top(FullLevelLen,T,lists:append(Accumulated,[erlang:phash2(string:concat([F],[S]))]),Level);
%% Missing case:
% publ_top(_, [H], Accumulated, Level) ->
% ...

Resources