I've found a basic example of an f# recursive function that takes a list and returns a list of only even integers. I understand it for the most part, but there's a little i'm confused about.
let numbers = [1..4]
let rec even ls =
match ls with
| [] -> []
|head :: tail when head % 2 = 0 -> head :: even tail
|_::tail -> even tail
The line that matches head confuses me. This is how I read it. Append head to tail when head is even, then call even tail again. Because we appended head to tail, wouldn't that just get caught in a loop of adding head over and over again?
Also, the final line _::tail I assume means "do nothing, recurse again", but
I looked up the operator _ in f# and it says it's a wildcard pattern. Does that essentially mean if not covered by either of my first two matches, do this thing?
Hoping someone can clarify! f# looks like a lot of fun, but coming from an imperative background it's difficult to understand.
This is a pattern matching expression. The way to read each line is this: to the left of the arrow -> is the instruction of how to inspect the data, and to the right of the arrow is the instruction of what to do with the results of inspection.
Applying this to your case, follow the comments (I've inserted some newlines for clarity):
match ls with // Look at `ls`
| [] -> // when `ls` is an empty list,
[] // return an empty list
| head :: tail // when `ls` is a `head` attached to a `tail`,
when head % 2 = 0 -> // AND `head` is an even number,
head :: even tail // call `even tail`, attach `head` to it, and return that result
| _::tail -> // when `ls` is "something" attached to a `tail`,
even tail // call `even tail` and return that result
Note that the very last case will apply in all situations in which the second case applies. This ambiguity is resolved by the order of the cases: the program will attempt to match ("look at", "inspect") the datum according to each case in turn, and will execute code for the first case that matches.
Another subtle point that you seem to be missing: immutability. The list is immutable. You cannot "update" ("change", "alter") the list in place. If you have a list, it will always stay that way, the compiler guarantees it. Instead, the way data evolves through the program is by creating new data based on the old ones. In particular, if you say a :: b, this does not modify the list b, but instead creates a new list, in which a is head and b is tail.
These are pretty basic concepts in F#, and the fact that you're missing them suggests to me that you've only just started looking at the language (and perhaps at functional programming in general). If this is true, I recommend first reading some introductory material to familiarize yourself with basic concepts. My favourite is fsharpforfunandprofit.com, I can't recommend it enough.
Append head to tail when head is even, then call even tail again. Because we appended head to tail, wouldn't that just get caught in a loop of adding head over and over again?
Close but not quite. It's prepending head onto the list returned by recursing even tail. With each recursion the tail value in this pattern matching expression has one less item in it.
Also, the final line _::tail I assume means "do nothing, recurse again", but I looked up the operator _ in F# and it says it's a wildcard pattern. Does that essentially mean if not covered by either of my first two matches, do this thing?
_ can be a wildcard pattern, but in this pattern matching example it simply indicates we don't care about the head value when we destructure the list; we only care about the tail. This clause (because it comes after the clause that tests for evenness) causes non-even head values to be discarded.
You are dealing with immutable data-types (immutable list) here. This means lists are not changed, every operations creates and returns a new list.
Pattern Matching is a way to deconstruct data into multiple pieces. As an example. If you have the list [1;2;3;4] and you write.
let list = [1;2;3;4]
let (head :: tail) = list
Then head is the value 1. tail is the list [2;3;4] and list is still [1;2;3;4].
Let's go through your example step-by-step.
even [1;2;3;4] is called. Then there are three cases with Pattern Matching. The first one checks if [1;2;3;4] is an empty list. Its not so it checks the next one. head :: tail extracts the list into two pieces like above. So head is 1 and tail represents [2;3;4], but when head % 2 = 0 adds a conditional. It checks if head (currently 1) is dividable by two. It isn't, so it goes to the last Pattern Matching.
The last one is _::tail. First it does the exact same as head::tail. It extracts the first value 1 and stores it in the variable _. The reason to use _ as a variable name is to clarify that the first name is never used. You also could change _ to head and the code works the same.
The last Pattern Match matches, and now you have 1 stored in _ and [2;3;4] stored in tail. And all you do is call even tail. Or in that case, you call even [2;3;4] and return the result of this function call as your result.
When even [2;3;4] is called it does the same above.
It checks if its an empty list. It's not. Then it extract the first value 2 in head and [3;4] into tail. It checks the when condition. This time it is true. So now it executes head :: even tail
Or if we replace the values we get 2 :: even [3;4]
:: is a concatenation of a list, but before we can concatenate a list, first the function call even [3;4] needs to return a list. So even [3;4] is called.
This then checks again.
Is [3;4] an empty list. Nope.
Is the head the value 3 dividable by 2. Nope.
Extract 3 into _ and [4] into tail, and call even [4].
even [4] then does the same:
Is [4] an empty list. Nope.
Is the value 4 assigned to head an even number? Yes it is. So 4 :: even [] is called.
even [] then does the same:
Is [] an empty list. Yes it is. So return the empty list. []
Then it goes backwards.
-> means "returns"
even [] -> []
4 :: even [] -> [4]
even [3;4] -> [4]
2 :: even [3;4] -> [2;4]
even [2;3;4] -> [2;4]
even [1;2;3;4] -> [2;4]
Related
I want to understand this implementation of finding a prefix in a string, which is implemented without using any built-in list functions, but using recursion to iterate through the string.
attempt:
checkStringPrefix([C|StringTail],[C|LookupTail]) ->
checkStringPrefix(StringTail,LookupTail);
checkStringPrefix(_,[]) ->
"***";
checkStringPrefix(_String,_Lookup) ->
"".
It works, the function calls itself recursively separating the first Character from the Tail.
Example calls:
1> stringUtil:checkStringPrefix("test xxxxx", "test").
"***"
2> stringUtil:checkStringPrefix("test xxxxx", "testtt").
[]
In the case of two non equal characters, the last function variant gets called.
I dont fully grasp this concept, I would appreciate an explanation. I understand the process of recursive iteration, what I dont understand is why the second variants gets called in the correct moment.
Consider what happens when passing either single character strings or empty strings as arguments:
Passing "a" and "a": this calls the first clause of checkStringPrefix/2 because explicitly matches the first elements of both arguments in its function head, which also enforces that neither argument can be the empty list. This clause calls the function recursively, and since neither argument has a tail, the second function clause gets called because the second argument matches the empty list, so the result is "***".
Passing "a" and "b": this won't call the first clause because the first elements do not match, and it won't call the second clause because the second argument isn't the empty list. It therefore calls the third clause, so the result is "".
Passing "" and "": this won't call the first clause because both arguments are empty lists; it calls the second clause because the second argument matches the empty list, so the result is "***". In fact, since the second clause treats its first argument as a don't-care, this same analysis applies even if the first argument is non-empty.
Passing "" and "a": this won't call the first clause because the first argument is empty, and it won't call the second clause because the second argument is not empty, so it calls the third clause and the result is "".
No matter the lengths of the two argument strings, these are the choices for each invocation.
To answer your specific question about when the second function clause gets called, it happens any time the second argument is the empty list because of the specific match for that case in the function head, and that function head also ignores the first argument. Matching occurs in order of declaration; the first function clause requires both arguments to have at least one element, so when the second argument is the empty list, it can't match the first clause and so matches the second clause.
I would like to see a code snippet of julia that will read a file and return lines (string type) that match a regular expression.
I welcome multiple techniques, but output should be equivalent to the following:
$> grep -E ^AB[AJ].*TO' 'webster-unabridged-dictionary-1913.txt'
ABACTOR
ABATOR
ABATTOIR
ABJURATORY
I'm using GNU grep 3.1 here, and the first line of each entry in the file is the all caps word on its own.
You could also use the filter function to do this in one line.
filter(line -> ismatch(r"^AB[AJ].*TO",line),readlines(open("webster-unabridged-dictionary-1913.txt")))
filter applies a function returning a Boolean to an array, and only returns those elements of the array which are true. The function in this case is an anonymous function line -> ismatch(r"^AB[AJ].*TO",line)", which basically says to call each element of the array being filtered (each line, in this case) line.
I think this might not be the best solution for very large files as the entire file needs to be loaded into memory before filtering, but for this example it seems to be just as fast as the for loop using eachline. Another difference is that this solution returns the results as an array rather than printing each of them, which depending on what you want to do with the matches might be a good or bad thing.
My favored solution uses a simple loop and is very easy to understand.
julia> open("webster-unabridged-dictionary-1913.txt") do f
for i in eachline(f)
if ismatch(r"^AB[AJ].*TO", i) println(i) end
end
end
ABACTOR
ABATOR
ABATTOIR
ABJURATORY
notes
Lines with tab separations have the tabs preserved (no literal output of '\t')
my source file in this example has the dictionary words in all caps alone on one line above the definition; the complete line is returned.
the file I/O operation is wrapped in a do block syntax structure, which expresses an anonymous function more conveniently than lamba x -> f(x) syntax for multi-line functions. This is particularly expressive with the file open() command, defined with a try-finally-close operation when called with a function as an argument.
Julia docs: Strings/Regular Expressions
regex objects take the form r"<regex_literal_here>"
the regex itself is a string
based on perl PCRE library
matches become regex match objects
example
julia> reg = r"^AB[AJ].*TO";
julia> typeof(reg)
Regex
julia> test = match(reg, "ABJURATORY")
RegexMatch("ABJURATO")
julia> typeof(test)
RegexMatch
Just putting ; in front is Julia's way to using commandline commands so this works in Julia's REPL
;grep -E ^AB[AJ].*TO' 'webster-unabridged-dictionary-1913.txt'
I'm trying to implement a sliding window algorithm for matching words in a text file. I come from a procedural background and my first attempt to do this in a functional language like Erlang seems to require time O(n^2) (or even more). How would one do this in a functional language?
-module(test).
-export([readText/1,patternCount/2,main/0]).
readText(FileName) ->
{ok,File} = file:read_file(FileName),
unicode:characters_to_list(File).
patternCount(Text,Pattern) ->
patternCount_(Text,Pattern,string:len(Pattern),0).
patternCount_(Text,Pattern,PatternLength,Count) ->
case string:len(Text) < PatternLength of
true -> Count;
false ->
case string:equal(string:substr(Text,1,PatternLength),Pattern) of
true ->
patternCount_(string:substr(Text,2),Pattern,PatternLength,Count+1);
false ->
patternCount_(string:substr(Text,2),Pattern,PatternLength,Count)
end
end.
main() ->
test:patternCount(test:readText("file.txt"),"hello").
Your question is a bit too broad, since it asks about implementing this algorithm in functional languages but how best to do that is language-dependent. My answer therefore focuses on Erlang, given your example code.
First, note that there's no need to have separate patternCount and patternCount_ functions. Instead, you can just have multiple patternCount functions with different arities as well as multiple clauses of the same arity. First, let's rewrite your functions to take that into account, and also replace calls to string:len/1 with the length/1 built-in function:
patternCount(Text,Pattern) ->
patternCount(Text,Pattern,length(Pattern),0).
patternCount(Text,Pattern,PatternLength,Count) ->
case length(Text) < PatternLength of
true -> Count;
false ->
case string:equal(string:substr(Text,1,PatternLength),Pattern) of
true ->
patternCount(string:substr(Text,2),Pattern,PatternLength,Count+1);
false ->
patternCount(string:substr(Text,2),Pattern,PatternLength,Count)
end
end.
Next, the multi-level indentation in the patternCount/4 function is a "code smell" indicating it can be done better. Let's split that function into multiple clauses:
patternCount(Text,Pattern,PatternLength,Count) when length(Text) < PatternLength ->
Count;
patternCount(Text,Pattern,PatternLength,Count) ->
case string:equal(string:substr(Text,1,PatternLength),Pattern) of
true ->
patternCount(string:substr(Text,2),Pattern,PatternLength,Count+1);
false ->
patternCount(string:substr(Text,2),Pattern,PatternLength,Count)
end.
The first clause uses a guard to detect that no more matches are possible, while the second clause looks for matches. Now let's refactor the second clause to use Erlang's built-in matching. We want to advance through the input text one element at a time, just as the original code does, but we also want to detect matches as we do so. Let's perform the matches in our function head, like this:
patternCount(_Text,[]) -> 0;
patternCount(Text,Pattern) ->
patternCount(Text,Pattern,Pattern,length(Pattern),0).
patternCount(Text,_Pattern,_Pattern,PatternLength,Count) when length(Text) < PatternLength ->
Count;
patternCount(Text,[],Pattern,PatternLength,Count) ->
patternCount(Text,Pattern,Pattern,PatternLength,Count+1);
patternCount([C|TextTail],[C|PatternTail],Pattern,PatternLength,Count) ->
patternCount(TextTail,PatternTail,Pattern,PatternLength,Count);
patternCount([_|TextTail],_,Pattern,PatternLength,Count) ->
patternCount(TextTail,Pattern,Pattern,PatternLength,Count).
First, note that we added a new argument to the bottom four clauses: we now pass Pattern as both the second and third arguments to allow us to use one of them for matching and one of them to maintain the original pattern, as explained more fully below. Note also that we added a new clause at the very top to check for an empty Pattern and just return 0 in that case.
Let's focus only on the bottom three patternCount/5 clauses. These clauses are tried in order at runtime, but let's look at the second of these three clauses first, then the third clause, then the first of the three:
In the second of these three clauses, we write the first and second arguments in [Head|Tail] list notation, which means Head is the first element of the list and Tail is the rest of the list. We use the same variable for the head of both lists, which means that if the first elements of both lists are equal, we have a potential match in progress, so we then recursively call patternCount/5 passing the tails of the lists as the first two arguments. Passing the tails allows us to advance through both the input text and the pattern an element at a time, checking for matching elements.
In the last clause, the heads of the first two arguments do not match; if they did, the runtime would execute the second clause, not this one. This means that our pattern match has failed, and so we no longer care about the first element of the first argument nor about the second argument, and we have to advance through the input text to look for a new match. Note that we write both the head of the input text and the second argument as the _ "don't care" variable, as they are no longer important to us. We recursively call patternCount/5, passing the tail of the input text as the first argument and the full Pattern as the second argument, allowing us to start looking for a new match.
In the first of these three clauses, the second argument is the empty list, which means we've gotten here by successfully matching the full Pattern, element by element. So we recursively call patternCount/5 passing the full Pattern as the second argument to start looking for a new match, and we also increment the match count.
Try it! Here's the full revised module:
-module(test).
-export([read_text/1,pattern_count/2,main/0]).
read_text(FileName) ->
{ok,File} = file:read_file(FileName),
unicode:characters_to_list(File).
pattern_count(_Text,[]) -> 0;
pattern_count(Text,Pattern) ->
pattern_count(Text,Pattern,Pattern,length(Pattern),0).
pattern_count(Text,_Pattern,_Pattern,PatternLength,Count)
when length(Text) < PatternLength ->
Count;
pattern_count(Text,[],Pattern,PatternLength,Count) ->
pattern_count(Text,Pattern,Pattern,PatternLength,Count+1);
pattern_count([C|TextTail],[C|PatternTail],Pattern,PatternLength,Count) ->
pattern_count(TextTail,PatternTail,Pattern,PatternLength,Count);
pattern_count([_|TextTail],_,Pattern,PatternLength,Count) ->
pattern_count(TextTail,Pattern,Pattern,PatternLength,Count).
main() ->
pattern_count(read_text("file.txt"),"hello").
A few final recommendations:
Searching through text element by element is slower than necessary. You should have a look at the Boyer-Moore algorithm and other related algorithms to see ways of advancing through text in larger chunks. For example, Boyer-Moore attempts to match at the end of the pattern first, since if that's not a match, it can advance through the text by as much as the full length of the pattern.
You might want to also looking into using Erlang binaries rather than lists, as they are more compact memory-wise and they allow for matching more than just their first elements. For example, if Text is the input text as a binary and Pattern is the pattern as a binary, and assuming the size of Text is equal to or greater than the size of Pattern, this code attempts to match the whole pattern:
case Text of
<<Pattern:PatternLength/binary, TextTail/binary>> = Text ->
patternCount(TextTail,Pattern,PatternLength,Count+1);
<<_/binary,TextTail/binary>> ->
patternCount(TextTail,Pattern,PatLen,Count)
end.
Note that this code snippet reverts to using patternCount/4 since we no longer need the extra Pattern argument to work through element by element.
As shown in the full revised module, when calling functions in the same module, you don't need the module prefix. See the simplified main/0 function.
As shown in the full revised module, conventional Erlang style does not use mixed case function names like patternCount. Most Erlang programmers would use pattern_count instead.
I just started learning Prolog. One of the exercise wants me to write backwards/2, a predicate that is true when the second list argument is the reverse of the first list argument. And I have to use a paste predicate to write backwards.
paste([], L, L).
paste([X|L1], L2, [X|L3]) :- paste(L1,L2,L3).
I worked on it for hours but still haven't gotten the answer.
Here's what I have:
backwards([H|T], Lrev) :- paste([X|T], X, Lrev).
backwards([],[]).
Prolog lists are a very simple data structure.
An empty list is denoted by the atom [].
A non-empty list is denoted by the structure '.'/2, where the first argument is the list's head and the second argument is the list's tail, another list, empty or non-empty.
The Prolog list notation is syntactic sugar over that data structure:
[] is the empty list in both notations.
For non-empty lists,
[a] is the structure '.'(a,[]).
[a,b] is the structure '.'(a,'.'(b,[]))
[a,b,c] is the structure '.'(a,'.'(b,'.'(c,[])))
[Head|Tail] is exactly equivalent to '.'(Head,Tail).
[a|[b,c]] is exactly equivalent to [a,b,c] and to '.'(a,'.'(b,'.'(c,[])))
It is easy to see why one might prefer a little syntactic sugar.
The structure of a list then, like a classic singly linked list in more ... conventional ... languages, makes it trivial to add and remove items from the head (left) end of an existing list.
One should note that if, one at a time you remove items from one list and add items to another, you produce the first list in reverse order.
A common Prolog idiom is the use of a helper or accumulator argument that carries state through the recursion. Often this helper will be seeded with an initial value. One might also note that most recursive problems have one or two special cases and the broader general case.
Such a predicate that reverses a list might look something like this:
paste( [] , Rs , Rs ) . % Special Case: if the source list is empty, unify the accumulator with the result.
paste( [X|Xs] , T , Rs ) :- % General Case:
T1 = [X|T] , % - prepend the current list head to the temporary, creating a new temporary, and
paste(Xs,T1,Rs) % - recurse down.
.
One might note that the second clause above could be refactored for simplicity to remove the explicit creation of the new temporary:
paste( [] , Rs , Rs ) . % Special Case: if the source list is empty, unify the accumulator with the result.
paste( [X|Xs] , T , Rs ) :- % General Case:
paste(Xs,[X|T],Rs) % - prepend the current head to the temporary and recurse down.
.
Either way, this predicate REQUIRES that the temporary be seeded with an initial empty list ([]).
Calling it thusly, then:
paste( [a,b,c] , [] , R ).
produces
R = [c,b,a]
Once you have your paste/3 predicate, creating your backwards/2 predicate is simplicity itself:
backwards( L , R ) :- paste( L , [] , R ) .
This exposes another common Prolog idiom: a trivial "public" predicate that invokes a "private" worker predicate that actually does all the work.
Heres a snippet:
translate("a", "4").
translate("m", "/\\/\\").
tol33t([], []).
tol33t([Upper|UpperTail], [Lower|LowerTail]) :-
translate([Upper], [Lower]),
tol33t(UpperTail, LowerTail).
Basically what i want to do is look up in the table for a letter and then get that letter and add it to the new list.
What i have works if its a character, but I'm not sure how to append the new list of characters with the old.
Example input:
l33t("was", L).
It will be put through like this:
l33t([119,97,115], L).
Now that should come back as:
[92,47,92,47]++[52]++[53] or [92,47,92,47,52,53]
Problem is i don't know how to append it like that.
Consider these modifications to tol33t/2:
tol33t([], []).
tol33t([Code|Codes], Remainder) :-
translate([Code], Translation), !,
tol33t(Codes, Rest),
append(Translation, Rest, Remainder).
tol33t([Code|Codes], [Code|Remainder]) :-
tol33t(Codes, Remainder).
The first clause is the base case.
The second clause will succeed iff there is a translation for the current Code via translate/2, as a list of characters of arbitrary length (Translation - note you had [Lower] instead, which restricted results to lists of length 1 only). The cut (!) after the check for a code translation commits to finding the Rest of the solution recursively and then appends the Translation to the front, as the Remainder to return.
The third clause is executed iff there was no translation for the current Code (i.e., the call to translate/2) in the second clause. In this case, no translation for the Code means we just return it as is and compute the rest.
EDIT:
If you don't have cut (!), the second and third clauses can be combined to become:
tol33t([Code|Codes], Remainder) :-
tol33t(Codes, Rest),
(translate([Code], Translation) ->
append(Translation, Rest, Remainder)
; Remainder = [Code|Rest]
).
This (unoptimized) version checks, at every Code in the character list, if there is a translate/2 that suceeds; if so, the Translation is appended to the Rest, else the Code is passed through unchanged. Note that this has the same semantics as the implementation above, in that solutions are commited to (i.e., simulating a cut !) if the antecedent to -> (translate/2) succeeds. Note that the cut in both implementations is strictly necessary; without it, the program will backtrack to find solutions where Code bindings are not translated where there exists an applicable translate/2 clause.