This is from the SICP book that I am sure many of you are familiar with. This is an early example in the book, but I feel an extremely important concept that I am just not able to get my head around yet. Here it is:
(define (cons x y)
(define (dispatch m)
(cond ((= m 0) x)
((= m 1) y)
(else (error "Argument not 0 or 1 - CONS" m))))
dispatch)
(define (car z) (z 0))
(define (cdr z) (z 1))
So here I understand that car and cdr are being defined within the scope of cons, and I get that they map some argument z to 1 and 0 respectively (argument z being some cons). But say I call (cons 3 4)...how are the arguments 3 and 4 evaluated, when we immediately go into this inner-procedure dispatch which takes some argument m that we have not specified yet? And, maybe more importantly, what is the point of returning 'dispatch? I don't really get that part at all. Any help is appreciated, thanks!
This is one of the weirder (and possibly one of the more wonderful) examples of exploiting first-class functions in Scheme. Something similar is also in the Little Schemer, which is where I first saw it, and I remember scratching my head for days over it. Let me see if I can explain it in a way that makes sense, but I apologize if it's not clear.
I assume you understand the primitives cons, car, and cdr as they are implemented in Scheme already, but just to remind you: cons constructs a pair, car selects the first component of the pair and returns it, and cdr selects the second component and returns it. Here's a simple example of using these functions:
> (cons 1 2)
(1 . 2)
> (car (cons 1 2))
1
> (cdr (cons 1 2))
2
The version of cons, car, and cdr that you've pasted should behave exactly the same way. I'll try to show you how.
First of all, car and cdr are not defined within the scope of cons. In your snippet of code, all three (cons, car, and cdr) are defined at the top-level. The function dispatch is the only one that is defined inside cons.
The function cons takes two arguments and returns a function of one argument. What's important about this is that those two arguments are visible to the inner function dispatch, which is what is being returned. I'll get to that in a moment.
As I said in my reminder, cons constructs a pair. This version of cons should do the same thing, but instead it's returning a function! That's ok, we don't really care how the pair is implemented or laid out in memory, so long as we can get at the first and second components.
So with this new function-based pair, we need to be able to call car and pass the pair as an argument, and get the first component. In the definition of car, this argument is called z. If you were to execute the same REPL session I had above with these new cons ,car, and cdr functions, the argument z in car will be bound to the function-based pair, which is what cons returns, which is dispatch. It's confusing, but just think it through carefully and you'll see.
Based on the implementation of car, it appears to be that it take a function of one argument, and applies it to the number 0. So it's applying dispatch to 0, and as you can see from the definition of dispatch, that's what we want. The cond inside there compares m with 0 and 1 and returns either x or y. In this case, it returns x, which is the first argument to cons, in other words the first component of the pair! So car selects the first component, just as the normal primitive does in Scheme.
If you follow this same logic for cdr, you'll see that it behaves almost the same way, but returns the second argument to cons, y, which is the second component of the pair.
There are a couple of things that might help you understand this better. One is to go back to the description of the substitution model of evaluation in Chapter 1. If you carefully and meticulously follow that substitution model for some very simple example of using these functions, you'll see that they work.
Another way, which is less tedious, is to try playing with the dispatch function directly at the REPL. Below, the variable p is defined to refer to the dispatch function returned by cons.
> (define p (cons 1 2))
#<function> ;; what the REPL prints here will be implementation specific
> (p 0)
1
> (p 1)
2
The code in the question shows how to redefine the primitive procedure cons that creates a cons-cell (a pair of two elements: the car and the cdr), using only closures and message-dispatching.
The dispatch procedure acts as a selector for the arguments passed to cons: x and y. If the message 0 is received, then the first argument of cons is returned (the car of the cell). Likewise, if 1 is received, then the second argument of cons is returned (the cdr of the cell). Both arguments are stored inside the closure defined implicitly for the dispatch procedure, a closure that captures x and y and is returned as the product of invoking this procedural implementation of cons.
The next redefinitions of car and cdr build on this: car is implemented as a procedure that passes 0 to a closure as returned in the above definition, and cdr is implemented as a procedure that passes 1 to the closure, in each case ultimately returning the original value that was passed as x and y respectively.
The really nice part of this example is that it shows that the cons-cell, the most basic unit of data in a Lisp system can be defined as a procedure, therefore blurring the distinction between data and procedure.
This is the "closure/object isomorphism", basically.
The outer function (cons) is a class constructor. It returns an object, which is a function of one argument, where the argument is equivalent to the name of a method. In this case, the methods are getters, so they evaluate to values. You could just as easily have stored more procedures in the object returned by the constructor.
In this case, numbers where chosen as method names and sugary procedures defined outside the object itself. You could have used symbols:
(define (cons x y)
(lambda (method)
(cond ((eq? method 'car) x)
((eq? method 'cdr) y)
(else (error "unknown method")))))
In which case what you have more closely resembles OO:
# (define p (cons 1 2))
# (p 'car)
1
# (p 'cdr)
2
Related
I'm trying to implement higher level functions in my Racket code, specifically with regards to this function:
(define (indivisible e L)
(map (remove 0 ((map ((lambda (x y) (modulo x y))) L e)))))
Essentially, I'm trying to remove all the elements that are divisible by e from the list. However, it keeps giving me an error that says that "the expected number of arguments did not match the given number (0 vs 2)". Why is this so?
Several places you have two sets of parentheses. Unless the parentheses are a part of a special form or macro, eg. let, it represent an application. Ie.
((lambda (x y) (modulo x y)))
Here the form (lambda ...) is evaluated and become a function. The second set of parentheses calls this function with no arguments. Since you have two arguments, x and y and not supplying any in your application it signals an error.
Another place where you do the same is around (map ....). Since I know map always evaluates to a list or null it looks kind of strange that you call it as a function ((map ...)).
If you are more familiar with algol languages like python, what you are doing is like someFunc(arg1 args2)() where you clearly see someFunc needs to return a function wince it's immediately called afterwards. The same in Scheme looks like ((some-func arg1 arg2)).
remove removes the first argument from the second argument list. It does not return a function so the outer map won't work.
To solve this I think you are looking for filter. You only need to make a predicate that is #f for the elements you don't want and you're done.
For Racket programming language, why is lambda not considered a function?
For example, it can't be defined as a higher order function like this.
(define (my-lambda args body)
(lambda args body))
There's a key distinction that your question is missing:
lambda is syntax.
Procedures are values.
A lambda form is a form of expression whose value is a procedure. The question whether "lambda is a function" starts off with a type error, so to speak, because lambdas and procedures don't live in the same world.
But let's set that aside. The other way to look at this is by thinking of it in terms of evaluation rules. The default Scheme evaluation rule, for the application of a procedure to arguments, can be expressed in pseudo-code like this:
(define (eval-application expr env)
(let ((values
;; Evaluate each subexpression in the same environment as the
;; enclosing expression, and collect the result values.
(map (lambda (subexpr) (eval subexpr env))
expr)))
;; Apply the first value (which must be a procedure) to the
;; other ones in the results.
(apply (car values) (cdr values))))
In English:
Evaluate all of the subexpressions in the same environment as the "parent".
apply the first result (which must have evaluated to a procedure) to the list of the rest.
And now, another reason lambda can't be a procedure is that this evaluation rule doesn't work for lambda expressions. In particular, the point of lambda is to not evaluate its body right away! This, in particular, is what afflicts your my-lambda—if you try to use it this way:
(my-lambda (x) (+ x x))
...the (x) in the middle must be immediately evaluated as an invocation of a procedure named x in the environment where the whole expression appears. The (+ x x) must also be immediately evaluated.
So lambda requires its own evaluation rule. As Basile's answer points out, this is normally implemented as a primitive in the Scheme system implementation, but we can sketch it in pseudocode with something like this:
;;;
;;; Evaluate an expression of this form, returning a procedure:
;;;
;;; (lambda <formals> <body> ...)
;;;
(define (eval-lambda expr env)
(let ((formals (second expr))
(body (cddr expr)))
;; We don't evaluate `body` right away, we return a procedure.
(lambda args
;; `formals` is never evaluated, since it's not really an
;; expression on its own, but rather a subpart that cannot
;; be severed from its enclosing `lambda`. Or if we want to
;; say it all fancy, the `formals` is *syncategorematic*...
(let ((bindings (make-bindings formals args)))
;; When the procedure we return is called, *then* we evaluate
;; the `body`--but in an extended environment that binds its
;; formal parameters to the arguments supplied in that call.
(eval `(begin ,#body) (extend-environment env bindings))))))
;;;
;;; "Tie" each formal parameter of the procedure to the corresponding
;;; argument values supplied in a given call. Returns the bindings
;;; as an association list.
;;;
(define (make-bindings formals args)
(cond ((symbol? formals)
`((,formals . args)))
((pair? formals)
`((,(car formals) . ,(car args))
,#(make-bindings (cdr formals) (cdr args))))))
To understand this pseudocode, the time-tested thing is to study one of the many Scheme books that demonstrate how to build a meta-circular interpreter (a Scheme interpreter written in Scheme). See for example this section of Structure and Interpretation of Computer programs.
lambda needs to be a core language feature (like if, let, define are) in Scheme because it is constructing a closure so needs to manage the set of closed or free variables (and somehow put their binding in the closure).
For example:
(define (translate d) (lambda (x) (+ d x)))
When you invoke or evaluate (translate 3) the d is 3 so the dynamically constructed closure should remember that d is bound to 3. BTW, you generally want the result of (translate 3) and of (translate 7) be two different closures sharing some common code (but having different bindings for d).
Read also about λ-calculus.
Explaining that all in details requires an entire book. Fortunately, C. Queinnec has written it, so read his Lisp In Small Pieces book.
(If you read French, you could read the latest French version of that book)
See also the Kernel programming language.
Read also wikipage about evaluation strategy.
PS. You could, and some Lisp implementations (notably MELT and probably SBCL) do that, define lambda as some macro -e.g. which would expand to building some closure in an implementation specific way (but lambda cannot be defined as a function).
A function call (e0 e1 e2) is evaluated like this
e0 is evaluated, the result is (hopefully) a function f
e1 is evaluated, the result is a value v1
e2 is evaluated, the result is a value v2
The function body of f is evaluated in an environment in which
the formal parameters are bound to the values v1 and v2.
Note that all expressions e0, e1, and, e2 are evaluated before the body of the function is activated.
This means that a function call like (foo #t 2 (/ 3 0)) will result in an error when (/ 3 0) is evaluated - before control is handed over to the body of foo.
Now consider the special form lambda. In (lambda (x) (+ x 1)) this creates a function of one variable x which when called with a value v will compute (+ v 1).
If in contrast lambda were a function, then the expressions (x) and (+ x 1) are evaluated before the body of lambda is activated. And now (x) will most likely produce an error - since (x) means call the function x with no arguments.
In short: Function calls will always evaluate all arguments, before the control is passed to the function body. If some expressions are not to be evaluated a special form is needed.
Here lambda is a form, that don't evaluate all subforms - so lambda needs to be a special form.
In Scheme lingo we use the term procedure instead of function throughout the standard report. Thus since this is about scheme dialects I'll use the term procedure.
In eager languages like standard #!racket and #!r6rs procedures get their arguments evaluated before the body is evaluated with the new lexical environment. Thus since if and lambda has special evaluation rules than for procedures special forms and macros are the way to introduce new syntax.
In a lazy language like #!lazy racket evaluation is by need and thus many forms that are implemented as macros/special forms in an eager language can be implemented as procedure. eg. you can make if as a procedure using cond but you cannot make cond using if because the terms themselves would be evaluated as forms on access and eg (cond (#t 'true-value)) would fail since #t is not a procedure. lambda has similar issue with the argument list.
(defun triangle-using-cond (number)
(cond
((<= number 0) 0) ; 1st
((= number 1) 1) ; 2nd
((> number 1) ; 3rd
;; 4th
(+ number
(triangle-using-cond (1- number))))))
Things that I know about Cond
It allows multiple test and alternative expressions
It has pre-specified evaluation order. For instance, the first condition will always evaluated whether it is right or not
One thing that I cannot distinguish is that what makes cond different from a function!
A function call (e0 e1 e2) is evaluated like this
1. e0 is evaluated, the result is (hopefully) a function f
2. e1 is evaluated, the result is a value v1
3. e2 is evaluated, the result is a value v2
4. The function body of `f` is evaluated in an environment in which
the formal parameters are bound to the values `v1` and `v2`.
Note that all expressions e0, e1, and, e2 are evaluated before the body of the function is activated.
This means that a function call like (foo #t 2 (/ 3 0)) will result in an error when (/ 3 0) is evaluated - before control is handed over to the body of foo.
Now consider the special form if. In (if #t 2 (/ 3 0)) the expressions #t is evaluated and since the value non-false, the second expression 2 is evaluated and the resulting value is 2. Here (/ 3 0) is never evaluated.
If in contrast if were a function, then the expressions #t, 2, and, (/ 3 0) are evaluated before the body of is activated. And now (/ 3 0) will produce an error - even though the value of that expressions is not needed.
In short: Function calls will always evaluate all arguments, before the control is passed to the function body. If some expressions are not to be evaluated a special form is needed.
Here if and cond are examples of forms, that don't evaluate all subexpressions - so they they need to be special forms.
If cond were not a special form then the expression:
((> number 1) ;;3rd
(+ number (triangle-using-cond (1- number))))
would cause either:
an infinite loop because triangle-using-cond would keep calling itself recursively via the tail call (triangle-using-cond (1- number)).
or, the last expression would try to apply the value #f or #t as a function (which in a type-2 Lisp such as ANSI Common Lisp is possible, but not possible in a type-1 Lisp such as Racket or Scheme) and produce an error.
What makes cond a special form is that its arguments are evaluated lazily whereas functions in Scheme or Common Lisp evaluate their arguments eagerly.
As already answered, all arguments to a call to some function f are evaluated before the result of applying f is computed. Does it however mean that cond, or if, or both should be special forms?
Well, first, if you have if, you can easily simulate a cond with nested tests. Conversely, if is just a degenerate form of cond. So you can say that it is sufficient to have one of them a special form. Let's choose if because it is simpler to specify.
So shall if be special?
It doesn't really need to...
If the underlying question is "is if expressible in terms of a smaller set of special forms?", then the answers is yes: just implement if in terms of functions:
(define true-fn (lambda (then else) (then)))
(define false-fn (lambda (then else) (else)))
Whenever you can return a boolean, you return one of the above function instead.
You could for example decide to bind #t and #f to those functions.
Notice how they call one of the two input parameters.
((pair? x) ;; returns either true-fn or false-fn
(lambda () (+ x 1))
(lambda () x))
...but why code in lambda calculus?
Evaluating code conditionally is really a fundamental operation of computing. Trying to find a minimal special forms where you cannot express that directly leads to a poorer programming language from the perspective of the programmer, however "clean" the core language is.
From a certain point of view, the if form (or cond) is necessary because without them it becomes really hard to express conditional execution in a way that a compiler/interpreter can handle efficiently.
This document referenced by uselpa discuss using closures to implement if, and concludes:
However, the syntactic inconvenience would be so great that even
Scheme defines if as a special form.
In Haskell, like in many other functional languages, the function foldl is defined such that, for example, foldl (-) 0 [1,2,3,4] = -10.
This is OK, because foldl (-) 0 [1, 2,3,4] is, by definition, ((((0 - 1) - 2) - 3) - 4).
But, in Racket, (foldl - 0 '(1 2 3 4)) is 2, because Racket "intelligently" calculates like this: (4 - (3 - (2 - (1 - 0)))), which indeed is 2.
Of course, if we define auxiliary function flip, like this:
(define (flip bin-fn)
(lambda (x y)
(bin-fn y x)))
then we could in Racket achieve the same behavior as in Haskell: instead of (foldl - 0 '(1 2 3 4)) we can write: (foldl (flip -) 0 '(1 2 3 4))
The question is: Why is foldl in racket defined in such an odd (nonstandard and nonintuitive) way, differently than in any other language?
The Haskell definition is not uniform. In Racket, the function to both folds have the same order of inputs, and therefore you can just replace foldl by foldr and get the same result. If you do that with the Haskell version you'd get a different result (usually) — and you can see this in the different types of the two.
(In fact, I think that in order to do a proper comparison you should avoid these toy numeric examples where both of the type variables are integers.)
This has the nice byproduct where you're encouraged to choose either foldl or foldr according to their semantic differences. My guess is that with Haskell's order you're likely to choose according to the operation. You have a good example for this: you've used foldl because you want to subtract each number — and that's such an "obvious" choice that it's easy to overlook the fact that foldl is usually a bad choice in a lazy language.
Another difference is that the Haskell version is more limited than the Racket version in the usual way: it operates on exactly one input list, whereas Racket can accept any number of lists. This makes it more important to have a uniform argument order for the input function).
Finally, it is wrong to assume that Racket diverged from "many other functional languages", since folding is far from a new trick, and Racket has roots that are far older than Haskell (or these other languages). The question could therefore go the other way: why is Haskell's foldl defined in a strange way? (And no, (-) is not a good excuse.)
Historical update:
Since this seems to bother people again and again, I did a little bit of legwork. This is not definitive in any way, just my second-hand guessing. Feel free to edit this if you know more, or even better, email the relevant people and ask. Specifically, I don't know the dates where these decisions were made, so the following list is in rough order.
First there was Lisp, and no mention of "fold"ing of any kind. Instead, Lisp has reduce which is very non-uniform, especially if you consider its type. For example, :from-end is a keyword argument that determines whether it's a left or a right scan and it uses different accumulator functions which means that the accumulator type depends on that keyword. This is in addition to other hacks: usually the first value is taken from the list (unless you specify an :initial-value). Finally, if you don't specify an :initial-value, and the list is empty, it will actually apply the function on zero arguments to get a result.
All of this means that reduce is usually used for what its name suggests: reducing a list of values into a single value, where the two types are usually the same. The conclusion here is that it's serving a kind of a similar purpose to folding, but it's not nearly as useful as the generic list iteration construct that you get with folding. I'm guessing that this means that there's no strong relation between reduce and the later fold operations.
The first relevant language that follows Lisp and has a proper fold is ML. The choice that was made there, as noted in newacct's answer below, was to go with the uniform types version (ie, what Racket uses).
The next reference is Bird & Wadler's ItFP (1988), which uses different types (as in Haskell). However, they note in the appendix that Miranda has the same type (as in Racket).
Miranda later on switched the argument order (ie, moved from the Racket order to the Haskell one). Specifically, that text says:
WARNING - this definition of foldl differs from that in older versions of Miranda. The one here is the same as that in Bird and Wadler (1988). The old definition had the two args of `op' reversed.
Haskell took a lot of stuff from Miranda, including the different types. (But of course I don't know the dates so maybe the Miranda change was due to Haskell.) In any case, it's clear at this point that there was no consensus, hence the reversed question above holds.
OCaml went with the Haskell direction and uses different types
I'm guessing that "How to Design Programs" (aka HtDP) was written at roughly the same period, and they chose the same type. There is, however, no motivation or explanation — and in fact, after that exercise it's simply mentioned as one of the built-in functions.
Racket's implementation of the fold operations was, of course, the "built-ins" that are mentioned here.
Then came SRFI-1, and the choice was to use the same-type version (as Racket). This decision was question by John David Stone, who points at a comment in the SRFI that says
Note: MIT Scheme and Haskell flip F's arg order for their reduce and fold functions.
Olin later addressed this: all he said was:
Good point, but I want consistency between the two functions.
state-value first: srfi-1, SML
state-value last: Haskell
Note in particular his use of state-value, which suggests a view where consistent types are a possibly more important point than operator order.
"differently than in any other language"
As a counter-example, Standard ML (ML is a very old and influential functional language)'s foldl also works this way: http://www.standardml.org/Basis/list.html#SIG:LIST.foldl:VAL
Racket's foldl and foldr (and also SRFI-1's fold and fold-right) have the property that
(foldr cons null lst) = lst
(foldl cons null lst) = (reverse lst)
I speculate the argument order was chosen for that reason.
From the Racket documentation, the description of foldl:
(foldl proc init lst ...+) → any/c
Two points of interest for your question are mentioned:
the input lsts are traversed from left to right
And
foldl processes the lsts in constant space
I'm gonna speculate on how the implementation for that might look like, with a single list for simplicity's sake:
(define (my-foldl proc init lst)
(define (iter lst acc)
(if (null? lst)
acc
(iter (cdr lst) (proc (car lst) acc))))
(iter lst init))
As you can see, the requirements of left-to-right traversal and constant space are met (notice the tail recursion in iter), but the order of the arguments for proc was never specified in the description. Hence, the result of calling the above code would be:
(my-foldl - 0 '(1 2 3 4))
> 2
If we had specified the order of the arguments for proc in this way:
(proc acc (car lst))
Then the result would be:
(my-foldl - 0 '(1 2 3 4))
> -10
My point is, the documentation for foldl doesn't make any assumptions on the evaluation order of the arguments for proc, it only has to guarantee that constant space is used and that the elements in the list are evaluated from left to right.
As a side note, you can get the desired evaluation order for your expression by simply writing this:
(- 0 1 2 3 4)
> -10
Sorry for the vague title, I guess I just don't understand my problem well enough to ask it yet but here goes. I want to write a recursive function which takes a sequence of functions to evaluate and then calls itself with their results & so on. The recursion stops at some function which returns a number.
However, I would like the function being evaluated at any point in the recursion, f, to be wrapped in a function, s, which returns an initial value (say 0, or the result of another function i) the first time it is evaluated, followed by the result of evaluating f (so that the next time it is evaluated it returns the previously evaluated result, and computes the next value). The aim is to decouple the recursion so that it can proceed without causing this.
I think I'm asking for a lazy-seq. It's a pipe that's filling-up with evaluations of a function at one end, and historical results are coming out of the other.
Your description reminds me some of reductions? Reductions will perform a reduce and return all the intermediate results.
user> (reductions + (range 10))
(0 1 3 6 10 15 21 28 36 45)
Here (range 10) creates a seq of 0 to 9. Reductions applies + repeatedly, passing the previous result of + and the next item in the sequence. All of the intermediate results are returned. You might find it instructive to look at the source of reductions.
If you need to build a test (check for value) into this, that's easy to do with an if in your function (although it won't stop traversing the seq). If you want early exit on a condition being true, then you'll need to write your own loop/recur which amalloy has already done well.
I hate to say it, but I suspect this might also be a case for the State Monad but IANAMG (I Am Not A Monad Guy).
I don't understand your entire goal: a lot of the terms you use are vague. Like, what do you mean you want to evaluate a sequence of functions and then recur on their results? These functions must be no-arg functions (thunks), then, I suppose? But having a thunk which first returns x, and then returns y the next time you call it, is pretty vile and stateful. Perhaps trampoline will solve part of your problem?
You also linked to something you want to avoid, but seem to have pasted the wrong link - it's just a link back to this page. If what you want to avoid is stack overflow, then trampoline is likely to be an okay way to go about it, although it should be possible with just loop/recur. This notion of thunks returning x unless they return y is madness, if avoiding stack overflow is your primary goal. Do not do that.
I've gone ahead and taken a guess at the most plausible end goal you might have, and here's my implementation:
(defn call-until-number [& fs]
(let [numeric (fn [x] (when (number? x) x))]
(loop [fs fs]
(let [result (map #(%) fs)]
(or (some numeric result)
(recur result))))))
(call-until-number (fn [] (fn [] 1))) ; yields 1
(call-until-number (fn [] (fn [] 1)) ; yields 2
(fn [] 2))
(call-until-number (fn f [] f)) ; never returns, no overflow