My understanding is that tail recursion is recursion where a return value is not necessary to finish the operation; that is, the recursion is the last step in the function, and the rest of the function is done once it makes the recursive call.
To that, I ask if this example (from Mr. Norvig) is tail recursion:
(defparameter *titles*
'(Mr Mrs Miss Ms Sir Madam Dr Admiral Major General)
"A list of titles that can appear at the start of a name.")
(defun first-name (name)
"Select the first name from a name represented as a list."
(if (member (first name) *titles*)
(first-name (rest name))
(first name)))
Once the final first-name is called as a branch of the if statement, there is nothing else that function does; therefore, it is tail recursion?
Yup, that is an example.
Tail recursion optimization is available in many implementations of Common Lisp but it is not required by the spec.
This means you can have a Common Lisp without tail recursion optimization.
You may also find that the version you are using needs to be poked a bit to perform this optimization.
So in some implementation you may need to use 'declare' to inform your compiler that you want to optimize for speed.
(defun first-name (name)
"Select the first name from a name represented as a list."
(declare (optimize (speed 3) (compilation-speed 0) (debug 0) (safety 1)))
(if (member (first name) *titles*)
(first-name (rest name))
(first name)))
Edit:
This site is a few years old now but may provide some info.
Also be sure to read the comments as Joshua and Rainer massively improve the detail here.
Yes and no. Usually yes. It will also be optimized if the compiler supports TCO and the right optimization settings are active. But sometimes the compiler will not be able to optimize it.
If name would have been declared special, then possibly not.
If there would be something like
(defvar name '(susanne mustermann))
then the parameter name of the function would be declared special (it would use dynamic binding). Then a compiler might not use tail call optimization in the first-name function.
This means that you also need to know whether variable symbols are declared special or not.
That's one of the reasons, global special variables should be written like *name* to prevent special declaration of those local variables which should be lexical variables. In this case a special declaration would also prevent TCO.
We better write:
(defvar *name* '(susanne mustermann))
Related
When developing with Common Lisp, we have three possibilities to define new setf-forms:
We can define a function whose name is a list of two symbols, the first one being setf, e.g. (defun (setf some-observable) (…)).
We can use the short form of defsetf.
We can use the long form of defsetf.
We can use define-setf-expander.
I am not sure what is the right or intended use-case for each of these possibilities.
A response to this question could hint at the most generic solution and outline contexts where other solutions are superior.
define-setf-expander is the most general of these. All of setf's functionality is encompassed by it.
Defining a setf function works fine for most accessors. It is also valid to use a generic function, so polymorphism is insufficient to require using something else. Controlling evaluation either for correctness or performance is the main reason to not use a setf function.
For correctness, many forms of destructuring are not possible to do with a setf function (e.g. (setf (values ...) ...)). Similarly I've seen an example that makes functional data structures behave locally like a mutable one by changing (setf (dict-get key some-dict) 2) to assign a new dictionary to some-dict.
For performance, consider the silly case of (incf (nth 10000 list)) which if the nth writer were implemented as a function would require traversing 10k list nodes twice, but in a setf expander can be done with a single traversal.
At the end of section 6.5 in the current SBCL manual, we have the following quote:
If your system's performance is suffering because of some construct which could in principle be compiled efficiently, but which the SBCL compiler can't in practice compile efficiently, consider writing a patch to the compiler and submitting it for inclusion in the main sources. Such code is often reasonably straightforward to write; search the sources for the string “deftransform” to find many examples (some straightforward, some less so).
I've been playing around and found the likes of sb-c::defknown and sb-c::deftransform but thus far have had little luck in successfully adding any new transforms that do anything.
Lets pretend i have the following 3 toy functions:
(defun new-+ (x y)
(+ x y))
(defun fixnum-+ (x y)
(declare (optimize (speed 3) (safety 0))
(fixnum x y))
(+ x y))
(defun string-+ (x y)
(declare (optimize (speed 3) (safety 0))
(string x y))
(concatenate 'string x y))
As a purely toy example, lets say we wanted to tell the compiler that it could transform calls to my user defined function new-+ into calls to either fixnum-+ or string-+.
The condition for the compiler transforming (new-+ x y) into (fixnum-+ x y) would be knowing that the arguments x and y are both of type fixnum, and the conditions for transforming into (string-+ x y) would be knowing that the arguments x and y are both of type string.
So the questions:
Can I actually do this?
What are the actual mechanics of doing so and generating other user based transforms/extensions?
Any reading or sources apart from manually reading through the source to discover more info regarding this?
If i can't do this using the likes of deftransform, is there any other way I could do so?
Note: I'm aware of the operations and nature of macros and generic functions in general common lisp coding, and don't consider using them an answer to this question, since I'm specifically curious about extending the SBCL internals and interacting with its compiler.
You achieve what you want in portable Common Lisp using define-compiler-macro
AFAIK reading the SBCL sources is the only way to learn how deftransform works. But before diving into SBCL sources checkout Paul Khuong's Starting to Hack on SBCL or at the very least The Python Compiler for CMU Common Lisp it links to to have an overview of how SBCL works.
I now attempt to provide a broad overview that answers my questions and may point others towards constructively investigating similar directions.
Can I actually do this?
Yes. Though depending on the specifics of how and why, you may have a choice of options available to you, and they may have variable levels of portability between Common Lisp implementations.
What are the actual mechanics of doing so and generating other user based transforms/extensions?
I answer this with respect to two possible methods that the programmer may choose to get started, and which seem most applicable.
For both examples, I reiterate that with limited reflection on the topic, i think it bad form to transform relationships between the input/output mappings of a function. I do so here for demonstration purposes only, to verify that the transformations I'm implementing are actually taking place.
I actually had quite a difficult time testing my transformations were actually happening: SBCL especially seems quite happy to optimise certain expressions and forms, there are additional pieces of information you can make available to the compiler not covered here. Additionally, there may be other transformations available, and so just because your transform isn't used, doesn't necessarily mean it isn't "working".
Environments and Define-Compiler-Macro Extensions using Common Lisp the Language 2
I was previously under the impression that DEFINE-COMPILER-MACRO was relatively limited in its abilities, working only on types connected with literal values, but this is not necessarily the case.
To demonstrate this, i use three user-defined functions and a compiler macro.
First: We will begin with a general addition function gen+ that decides at run-time to either add two numbers together, or concatenate two strings:
(defun gen+ (x y)
(if (and (numberp x)
(numberp y))
(+ x y)
(concatenate 'string x y)))
But say we know at compile time that in certain instances, only strings will be fed to this function. Let's define our specialised string addition function, and to prove its actually being used, we'll do a very bad thing stylistically and additionally concatenate the string "kapow" as well:
(defun string+ (x y)
(declare (optimize (speed 3) (safety 0))
(string x y))
(concatenate 'string x y "kapow"))
The following function is a very simple convenience function that checks an environment to establish whether the declared type of the variable bound in that environment is eq to STRING. We're using a NON-ANSI function here from Common Lisp the Language 2. In sbcl, the function VARIABLE-INFORMATION, and other cltl2 functions are available in the sb-ctlt2 package.
(defun env-stringp (symbol environment)
(eq 'string
(cdr (assoc 'type
(nth-value 2 (sb-cltl2:variable-information symbol environment))))))
Lastly, we use DEFINE-COMPILER-MACRO to generate the transformation. I've tried to name things in this code differently from other examples I've seen so that people can follow along and not get mixed up with what variable/symbol is in which scope/context. A couple of things I didn't know previously about DEFINE-COMPILER-MACRO.
The variable that immediately follows the &whole parameter is a variable which represents the form of the initial call. In our example it will be bound to the list (GEN+ A B)
arg1 is bound to the symbol A
arg2 is bound to the symbol B
The &environment parameter says that within this macro, the symbol ENV will be bound to the environment in which the macro is being evaluated. This is what lets us "kind of step back out of the macro" and check the surrounding code for declarations regarding the type of the variables represented by the symbols bound to 'ARG1' and 'ARG2'
In this definition, we tell the compiler macro that if the user has declared the parameters of GEN+ to be strings, then replace the call to (GEN+ ARG1 ARG2) with a call to (STRING+ ARG1 ARG2).
Note that because the condition of this transformation is the result of a user-defined operation on the environment, if the parameters to GEN+ are literal strings, the transformation will not be triggered, because the environment does not see that the variables have been declared strings. To do that, you would have to add another option and transformation to explicitly check the types of the values in ARG1 and ARG2 as per a traditional use of DEFINE-COMPILER-MACRO. This can be left as an exercise for the reader. But beware about the utility of doing so, because SBCL, for instance, might constant-fold your expression rather than use your transformation anyway.
(define-compiler-macro gen+ (&whole form arg1 arg2 &environment env)
(cond ((and (env-stringp arg1 env)
(env-stringp arg2 env))
`(string+ ,arg1 ,arg2))
(t form)))
Now we can test it with a simple call with type declarations:
(let ((a "bob")
(b "dole"))
(declare (string a b))
(gen+ a b))
This should return the string "bobdolekapow" as the call to GEN+ was transformed into a call to STRING+ based on the declared types of the variables A and B, not just literal types.
Using Basic (defknown)/(deftransform) Combinations with the SBCL Implementation Compiler
The previous technique is indeed potentially useful, more powerful and flexible than transforming on the types of literals, and while not standard ANSI Common Lisp, is more portable/adaptable to other implementations than the technique that follows.
A reason you might forego the former technique in preference of the one that follows, is that the former doesn't get you everything. You still had to declare the types of the variables a and b and write the user-defined function to extract the declared type information from the environment.
If you can interact directly with the SBCL compiler however, with the cost of potentially some brittle-ness and extreme non-portability, you now gain the ability to hack into the compiler itself and gain the benefits of things like type propagation: you might not need to explicitly inform the compiler of the types of A and B for it to implement your transformation.
For our example, we will implement a very basic transformation on the functions wat and string-wat, which are identical in form to our previous functions gen+ and string+.
Understand there are many more pieces of information and optimisation you can feed the SBCL compiler not covered here. And if anyone more experienced with SBCL internals wants to correct/extent anything regarding my impressions here, please comment and i'll be happy to update my answer:
First we tell the compiler about the existence and type signature of wat. We do this by calling defknown in the sb-c package and inform it that wat takes two parameters of any type: (T T) and that it returns a single value of any type: *
(sb-c:defknown wat (T T) *)
Then we define a simple transform using sb-c:deftransform, essentially saying when the two parameters fed to wat are strings, we transform the code into a call to string-wat.
(sb-c:deftransform wat ((x y) (string string) *)
`(string-wat x y))
The forms of wat and string-wat for completeness:
(defun wat (x y)
(if (and (numberp x)
(numberp y))
(+ x y)
(concatenate 'string x y)))
(defun string-wat (x y)
(declare (optimize (speed 3) (safety 0))
(string x y))
(concatenate 'string x y "watpow"))
And this time a demonstration in SBCL using bound variables but no explicit type declarations:
(let ((a (concatenate 'string "bo" "b"))
(b (concatenate 'string "dole")))
(wat a b))
And the returned string should be "bobdolewatpow".
Any reading or sources apart from manually reading through the source to discover more info regarding this?
I haven't been able to find anything much about this out there, and would say that to get much deeper, you're going to have to start trawling through some source code.
SBCL github mirror is currently available here.
User #PuercoPop has suggested background reading of Starting to Hack on SBCL and The Python Compiler for CMU Common Lisp, albeit I am including a link to a .pdf version rather than a .ps version commonly linked to.
I am trying to write a program in Common Lisp using GNU ClISP to compile it. I would like to enter a list such as (A(B (C) ()) (D (E) (F (G) ()))) and depending on the first word print out the pre-, in-, or post-order traversal. Example:
(pre '(A(B (C)... etc))
I am having trouble putting my logic into Clisp notation. I currently have the following code:
(defun leftchild (L)(cadr L))
(defun rightchild (L)(caddr L))
(defun data (L)(car L))
(defun pre (L)(if (null L) '()((data L)(pre(leftchild L))(pre(rightchild L)))))
... similar in and post functions
I get compiling errors saying that I should use a lambda in my pre function. I think this is due to the double (( infront of data because it is expecting a command, but I am not sure what I should put there. I don't think cond would work, because that would hinder the recursive loop. Also, will data L print as it is now? The compiler did not recognize (print (data L)).
I have been working on this code for over a week now, trying to troubleshoot it myself, but I am at a loss. I would greatly appreciate it if someone could explain what I am doing incorrectly.
Another question that I have is how can I make the program prompt a line to the user to enter the (pre '(A... etc)) so that when I run the compiled file the program will run instead of giving a funcall error?
Thank you for your time.
Short answer: If you want to use if, note that you'll need a progn in order to have more than one form in the consequent and alternative cases.
Long answer – also explains how to traverse accumulating the visited nodes in a list:
I guess this is homework, so I won't give you a full solution, but your question shows that you have basically the right idea, so I'll show you an easy, idiomatic way to do this.
First, you're right: The car of an unquoted form should be a function, so basically anything like (foo ...), where foo is not a function (or macro, special form ...), and the whole thing is to be evaluated, will be an error. Note that this does not hold inside special forms and macros (like cond, for example). These can change the evaluation rules, and not everything that looks like (foo bar) has to be a form that is to be evaluated by the normal evaluation rules. The easiest example would be quote, which simply returns its argument unevaluated, so (quote (foo bar)) will not be an error.
Now, about your problem:
An easy solution would be to have an accumulator and a recursive helper function that traverses the tree, and pushes the values in the accumulator. Something like this:
(defun pre (node)
(let ((result (list)))
(labels ((rec (node)
(cond (...
...
...))))
(rec node)
(nreverse result))))
The labels just introduces a local helper function, which will do the actual recursion, and the outer let gives you an accumulator to collect the node values. This solution will return the result as a list. If you just want to print each nodes value, you don't need the accumulator or the helper function. Just print instead of pushing, and make the helper your toplevel function.
Remember, that you'll need a base case where the recursion stops. You should check for that in the cond. Then, you'll need the recursive steps for each subtree and you'll need to push the node's value to the results. The order in which you do these steps decides whether you're doing pre-, in-, or post-order traversal. Your code shows that you already understand this principle, so you'll just have to make it work in Lisp-code. You can use push to push values to result, and consp to check whether a node is a non-empty list. Since there's nothing to do for empty lists, you'll basically only need one test in the cond, but you can also explicitly check whether the node is null, as you did in your code.
I've been learning scheme, and I just realized that I don't really know how to properly comment my functional scheme code. I know how to add a comment of course - you add a ; and put your comment after it. My question is what should I put in my comments, and where should I comment for maximum readability and comprehensability for other programmers reading my code?
Here's a code snippet I wrote. It's a function called display-n. It can be called with any number of arguments and outputs each argument to the screen in the order that they are provided.
(define display-n
(lambda nums
(letrec ((display-n-inner
(lambda (nums)
(display (car nums))
(if (not (equal? (cdr nums) (quote ()))
(display-n-inner (cdr nums))))))
(display-n-inner nums))))
Edit: Improved tabbing and replaced '() with (quote ()) to avoid SO messing up the formatting.
I'm just not sure how/where to add comments to make it more understandable. Some scheme code I've seen just has comments at the top, which is great if you want to use the code, but not helpful if you want to understand/modify it.
Also - how should I comment macros?
The common style for Lisp comments is
Four semicolons for commentary on a whole subsection of a file.
Three semicolons for introducing a single procedure.
Two semicolons for a description of the expression/procedure definition on the following line.
One semicolon for an endline comment.
Procedure overview comments should probably follow the style of RnRS documens, so to just add comments to your procedure as-is, would look something like
;;; Procedure: display-n NUM ...
;; Output each argument to the screen in the order they are provided.
(define
display-n (lambda nums
(letrec ((display-n-inner (lambda (nums)
(display (car nums))
(if (not (equal? (cdr nums) '()))
(display-n-inner (cdr nums))))))
(display-n-inner nums))))
N.B. I don't use three semicolons for the whole procedure description, since it screws up fill-paragraph in Emacs.
Now about the code, I would ditch the whole define-variable-as-a-lambda thing. Yes, I get that this is the "purest" way to define a function, and it makes for a nice consistency with defining procedures are the results of LETs and other procedures, but there's a reason for syntactic sugar, and it's to make things more readable. Same for the LETREC—just use an internal DEFINE, which is the same thing but more readable.
It's not a huge deal that DISPLAY-N-INNER's parameter is called NUMS, since the procedure's so short and DISPLAY-N just hands its NUMS straight to it anyways. "DISPLAY-N-INNER" is sort of a lame name, though. You would give it something with more semantic meaning, or give it a simple name like "ITER" or "LOOP".
Now about the logic of the procedure. First, (equal? (cdr nums) '()) is silly, and is better as (null? (cdr nums)). Actually, when you are operating over an entire list, it's best to make the base case a test of whether the list itself, and not its CDR, is empty. This way the procedure won't error if you pass it no arguments (unless you want it to do that, but I think it makes more sense for DISPLAY-N to do nothing if it gets nothing). Furthermore, you should test whether to stop the procedure, not whether to continue:
(define (display-n . nums)
(define (iter nums)
(if (null? nums)
#t ; It doesn't matter what it returns.
(begin (display (car nums))
(iter (cdr nums)))))
(iter nums))
But for all that, I would say the the procedure itself is not the best way to accomplish the task it does, since it is too concerned with the details of traversing a list. Instead you would use the more abstract FOR-EACH method to do the work.
(define (display-n . nums)
(for-each display nums))
This way, instead of a reader of the procedure getting mired in the details of CARs and CDRs, he can just understand that FOR-EACH will DISPLAY each element of NUMS.
Some random notes:
Traditionally, Scheme and Lisp code has used ;;; for toplevel comments, ;; for comments in the code, and ; for comments on the same line as the code they're commenting on. Emacs has support for this, treating each of these a little differently. But especially on the Scheme side this is no longer as popular as it was, but the difference between ;; and ; is still common.
Most modern Schemes have adopted new kinds of comments: theres:
#|...|# for a block comment -- useful for long pieces of text that comment on the whole file.
#;<expr> is a comment that makes the implementation ignore the expression, which is useful for debugging.
As for the actual content of what to write, that's not different than any other language, except that with a more functional approach you usually have more choices on how to lay out your code. It also makes it more convenient to write smaller functions that are combined into larger pieces of functionality -- and this changes the documentation style too, since many such small functions will be "self documenting" (in that they're easy to read and very obvious in how they're working).
I hate to sound like a broken record, but I still think that you should spend some time with HtDP. One thing that it encourages in its design recipe is to write examples first, then the documentation, and then expand that to actual code. Furthermore, this recipe leaves you with code that has a very standard set of comments: the input/output types, a purpose statement, some documentation about how the function is implemented when necessary, and the examples can be considered as another kind of documentation (which would turn to commented code in "real" code). (There are other books that take a similar position wrt documentation.)
Finally, documenting macros is not different than documenting any other code. The only thing that can be very different i what's written in the comments: instead of describing what some function is doing, you tend to describe what code it expands too, so the comments are also more on the meta level. A common approach to macros is to to minimal work inside the macro -- just what's needed at that level (eg, wrap expressions in (lambda () ...)), and leave the actual implementation to a function. This helps in documenting too, since the two related pieces will have comments on how the macro expands and how it runs, independently.
I follow an approach similar to what's posted here:
http://www.cc.gatech.edu/computing/classes/cs2360/ghall/style/commenting.html
Note: this is for Common Lisp.
Specifically:
" Four Semicolons(;;;;)
...denote a sub heading in the file...
Three Semicolons(;;;)
...denote a description of the succeeding function, macro, or
variable definition...
[I usually just most of the description into the "docstring"
of the function or variable.]
Two Semicolons(;;)
...denote a description of the succeeding expression...
One Semicolon(;)
...denotes an in-line comment that explains a particular element
of the expression on that line... Brevity is important for
inline comments"
I think a great place to start would be to put your one-sentence description of what the function does
It can be called with any number of arguments and outputs each argument to the screen in the order that they are provided.
as a comment at the beginning.
I'm not particularly conversant in scheme, so I can't comment (:-) on whether additional line-by-line comments explaining the mechanics of how the function achieves that result would be expected according to normal scheme style (but I suspect not).
I've been writing more Lisp code recently. In particular, recursive functions that take some data, and build a resulting data structure. Sometimes it seems I need to pass two or three pieces of information to the next invocation of the function, in addition to the user supplied data. Lets call these accumulators.
What is the best way to organize these interfaces to my code?
Currently, I do something like this:
(defun foo (user1 user2 &optional acc1 acc2 acc3)
;; do something
(foo user1 user2 (cons x acc1) (cons y acc2) (cons z acc3)))
This works as I'd like it to, but I'm concerned because I don't really need to present the &optional parameters to the programmer.
3 approaches I'm somewhat considering:
have a wrapper function that a user is encouraged to use that immediately invokes the extended definiton.
use labels internally within a function whose signature is concise.
just start using a loop and variables. However, I'd prefer not since I'd like to really wrap my head around recursion.
Thanks guys!
If you want to write idiomatic Common Lisp, I'd recommend the loop and variables for iteration. Recursion is cool, but it's only one tool of many for the Common Lisper. Besides, tail-call elimination is not guaranteed by the Common Lisp spec.
That said, I'd recommend the labels approach if you have a structure, a tree for example, that is unavoidably recursive and you can't get tail calls anyway. Optional arguments let your implementation details leak out to the caller.
Your impulse to shield implementation details from the user is a smart one, I think. I don't know common lisp, but in Scheme you do it by defining your helper function in the public function's lexical scope.
(define (fibonacci n)
(let fib-accum ((a 0)
(b 1)
(n n))
(if (< n 1)
a
(fib-accum b (+ a b) (- n 1)))))
The let expression defines a function and binds it to a name that's only visible within the let, then invokes the function.
I have used all the options you mention. All have their merits, so it boils down to personal preference.
I have arrived at using whatever I deem appropriate. If I think that leaving the &optional accumulators in the API might make sense for the user, I leave it in. For example, in a reduce-like function, the accumulator can be used by the user for providing a starting value. Otherwise, I'll often rewrite it as a loop, do, or iter (from the iterate library) form, if it makes sense to perceive it as such. Sometimes, the labels helper is also used.