On page 329 of Land of Lisp, Conrad Barski explains the technique of memoization with the following example code
(let ((old-neighbors (symbol-function 'neighbors))
(previous (make-hash-table)))
(defun neighbors (pos)
(or (gethash pos previous)
(setf (gethash pos previous) (funcall old-neighbors pos)))))
The idea is nice: when I call the neighbors function, I save the result into a hash table, so that the next time calling neighbors with the same value of pos, I can just look-up the result without having to compute it again.
So I was wondering, whether it would not be easier to rename the function neighbors to old-neighbors by editing and recompiling its source code (given on page 314 of the book). Then the memoization example could be simplified to
(let ((previous (make-hash-table)))
(defun neighbors (pos)
(or (gethash pos previous)
(setf (gethash pos previous) (funcall old-neighbors pos)))))
or, by turning previous into a global variable *previous-neighbors* beforehand, even into
(defun neighbors (pos)
(or (gethash pos *previous-neighbors*)
(setf (gethash pos *previous-neighbors*) (funcall old-neighbors pos))))
thus rendering the closure unnecessary.
So my question is: what is the reason for doing it this way?
Reasons I could imagine:
It's didactical, showing what could be done with a closure (which had been explained just before) and providing an example of symbol-function.
This technique is applicable even in situations, where you cannot or may not change the source code of neighbors.
I am missing something.
You have made some good observations.
Generally the style to use closures like that is more likely to be found in Scheme code - where Scheme developers often use functions to return functions.
Generally as you have detected it makes little sense to have a memoize function foo calling an function old-foo. Using global variables reduces encapsulation (-> information hiding), but increases the ability to debug a program, since one can more easily inspect the memoized values.
A similar, but potentially more useful, pattern would be this:
(defun foo (bar)
<does some expensive computation>)
(memoize 'foo)
Where ˋmemoizeˋ would be something like this
(defun memoize (symbol)
(let ((original-function (symbol-function symbol))
(values (make-hash-table)))
(setf (symbol-function symbol)
(lambda (arg &rest args)
(or (gethash arg values)
(setf (gethash arg values)
(apply original-function arg args)))))))
The advantage is that you don't need to write the memoizing code for each function. You only need one function memoize. In this case the closure also makes sense - though you could also have a global table storing memoize tables.
Note the limitations of above: the comparison uses EQL and only the first argument of the function to memoize.
There are also more extensive tools to provide similar functionality.
See for example:
https://github.com/fare/fare-memoization
https://github.com/sharplispers/cormanlisp/blob/master/examples/memoize.lisp
https://github.com/AccelerationNet/function-cache
Using my memoize from above:
CL-USER 22 > (defun foo (n)
(sleep 3)
(expt 2 n))
FOO
CL-USER 23 > (memoize 'foo)
#<Closure 1 subfunction of MEMOIZE 40600008EC>
The first call with arg 10 runs three seconds:
CL-USER 24 > (foo 10)
1024
The second call with arg 10 runs faster:
CL-USER 25 > (foo 10)
1024
The first call with arg 2 runs three seconds:
CL-USER 26 > (foo 2)
4
The second call with arg 2 runs faster:
CL-USER 27 > (foo 2)
4
The third call with arg 10 runs fast:
CL-USER 28 > (foo 10)
1024
Related
I am trying to compare the performance of a function and a macro.
EDIT: Why do I want to compare the two?
Paul Graham wrote in his ON LISP book that macros can be used to make a system more efficient because a lot of the computation can be done at compile time. so in the example below (length args) is dealt with at compile time in the macro case and at run time in the function case. So, I just wanted how much faster did (avg2 super-list) get computed relative to (avg super-list).
Here is the function and the macro:
(defun avg (args)
(/ (apply #'+ args) (length args)))
(defmacro avg2 (args)
`(/ (+ ,#args) ,(length args)))
I have looked at this question How to pass a list to macro in common lisp? and a few other ones but they do not help because their solutions do not work; for example, in one of the questions a user answered by saying to do this:
(avg2 (2 3 4 5))
instead of this:
(avg2 '(2 3 4))
This works but I want a list containg 100,000 items:
(defvar super-list (loop for i from 1 to 100000 collect i))
But this doesnt work.
So, how can I pass super-list to avg2?
First of all, it simply makes no sense to 'compare the performance of a function and a macro'. It only makes sense to compare the performance of the expansion of a macro with a function. So that's what I'll do.
Secondly, it only makes sense to compare the performance of a function with the expansion of a macro if that macro is equivalent to the function. In other words the only places this comparison is useful is where the macro is being used as a hacky way of inlining a function. It doesn't make sense to compare the performance of something which a function can't express, like if or and say. So we must rule out all the interesting uses of macros.
Thirdly it makes no sense to compare the performance of things which are broken: it is very easy to make programs which do not work be as fast as you like. So I'll successively modify both your function and macro so they're not broken.
Fourthly it makes no sense to compare the performance of things which use algorithms which are gratuitously terrible, so I'll modify both your function and your macro to use better algrorithms.
Finally it makes no sense to compare the performance of things without using the tools the language provides to encourage good performance, so I will do that as the last step.
So let's address the third point above: let's see how avg (and therefore avg2) is broken.
Here's the broken definition of avg from the question:
(defun avg (args)
(/ (apply #'+ args) (length args)))
So let's try it:
> (let ((l (make-list 1000000 :initial-element 0)))
(avg l))
Error: Last argument to apply is too long: 1000000
Oh dear, as other people have pointed out. So probably I need instead to make avg at least work. As other people have, again, pointed out, the way to do this is reduce:
(defun avg (args)
(/ (reduce #'+ args) (length args)))
And now a call to avg works, at least. avg is now non-buggy.
We need to make avg2 non-buggy as well. Well, first of all the (+ ,#args) thing is a non-starter: args is a symbol at macroexpansion time, not a list. So we could try this (apply #'+ ,args) (the expansion of the macro is now starting to look a bit like the body of the function, which is unsurprising!). So given
(defmacro avg2 (args)
`(/ (apply #'+ ,args) (length ,args)))
We get
> (let ((l (make-list 1000000 :initial-element 0)))
(avg2 l))
Error: Last argument to apply is too long: 1000000
OK, unsurprising again. let's fix it to use reduce again:
(defmacro avg2 (args)
`(/ (reduce #'+ ,args) (length ,args)))
So now it 'works'. Except it doesn't: it's not safe. Look at this:
> (macroexpand-1 '(avg2 (make-list 1000000 :initial-element 0)))
(/ (reduce #'+ (make-list 1000000 :initial-element 0))
(length (make-list 1000000 :initial-element 0)))
t
That definitely is not right: it will be enormously slow but also it will just be buggy. We need to fix the multiple-evaluation problem.
(defmacro avg2 (args)
`(let ((r ,args))
(/ (reduce #'+ r) (length r))))
This is safe in all sane cases. So this is now a reasonably safe 70s-style what-I-really-want-is-an-inline-function macro.
So, let's write a test-harness both for avg and avg2. You will need to recompile av2 each time you change avg2 and in fact you'll need to recompile av1 for a change we're going to make to avg as well. Also make sure everything is compiled!
(defun av0 (l)
l)
(defun av1 (l)
(avg l))
(defun av2 (l)
(avg2 l))
(defun test-avg-avg2 (nelements niters)
;; Return time per call in seconds per iteration per element
(let* ((l (make-list nelements :initial-element 0))
(lo (let ((start (get-internal-real-time)))
(dotimes (i niters (- (get-internal-real-time) start))
(av0 l)))))
(values
(let ((start (get-internal-real-time)))
(dotimes (i niters (float (/ (- (get-internal-real-time) start lo)
internal-time-units-per-second
nelements niters)))
(av1 l)))
(let ((start (get-internal-real-time)))
(dotimes (i niters (float (/ (- (get-internal-real-time) start lo)
internal-time-units-per-second
nelements niters)))
(av2 l))))))
So now we can test various combinations.
OK, so now the fouth point: both avg and avg2 use awful algorithms: they traverse the list twice. Well we can fix this:
(defun avg (args)
(loop for i in args
for c upfrom 0
summing i into s
finally (return (/ s c))))
and similarly
(defmacro avg2 (args)
`(loop for i in ,args
for c upfrom 0
summing i into s
finally (return (/ s c))))
These changes made a performance difference of about a factor of 4 for me.
OK so now the final point: we should use the tools the language gives us. As has been clear throughout this whole exercise only make sense if you're using a macro as a poor-person's inline function, as people had to do in the 1970s.
But it's not the 1970s any more: we have inline functions.
So:
(declaim (inline avg))
(defun avg (args)
(loop for i in args
for c upfrom 0
summing i into s
finally (return (/ s c))))
And now you will have to make sure you recompile avg and then av1. And when I look at av1 and av2 I can now see that they are the same code: the entire purpose of avg2 has now gone.
Indeed we can do even better than this:
(define-compiler-macro avg (&whole form l &environment e)
;; I can't imagine what other constant forms there might be in this
;; context, but, well, let's be safe
(if (and (constantp l e)
(listp l)
(eql (first l) 'quote))
(avg (second l))
form))
Now we have something which:
has the semantics of a function, so, say (funcall #'avg ...) will work;
isn't broken;
uses a non-terrible algorithm;
will be inlined on any competent implementation of the language (which I bet is 'all implementations' now) when it can be;
will detect (some?) cases where it can be compiled completely away and replaced by a compile-time constant.
Since the value of super-list is known, one can do all computation at macro expansion time:
(eval-when (:execute :compile-toplevel :load-toplevel)
(defvar super-list (loop for i from 1 to 100000 collect i)))
(defmacro avg2 (args)
(setf args (eval args))
(/ (reduce #'+ args) (length args)))
(defun test ()
(avg2 super-list))
Trying the compiled code:
CL-USER 10 > (time (test))
Timing the evaluation of (TEST)
User time = 0.000
System time = 0.000
Elapsed time = 0.000
Allocation = 0 bytes
0 Page faults
100001/2
Thus the runtime is near zero.
The generated code is just a number, the result number:
CL-USER 11 > (macroexpand '(avg2 super-list))
100001/2
Thus for known input this macro call in compiled code has a constant runtime of near zero.
I don't think you really want a list of 100,000 items. That would have terrible performance with all that cons'ing. You should consider a vector instead, e.g.
(avg2 #(2 3 4))
You didn't mention why it didn't work; if the function never returns, it's likely a memory issue from such a large list, or attempting to apply on such a large function argument list; there are implementation defined limits on how many arguments you can pass to a function.
Try reduce on a super-vector instead:
(reduce #'+ super-vector)
I have a code which takes a list and returns all possible permutations by the parameter result.
But when I compile I have an error which says *** - =: (1+ INDEX) is not a number.
Is this message true or I messed up the code generally?
I am new to lisp I can looking for a fix and also open to suggestions from fucntional programmers.
;; Creates permutatiions of a given list and returns it via parameter
(defun create-permuations (source)
(setf result (list))
(create-permuations-helper source 0 '() result)
result)
(defmacro create-permuations-helper (source index cur result)
(if (= (list-length cur) index)
(cons cur result)
(loop for i from 0 to (list-length cur) do
(create-permuations-helper source (1+ index)
(append cur (list (nth i source))) result))))
99% of times when a compiler reports an error you can trust it to be true. Here Index is the list (1+ index), literally the 1+ symbol followed by the index symbol. This is so because you are using a macro, and macros operate on code.
In your macro, you do not return a form to be evaluated, you execute code during macro-expansion that depends on itself. That alone is an undefined behaviour. For example:
(defmacro a (x)
(if (plusp x)
(a (- x 1))
nil))
In the body of a, you want to expand code using a recursive call to itself. But the macro is not yet fully known and cannot be until the whole macro is defined.
Maybe the particular lisp implementation binds a to the macro function in body of the macro, which is a strange thing to do, or you evaluated the definition twice. The first time the compiler assumes a is an unknown function, then binds a to a macro, and the second time it tries to expand the macro.
Anyway macro are not supposed to be recursive.
In the example, since the macro does not evaluate its argument, the nested call to the macro is given the literal expression (- x 1), and not its actual value, which cannot be known anyway since x is unknown. You are crossing a level of abstraction here by trying to evaluate things at macroexpansion time.
But, macros can expand into code that refers to themselves.
(defmacro a (x)
(if (plusp x)
`(b (a ,(- x 1)))
nil))
Now, (a 2) expands into (b (a 1)), which itself macroexpands into (b (b (a 0))), and finally reaches a fixpoint which is (b (b nil)).
The difference is that the macro produces a piece of code and returns, which the compiler macroexpands again, whereas in the first example, the macro must already be expanded in the body of its own definition.
Possible implementation
One way to solve your problem is to define a local function that has access to a variable defined in your main function. Then, the local function can set it, and you do not need to pass a variable by reference (which is not possible to do):
(defun permut (list)
(let (result)
(labels ((recurse (stack list)
(if list
(dolist (x list)
(recurse (cons x stack)
(remove x list :count 1)))
(push stack result))))
(recurse nil list))
result))
Alternatively, you can split the process in two; first, define permut-helper, which is a higher-order function that takes a callback function; it generates permutations and calls the callback for each one:
(defun permut-helper (stack list callback)
(if list
(dolist (x list)
(permut-helper (cons x stack)
(remove x list :count 1)
callback))
(funcall callback stack)))
You call it with a function that pushes results into a list of permutations:
(defun permut (list)
(let (result)
(flet ((add-result (permutation)
(push permutation result)))
(permut-helper nil list #'add-result))
result))
This is not a homework assignment. In the following code:
(defparameter nums '())
(defun fib (number)
(if (< number 2)
number
(push (+ (fib (- number 1)) (fib (- number 2))) nums))
return nums)
(format t "~a " (fib 100))
Since I am quite inexperienced with Common Lisp, I am at a loss as to why the function does not return an value. I am a trying to print first 'n' values, e.g., 100, of the Fibonacci Sequence.
Thank you.
An obvious approach to computing fibonacci numbers is this:
(defun fib (n)
(if (< n 2)
n
(+ (fib (- n 1)) (fib (- n 2)))))
(defun fibs (n)
(loop for i from 1 below n
collect (fib i)))
A little thought should tell you why no approach like this is going to help you compute the first 100 Fibonacci numbers: the time taken to compute (fib n) is equal to or a little more than the time taken to compute (fib (- n 1)) plus the time taken to compute (fib (- n 2)): this is exponential (see this stack overflow answer).
A good solution to this is memoization: the calculation of (fib n) repeats subcalculations a huge number of times, and if we can just remember the answer we computed last time we can avoid doing so again.
(An earlier version of this answer has an overcomplex macro here: something like that may be useful in general but is not needed here.)
Here is how you can memoize fib:
(defun fib (n)
(check-type n (integer 0) "natural number")
(let ((so-far '((2 . 1) (1 . 1) (0 . 0))))
(labels ((fibber (m)
(when (> m (car (first so-far)))
(push (cons m (+ (fibber (- m 1))
(fibber (- m 2))))
so-far))
(cdr (assoc m so-far))))
(fibber n))))
This keeps a table – an alist – of the results it has computed so far, and uses this to avoid recomputation.
With this memoized version of the function:
> (time (fib 1000))
Timing the evaluation of (fib 1000)
User time = 0.000
System time = 0.000
Elapsed time = 0.000
Allocation = 101944 bytes
0 Page faults
43466557686937456435688527675040625802564660517371780402481729089536555417949051890403879840079255169295922593080322634775209689623239873322471161642996440906533187938298969649928516003704476137795166849228875
The above definition uses a fresh cache for each call to fib: this is fine, because the local function, fibber does reuse the cache. But you can do better than this by putting the cache outside the function altogether:
(defmacro define-function (name expression)
;; Install EXPRESSION as the function value of NAME, returning NAME
;; This is just to avoid having to say `(setf ...)`: it should
;; probably do something at compile-time too so the compiler knows
;; the function will be defined.
`(progn
(setf (fdefinition ',name) ,expression)
',name))
(define-function fib
(let ((so-far '((2 . 1) (1 . 1) (0 . 0))))
(lambda (n)
(block fib
(check-type n (integer 0) "natural number")
(labels ((fibber (m)
(when (> m (car (first so-far)))
(push (cons m (+ (fibber (- m 1))
(fibber (- m 2))))
so-far))
(cdr (assoc m so-far))))
(fibber n))))))
This version of fib will share its cache between calls, which means it is a little faster, allocates a little less memory but may be less thread-safe:
> (time (fib 1000))
[...]
Allocation = 96072 bytes
[...]
> (time (fib 1000))
[...]
Allocation = 0 bytes
[...]
Interestingly memoization was invented (or at least named) by Donald Michie, who worked on breaking Tunny (and hence with Colossus), and who I also knew slightly: the history of computing is still pretty short!
Note that memoization is one of the times where you can end up fighting a battle with the compiler. In particular for a function like this:
(defun f (...)
...
;; no function bindings or notinline declarations of F here
...
(f ...)
...)
Then the compiler is allowed (but not required) to assume that the apparently recursive call to f is a recursive call into the function it is compiling, and thus to avoid a lot of the overhead of a full function call. In particular it is not required to retrieve the current function value of the symbol f: it can just call directly into the function itself.
What this means is that an attempt to write a function, memoize which can be used to mamoize an existing recursive function, as (setf (fdefinition 'f) (memoize #'f)) may not work: the function f still call directly into the unmemoized version of itself and won't notice that the function value of f has been changed.
This is in fact true even if the recursion is indirect in many cases: the compiler is allowed to assume that calls to a function g for which there is a definition in the same file are calls to the version defined in the file, and again avoid the overhead of a full call.
The way to deal with this is to add suitable notinline declarations: if a call is covered by a notinline declaration (which must be known to the compiler) then it must be made as a full call. From the spec:
A compiler is not free to ignore this declaration; calls to the specified functions must be implemented as out-of-line subroutine calls.
What this means is that, in order to memoize functions you have to add suitable notinline declarations for recursive calls, and this means that memoizing either needs to be done by a macro, or must rely on the user adding suitable declarations to the functions to be memoized.
This is only a problem because the CL compiler is allowed to be smart: almost always that's a good thing!
Your function unconditionally returns nums (but only if a variable called return exists). To see why, we can format it like this:
(defun fib (number)
(if (< number 2)
number
(push (+ (fib (- number 1)) (fib (- number 2))) nums))
return
nums)
If the number is less than 2, then it evaluates the expression number, uselessly, and throws away the result. Otherwise, it pushes the result of the (+ ....) expression onto the nums list. Then it uselessly evaluates return, throwing away the result. If a variable called return doesn't exist, that's an error situation. Otherwise, it evaluates nums and that is the return value.
In Common Lisp, there is a return operator for terminating and returning out of anonymous named blocks (blocks whose name is the symbol nil). If you define a named function with defun, then an invisible block exists which is not anonymous: it has the same name as that function. In that case, return-from can be used:
(defun function ()
(return-from function 42) ;; function terminates, returns 42
(print 'notreached)) ;; this never executes
Certain standard control flow and looping constructs establish a hidden anonymous block, so return can be used:
(dolist (x '(1 2 3))
(return 42)) ;; loop terminates, yields 42 as its result
If we use (return ...) but there is no enclosing anonymous block, that is an error.
The expression (return ...) is different from just return, which evaluates a variable named by the symbol return, retrieving its contents.
It is not clear how to repair your fib function, because the requirements are unknown. The side effect of pushing values into a global list normally doesn't belong inside a mathematical function like this, which should be pure (side-effect-free).
So you might know that if you know the two previous numbers you can compute the next. What comes after 3, 5? If you guess 8 you have understood it. Now if you start with 0, 1 and roll 1, 1, 1, 2, etc you collect the first variable until you have the number of numbers you'd like:
(defun fibs (elements)
"makes a list of elements fibonacci numbers starting with the first"
(loop :for a := 0 :then b
:for b := 1 :then c
:for c := (+ a b)
:for n :below elements
:collect a))
(fibs 10)
; ==> (0 1 1 2 3 5 8 13 21 34)
Every form in Common Lisp "returns" a value. You can say it evaluates to. eg.
(if (< a b)
5
10)
This evaluates either to 5 or 10. Thus you can do this and expect that it evaluates to either 15 or 20:
(+ 10
(if (< a b)
5
10))
You basically want your functions to have one expression that calculates the result. eg.
(defun fib (n)
(if (zerop n)
n
(+ (fib (1- n)) (fib (- n 2)))))
This evaluates to the result og the if expression... loop with :collect returns the list. You also have (return expression) and (return-from name expression) but they are usually unnecessary.
Your global variable num is actually not that a bad idea.
It is about to have a central memory about which fibonacci numbers were already calculated. And not to calculate those already calculated numbers again.
This is the very idea of memoization.
But first, I do it in bad manner with a global variable.
Bad version with global variable *fibonacci*
(defparameter *fibonacci* '(1 1))
(defun fib (number)
(let ((len (length *fibonacci*)))
(if (> len number)
(elt *fibonacci* (- len number 1)) ;; already in *fibonacci*
(labels ((add-fibs (n-times)
(push (+ (car *fibonacci*)
(cadr *fibonacci*))
*fibonacci*)
(cond ((zerop n-times) (car *fibonacci*))
(t (add-fibs (1- n-times))))))
(add-fibs (- number len))))))
;;> (fib 10)
;; 89
;;> *fibonacci*
;; (89 55 34 21 13 8 5 3 2 1 1)
Good functional version (memoization)
In memoization, you hide the global *fibonacci* variable
into the environment of a lexical function (the memoized version of a function).
(defun memoize (fn)
(let ((cache (make-hash-table :test #'equal)))
#'(lambda (&rest args)
(multiple-value-bind (val win) (gethash args cache)
(if win
val
(setf (gethash args cache)
(apply fn args)))))))
(defun fib (num)
(cond ((zerop num) 1)
((= 1 num) 1)
(t (+ (fib (- num 1))
(fib (- num 2))))))
The previously global variable *fibonacci* is here actually the local variable cache of the memoize function - encapsulated/hidden from the global environment,
accessible/look-up-able only through the function fibm.
Applying memoization on fib (bad version!)
(defparameter fibm (memoize #'fib))
Since common lisp is a Lisp 2 (separated namespace between function and variable names) but we have here to assign the memoized function to a variable,
we have to use (funcall <variable-name-bearing-function> <args for memoized function>).
(funcall fibm 10) ;; 89
Or we define an additional
(defun fibm (num)
(funcall fibm num))
and can do
(fibm 10)
However, this saves/memoizes only the out calls e.g. here only the
Fibonacci value for 10. Although for that, Fibonacci numbers
for 9, 8, ..., 1 are calculated, too.
To make them saved, look the next section!
Applying memoization on fib (better version by #Sylwester - thank you!)
(setf (symbol-function 'fib) (memoize #'fib))
Now the original fib function is the memoized function,
so all fib-calls will be memoized.
In addition, you don't need funcall to call the memoized version,
but just do
(fib 10)
Let us suppose we have a function func1 :
(defun func1 (&rest values)
; (do something with values...)
(loop for i in values collect i))
Now, we have a function func2 which calls func1 :
(defun func2 (&rest values)
; (do something with values...)
(func1 ???))
What should I put instead of ??? to "copy" all the parameters of func2's values to func1's values ?
For instance, I would have the following behavior :
(func2 1 2 3 4) ; result is (1 2 3 4) and not ((1 2 3 4)).
In an earlier question I tried to do something like this :
(defun func2 (&rest values)
(macrolet ((my-macro (v)
`(list ,#v)))
(func1 (my-macro values))))
But the defun cannot get the value because it is not runtime. In this answer, he suggested that I use apply, but this function takes a &rest parameter too, so it doesn't solve my problem...
If possible, I would rather avoid to change the prototype of both functions, and the behavior of func1.
In common lisp, it has to be
(apply #'func1 values) ;; since `func1` has to be looked up in function namespace
remember, Clojure and Racket/Scheme are Lisp1, and common lisp is Lisp2.
Alternative solution (just for the sake)
I was asking myself, how to get it done without apply - just for the sake.
The problem with
`(func2 ,#values)
is, that if e.g.
(func2 (list 1 2 3) (list 4) 5)
is called, the values variable is ((1 2 3) (4) 5)
But when it is spliced into (func1 ,#values), what is created is
(func1 (1 2 3) (4) 5). But if we compare this with the func2 call,
it should be rather (func1 (list 1 2 3) (list 4) 5) which is perhaps not possible, because when (func2 (list 1 2 3) (list 4) 5) is called -
in the lisp manner - the arguments of func2 are each evaluated, before they enter the function body of func2, so we end up with values as a list of already evaluated arguments, namely ((1 2 3) (4) 5).
So somehow, concerning the arguments for func1 in the last expression, we are one evaluation-step offbeat.
But there is a solution with quote, that we manage to quote each of the arguments before giving it to func1 in the last expression, to "synchronize" the func1 function call - to let the arguments' evaluation pause for one round.
So my first aim was to generate a new values list inside the func2 body where each of the values list's argument is quoted (this is done in the let-binding).
And then at the end to splice this quoted-values list into the last expression: (func1 '(1 2 3) '(4) '5) which can be regarded as equivalent to (func1 (list 1 2 3) (list 4) 5) for this kind of problems / for this kind of calls.
This was achieved by this code:
(defun func2 (&rest vals)
(let ((quoted-values (loop for x in vals
collect `',x)))
; do sth with vals here - the func2 function -
(eval `(func1 ,#quoted-values))))
This is kind of a macro (it creates code btw. it organizes new code) but executed and created in run-time - not in pre-compile time. Using an eval we execute that generated code on the fly.
And like macroexpand-1, we can look at the result - the code - to which the func1 expression "expands", by removing eval around it - I call it func2-1:
(defun func2-1 (&rest vals)
(let ((quoted-values (loop for x in vals
collect `',x)))
; do sth with vals here - the func2 function -
`(func1 ,#quoted-values)))
And if we run it, it returns the last expression as code immediately before it is evluated in the func2 version:
(func2-1 (list 1 2 3) (list 4) 5)
;; (FUNC1 '(1 2 3) '(4) '5) ;; the returned code
;; the quoted arguments - like desired!
And this happens if we call it using func2 (so with evaluation of the func1 all:
(func2 (list 1 2 3) (list 4) 5)
;; ((1 2 3) (4) 5) ;; the result of (FUNC1 '(1 2 3) '(4) '5)
So I would say this is exactly what you desired!
lists vs. spread arguments
In Common Lisp it is good style to pass lists as lists and not as spread arguments:
(foo (list 1 2 3)) ; better interface
(foo 1 2 3) ; interface is not so good
The language has been defined in a way that efficient function calling can be used by a compiler and this means that the number of arguments which can be passed to a function is limited. There is a standard variable which will tell us how many arguments a particular implementation supports:
This is LispWorks on my Mac:
CL-USER 13 > call-arguments-limit
2047
Some implementations allow much larger number of arguments. But this number can be as low as 50 - for example ABCL, Common Lisp on the JVM, allows only 50 arguments.
Computing with argument lists
But sometimes we want the arguments as a list and then we can use the &rest parameter:
(lambda (&rest args)
(print args))
This is slightly in-efficient, since a list will be consed for the arguments. Usually Lisp tries to avoid to cons lists for arguments - they will be passed in registers or on the stack - if possible.
If we know that the argument list will not be used, then we can give the compiler a hint to use stack allocation - if possible:
(lambda (&rest args)
(declare (dynamic-extent args))
(reduce #'+ args))
In above function, the list of arguments can be deallocated when leaving the function - because the argument list is no longer used then.
If you want to pass these arguments to another function you can use FUNCALL and usually more useful APPLY:
(lambda (&rest args)
(funcall #'write (first args) (second args) (third args)))
or more useful:
(lambda (&rest args)
(apply #'write args))
One can also add additional arguments to APPLY before the list to apply:
CL-USER 19 > ((lambda (&rest args)
(apply #'write
(first args) ; the object
:case :downcase ; additional args
(rest args))
(values))
'(defun foo () 'bar)
:pretty t
:right-margin 15)
(defun foo ()
'bar)
I have been searching everywhere for the following functionality in Lisp, and have gotten nowhere:
find the index of something in a list. example:
(index-of item InThisList)
replace something at a specific spot in a list. example:
(replace item InThisList AtThisIndex) ;i think this can be done with 'setf'?
return an item at a specific index. example:
(return InThisList ItemAtThisIndex)
Up until this point, I've been faking it with my own functions. I'm wondering if I'm just creating more work for myself.
This is how I've been faking number 1:
(defun my-index (findMe mylist)
(let ((counter 0) (found 1))
(dolist (item mylist)
(cond
((eq item findMe) ;this works because 'eq' checks place in memory,
;and as long as 'findMe' was from the original list, this will work.
(setq found nil)
(found (incf counter))))
counter))
You can use setf and nth to replace and retrieve values by index.
(let ((myList '(1 2 3 4 5 6)))
(setf (nth 4 myList) 101); <----
myList)
(1 2 3 4 101 6)
To find by index you can use the position function.
(let ((myList '(1 2 3 4 5 6)))
(setf (nth 4 myList) 101)
(list myList (position 101 myList)))
((1 2 3 4 101 6) 4)
I found these all in this index of functions.
find the index of something in a list.
In Emacs Lisp and Common Lisp, you have the position function:
> (setq numbers (list 1 2 3 4))
(1 2 3 4)
> (position 3 numbers)
2
In Scheme, here's a tail recursive implementation from DrScheme's doc:
(define list-position
(lambda (o l)
(let loop ((i 0) (l l))
(if (null? l) #f
(if (eqv? (car l) o) i
(loop (+ i 1) (cdr l)))))))
----------------------------------------------------
> (define numbers (list 1 2 3 4))
> (list-position 3 numbers)
2
>
But if you're using a list as a collection of slots to store structured data, maybe you should have a look at defstruct or even some kind of Lisp Object System like CLOS.
If you're learning Lisp, make sure you have a look at Practical Common Lisp and / or The Little Schemer.
Cheers!
Answers:
(position item sequence &key from-end (start 0) end key test test-not)
http://lispdoc.com/?q=position&search=Basic+search
(setf (elt sequence index) value)
(elt sequence index)
http://lispdoc.com/?q=elt&search=Basic+search
NOTE: elt is preferable to nth because elt works on any sequence, not just lists
Jeremy's answers should work; but that said, if you find yourself writing code like
(setf (nth i my-list) new-elt)
you're probably using the wrong datastructure. Lists are simply linked lists, so they're O(N) to access by index. You might be better off using arrays.
Or maybe you're using lists as tuples. In that case, they should be fine. But you probably want to name accessors so someone reading your code doesn't have to remember what "nth 4" is supposed to mean. Something like
(defun my-attr (list)
(nth 4 list))
(defun (setf my-attr) (new list)
(setf (nth 4 list) new))
+2 for "Practical Common Lisp". It is a mixture of a Common Lisp Cookbook and a quality Teach Yourself Lisp book.
There's also "Successful Common Lisp" (http://www.psg.com/~dlamkins/sl/cover.html and http://www.psg.com/~dlamkins/sl/contents.html) which seemed to fill a few gaps / extend things in "Practical Common Lisp".
I've also read Paul Graham's "ANSI Common Lisp" which is more about the basics of the language, but a bit more of a reference manual.
I have to agree with Thomas. If you use lists like arrays then that's just going to be slow (and possibly awkward). So you should either use arrays or stick with the functions you've written but move them "up" in a way so that you can easily replace the slow lists with arrays later.