For 2D graphics I need to optimize my functions, but in SBCL I get lots of comments about SBCL not being able to inline arithmetic operations. I tried all sorts of declarations but it doesn't seem to make the compiler happy. Here is a simple example:
(defun test-floor (x div)
(declare (type single-float x)
(type (signed-byte 64) div)
(optimize (speed 3)))
(floor x div))
gives the following 4 notes below. I'm completely lost as #'floor is a builtin function. I tried to find information/tutorials on how to properly give the compiler hints in SBCL and didn't find the right information, so any info would be very appreciated! Unfortunately optimization in Common Lisp is quite unknown territory for me. I'm using SBCL 1.3.20 on a Linux machine.
; file: /tmp/file595dqU
; in: defun test-floor
; (FLOOR CL-FLOCKS::X CL-FLOCKS::DIV)
; --> MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL TRUNCATE LET*
; ==>
; (SB-KERNEL:%UNARY-TRUNCATE/SINGLE-FLOAT (/ SB-C::X SB-C::F))
;
; note: forced to do full call
; unable to do inline float truncate (cost 5) because:
; The result is a (values integer &optional), not a (values
; (signed-byte 64) &rest
; t).
; --> MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL TRUNCATE LET* VALUES - *
; ==>
; (SB-KERNEL:%SINGLE-FLOAT SB-C::RES)
;
; note: forced to do full call
; unable to do inline float coercion (cost 5) because:
; The first argument is a integer, not a (signed-byte 64).
; --> MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL FUNCTION IF VALUES 1-
; ==>
; (- SB-C::TRU 1)
;
; note: forced to do generic-- (cost 10)
; unable to do inline fixnum arithmetic (cost 1) because:
; The first argument is a integer, not a fixnum.
; The result is a (values integer &optional), not a (values fixnum &rest t).
; unable to do inline fixnum arithmetic (cost 2) because:
; The first argument is a integer, not a fixnum.
; The result is a (values integer &optional), not a (values fixnum &rest t).
; etc.
; --> MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL FUNCTION IF VALUES
; ==>
; (+ REM SB-C::DIVISOR)
;
; note: doing signed word to integer coercion (cost 20) from div, for:
; the second argument of generic-+
;
; compilation unit finished
; printed 4 notes
CL-USER>
When you call floor, you have to deal with different subtypes of numbers: the code divides a float by an integer (that may involve coercing the integer as a float), and then have to coerce the result back to an integer. This is the amount of work that must be done, correctly, and you are unlikely to bypass it if you don't restrict your input types.
If instead you use ffloor, then the primary result is a float (you can still round it to an integer later, when you really need it (e.g. convert to pixels coordinates)). The following code does not give compilation notes:
(defun test-floor (x div)
(declare (type single-float x)
(type fixnum div)
(optimize (speed 3)))
(ffloor x div))
You may even declare div as being a float, which would push the responsibility of providing appropriately typed values (and performing runtime checks) to the callers.
Note also that you should probably (declaim (inline test-floor)) before you define the function; that helps because then the compiler can put shortcuts in the code to avoid checking input parameter types and boxing results.
Edit:
The range of a float cover a large possible domain (due to the exponent): more densely packed near zero, more spaced towards infinity. Integer values are linearly spaced, but cover a smaller range with the same number of bits. So if you want to guarantee your output fits in a fixnum, you have to make sure the float in your inputs does not go outside the range of fixnum, too. I tried the following:
(defun test-round (x)
(declare (type (single-float #.(float most-negative-fixnum 0f0)
#.(float (/ most-positive-fixnum 2) 0f0)) x)
(optimize (speed 3)))
(round x))
I had to half the upper range of the floats because when you test:
(typep (round (coerce most-positive-fixnum 'single-float)) 'fixnum)
... it returns NIL. I don't have much time to see why this happens, but this depends on your implementation and architecture. Taking half the most positive fixnum ensures the value is low enough to be converted as a fixnum. Now, I have no more compilation notes.
(the same goes for (signed-byte 64)))
NB. Unlike in the above example, you should use deftype and avoid repeating the same declarations everywhere.
If you want to specify the return value of an expression you can use THE:
(the fixnum (1+ 3))
But you really want to make sure that the value is actually a fixnum. If you 'lie', then Lisp may believe you and you have unspecified runtime effects. SBCL may warn at compile time, but you really should take care of that. If you provide wrong types are return wrong types, data may get corrupted and/or Lisp may crash.
Another way to specify a return value is the FTYPE declaration:
For example a function ith may take an integer and a list as arguments. It returns an arbitrary type -> T or any subtype.
(declaim (ftype (function (integer list) t)
ith))
For example:
(the fixnum (+ (the fixnum a) (the fixnum b)))
There you need to be sure that:
a is an fixnum
b is an fixnum
the sum of a and b is also always a fixnum
Here this is easier, since the sum of a and b is definitely a fixnum:
CL-USER 3 > (let ((a 3) (b 12))
(the fixnum (+ (the (integer 0 10) a)
(the (integer 3 20) b))))
15
Lisp may check that at runtime and/or compile-time. The addition now can be a simple fixnum operation and does not need to deal with fixnum overflows and bignums. If you set the safety value to something low, then the runtime check may also be omitted. But: you should never call this code with the wrong types.
SBCL is claiming that it can’t optimise the call to floor because it isn’t sure the return value will be small enough to fit into a 64-bit integer.
CL-USER> (test-floor 1f25 1234)
8103727629894569426944 ;
0.0
CL-USER> (format nil “~b” *)
;; a long binary string
CL-USER> (length *)
73
A 73 bit Integer may be returned but does not fit into 64 bits.
Se also the SBCL manual:
4 Compiler
4.1.3 Understanding Compiler Diagnostics
6 Efficiency
Edit: after some searching I have found the transformation for floor. It is here. I reproduce it below:
(deftransform floor ((number divisor))
`(multiple-value-bind (tru rem) (truncate number divisor)
(if (and (not (zerop rem))
(if (minusp divisor)
(plusp number)
(minusp number)))
(values (1- tru) (+ rem divisor))
(values tru rem))))
So that explains what the compiler messages were talking about
Related
After googling for about an hour, I have to confess, that while I find a lot of documentation about functions operating on bit arrays, I cannot find a single reference on how to actually create a bit array.
Right now, it seems to me that either, some arrays with other element types can be handled as bit arrays OR that one could use (make-array :element-type (???)) where I could not find any explanation as to what to put where I wrote the "???".
So, while it is probably obvious to anyone else, I have no idea how to create a bit array. I know about how to write a literal bit array - but if I need a bit array with, say 2^16 bits - how would I do it?
You are right about using make-array, just use 'bit as the element type. Try
(make-array initial-size :element-type 'bit).
The symbol BIT names the bit type and could be replaced with any other type specifier to make an array holding objects of that type. In this example initial-size is just a variable holding a whole number.
Another way to create a bit vector:
> (make-sequence '(vector bit) 10)
#*0000000000
There's also a literal syntax using the #* reader macro, and notice that the concrete type may differ between using make-array and make-sequence, though I am not sure if performance may be different depending on that...
Tested with SBCL:
CL-USER> (defvar arr (make-array 10 :element-type 'bit :fill-pointer 0))
ARR
CL-USER> (type-of arr)
(VECTOR T 10)
CL-USER> (defvar arr3 (make-sequence '(vector bit) 10))
ARR3
CL-USER> (type-of arr3)
(SIMPLE-BIT-VECTOR 10)
CL-USER> (type-of #*0101010100)
(SIMPLE-BIT-VECTOR 10)
How about this:
(setq x 10)
10
(setq y (read-from-string (format nil "#*~7,'0b" x)))
#*0001010
The 7 is an arbitrary length, which could be set by
(setq z 8)
8
(setq y (read-from-string (format nil (concatenate 'string "#*~" (write-to-string z) ",'0b") x)))
#*00001010
A large bit array may be better handled as an unsigned integer.
I've written two versions of a lisp function. The main difference between the two is that one is done with recursion, while the other is done with iteration.
Here's the recursive version (no side effects!):
(defun simple-check (counter list)
"This function takes two arguments:
the number 0 and a list of atoms.
It returns the number of times the
atom 'a' appears in that list."
(if (null list)
counter
(if (equal (car list) 'a)
(simple-check (+ counter 1) (cdr list))
(simple-check counter (cdr list)))))
Here's the iterative version (with side effects):
(defun a-check (counter list)
"This function takes two arguments:
the number 0 and a list of atoms.
It returns the number of times the
atom 'a' appears in that list."
(dolist (item list)
(if (equal item 'a)
(setf counter (+ counter 1))
(setf counter (+ counter 0))))
counter)
As far as I know, they both work. But I'd really like to avoid side-effects in the iterative version. Two questions I'd like answered:
Is it possible to avoid side effects and keep iteration?
Assuming the answer to #1 is a yes, what are the best ways to do so?
For completeness, note that Common Lisp has a built-in COUNT:
(count 'a list)
In some ways, the difference between side-effect or no side-effect is a bit blurred. Take the following loop version (ignoring that loop also has better ways):
(loop :for x :in list
:for counter := (if (eq x 'a) (1+ counter) counter)
:finally (return counter))
Is counter set at each step, or is it rebound? I. e., is an existing variable modified (like in setf), or is a new variable binding created (as in a recursion)?
This do version is very much like the recursive version:
(do ((list args (rest list))
(counter 0 (+ counter (if (eq (first list) 'a) 1 0))))
((endp list) counter))
Same question as above.
Now the “obvious” loop version:
(loop :for x :in list
:count (eq x 'a))
There isn't even an explicit variable for the counter. Are there side-effects?
Internally, of course there are effects: environments are created, bindings established, and, especially if there is tail call optimization, even in the recursive version destroyed/replaced at each step.
I see as side effects only effects that affect things outside of some defined scope. Of course, things appear more elegant if you can also on the level of your internal definition avoid the explicit setting of things, and instead use some more declarative expression.
You can also iterate with map, mapcar and friends.
https://lispcookbook.github.io/cl-cookbook/iteration.html
I also suggest a look at remove-if[-not] and other reduce and apply:
(length (remove-if-not (lambda (x) (equal :a x)) '(:a :b :a))) ;; 2
Passing counter to the recursive procedure was a means to enable a tail recursive definition. This is unnecessary for the iterative definition.
As others have pointed out, there are several language constructs which solve the stated problem elegantly.
I assume you are interested in this in a more general sense such as when you cannot find
a language feature that solves a problem directly.
In general, one can maintain a functional interface by keeping the mutation private as below:
(defun simple-check (list)
"return the number of times the symbol `a` appears in `list`"
(let ((times 0))
(dolist (elem list times)
(when (equal elem 'a)
(incf times)))))
As an easy way to overflow a floating point (I'm using double floats in my code, so I'll do so here as well):
(setq *read-default-float-format* 'double-float)
(defun example-float-overflow (x)
(example-float-overflow (* x x)))
(example-float-overflow 4.4)
Very quickly, x grows larger and larger. Pretty soon it reaches 5.295234290518905e164 and overflows. Which, is that even a double float anymore?
Anyway, what is the best way to identify the point right before it overflows? Right now I'm doing something like:
(defun example-float-overflow-no-error (x)
(if (> (* x x) 1.0e20)
x
(example-float-overflow-no-error (* x x))))
(example-float-overflow 4.4)
=> 1.973525870240772e10
Note: I'm not actually interested in the result, but the rest of my code depends on it to run as many times as it can before overflowing.
Barmar suggested handling the floating-point-overflow condition just after the overflow occurs. This is a little bit than detecting when it is about to occur, but it's probably the easiest thing to do. For instance, here's an add function that adds just like +, except that if something goes wrong, you can use the use-value restart to provide a different value:
(defun add (&rest numbers)
"Add numbers with +, but with a USE-VALUE restart
available in case of an overflow (or other condition)."
(restart-case (reduce '+ numbers)
(use-value (value &optional condition) value)))
Then you can establish use-value restarts that can be used to provide a value if a call to add fails:
;; Attempt to some arithmetic, but with a handler bound that
;; will return 42 if an floating point-overflow occurs.
(handler-bind ((floating-point-overflow
(lambda (condition)
(use-value 42 condition))))
(+ 5 (add most-positive-double-float most-positive-double-float)))
;; |----------- this ends up evaluating to 42 ---------------|
;;|------------- and 42 + 5 is 47 --------------------------------|
;=> 47
When I run this function from a listener in LispWorks, it either crashes the listener or gives an exception and assembly language data. Can anyone tell me what's wrong with it?
(defun should-flip-block (rowlist)
(declare ((vector number) rowlist))
(if (= (length rowlist) 0) (eval nil)
(let* ((exithigh (= (car (last rowlist)) 2))
(enterhigh (= (first rowlist) 2)))
(and exithigh enterhigh))))
It's called as (should-flip-block '(1 2 1 2 1)).
Problematic declaration
Note that not all Common Lisp implementations will think that (declare ((vector number)) rowvector) is a valid declaration.
Write instead (declare (type (vector number) rowvector)).
Wrong: a list is not a vector
The problems you see is because you lied to the implementation and safety is set low. You told Lisp that the argument is a vector, but you pass a list (which is not a vector).
The function then use calls to FIRST and LAST, which don't work on vectors, but lists.
Run code with higher safety value
Don't run Common Lisp by default with low safety. Use a default safety value of 2 or 3.
Using LispWorks 6.1.1:
CL-USER 43 > (proclaim '(optimize (safety 2)))
NIL
now I recompile the function and then call it:
CL-USER 44 > (should-flip-block '(1 2 1 2 1))
Error: Variable ROWLIST was declared type (VECTOR NUMBER) but is being
bound to value (1 2 1 2 1)
1 (abort) Return to level 0.
2 Return to top loop level 0.
Now you see an useful error and not a segment violation.
Literal Vectors
#(1 2 1 2 1) is a vector.
Note: LIST type has no parameter
Note that a type (list number) does not exist in Common Lisp and can't be defined. The type list can't have a parameter. It's also not possible to define such a type based on the type cons - recursive types don't work.
You declare that rowlist is a vector (but treat it is a list — using last and first).
This means that the compiler assumes that the object you pass to it is a vector, so, when you pass it a list, you get undefined behavior.
The most important thing to know about declarations in Lisp is: do not lie to the compiler.
I.e., if you violate your declarations (like you just did), you will get burned.
(Additionally, you do not need to eval nil, and there is no need for the let* since you are using the variables it binds just once).
This recursive definition of a macro does what it should (sum integers from 1 to n):
(defmacro sum-int-seq (n)
`(cond
((equal 0 ,n) 0)
(t (+ ,n (sum-int-seq (- ,n 1))))))
For example (sum-int-seq 5) gives 15.
But why does it work? When the macro gets expanded i get this:
(macroexpand '(sum-int-seq 5))
(IF (EQUAL 0 5) 0 (+ 5 (SUM-INT-SEQ (- 5 1))))
But because sum-int-seq is a macro the macro evaluation should become an infinite loop. Does the compiler create a recursive function instead? If this definition creates a recursive function is there any way to define macros recursively?
(This is a silly example for the sake of brevity, a function would of course work better for this)
Your example does not work.
It may work in an interpreter. But with a compiler you'll see an endless loop during compilation.
CL-USER 23 > (defun test (foo)
(sum-int-seq 5))
TEST
Let's use the LispWorks interpreter:
CL-USER 24 > (test :foo)
15
Let's try to compile the function:
CL-USER 25 > (compile 'test)
Stack overflow (stack size 15997).
1 (continue) Extend stack by 50%.
2 Extend stack by 300%.
3 (abort) Return to level 0.
4 Return to top loop level 0.
Type :b for backtrace or :c <option number> to proceed.
Type :bug-form "<subject>" for a bug report template or :? for other options.
So, now the next question: why does it work in the interpreter, but the compiler can't compile it?
Okay, I'll explain it.
Let's look at the interpreter first.
it sees (sum-int-seq 5).
it macroexpands it to (COND ((EQUAL 0 5) 0) (T (+ 5 (SUM-INT-SEQ (- 5 1))))).
it then evaluates above form. It determines that it needs to compute (+ 5 (SUM-INT-SEQ (- 5 1))). For that it needs to macroexpand (SUM-INT-SEQ (- 5 1)).
eventually it will expand into something like (cond ((EQUAL 0 (- (- (- (- (- 5 1) 1) 1) 1) 1)) 0) .... Which then will return 0 and the computation can use this result and add the other terms to it.
The interpreter takes the code, evaluates what it can and macroexpands if necessary. The generated code is then evaluated or macroexpanded. And so on.
Now let's look at the compiler.
it sees (sum-int-seq 5) and macroexpands it into (COND ((EQUAL 0 5) 0) (T (+ 5 (SUM-INT-SEQ (- 5 1))))).
now the macroexpansion will be done on the subforms, eventually.
the compiler will macroexpand (SUM-INT-SEQ (- 5 1)). note that the code never gets evaluated, only expanded.
the compiler will macroexpand (SUM-INT-SEQ (- (- 5 1) 1)) and so forth. finally you'll see a stack overflow.
The compiler walks (recursively compiles / expands) the code. It may not execute the code (unless it does optimizations or a macro actually evaluates it explicitly).
For a recursive macro you'll need to actually count down. If you eval inside the macro, then something like (sum-int-seq 5) can made work. But for (defun foo (n) (sum-int-seq n)) this is hopeless, since the compiler does not know what the value of n is.
One other thing to add: in your example, the occurrence of sum-int-seq inside the macro is inside a quoted expression, so it doesn't get expanded when the macro is evaluated. It's just data until the macro is called. And since it is nested inside a cond, at run-time the inner macro only gets called when the condition is true, same as in a regular function.
Expanding a macro generates Lisp code that is then evaluated. Calling a function diverts the execution flow to a copy of pre-existing lisp code which is then run. Other than that, the two are pretty similar, and recursion works in the same way. In particular, macro expansion stops for the same reason that a properly written recursive function stops: because there is a termination condition,and the transformation between one call and the next has been written so that the this condition is actually reached. If it weren't reached, the macro expansion would enter a loop, just like an improperly written recursive function.
To the answer of Kilan I'd add, that macroexpand doesn't have to produce a full expansion of all macros in your form, until there's no macro left :) If you look at Hyperspec, you'll see, that it evaluates the whole form until it's not a macro (in your case it stops at if). And during compilation all the macros are expanded, as if macroexpand was applied to each element of the source tree, not only to its root.
Here's an implementation that does work:
(defmacro sum-int-seq (n)
(cond
((equal 0 n) `0)
(t `(+ ,n (sum-int-seq ,(- n 1))))))
It is possible to write a recursive macro, but (as was mentioned), the expansion must be able to hit the base case at compile time. So the values of all arguments passed to the macro must be known at compile time.
(sum-int-seq 5)
Works, but
(sum-int-seq n)
Does not.