How many memory the bit-vector using in sbcl?
Does per bit spend 1 bit memory?
Does per bit spend 1 byte memory?
Does per bit spend 1 word memory?
Bit vectors in SBCL are stored efficiently with one bit per bit, plus some small housekeeping overhead per vector.
They are also very efficient at bitwise operations, working a full word at a time. For example, BIT-XOR on a 64-bit platform will work on 64 bits of a bit-vector at once.
From Common Lisp one can ask if there is a special array type for bit vectors:
* (UPGRADED-ARRAY-ELEMENT-TYPE 'bit)
BIT
That means that when you ask for a bit vector, then CL provides you with a bit vector and not a, say, vector with 8 bit elements.
Size of an object in SBCL
Alastair Bridgewater provided this function as an attempt to get the 'size' of an object in SBCL:
(defun get-object-size/octets (object)
(sb-sys:without-gcing
(nth-value 2 (sb-vm::reconstitute-object
(ash (logandc1 sb-vm:lowtag-mask
(sb-kernel:get-lisp-obj-address object))
(- sb-vm:n-fixnum-tag-bits))))))
* (get-object-size/octets (make-array 40 :element-type 'bit :initial-element 1))
32
* (get-object-size/octets (make-array 400 :element-type 'bit :initial-element 1))
80
* (get-object-size/octets (make-array 4000 :element-type 'bit :initial-element 1))
528
Related
So I have some code that needs a subset of a multidimensional array in such a way that it works a bit more like taking a subsection of a matrix, ideally it would work like a displaced array.
So let's say I have something that looks like this
(defvar *a* (make-array '(3 3) :initial-contents
'((1 2 3) (2 3 1) (3 1 2))
And I want it to be accessible with an array *b*
(defvar *b* (make-array '(2 2) :displaced-to *a* :displaced-index-offset
(array-major-row-index *a* '(1 1)))
Such that *b* will point to
#2a((3 1) (1 2))
instead of
#2a((3 1) (3 1))
I've already written myself a multidimensional slice function that copies the parts of the array I want, but it would be ideal to not need to copy back and forth manually, is there any solution that works like this in vanilla common lisp?
I understand that the way that displaced multidimensional arrays work in a way that coheres directly with (array-major-row-index) (namely that #2a((1 2 3) (2 3 4)) has row indices (0 1 2 3 4 5) and therefore the displaced array at '(1 0) of dimensions '(2 2) will point to #2a((2 3) (2 3)), so I need to wrap the new array such that it refers to specific places in the old one, but so far I don't know how to capture such a reference.
I'm not entirely sure that it is possible to get pointers to places in the array, so I would appreciate if that could be cleared up.
You can't directly do it, but FYI there used to be support for this in Symbolics Lisp Machines.
From Kent Pitman:
What was new with the LispM, and which did not carry into CL (perhaps
because of the lack of microcode assist for speed) was conformally
displaced arrays (I think you said :displaced-conformally t, or some
such) in which case you got a displaced region of the original square
(cube, etc) rather than a region of the linearized storage. This was
useful for displacing to screen memory, especially since the LispM
used DMA (direct memory access) display from a raster array that was,
I think, specially known by the screen to mean "this array's memor IS
the screen" and doing a SETF of AREF into that special array made
something appear on the screen. All windows had conformally displaced
indirect arrays that represented their part of the screen.
As pointed out by Rainer Joswig, there is a video on Youtube, from Kalman Reti, demonstrating conformally displaced arrays. It might be possible for implementations to provide support for this, but I don't know if any current one provides such displaced arrays. But other answers are fine suggesting alternatives.
Multidimensional arrays are stored in memory as a one-dimensional array in row-major order. That is, #2a((1 2 3) (2 3 1) (3 1 2)) is actually the same as #(1 2 3 2 3 1 3 1 2).
CL-USER> (let ((a (make-array '(3 3) :initial-contents
'((1 2 3) (2 3 1) (3 1 2)))))
(make-array 9 :displaced-to a))
#(1 2 3 2 3 1 3 1 2)
A displaced array is a contiguous subset of the actual array (sharing memory with it). Your desired *B* would not be contiguous, since it would have to arbitrarily jump over the last 3 in the array.
*B*
/ \
--- ---
1 2 3 2 3 1 3 1 2
You would have to either include the skipped over 3 in the displaced array, or use two separate displaced arrays.
As explained in the answer of jkiiski you cannot obtain directly what you want, but you could “approximate” such result by using array of arrays, instead of multi-dimensional arrays.
For instance:
(defvar *a* (make-array 3 :initial-contents '(#(1 2 3) #(2 3 1) #(3 1 2))))
and then *b* could be defined as an array whose elements are arrays displaced on the appropriate arrays of *a*:
(defvar *b*
(make-array 2 :initial-contents
(loop for row from 1 to 2
collect (make-array 2 :displaced-to (aref *a1* row) :displaced-index-offset 1))))
The main difference with respect to the multi-dimensional arrays is that instead of using:
(aref *b* 1 1)
you should use:
(aref (aref *b* 1) 1) ; => produces 2 for the example above
And of course you could define macros or reader macros to simplify this notation.
I was wondering if there was a way to take a list of numbers (digits), and truncate the numbers together to be one large number (not addition) in Scheme. For example, I would want
(foo '(1 2 3 4))
;=> 1234
Does Scheme have a built in function to do this?
There are a number of languages that are in the Scheme family, and there are a few versions of Scheme, too. If you're using one, e.g., Racket, that includes a left associative fold (often called foldl, fold, or reduce, though there are other variations, too), this is pretty straightfoward to implement in terms of the fold. Folds have been described in more detail in these questions and answers:
Finding maximum distance between two points in a list (scheme) This question includes a description of how fold can be viewed as an iterative construct (and in Scheme, which mandates tail call optimization, is compiled to iterative code), and also includes an implementation of foldl for Schemes that don't have it.
Flattening a List of Lists This question is about a somewhat unusual fold, and how it (or a standard fold) can be used to flatten a list.
scheme structures and lists This question has an example of how you might adjust the function that you pass to a fold to achieve slightly different behavior. (I also include an opinionated (but true ;), I assure you) comment about how Common Lisp's reduce provides a somewhat more convenient interface than what's provided in some of the Scheme libraries.
Here's what the code looks like in terms of foldl:
(define (list->num digits)
(foldl (lambda (digit n)
(+ (* 10 n) digit))
0
digits))
> (list->num '(1 2 3 4))
1234
If your language doesn't have it, foldl is pretty easy to write (e.g., my answer to the one of the questions above includes an implementation) and use the preceding code, or you can write the whole function (using the same approach) yourself:
(define (list->num-helper digits number-so-far)
(if (null? digits)
number-so-far
(list->num-helper (cdr digits)
(+ (* 10 number-so-far)
(car digits)))))
(define (list->num digits)
(list->num-helper digits 0))
You can make that a bit more concise by using a named let:
(define (list->num digits)
(let l->n ((digits digits)
(number 0))
(if (null? digits)
number
(l->n (cdr digits)
(+ (* 10 number)
(car digits))))))
I read a lot of documentation about Clojure (and shall need to read it again) and read several Clojure questions here on SO to get a "feel" of the language. Besides a few tiny functions in elisp I've never written in any Lisp language before. I wrote my first project Euler solution in Clojure and before going further I'd like to better understand something about map and reduce.
Using a lambda, I ended up with the following (to sum all multiple of either 3 or 5 or both between 1 and 1000 inclusive):
(reduce + (map #(if (or (= 0 (mod %1 3)) (= 0 (mod %1 5))) %1 0) (range 1 1000)))
I put it on one line because I wrote it on the REPL (and it gives the correct solution).
Without the lambda, I wrote this:
(defn val [x] (if (or (= 0 (mod x 3)) (= 0 (mod x 5))) x 0))
And then I compute the solution doing this:
(reduce + (map val (range 1 1000)))
In both cases, my question concerns what the map should return, before doing the reduce. After doing the map I noticed I ended up with a list looking like this: (0 0 3 0 5 6 ...).
I tried removing the '0' at the end of the val definition but then I received a list made of (nil nil 3 nil 5 6 etc.). I don't know if the nil are an issue or not. I figured out that I was going to sum while doing a fold-left anyway so that the zero weren't really an issue.
But still: what's a sensible map to return? (0 0 3 0 5 6 ...) or (nil nil 3 nil 5 6...) or (3 5 6 ...) (how would I go about this last one?) or something else?
Should I "filter out" the zeroes / nils and if so how?
I know I'm asking a basic question but map/reduce is obviously something I'll be using a lot so any help is welcome.
It sounds like you already have an intuative undestanding of the need to seperate mapping concerns form the reducing It's perfectly natural to have data produced by map that is not used by the reduce. infact using the fact that zero is the identity value for addition make this even more elegant.
mappings job is to produce the new data (in this case 3 5 or "ignore")
reduces job is to decide what to include and to produce the final result.
what you started with is idiomatic clojure and there is no need to complicate it any more,
so this next example is just to illustrate the point of having map decide what to include:
(reduce #(if-not (zero? %1) (+ %1 %2) %2) (map val (range 10)))
in this contrived example the reduce function ignores the zeros. In typical real world code if the idea was as simple as filtering out some value then people tend to just use the filter function
(reduce + (filter #(not (zero? %)) (map val (range 10))))
you can also just start with filter and skip the map:
(reduce + (filter #(or (zero? (rem % 3)) (zero? (rem % 5))) (range 10)))
The watchword is clarity.
Use filter, not map. Then you don't have to choose a null
value that you later have to decide not to act on.
Naming the filtering/mapping function can help. Do so with let
or letfn, not defn, unless you have use for the function elsewhere.
Acting on this advice brings us to ...
(let [divides-by-3-or-5? (fn [n] (or (zero? (mod n 3)) (zero? (mod n 5))))]
(reduce + (filter divides-by-3-or-5? (range 1 1000))))
You may want to stop here for now.
This reads well, but the divides-by-3-or-5? function sticks in the throat. Change the factors and we need a completely new function. And that repeated phrase (zero? (mod n ...)) jars. So ...
We want a function, that - given a list (or other collection) of possible factors - tells us whether any of them apply to a given number. In other words, we want
a function of a collection of numbers - the possible factors - ...
that returns a function of one number - the candidate - ...
that tells us whether the candidate is divisible by any of the possible factors.
One such function is
(fn [ns] (fn [n] (some (fn [x] (zero? (mod n x))) ns)))
... which we can employ thus
(let [divides-by-any? (fn [ns] (fn [n] (some (fn [x] (zero? (mod n x))) ns)))]
(reduce + (filter (divides-by-any? [3 5]) (range 1 1000))))
Notes
This "improvement" has made the program a little slower.
divides-by-any? might prove useful enough to be promoted to a
defn.
If the operation were critical, you could consider stripping out
redundant factors. For example [2 3 6] could be reduced to [6].
If the operation were really critical, and the factors were supplied
as constants, you could consider creating the filter function with a
macro that went back to using or.
This is a bit of a shaggy-dog story, but it recounts the thoughts prompted by the problem you refer to.
In your case I would use keep instead of map. It is similar to map except that it keeps only the non-nil values.
If I understand correctly Clojure can return lists (as in other Lisps) but also vectors and sets.
What I don't really get is why there's not always a collection that is returned.
For example if I take the following code:
(loop [x 128]
(when (> x 1)
(println x)
(recur (/ x 2))))
It does print 128 64 32 16 8 4 2. But that's only because println is called and println has the side-effect (?) of printing something.
So I tried replacing it with this (removing the println):
(loop [x 128]
(when (> x 1)
x
(recur (/ x 2))))
And I was expecting to get some collecting (supposedly a list), like this:
(128 64 32 16 8 4 2)
but instead I'm getting nil.
I don't understand which determines what creates a collection and what doesn't and how you switch from one to the other. Also, seen that Clojure somehow encourages a "functional" way of programming, aren't you supposed to nearly always return collections?
Why are so many functions that apparently do not return any collection? And what would be an idiomatic way to make these return collections?
For example, how would I solve the above problem by first constructing a collection and then iterating (?) in an idiomatic way other the resulting list/vector?
First I don't know how to transform the loop so that it produces something else than nil and then I tried the following:
(reduce println '(1 2 3))
But it prints "1 2nil 3nil" instead of the "1 2 3nil" I was expecting.
I realize this is basic stuff but I'm just starting and I'm obviously missing basic stuff here.
(P.S.: retag appropriately, I don't know which terms I should use here)
A few other comments have pointed out that when doesn't really work like if - but I don't think that's really your question.
The loop and recur forms create an iteration - like a for loop in other languages. In this case, when you are printing, it is indeed just for the side effects. If you want to return a sequence, then you'll need to build one:
(loop [x 128
acc []]
(if (< x 1)
acc
(recur (/ x 2)
(cons x acc))))
=> (1 2 4 8 16 32 64 128)
In this case, I replaced the spot where you were calling printf with a recur and a form that adds x to the front of that accumulator. In the case that x is less than 1, the code returns the accumulator - and thus a sequence. If you want to add to the end of the vector instead of the front, change it to conj:
(loop [x 128
acc []]
(if (< x 1)
acc
(recur (/ x 2)
(conj acc x))))
=> [128 64 32 16 8 4 2 1]
You were getting nil because that was the result of your expression -- what the final println returned.
Does all this make sense?
reduce is not quite the same thing -- it is used to reduce a list by repeatedly applying a binary function (a function that takes 2 arguments) to either an initial value and the first element of a sequence, or the first two elements of the sequence for the first iteration, then subsequent iterations are passed the result of the previous iteration and the next value from the sequence. Some examples may help:
(reduce + [1 2 3 4])
10
This executes the following:
(+ 1 2) => 3
(+ 3 3) => 6
(+ 6 4) => 10
Reduce will result in whatever the final result is from the binary function being executed -- in this case we're reducing the numbers in the sequence into the sum of all the elements.
You can also supply an initial value:
(reduce + 5 [1 2 3 4])
15
Which executes the following:
(+ 5 1) => 6
(+ 6 2) => 8
(+ 8 3) => 11
(+ 11 4) => 15
HTH,
Kyle
The generalized abstraction over collection is called a sequence in Clojure and many data structure implement this abstraction so that you can use all sequence related operations on those data structure without thinking about which data structure is being passed to your function(s).
As far as the sample code is concerned - the loop, recur is for recursion - so basically any problem that you want to solve using recursion can be solved using it, classic example being factorial. Although you can create a vector/list using loop - by using the accumulator as a vector and keep appending items to it and in the exist condition of recursion returning the accumulated vector - but you can use reductions and take-while functions to do so as shown below. This will return a lazy sequence.
Ex:
(take-while #(> % 1) (reductions (fn [s _] (/ s 2)) 128 (range)))
I've been slowly working my way through the list of Project Euler problems, and I've come to one that I know how to solve, but it seems like I can't (given the way my solution was written).
I am using Common Lisp to do this with and my script has been running for over 24 hours (well over their one minute goal).
For the sake of conciseness, here's my solution (it's a spoiler, but only if you have one hell of a fast processor):
(defun square? (num)
(if (integerp (sqrt num)) T))
(defun factors (num)
(let ((l '()))
(do ((current 1 (1+ current)))
((> current (/ num current)))
(if (= 0 (mod num current))
(if (= current (/ num current))
(setf l (append l (list current)))
(setf l (append l (list current (/ num current)))))))
(sort l #'< )))
(defun o_2 (n)
(reduce #'+ (mapcar (lambda (x) (* x x)) (factors n))))
(defun sum-divisor-squares (limit)
(loop for i from 1 to limit when (square? (o_2 i)) summing i))
(defun euler-211 ()
(sum-divisor-squares 64000000))
The time required to solve the problem using smaller, more friendly, test arguments seems to grow larger than exponentialy... which is a real problem.
It took:
0.007 seconds to solve for 100
0.107 seconds to solve for 1000
2.020 seconds to solve for 10000
56.61 seconds to solve for 100000
1835.385 seconds to solve for 1000000
24+ hours to solve for 64000000
I'm really trying to figure out which part(s) of the script is causing it to take so long. I've put some thought into memoizing the factors function, but I'm at a loss as to how to actually implement that.
For those that want to take a look at the problem itself, here it be.
Any ideas on how to make this thing go faster would be greatly appreciated.
**sorry if this is a spoiler to anyone, it's not meant to be.... but if you have the computing power to run this in a decent amount of time, more power to you.
Here's a solution, keeping in mind the spirit of [Project] Euler. [Warning: spoiler. I've tried to keep the hints slow, so that you can read only part of the answer and think on your own if you want. :)]
When you are confronted with a problem having to do with numbers, one good strategy (as you probably already know from solving 210 Project Euler problems) is to look at small examples, find a pattern, and prove it. [The last part may be optional depending on your attitude to mathematics ;-)]
In this problem, though, looking at small examples -- for n=1,2,3,4,... will probably not give you any hint. But there is another sense of "small examples" when dealing with number-theoretic problems, which you also probably know by now -- primes are the building blocks of the natural numbers, so start with the primes.
For a prime number p, its only divisors are 1 and p, so the sum of the squares of its divisors is 1+p2.
For a prime power pk, its only divisors are 1, p, p2, … pk, so the sum of the squares of its divisors is 1+p+p2+…+pk=(pk+1-1)/(p-1).
That was the simplest case: you've solved the problem for all numbers with only one prime factor.
So far nothing special. Now suppose you have a number n that has two prime factors, say n=pq. Then its factors are 1, p, q, and pq, so the sum of the squares of its divisors is 1+p2+q2+p2q2=(1+p2)(1+q2).
What about n=paqb? What is the sum of the squares of its factors?
[............................Dangerous to read below this line...................]
It is ∑0≤c≤a, 0≤d≤b(pcqd)2 = ((pa+1-1)/(p-1))((qb+1-1)/(q-1)).
That should give you the hint, both on what the answer is and how to prove it: the sum of the divisors of n is simply the product of the (answer) for each of the prime powers in its factorization, so all you need to do is to factorize 64000000 (which is very easy to do even in one's head :-)) and multiply the answer for each (=both, because the only primes are 2 and 5) of its prime powers.
That solves the Project Euler problem; now the moral to take away from it.
The more general fact here is about multiplicative functions -- functions on the natural numbers such that f(mn) = f(m)f(n) whenever gcd(m,n)=1, i.e. m and n have no prime factors in common. If you have such a function, the value of the function at a particular number is completely determined by its values at prime powers (can you prove this?)
The slightly harder fact, which you can try to prove[it's not that hard], is this: if you have a multiplicative function f [here, f(n)=n2] and you define the function F as F(n) = ∑d divides nf(d), (as the problem did here) then F(n) is also a multiplicative function.
[In fact something very beautiful is true, but don't look at it just yet, and you'll probably never need it. :-)]
I think that your algorithm is not the most efficient possible. Hint: you may be starting from the wrong side.
edit: I'd like to add that choosing 64000000 as the upper limit is likely the problem poster's way of telling you to think of something better.
edit: A few efficiency hints:
instead of
(setf l (append l (...)))
you can use
(push (...) l)
which destructively modifies your list by consing a new cell with your value as car and the former l as cdr, then points l to this cell. This is much faster than appending which has to traverse the list once each. If you need the list in the other order, you can nreverse it after it is complete (but that is not needed here).
why do you sort l?
you can make (> current (/ num current)) more efficient by comparing with the square root of num instead (which only needs to be computed once per num).
is it perhaps possible to find the factors of a number more efficiently?
And a style hint: You can put the scope of l into the do declaration:
(do ((l ())
(current 1 (+ current 1)))
((> current (/ num current))
l)
...)
I would attack this by doing the prime factorization of the number (for example: 300 = 2^2 * 3^1 * 5^2), which is relatively fast, especially if you generate this by sieve. From this, it's relatively simple to generate the factors by iterating i=0..2; j=0..1; k=0..2, and doing 2^i * 3^j * 5^k.
5 3 2
-----
0 0 0 = 1
0 0 1 = 2
0 0 2 = 4
0 1 0 = 3
0 1 1 = 6
0 1 2 = 12
1 0 0 = 5
1 0 1 = 10
1 0 2 = 20
1 1 0 = 15
1 1 1 = 30
1 1 2 = 60
2 0 0 = 25
2 0 1 = 50
2 0 2 = 100
2 1 0 = 75
2 1 1 = 150
2 1 2 = 300
This might not be fast enough
The clever trick you are missing is that you don't need to factor the numbers at all
How many numbers from 1..N are multiples of 1? N
How many numbers from 1..N are multiples of 2? N/2
The trick is to sum each number's factors in a list.
For 1, add 1^2 to every number in the list. For 2, add 2^2 to every other number.
For 3, add 3^2 to every 3rd number.
Don't check for divisibility at all.
At the end, you do have to check whether the sum is a perfect square, and that's it.
In C++, this worked in 58 seconds for me.
Sorry, I don't understand LISP well enough to read your answer. But my first impression is that the time cost of the brute force solution should be:
open bracket
sqrt(k) to find the divisors of k (by trial division), square each one (constant time per factor), and sum them (constant time per factor). This is σ2(k), which I will call x.
plus
not sure what the complexity of a good integer square root algorithm is, but certainly no worse than sqrt(x) (dumb trial multiplication). x might well be big-O larger than k, so I reserve judgement here, but x is obviously bounded above by k^3, because k has at most k divisors, each itself no bigger than k and hence its square no bigger than k^2. It's been so long since my maths degree that I have no idea how fast Newton-Raphson converges, but I suspect it's faster than sqrt(x), and if all else fails a binary chop is log(x).
close bracket
multiplied by n (as k ranges 1 .. n).
So if your algorithm is worse than O(n * sqrt(n^3)) = O(n ^ (5/2)), in the dumb-sqrt case, or O(n * (sqrt(n) + log(n^3)) = O(n ^ 3/2) in the clever-sqrt case, I think something has gone wrong which should be identifiable in the algorithm. At this point I'm stuck because I can't debug your LISP.
Oh, I've assumed that arithmetic is constant-time for the numbers in use. It darn well should be for numbers as small as 64 million, and the cube of that fits in a 64bit unsigned integer, barely. But even if your LISP implementation is making arithmetic worse than O(1), it shouldn't be worse than O(log n), so it won't have much affect on the complexity. Certainly won't make it super-polynomial.
This is where someone comes along and tells me just how wrong I am.
Oops, I just looked at your actual timing figures. They aren't worse than exponential. Ignoring the first and last values (because small times aren't accurately measurable and you haven't finished, respectively), multiplying n by 10 multiplies time by no more than 30-ish. 30 is about 10^1.5, which is about right for brute force as described above.
I think you can attack this problem with something like a prime sieve. That's only my first impression though.
I've reworked the program with some notes taken from the comments here. The 'factors' function is now ever so slightly more efficient and I also had to modify the σ_(2)(n) function to accept the new output.
'factors' went from having an output like:
$ (factors 10) => (1 2 5 10)
to having one like
$ (factors 10) => ((2 5) (1 10))
Revised function looks like this:
(defun o_2 (n)
"sum of squares of divisors"
(reduce #'+ (mapcar (lambda (x) (* x x)) (reduce #'append (factors n)))))
After the modest re-writes I did, I only saved about 7 seconds in the calculation for 100,000.
Looks like I'm going to have to get off of my ass and write a more direct approach.