How to prevent close!-ing before put-ing in onto-chan - asynchronous

I'd like to run a code like
(->> input
(partition-all 5)
(map a-side-effect)
dorun)
asynchronously dividing input and output(a-side-effect).
Then I've written the code to experiment below.
;; using boot-clj
(set-env! :dependencies '[[org.clojure/core.async "0.2.374"]])
(require '[clojure.core.async :as async :refer [<! <!! >! >!!]])
(let [input (range 18)
c (async/chan 1 (comp (partition-all 5)
(map prn)))]
(async/onto-chan c input false)
(async/close! c))
explanation for this code:
Actually elements in input and its quantity is not defined before running and elements in input is able to be taken by some numbers from 0 to 10.
async/onto-chan is used to put a Seq of elements (a fragment of input) into the channel c and will be called many times thus the 3rd argument is false.
prn is a substitute for a-side-effect.
I expected the code above prints
[0 1 2 3 4]
[5 6 7 8 9]
[10 11 12 13 14]
[15 16 17]
in REPL however it prints no characters.
And then I add a time to wait, like this
(let [c (async/chan 1 (comp (partition-all 5)
(map prn)))]
(async/onto-chan c (range 18) false)
(Thread/sleep 1000) ;wait
(async/close! c))
This code gave my expected output above.
And then I inspect core.async/onto-chan.
And I think what happend:
the channel c was core.async/close!ed in my code.
each item of the argument of core.async/onto-chan was put(core.async/>!) in vain in the go-loop in onto-chan because the channel c was closed.
Are there sure ways to put items before close!ing?
write a synchronous version of onto-chan not using go-loop?
Or is my idea wrong?

Your second example with Thread.sleep only ‘works’ by mistake.
The reason it works is that every transformed result value that comes out of c’s transducer is nil, and since nils are not allowed in channels, an exception is thrown, and no value is put into c: this is what allows the producer onto-chan to continue putting into the channel and not block waiting. If you paste your second example into the REPL you’ll see four stack traces – one for each partition.
The nils are of course due to mapping over prn, which is a side-effecting function that returns nil for all inputs.
If I understand your design correctly, your goal is to do something like this:
(defn go-run! [ch proc]
(async/go-loop []
(when-let [value (<! ch)]
(proc value)
(recur))))
(let [input (range 18)
c (async/chan 1 (partition-all 5))]
(async/onto-chan c input)
(<!! (go-run! c prn)))
You really do need a producer and a consumer, else your program will block. I’ve introduced a go-loop consumer.
Very generally speaking, map and side-effects don’t go together well, so I’ve extracted the side-effecting prn into the consumer.
onto-chan cannot be called ‘many times’ (at least in the code shown) so it doesn’t need the false argument.

taking megakorre's idea:
(let [c (async/chan 1 (comp (partition-all 5)
(map prn)))
put-ch (async/onto-chan c (range 18) false)]
(async/alts!! [put-ch])
(async/close! c))

Related

Clojure swap! atom executed in parallel

I'm playing with clojure to do a script to read as input a sequence of URIs from a file and do a report on the status code for them.
I've implemented this using clojure.core.async/pipeline-async to execute the HTTP call to the URI (using an httpkit async call).
I want to monitor the execution of the script so I've an atom for the state:
(let [processing (atom [(System/currentTimeMillis) 0])]
and a function to track the progress.
(defn track-progress [total progress]
(swap! progress
(fn [[time count]]
(let [incremented-count (inc count)
now (System/currentTimeMillis)]
(if (= 0 (mod incremented-count (max 1 (int (/ total 20)))))
(do
(println (str "Progress " incremented-count "/" total " | " (- now time) "ms"))
[now incremented-count])
[time incremented-count])))))
Using it after the HTTP call:
(a/pipeline-async
parallelism
output-chan
(fn [[an-id uri] result]
(http/head uri {:throw-exceptions false
:timeout timeout}
(fn [{:keys [status error]}]
(track-progress total processing)
(a/go
(if (nil? error)
(do (a/>! result [an-id (keyword (str status))])
(a/close! result))
(do (a/>! result [an-id :error])
(a/close! result)))))))
input-chan)
The processing atom is created in a let expression, using that pipeline-async part.
Everything seems working fine, apart from that log.
I found out that sometimes the logging is very weird, having stuffs like this:
Progress 500/10000 | 11519ms
Progress 500/10000 | 11519msProgress 500/10000 | 11519ms
Progress 1000/10000 | 11446ms
Progress 1000/10000 | 11446ms
Progress 1500/10000 | 9503ms
Progress 2000/10000 | 7802ms
Progress 2500/10000 | 12822ms
Progress 2500/10000 | 12822msProgress 2500/10000 | 12822ms
Progress 2500/10000 | 12822ms
Progress 3000/10000 | 10623ms
Progress 3500/10000 | 9018ms
Progress 4000/10000 | 9618ms
Progress 4500/10000 | 13544ms
Progress 5000/10000 | 10541ms
Progress 5500/10000 | 10817ms
Progress 6000/10000 | 8921ms
Progress 6500/10000 | 9078ms
Progress 6500/10000 | 9078ms
Progress 7000/10000 | 9270ms
Progress 7500/10000 | 11826msProgress 7500/10000 | 11826msProgress 7500/10000 | 11826ms
The output is formatted as it is wrote in the shell, it seems that sometimes the same println is executed multiple times, or the fn passed to the swap! function is executed in parallel (no concurrency) in the atom.
(If the the println I remove the str to create the string to print, the lines in which I have the same progress multiple times are totally mixed up like ProgressProgress 7500/10000 | 11826ms7500/100007500 | 11826msProgress/10000 | 11826ms)
Is it something wrong with my code?
Or I am getting the atom wrong, as I supposed it not allows the parallel execution of a function changing its state?
A Clojure atom is designed specifically so that in a multi-threaded program, there can be multiple threads executing swap! on a single atom, and if your program does this, those update functions f given to swap! can run simultaneously. The only part of swap! that is synchronized is a 'compare and swap' operation that effectively does:
lock the atom's state
check if its current value is identical? to the reference it contained before f began executing, and if it is, replace it with the new object returned by f.
Unlock the atom's state".
The function f may take a long time to calculate a new value from the current one, but the critical section above is merely a pointer comparison, and if equal, a pointer assignment.
That is why the doc string for swap! says "Note that f may be called multiple times, and thus should be free of side effects."
What you want is to serialize the output stream from a group of concurrently executing threads. You could use an agent to serialize access to a piece of mutable state, but here you have a degenerate case without state, only with side-effects. For this case, the locking function is all you need.
An example:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test))
(defn do-println
[& args]
(apply println args))
(def lock-obj (Object.))
(defn do-println-locking
[& args]
(locking lock-obj
(apply println args)))
(def sleep-millis 500)
(defn wait-and-print
[print-fn id]
(Thread/sleep sleep-millis)
(print-fn (format "wait-and-print %s is complete" id)))
(defn start-threads
[print-fn count]
(println "-----------------------------------------------------------------------------")
(let [futures (forv [i (range count)]
(future (wait-and-print print-fn i)))]
(doseq [future futures]
; block until future is complete
(deref future))))
(dotest
(start-threads do-println 10)
(start-threads do-println-locking 10))
Typical result:
--------------------------------------
Clojure 1.10.2-alpha1 Java 15
--------------------------------------
Testing tst.demo.core
-----------------------------------------------------------------------------
wait-and-print 4 is completewait-and-print 3 is completewait-and-print 2 is complete
wait-and-print 8 is completewait-and-print 9 is complete
wait-and-print 6 is completewait-and-print 1 is complete
wait-and-print 7 is complete
wait-and-print 0 is complete
wait-and-print 5 is complete
-----------------------------------------------------------------------------
wait-and-print 5 is complete
wait-and-print 8 is complete
wait-and-print 7 is complete
wait-and-print 9 is complete
wait-and-print 6 is complete
wait-and-print 3 is complete
wait-and-print 0 is complete
wait-and-print 4 is complete
wait-and-print 2 is complete
wait-and-print 1 is complete
So you can see the output without serialization from locking is jumbled, while each println in the 2nd case is allowed to complete one-at-a-time (even though the order is still random).
If println printed one char at a time instead of one string at a time, the results in the unsynchronized case would be even more jumbled. Modify the output functions to print each character separately:
(defn do-println
[& args]
(doseq [ch (str/join args)]
(print ch))
(newline))
(def lock-obj (Object.))
(defn do-println-locking
[& args]
(locking lock-obj
(apply do-println args)))
with typical result:
--------------------------------------
Clojure 1.10.2-alpha1 Java 15
--------------------------------------
Testing tst.demo.core
-----------------------------------------------------------------------------
wwwwwaaawwiiiattti--taaa--nnaiddnaa--dwpp-irrptaiir-niiantnttn -dw2ta- ani96ipds trn- i-pcndrota-impn nrpd4itl- n eipt5tr s e7i
incisots mc0cpo olmmieppstll ee
etctteo
e-
amnidps-l pectroeai
intt- a1n di-sip rcsio nmctmpo plm3lew etaiei
spt t-lceeatone
d
m-pplreitnet
8 is complete
-----------------------------------------------------------------------------
wait-and-print 3 is complete
wait-and-print 9 is complete
wait-and-print 8 is complete
wait-and-print 4 is complete
wait-and-print 6 is complete
wait-and-print 7 is complete
wait-and-print 0 is complete
wait-and-print 1 is complete
wait-and-print 5 is complete
wait-and-print 2 is complete
but we see that locking serializes the function calls so that the active call must complete before the next can begin.

How do I wrap a sample in an envelope in Overtone?

Some samples I use from Freesound.org have a slight click at the end, e.g.:
repl> (use 'overtone.live)
nil
repl> (def stick (freesound 82280))
#'repl/stick
repl> (stick)
So I'm trying to wrap this sample in an envelope, however all I get is silence. I suspect there's something wrong with my use of buf-rd...
(definst stick1
[amp 0.7]
(let [env (env-gen (perc) :action FREE)
phase (phasor:ar :start 0 :end 1 :rate 1)
index (* phase (buf-frames stick))
snd (buf-rd 1 stick index)]
(* amp env snd)))
(stick1)
play-buf is the correct function to encorporate a sample in an envelope. perc is used to set an attack of 0.01 seconds, and a release of 1 second before silence, thus killing the click.
(def stick (freesound 82280))
(definst stick1
[amp 0.7]
(let [env (env-gen (perc 0.01 1) :action FREE)
snd (play-buf 1 stick)]
(* amp env snd)))
(stick1)

What's the difference between count and length?

For example in Scheme (count '(1 2 3)) gives 3 and (length '(1 2 3)) also gives 3.
It depends on what interpreter you're using. In standard Scheme, only length is defined. In other interpreters (say, Racket) count exists but it's different, it receives a list and a predicate and returns the number of elements in the list that meet the condition.
I don't know in which interpreter count is defined as a single-parameter function that returns the length of the list, (In Racket (count '(1 2 3)) causes an error), but it seems to me that count is just an alias for length in your interpreter (in other words: they're the same thing) - to be sure, please check the documentation. If I had to choose one, I'd use length, which is standard and will work everywhere.
length returns the number of elements in a list.
count is not a standard procedure mentioned in any of the official Scheme reports (I searched R5RS, R6RS and R7RS) so it's not a part of Scheme. In many implementations you will get some sort of error saying that count does not exist. This is radically different than the expected result you have in your question but it is the more likely result if you were to test it in 5 Scheme implementations.
There is no reason to use a implementation dependent extension when its result is the same as length.
I have tried (count '(1 2 3)) in stalin (r4rs), scm (r5rs), chicken (r5rs), gambit (r5rs), racket (both r5rs and r6rs), ikarus (r6rs), chibi-scheme (r7rs), gauche/gosh (r7rs). None of them have count.
sylwester#pussycat:/p/n/sylwester$ csi
CHICKEN
(c) 2008-2013, The Chicken Team
(c) 2000-2007, Felix L. Winkelmann
Version 4.8.0.5 (stability/4.8.0) (rev 5bd53ac)
linux-unix-gnu-x86-64 [ 64bit manyargs dload ptables ]
compiled 2013-10-03 on aeryn.xorinia.dim (Darwin)
#;1> (count '(1 2 3))
Error: unbound variable: count
Call history:
<syntax> (count (quote (1 2 3)))
<syntax> (quote (1 2 3))
<syntax> (##core#quote (1 2 3))
<eval> (count (quote (1 2 3))) <--
#;1>
sylwester#pussycat:/p/n/sylwester$ gsi
Gambit v4.6.9
> (count '(1 2 3))
*** ERROR IN ##raise-unbound-global-exception -- Unbound variable: count
1>
>
*** EOF again to exit
sylwester#pussycat:/p/n/sylwester$ ikarus
Ikarus Scheme version 0.0.4-rc1+, 64-bit (revision 1870, build 2012-02-21)
Copyright (c) 2006-2009 Abdulaziz Ghuloum
> (count '(1 2 3))
Unhandled exception
Condition components:
1. &undefined
2. &who: eval
3. &message: "unbound variable"
4. &irritants: (count)
>
sylwester#pussycat:/p/n/sylwester$ plt-r5rs
Welcome to Racket v6.1.1.
R5RS legacy support loaded
> (count '(1 2 3))
count: undefined;
cannot reference undefined identifier
context...:
/usr/share/racket/collects/racket/private/misc.rkt:87:7
/usr/share/racket/pkgs/r5rs-lib/r5rs/run.rkt: [running body]
sylwester#pussycat:/p/n/sylwester$ echo "(import (rnrs))
(display (count '(1 2 3)))" > test.scm
sylwester#pussycat:/p/n/sylwester$ plt-r6rs test.scm
test.scm:2:10: count: unbound identifier in module
in: count
context...:
/usr/share/racket/pkgs/r6rs-lib/r6rs/run.rkt: [running body]
sylwester#pussycat:/p/n/sylwester$ chibi-scheme
> (count '(1 2 3))
ERROR on line 1: undefined variable: count
>
sylwester#pussycat:/p/n/sylwester$
sylwester#pussycat:/p/n/sylwester$ gosh
gosh> (length '(1 2 3))
3
gosh> (count '(1 2 3))
*** ERROR: unbound variable: count
Stack Trace:
_______________________________________
gosh>
sylwester#pussycat:/p/n/sylwester$ scm
SCM version 5e5, Copyright (C) 1990-2006 Free Software Foundation.
SCM comes with ABSOLUTELY NO WARRANTY; for details type `(terms)'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `(terms)' for details.
;loading /usr/share/slib/require
;done loading /usr/share/slib/require.scm
;loading /usr/share/slib/require
;done loading /usr/share/slib/require.scm
;loading /usr/lib/scm/Link
;done loading /usr/lib/scm/Link.scm
;loading /usr/lib/scm/Transcen
;done loading /usr/lib/scm/Transcen.scm
> (count '(1 2 3))
;ERROR: "/usr/lib/scm/Iedline.scm": unbound variable: count
; in expression: (count (quote (1 2 3)))
; in top level environment.
;STACK TRACE
1; (##let ((tail (##lambda (c) (##if (##char? ##c) ##c (##let* (( ...
2; (count (quote (1 2 3)))
>
sylwester#pussycat:/p/n/sylwester$ echo "(display (count '(1 2 3)))" > test.scm
sylwester#pussycat:/p/n/sylwester$ stalin -On test.scm
test.scm:1:10:Unbound variable

Lisp, why is this number not a float

Using Common Lisp I am trying loop through a list of students and if the GPA is greater than or equal to 3.0 I want to push a 1 onto another list called equal_names. The problem I am having is the interpreter keeps saying the GPA in the comparison list is "not of type (or rational float)". Why am I getting this error?
Yes, this is for homework. Also this is my first time asking on here, so if you need anything else please let me know.
Sample of the list I am getting the GPA from, where the GPA is 2.307...:
(SETQ students (LIST
(LIST (LIST 'Abbott 'Ashley 'J) '8697387888 'NONE 2.3073320999676614)))
The code I have written:
(setq gpa_count ())
(loop for x in students
if(>= 3.0 (cdr (cdr (cdr x))))
do(push '1 gpa_count))
Given a non-empty list cdr returns the tail of that list, i.e. the list that contains all the elements of the list but the first. The important thing to note is that it returns a list, not an element. That is (cdr (cdr (cdr x))) returns the list (2.30733...), not the float 2.30733.
The loop iterates the outer list. To understand the code in the loop you can look at the first element in students, which is:
'((Abbott Ashley J) 8697387888 NONE 2.3073320999676614)
Now we are going to orientate the list. Every time you pass an element add a d.
Every time you pick a value or go to a list in the list you add an a.
To find how to access the number 2.307.... You look at the first element element in the list:
(Abbott Ashley J) d
8697387888 d
NONE d
Now we are at the part that you are interested in, ie. (2.3073320999676614)), thus you add an a. Now order those in reverse and put a c in front and a r in the end.. It becomes cadddr In light of that your loop should be:
(setq students '(("Mary Larson" 333 NONE 1.1)
("Mina Morson" 333 NONE 2.5)
("Magnus Outsider" 333 NONE 4.1)))
(setq gpa_count ())
(loop for x in students
if (>= 3.0 (cadddr x))
do (push '1 gpa_count))
gpa_count ; ==> (1 1)
Another example:
(setq x (1 (2 3) (3 4 (5 6) 7))) ; ==> (1 (2 3) (3 4 (5 6) 7))
To get the 3*. We follow the parts. 1 == d, (2 3) == a, 2 ==d, 3* == a. In reverse: adad and add c and r to the ends ==> cadadr. thus:
(cadadr '(1 (2 3) (3 4 (5 6) 7))) ; ==> 3
To get the 5. we do the same 1 == d, (2 3) == d and then we have the list we want ==a.
Then 3 ==d, 4 ==d, (5 6) ==a. The 5 is the first element == a. In reverse aaddadd. Now CL guarantees 4 letters accessors so we need to split it up in 4s from the right. Thus it becomes:
(caadr (cdaddr '(1 (2 3) (3 4 (5 6) 7)))) ; ==> 5
Now, without describing you can pick any number or list. Eg. to get (5 6) ddadda, in reverse and split up becomes (cadr (cdaddr x))
Hope it helps.
If your data format is consistent then
(fourth x)
will return the GPA.
Going further,
(setf (symbol-function 'gpa)(function fourth))
would provide
(gpa x)
as "an accessor" for the gpa in the data structure.
My CLISP 2.49 gives this error message:
*** - >=: (2.307332) is not a real number
Let's look at that error message: >=: (2.307332) is not a real number.
The error happens at the call to >= and one argument is a list of a number, not a number.
Since you try to extract the number from a list, does that extract work?
We see that you call CDR. CDR of a list returns a list. So there is the error. You need to extract the number from the list.
Btw., CLISP has commands like help, where, backtrace, ... to further investigate the problem. Just type help and return, without anything else, and you see a list of commands.

How does these Pascal Triangle functions work?

I'm reading Concepts, Techniques, and Models of Computer Programming, and there's a code at the beginning that I just cannot understand no matter how hard I try.
declare Pascal AddList ShiftLeft ShiftRight
fun {Pascal N}
if N==1 then [1]
else
L in
L = {Pascal N-1} % Recursion
{AddList {ShiftLeft L}
{ShiftRight L}}
end
end
fun {ShiftLeft L}
case L of H|T then
H|{ShiftLeft T} % Recursion
else [0]
end
end
fun {ShiftRight L}
0 | L
end
fun {AddList L1 L2}
case L1 of H1|T1 then
case L2 of H2|T2
then
H1+H2|{AddList T1 T2} % Recursion
end
else nil
end
end
I kind of get the language constructs (this is the introduction to it), but the thing that really stands in my way is the recursion.
I'm trying to put a label on each recursion call that will abstractly say what goes in here, but I just can't figure it out.
What I ask for is a clear and easy explanations of how these functions work.
Start with N == 1: This is simple. The result is just [1].
Now check for N == 2:
First we calculate L = {Pascal N-1} = {Pascal 2-1} = {Pascal 1} = [1]
Now shifted to the left: [1 0]
Shifted to the right: [0 1]
AddList just adds elementwise. So the result for {Pascal 2} is [1 1].
Now for for N == 3:
{Pascal 2} = [1 1]
Shifted left: [1 1 0]
Shifted right: [0 1 1]
Added: [1 2 1]
Of course the program works the other way around: It starts with some larger N. But at the beginning of the Pascal function the program recurses repeatedly until the parameter N has become 1. Something like this:
{Pascal 3}
{Pascal 2}
{Pascal 1}
[1]
[1 1]
[1 2 1]
Edit: There are actually to kinds of recursion in the program. The first one in Pascal starts with some integer N and recurses down to 1.
The other (in the helper methods) starts with a list consisting of a head and a tail and stops as soon as the list is empty, i.e. cannot be split anymore. (This is using so-called cons lists, an intrinsically recursive data type.)
wmeyer's explanation is very nice. I just want to add a possibly helpful 'visualization' -->
First of all, I'm using the original version of the book (PDF), I beleive, and the functions look like this -->
declare Pascal AddList ShiftLeft ShiftRight
fun {Pascal N}
if N==1 then [1]
else
{AddList {ShiftLeft {Pascal N-1}} {ShiftRight {Pascal N-1}}}
end
end
fun {ShiftLeft L}
case L of H|T then
H|{ShiftLeft T}
else [0] end
end
fun {ShiftRight L} 0|L end
fun {AddList L1 L2}
case L1 of H1|T1 then
case L2 of H2|T2 then
H1+H2|{AddList T1 T2}
end
else nil end
end
Imagine that you want to see row eight of Pascal's triangle. You are going to enter:
{Browse {Pascal 8}}
i.e. you want to display the result of feeding 8 to the function Pascal as defined in the book/here.
First the function tests to see if the value it was just passed is 1 (which will not be true until the LAST iteration of the recursion (or the final recursive call(s)) at which time that [1] (from if N==1) will be returned as the output of THAT CALL OF Pascal and passed back up the 'chain' of executions (of Pascal) to the next most recent call first (where that result, [1], is added to the result of the matching ShiftLeft or ShiftRight, and then THAT result is sent back up the chain, and on and on, until it reaches the first one (Pascal 8). So the calls go 8 levels deep, then pass the answers back up those levels until you get the final answer... but I've jumped ahead.
Ok, since you fed an 8, the test N==1 fails, and therefore instead of being able to shift 'the lists' and add them together in the else clause right away, the function not being able to do that with undefined terms in the 'equations' says "I'll try N - 1! Maybe THAT will be the final answer!!" (for ShiftLeft AND ShiftRight - so this branching occurs each time the recursino happens)
So, the function waits for that answer from Pascal N-1 inside ShiftLeft and ShiftRight... waiting, waiting...
Well, {Pascal 7} won't be true for N==1 either, so the newer calls ("calls", 2nd AND 3rd calls, left and right!) of Pascal will BOTH also ask "What is Pascal N - 1" (7-1 this time) and they will both wait for the answer...
This goes on and on and on and on.... oh wait, until N==1!
Then [1], a list, is returned BACK UP THE CHAIN... so, each successive waiting function call, most recent first (keep in mind all these happen more and more on the way down to get here to the 'bottom' where N==1 as the splits increase (by calling ShiftLeft and ShiftRight at one time each call)) can finally make it's AddList calculation with the answers it has been waiting on from it's own personal, private calls to ShiftLeft and ShiftRight.
Everything goes all the way down to the bottom, splitting into more and more function calls, then we come back to the top and finally can get an answer returned. That final answer is the else clause of the first call to the Pascal function, {Pascal 8}, which now, inside, (since the 8th row of Pascal's triangle is [1 7 21 35 35 21 7 1]) will look like:
{AddList [1 7 21 35 35 21 7 0] [0 7 21 35 35 21 7 1]} <-- at least I think that's what the final lists to be added look like
Which once added is the one list returned as the final answer and displayed: [1 7 21 35 35 21 7 1]

Resources