Clojure: alternative to using a mutex/lock and a counter - networking

Scenario: I have a server listening to six active TCP/IP connections. When a "ready" message comes in, an event will be raised on its own thread. When the server has received a "ready" message from each connection, it needs to run the "start" function.
My object oriented solution would likely involve using a mutex and a counter. Something like:
int _countDown= 6;
object _lock;
void ReadyMessageReceivedForTheFirstTimeFromAConnection() {
lock(_lock) {
--_countDown; //
if (_countDown==0) Start();
}
}
How could this problem be solved in Clojure without resorting to locks/mutexes?

When you prefer a pure clojure version, you can use a promise to give your futures a go.
Every time you receive message you increment the conn-count
the watch checks if the treshold is reached and delivers :go to the barrier promise.
(def wait-barrier (promise))
(def conn-count (atom 0))
(add-watch conn-count :barrier-watch
(fn [key ref old-state new-state]
(when (== new-state 6)
(deliver wait-barrier :go))))
dummy-example:
(def wait-barrier (promise))
(def conn-count (atom 0))
(defn worker-dummy []
(when (= #wait-barrier :go)
(println "I'm a worker")))
(defn dummy-receive-msg []
(doall (repeatedly 6,
(fn []
(println "received msg")
(swap! conn-count inc)))))
(let [workers (doall (repeatedly 6 (fn [] (future (worker-dummy)))))]
(add-watch conn-count :barrier-watch
(fn [key ref old-state new-state]
(when (== new-state 6)
(deliver wait-barrier :go))))
(dummy-receive-msg)
(doall (map deref workers)))

You can use a CountDownLatch or a Phaser for this purpose.
In my futures library, imminent, I used both. CountDownLatch first and then replaced it with a Phaser for ForkJoin compatibility (might not be necessary in your case). You can see the change in this diff. Hopefully it gives you an idea of usage for both.
With latches the general idea would be:
(let [latch (CountDownLatch. 6)]
(on-receive-message this (fn [_] (.countDown latch)))
(.await latch)
...or something like that.

Since it hasn't been mentioned so far: you could easily do that with core.async. Have a look at this MCVE:
(let [conn-count 6
ready-chan (chan)]
;; Spawn a thread for each connection.
(doseq [conn-id (range conn-count)]
(thread
(Thread/sleep (rand-int 2000))
(>!! ready-chan conn-id)))
;; Block until all connections are established.
(doseq [total (range 1 (inc conn-count))]
(println (<!! ready-chan) "connected," total "overall"))
;; Invoke start afterwards.
(println "start"))
;; 5 connected, 1 overall
;; 3 connected, 2 overall
;; 4 connected, 3 overall
;; 0 connected, 4 overall
;; 1 connected, 5 overall
;; 2 connected, 6 overall
;; start
;;=> nil
You could also use a channel to implement a countdown latch (borrowed from Christophe Grand):
(defn count-down-latch-chan [n]
(chan 1 (comp (drop (dec n)) (take 1))))
For a short introduction into core.async, check out this Gist. For a longer one, read the corresponding chapter in "Clojure for the Brave and True".

Related

Core.async: map channel over async function

With Clojure's Core.Async one can map over a channel by using transducers:
(def my-chan (chan (buffer 10) (map inc)))
But what happens if the mapper function itself is async?
Say, we have a:
(defn async-inc [n]
(let [c (promise-chan)]
(put! (inc n))
c))
Is there a similar concise way to map the channel over this function? Or would one have to do something like this:
(def my-chan (chan (buffer 10)))
(def my-chan2 (chan (buffer 10)))
(go (while true
(>! my-chan2
(<! (async-inc (<! my-chan))))))
It would not really be mapping, since two channels are needed instead of one.
There is a general advice not to create channel inside a function and return it out because it forces user of that function to use core.async. You can either return an output through a promise or return through callback function.
Assuming what you want to do with an output from async-inc is to print it out using println function.
Return through a promise
(defn async-inc [n]
(let [p (promise)]
(deliver p (inc n))
p))
(println #(async-inc (<! my-chan)))
Return through callback
(defn async-inc [n callback]
(callback (inc n)))
(async-inc (<! my-chan) println)
But if you don't have a control over async-inc. Here are your options.
Use <!!
(println (<!! (go (<! (async-inc (<! my-chan))))))
or
Use take!
(take! (go (<! (async-inc (<! my-chan)))) println)

Add delay in clojure.core.async

When using clojure.core.async, is there a way to have one channel wait for the first item to be put on it, then wait some small amount of time, then get all the items currently on the channel (that could have arrived while waiting) and get all of them without blocking?
I.e. is there a way to implement get-available-items:
(defn delayer [ch ch2]
(go (loop []
(when-let [v (<! ch)]
(<! (timeout 500))
(let [vs (get-available-items ch)
items (cons v vs)]
(>! ch2 items))
(recur)))))
Basically, something like BlockingQueue.drain in Java.
There are plans to offer this feature with channels, though for now you can check for the presence of something on a channel with:
(alts!! [my-chan] :default :nothing-immediately-in-chan)
by iterating that you can drain a channel without blocking.
PS: extra thanks to tbaldridge and julianlevis on #clojure for helping with this one
You can just alt on the same timeout channel until you run out of "waiting time", collecting any incoming values meanwhile.
These seems to work:
(require '[clojure.core.async :as a :refer [<! >! go chan]])
(defn delayer [in out]
(a/go-loop []
(when-let [v (<! in)]
(loop [batch [v] timeout-ch (a/timeout 500)]
(let [[v ch] (a/alts! [in timeout-ch])]
(if (= in ch)
(recur (conj batch v) timeout-ch)
(>! out batch))))
(recur))))
Notice that we create the timeout channel just once and we reuse it. A simple test to prove that it works:
(def out (chan))
(def in (chan))
(delayer in out)
; print batches as soon as available
(a/go-loop []
(>pprint (str (java.util.Date.) (<! out)))
(recur))
; put a value every 100 millis
(a/go-loop [i 100]
(when-not (zero? i)
(<! (a/timeout 100))
(>! in i)
(recur (dec i))))

Simple Calculator (Addition function input) in Clojure

I'm trying to write a simple calculator with addition, subtraction, etc.
My problem is with getting the user input. How do I turn the string of numerical values into a vector? And also what is a better way to write the program?
(ns scalc.core)
(defn add
[numbers]
(println (apply + numbers)))
(defn numchoose
[]
(println "What numbers?: ")
(let [numbers (read-line)] numbers))
(defn opchoose
[]
(println "What operation would you like to do?: ")
(let [operation (read-line)]
(if (= operation "add")
(do
(println "You chose to add.")
(let [numvect (numchoose)]
(add [numvect]))))))
(defn -main
[& args]
(opchoose)
(numchoose))
And this is the error:
~/clj/scalc 1/7 % lein trampoline run -m scalc.core
What operation would you like to do?:
add
You chose to add.
What numbers?:
5 7
Exception in thread "main" java.lang.ClassCastException: Cannot cast java.lang.String to java.lang.Number
at java.lang.Class.cast(Class.java:3005)
at clojure.core$cast.invoke(core.clj:318)
at clojure.core$_PLUS_.invoke(core.clj:927)
at clojure.lang.AFn.applyToHelper(AFn.java:161)
at clojure.lang.RestFn.applyTo(RestFn.java:132)
at clojure.core$apply.invoke(core.clj:601)
at scalc.core$add.invoke(core.clj:5)
at scalc.core$opchoose.invoke(core.clj:21)
at scalc.core$_main.doInvoke(core.clj:27)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at clojure.lang.Var.invoke(Var.java:411)
at user$eval15.invoke(NO_SOURCE_FILE:1)
at clojure.lang.Compiler.eval(Compiler.java:6511)
at clojure.lang.Compiler.eval(Compiler.java:6501)
at clojure.lang.Compiler.eval(Compiler.java:6477)
at clojure.core$eval.invoke(core.clj:2797)
at clojure.main$eval_opt.invoke(main.clj:297)
at clojure.main$initialize.invoke(main.clj:316)
at clojure.main$null_opt.invoke(main.clj:349)
at clojure.main$main.doInvoke(main.clj:427)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at clojure.lang.Var.invoke(Var.java:419)
at clojure.lang.AFn.applyToHelper(AFn.java:163)
at clojure.lang.Var.applyTo(Var.java:532)
at clojure.main.main(main.java:37)
EDIT: the solved program now looks like this:
(ns scalc.core)
(defn add [numbers]
(reduce + numbers))
(defn numchoose []
(let [nums (re-seq #"\d+" (read-line))]
(map #(Integer/parseInt %) nums)))
(defn delegate []
(println "What operation would you like to do?: ")
(let [operation (read-line)]
(when (= operation "add")
(println "You chose to add.")
(println "What numbers? ")
(add (numchoose)))))
(defn -main
[& args]
(delegate))
For getting the numbers, you can use re-seq:
(re-seq #"\d+" "123 456 789") => ("123" "456" 789")
You still only have strings rather than numbers though. You can use read-string to get the numbers (read-string is convenient, but not safe in all cases. Here we make sure there are really only numbers in these strings so it's fine).
(read-string "5") => 5
Instead of (apply + numbers) you could use reduce: (reduce + numbers), also your add function really shouldn't print anything (you should try to separate functional functions from side-effecty functions whenever possible).
This (let [numbers (read-line)] numbers) is equal to (read-line). Don't overcomplicate things!
Instead of
(if (= operation "add")
(do ... ))
you can write
(when (= operation "add")
...)
when is just a macro that's useful when you don't need the else case in your ifs (it wraps everything after the condition in a do, and evaluates to nil when the condition evaluates to false).

How might you implement design-by-contract in Clojure specifically or functional languages in general?

I'd prefer examples to be in a Lisp variant (bonus points for Clojure or Scheme) since that's what I'm most familiar with, but any feedback regarding DBC in functional lanugages would of course be valuable to the greater community.
Here's an obvious way:
(defn foo [action options]
(when-not (#{"go-forward" "go-backward" "turn-right" "turn-left"} action)
(throw (IllegalArgumentException.
"unknown action")))
(when-not (and (:speed options) (> (:speed options) 0))
(throw (IllegalArgumentException.
"invalid speed")))
; finally we get to the meat of the logic)
What I don't like about this implementation is that the contract logic obscures the core functionality; the true purpose of the function is lost in conditional checks. This is the same issue I raised in this question. In an imperative language like Java, I can use annotations or metadata/attributes embedded in documentation to move the contract out of the method implementation.
Has anyone looked at adding contracts to metadata in Clojure? How would higher-order functions be used? What other options are there?
Clojure already has support for pre and post conditions, unfortunately not well documented:
Should I use a function or a macro to validate arguments in Clojure?
I could imagine something like this in Clojure:
(defmacro defnc
[& fntail]
`(let [logic# (fn ~#(next fntail))]
(defn ~(first fntail)
[& args#]
(let [metadata# (meta (var ~(first fntail)))]
(doseq [condition# (:preconditions metadata#)]
(apply condition# args#))
(let [result# (apply logic# args#)]
(doseq [condition# (:postconditions metadata#)]
(apply condition# result# args#))
result#)))))
(defmacro add-pre-condition!
[f condition]
`(do
(alter-meta! (var ~f) update-in [:preconditions] conj ~condition)
nil))
(defmacro add-post-condition!
[f condition]
`(do
(alter-meta! (var ~f) update-in [:postconditions] conj ~condition)
nil))
An example session:
user=> (defnc t [a test] (a test))
\#'user/t
user=> (t println "A Test")
A Test
nil
user=> (t 5 "A Test")
java.lang.ClassCastException: java.lang.Integer (NO_SOURCE_FILE:0)
user=> (add-pre-condition! t (fn [a _] (when-not (ifn? a) (throw (Exception. "Aaargh. Not IFn!")))))
nil
user=> (t 5 "A Test")
java.lang.Exception: Aaargh. Not IFn! (NO_SOURCE_FILE:0)
user=> (t println "A Test")
A Test
nil
So you can define the function and then afterwards define pre- and post-conditions whereever you like, without cluttering the function logic itself.
condition functions should throw an exception if something is wrong.

How can I use the Clojure REPL together with Qt Jambi?

I have not found a solution to use the Clojure REPL with Qt on the web.
Basically the problem is that the REPL hangs as soon as you call QApplication/exec in order to get the UI to display. You cannot C-c C-c back into the REPL, and closing the active Qt window seems to kill the whole Clojure process.
Now simply calling QApplication/processEvents from within an agent is not possible, unless the agent runs in exactly the same thread in which you created your Qt widgets. It took me two days to figure this out and I have seen others have the same issue/problem but without a solution.
So here is mine, in code:
(add-classpath "file:///usr/share/java/qtjambi.jar")
(ns qt4-demo
(:import (com.trolltech.qt.gui QApplication QPushButton QFont QFont$Weight)
(com.trolltech.qt.core QCoreApplication)
(java.util Timer TimerTask)
(java.util.concurrent ScheduledThreadPoolExecutor TimeUnit))
(:require swank.core))
(defn init []
(QApplication/initialize (make-array String 0)))
(def *gui-thread* (new java.util.concurrent.ScheduledThreadPoolExecutor 1))
(def *gui-update-task* nil)
(def *app* (ref nil))
(defn update-gui []
(println "Updating GUI")
(QApplication/processEvents))
(defn exec []
(.remove *gui-thread* update-gui)
(def *gui-update-task* (.scheduleAtFixedRate *gui-thread* update-gui 0 150 (. TimeUnit MILLISECONDS))))
(defn stop []
(.remove *gui-thread* update-gui)
(.cancel *gui-update-task*))
(defmacro qt4 [& rest]
`(do
(try (init) (catch RuntimeException e# (println e#)))
~#rest
))
(defmacro with-gui-thread [& body]
`(.get (.schedule *gui-thread* (fn [] (do ~#body)) (long 0) (. TimeUnit MILLISECONDS))))
(defn hello-world []
(with-gui-thread
(qt4
(let [app (QCoreApplication/instance)
button (new QPushButton "Go Clojure Go")]
(dosync (ref-set *app* app))
(doto button
(.resize 250 100)
(.setFont (new QFont "Deja Vu Sans" 18 (.. QFont$Weight Bold value)))
(.setWindowTitle "Go Clojure Go")
(.show)))))
(exec))
Basically it uses the ScheduledThreadPoolExecutor class in order to execute all Qt-code. You can use the with-gui-thread macro to make it easier to call functions from within the thread.
This makes it possible to change the Qt UI on-the-fly, without recompiling.
If you want to mess with Qt widgets from the REPL, QApplication/invokeLater or QApplication/invokeAndWait are probably what you want. You can use them in conjunction with agents. Given this:
(ns qt4-demo
(:import (com.trolltech.qt.gui QApplication QPushButton)
(com.trolltech.qt.core QCoreApplication)))
(def *app* (ref nil))
(def *button* (ref nil))
(def *runner* (agent nil))
(defn init [] (QApplication/initialize (make-array String 0)))
(defn exec [] (QApplication/exec))
(defn hello-world [a]
(init)
(let [app (QCoreApplication/instance)
button (doto (QPushButton. "Go Clojure Go") (.show))]
(dosync (ref-set *app* app)
(ref-set *button* button)))
(exec))
Then from a REPL:
qt4-demo=> (send-off *runner* hello-world)
#<Agent#38fff7: nil>
;; This fails because we are not in the Qt main thread
qt4-demo=> (.setText #*button* "foo")
QObject used from outside its own thread, object=QPushButton(0x8d0f55f0) , objectThread=Thread[pool-2-thread-1,5,main], currentThread=Thread[main,5,main] (NO_SOURCE_FILE:0)
;; This should work though
qt4-demo=> (QApplication/invokeLater #(.setText #*button* "foo"))
nil
qt4-demo=> (QApplication/invokeAndWait #(.setText #*button* "bar"))
nil
I've written about how to do this with SLIME on my blog (German) as well as on the Clojure mailing-list. The trick is to define appropriate functions on the Emacs side and tell SLIME to use those when making requests. Importantly, this frees you from having to do special incantations when invoking Qt code.
Quoting myself:
Given that we're talking Lisp here,
anyway, the solution seemed to be
obvious: Hack SLIME! So that's what I
did. The code below, when dropped
into your .emacs (at a point at which
SLIME is already fully loaded),
registers three new Emacs-Lisp
functions for interactive use. You
can bind them to whatever keys you
like, or you may even just set the
slime-send-through-qapplication
variable to t after your application
has started and not worry about key
bindings at all. Either should make
your REPL submissions and C-M-x-style
interactive evaluations indirect
through QCoreApplication/invokeAndWait.
Have fun!
(defvar slime-send-through-qapplication nil)
(defvar slime-repl-send-string-fn (symbol-function 'slime-repl-send-
string))
(defvar slime-interactive-eval-fn (symbol-function 'slime-interactive-
eval))
(defun qt-appify-form (form)
(concatenate 'string ;'
"(let [return-ref (ref nil)] "
" (com.trolltech.qt.core.QCoreApplication/invokeAndWait "
" (fn [] "
" (let [return-value (do "
form
" )] "
" (dosync (ref-set return-ref return-value))))) "
" (deref return-ref))"))
(defun slime-interactive-eval (string)
(let ((string (if slime-send-through-qapplication
(qt-appify-form string)
string)))
(funcall slime-interactive-eval-fn string)))
(defun slime-repl-send-string (string &optional command-string)
(let ((string (if slime-send-through-qapplication
(qt-appify-form string)
string)))
(funcall slime-repl-send-string-fn string command-string)))
(defun slime-eval-defun-for-qt ()
(interactive)
(let ((slime-send-through-qapplication t))
(slime-eval-defun)))
(defun slime-repl-closing-return-for-qt ()
(interactive)
(let ((slime-send-through-qapplication t))
(slime-repl-closing-return)))
(defun slime-repl-return-for-qt (&optional end-of-input)
(interactive)
(let ((slime-send-through-qapplication t))
(slime-repl-return end-of-input)))

Resources