TL;DR
how to reduce below repeated code, like create two job / trigger from job-inventory, instead of repeat twice and create terms
;; deps in project.clj
;; [clojurewerkz/quartzite "2.1.0"]
(ns hello.scheduler
(:require [clojurewerkz.quartzite.scheduler :as qs]
[clojurewerkz.quartzite.triggers :as t]
[clojurewerkz.quartzite.jobs :as j]
[clojurewerkz.quartzite.jobs :refer [defjob]]
[clojurewerkz.quartzite.schedule.cron :as cron])
(:use clojure.tools.logging)
(:gen-class))
(def job-inventory
[{:name "add" :task '(+ 1 1) :cron "0/5 * * ? * *"}
{:name "multiply" :task '(* 4 5) :cron "0/3 * * ? * *"}])
(defjob add [ctx] (info "add called, return" (+ 1 1)))
(defjob multiply [ctx] (info "multiply called, return" (* 2 3)))
(defn auto
[]
(let [s (-> (qs/initialize) qs/start)
_ (qs/clear! s)
job (j/build
(j/of-type add)
(j/with-identity (j/key "job.add")))
trigger (t/build
(t/with-identity (t/key "trigger.add"))
(t/start-now)
(t/with-schedule (cron/schedule
(cron/cron-schedule "0/5 * * ? * *"))))
_ (qs/schedule s job trigger)
job (j/build
(j/of-type multiply)
(j/with-identity (j/key "job.multiply")))
trigger (t/build
(t/with-identity (t/key "trigger.multiply"))
(t/start-now)
(t/with-schedule (cron/schedule
(cron/cron-schedule "0/3 * * ? * *"))))
_ (qs/schedule s job trigger)
]
))
similar to what's described in http://clojurequartz.info/articles/getting_started.html ,
I have block of code to create jobs and hooks them for execution
quesetion is, when I get more and more of them, I wonder if I could have a better way of manage them, like create / spawn from that job-inventory, instead of actually creating varaibles like add or multiply
so, asking for one more layers of looping
are there ways to utilize function programming, and avoid create new names ( in traditional language says python qt, if I had a sets of button, I could just smash into a giant dictionary, and loop over to create / disable, instead actually create each name as a top level varible )
I tried macro but it says unable to resolve class add, so guess I used it wrong
The key thing to remember is that functions are data. Whilst you can't dynamically create types very easily (as opposed to instances that implement an interface, via reify), you can statically create a class which then proxies your functions.
First let's make the :task of the job-inventory a function.
(def job-inventory
[{:name "add" :task (fn [] (println (+ 1 1))) :cron "0/5 * * ? * *"}
{:name "multiply" :task (fn [] (println (* 4 5))) :cron "0/3 * * ? * *"}])
Then we need the proxy job class. This executes a function it finds within the job data.
;; require clojurewerkz.quartzite.conversion :as qc
(defjob proxy-job [ctx]
(let [ctx (qc/from-job-data ctx)]
((ctx "proxied-fn"))))
Then we create a schedule function which takes a map from the job-inventory and schedules it using the proxy-job to indirect.
(defn schedule [scheduler {:keys [:name :task :cron]}]
(let [job (j/build
(j/of-type proxy-job)
(j/using-job-data {"proxied-fn" task})
(j/with-identity (j/key (str "job." name))))
trigger (t/build
(t/with-identity (t/key (str "trigger." name)))
(t/start-now)
(t/with-schedule (cron/schedule
(cron/cron-schedule "0/3 * * ? * *"))))]
(qs/schedule scheduler job trigger)
scheduler)
(reduce scheduler schedule job-inventory)
It's possible this approach will fall apart if quartz decides to serialise/deserialise the job data. I leave fixing this with another layer of indirection as an easy exercise for the interested reader - to be honest my initial thoughts are that we create named functions which we then refer to by symbol, but it complicates the proxy and makes you wonder why not just use defjob. If you're prepared to defjob your functions, you can still refer to them in a job-inventory and have a data-based job builder function.
Related
A simple reflection warning example:
lein repl
user=> (set! *warn-on-reflection* true)
true
user=> (eval '(fn [x] (.length x)))
Reflection warning, NO_SOURCE_PATH:1:16 - reference to field length can't be resolved.
#object[user$eval2009$fn__2010 0x487ba4b8 "user$eval2009$fn__2010#487ba4b8"]
I want to make this into a function. But where do reflection warnings go?
//clojure/compile.java 63
RT.errPrintWriter()
.format("Reflection warning, %s:%d:%d - reference to field %s can't be resolved.\n",
SOURCE_PATH.deref(), line, column, fieldName);
//clojure/RT.java 269
public static PrintWriter errPrintWriter(){
Writer w = (Writer) ERR.deref();
//clojure/RT.java 188
final static public Var ERR =
Var.intern(CLOJURE_NS, Symbol.intern("*err*"),
new PrintWriter(new OutputStreamWriter(System.err), true)).setDynamic();
Ok so they go to System.err. Lets capture it's output:
(def pipe-in (PipedInputStream.))
(def pipe-out (PipedOutputStream. pipe-in))
(System/setErr (PrintStream. pipe-out))
(defn reflection-check [fn-code]
(binding [*warn-on-reflection* true]
(let [x (eval fn-code)
;_ (.println (System/err) "foo") ; This correctly makes us return "foo".
n (.available pipe-in)
^bytes b (make-array Byte/TYPE n)
_ (.read pipe-in b)
s (apply str (mapv char b))]
s)))
However, calling it gives no warning, and no flushing seems to be useful:
(println "Reflection check:" (reflection-check '(fn [x] (.length x)))) ; no warning.
How can I extract the reflection warning?
You have correctly discovered how *err* is initialized, but since vars are rebindable this is no guarantee about its current value. The REPL often rebinds it to something else, e.g. a socket. If you want to redirect it yourself, you should simply rebind *err* to a Writer of your choosing.
Really I'm not sure your approach would work even if *err* were never rebound. The Clojure runtime has captured a pointer to the original value of System.err, and then you ask the Java runtime to use a new value for System.err. Clojure certainly won't know about this new value. Does the JRE maintain an extra level of indirection to allow it to do these swaps behind the scenes even for people who have already captured System.err? Maybe, but if so it's not documented.
I ran into a similar problem a while back and created some helper functions modelled on with-out-str. Here is a solution to your problem:
(ns tst.demo.core
(:use tupelo.core tupelo.test) )
(defn reflection-check
[fn-code]
(let [err-str (with-err-str
(binding [*warn-on-reflection* true]
(eval fn-code)))]
(spyx err-str)))
(dotest
(reflection-check (quote (fn [x] (.length x)))))
with result:
-------------------------------
Clojure 1.10.1 Java 14
-------------------------------
err-str => "Reflection warning, /tmp/form-init3884945788481466752.clj:12:36
- reference to field length can't be resolved.\n"
Note that binding and let forms can be in either order and still work.
Here is the source code:
(defmacro with-err-str
"Evaluates exprs in a context in which *err* is bound to a fresh
StringWriter. Returns the string created by any nested printing
calls."
[& body]
`(let [s# (new java.io.StringWriter)]
(binding [*err* s#]
~#body
(str s#))))
If you need to capture the Java System.err stream, it is different:
(defmacro with-system-err-str
"Evaluates exprs in a context in which JVM System/err is bound to a fresh
PrintStream. Returns the string created by any nested printing calls."
[& body]
`(let [baos# (ByteArrayOutputStream.)
ps# (PrintStream. baos#)]
(System/setErr ps#)
~#body
(System/setErr System/err)
(.close ps#)
(.toString baos#)))
See the docs here.
There are 5 variants (plus clojure.core/with-out-str):
with-err-str
with-system-out-str
with-system-err-str
discarding-system-out
discarding-system-err
Source code is here.
I am trying to save data into a collection of some sort, but the program that I have is saving everything into a separate map. I want to make it one map.
(defn readFile []
(map (fn [line] (clojure.string/split line #";"))
(with-open [rdr (reader "C:/Users/Rohil/Desktop/textfile.txt")]
(doseq [[idx line] (map-indexed vector(line-seq rdr))]
(if(.contains line "201609")
(if(not(.contains line "TBA"))
(println(assoc table :code(nth(clojure.string/split line #";")3) :instructor(nth(clojure.string/split line #";")19)))
)
)
)
)
)
)
)
Any help will be appreciated.
Looks like you are adapting to clojure :-) I went to the same process. Hang on, it will be worth it!
First: it is important to realize that map will save the result of the function into a new collection. Like cfrick mentions, println returns nil and assoc does not change a map.
I'm guessing a bit here what you are trying to do: You want to have a collection of dicts, where every dict has two keys, like so:
[
{ :code 1 :instructor "blah"}
{ :code 2 :instructor "boo" }
]
You need these values to come from a file, but you only want to save the lines where the line contains "201609" but not "TBA"
First some general remarks:
You probably want to split this function into smaller parts. One could be the check for lines (contains 201609 but not tba ), another could read the file...
I know it is the title of your question, but most likely there is a better way than to change a global variable. Maybe you could make the function readFile return the table?
try if you can pass in arguments to your function.
I'm not sure what you are trying to do with the line (doseq [[... Please give us more context there. I will ignore it
Here is a possible solution:
(ns test
(:require [clojure.string :as s]
[clojure.java.io :as io]))
(defn line-filter [include exclude line]
(and (not (s/includes? line exclude))
(s/includes? line include)))
(defn process-line [line]
(let [line-parts (s/split line #";")
code (nth line-parts 3)
instructor (nth line-parts 19)]
{:code code :instructor instructor}))
(defn read-file [file-name]
(s/split (slurp (io/resource file-name)) #"\n"))
(defn parse-lines [lines]
(map process-line lines))
(defn read-file-and-parse
"This function will read a file, process the lines, and output a collection of maps"
[filename search-for exclude]
(parse-lines
(filter #(line-filter search-for exclude %)
(read-file filename))))
you could now call this function like this: (read-file-and-parse "test.txt" "201609" "TBA")
If you want to add the result of this function into your table, you can use concat. But again, this will return a new version of your list (with new entries added) and not change the one you defined earlier.
Welcome to functional programming :-)))
This question already has an answer here:
Wrong number of args (0) passed to: PersistentVector on loop/recur function
(1 answer)
Closed 5 years ago.
I've written a recursive function to get a total cost of a journey.
costOfPath simply makes a call to ubergraph to get the cost for each journey, then this function adds them and displays it.
(defn routeCost [parcel cost]
"Calculate the total route cost"
(if (empty? parcel)
(print "Total Journey Cost: " cost)
((def first-parcel (first parcel))
(def start (:start first-parcel))
(def finish (:finish first-parcel))
(def value (costOfPath start finish))
(def parcel-two (rest parcel))
(routeCost parcel-two (+ cost value)))))
(routeCost task8 0)
Task 8 looks as such:
(def task8 [(Parcel. :main-office :r131 "Plastic Wallets" "Delivery" 1)
(Parcel. :r131 :r111 "CDs" "Delivery" 1)
(Parcel. :r111 :r121 "USBs" "Collection" 2)
(Parcel. :r121 :main-office "USBs" "Delivery" 2)])
The function prints out the correct cost, but gives a classCastException.
ClassCastException practice_ubergraph.core.Parcel cannot be cast to clojure.lang.IFn clojure.lang.Var.fn (Var.java:363)
The parcel record:
(defrecord Parcel [start
finish
package
run-type
weight
])
Why is this happening and how can i stop it?
EDIT: I think its something to do with the IF statement and the way i've put the brackets around the block.
As Tony says, it's a good idea to try to limit your use of defs to the top level.
The reason you see a ClassCastException is probably this line:
((def first-parcel (first parcel))
You're defining first-parcel and then calling it immediately with the outer set of parentheseis.
Compare it to this example which generates a similar exception:
((def a 1))
In this example a gets the value 1. def returns the var #'user/a, so the expression that is evaluated is:
(#'user/a)
The value of #'user/a is 1, and 1 is then treated as a function.
Generally, if you see cannot be cast to clojure.lang.IFn look for a double set of brackets.
Please do NOT use def in a function.
Here is a much better one
(defn route-cost [parcel cost]
"Calculate the total route cost"
(if (empty? parcel)
(print "Total Journey Cost: " cost)
(let [{:keys [start finish]} (first parcel)
value (cost-of-path start finish)]
(route-cost (rest parcel) (+ cost value)))))
The essence of clojure is that you could write your code as concise as possible . usually we use kebab-case in clojure to differentiate Java
Use let in your function will fix everything
I've just started learning Clojure, after many years of Java (and PHP/JavaScript) experience. What a challenge :-)
How do I update a map of values idiomatically? When I use the map function on a map it doesn't return a map, it returns a sequence.
I'm working on a small app where I have a list of tasks. What I'd like to do is alter some of the values in some of the individual tasks, then update the list of original tasks. Here are the tasks I'm testing with:
(defrecord Task [key name duration])
(def tasks
(atom
{
"t1" (->Task "t1" "Task 1" 10)
"t2" (->Task "t2" "Task 2" 20)
"t3" (->Task "t3" "Task 3" 30)
}
))
I've put the tasks in a hashmap, using a string key so it has fast, direct access to any task in the map. Each task holds the key as well, so I know what it's key is when I'm passing individual tasks to other functions.
To update the durations I'm using map and update-in to iterate over and selectively update the duration of each task, and returning the modified tasks.
Here's the function:
(defn update-task-durations
"Update the duration of each task and return the updated tasks"
[tasks]
; 1) Why do I have to convert the result of the map function,
; from a sequence then back to a map?
(into {}
(map
(fn [task]
(println task) ; debug
(update-in
task
; 2) Why do I have to use vector index '1' here
; to get the value of the map entry?
[1 :duration]
(fn [duration]
(if (< duration 20)
(+ duration 1)
(+ duration 2)
)
)
)
) tasks))
)
I print the before/after values with this:
(println "ORIGINAL tasks:")
(println #tasks)
(swap! tasks update-task-durations)
(println "\nUPDATED tasks:")
(println #tasks)
1) The main problem I'm having is that the map function returns a sequence, and not a map, so I'm having to convert the sequence back to a map again using into {} which seems to me to be unnecessary and inefficient.
Is there a better way to do this? Should I be using a function other than map?
Could I arrange my data structures better, while still being efficient for direct access to individual tasks?
Is it ok to convert a (potentially very large) sequence to a map using into {} ?
2) Also, inside my function parameter, that I pass to the map function, each task is given to me, by map, as a vector of the form [key value] when I would expect a map entry, so to get the value from the map entry I have to pass the following keys to my update-in [1 :duration] This seems a bit ugly, is there a better/clearer way to access the map entry rather than using index 1 of the vector?
A popular way to solve this mapping-over-maps problem is with zipmap:
(defn map-vals
"Returns the map with f applied to each item."
[f m]
(zipmap (keys m)
(map f (vals m))))
(defn update-task-durations
[tasks]
(let [update-duration (fn [duration]
(if (< duration 20)
(+ 1 duration)
(+ 2 duration)))]
(->> tasks
(map-vals #(update % :duration update-duration)))))
(swap! tasks update-task-durations)
For Clojure < 1.7, use (update-in % [:duration] ... instead.
Alternatively, you could also use destructuring to simplify your current solution without defining a utility function:
(->> tasks
(map (fn [[k task]]
[k (update task :duration update-duration)]))
(into {})
Why?
map only deals with sequences. If you're into type signatures, this means that map always has the same type (map :: (a -> b) -> [a] -> [b]), but it also means that all you'll get out of map is a seq-of-something.
map calls seq on its collection parameter before doing anything, and seq-ing a map gives you a sequence of key-val pairs.
Don't worry too much about efficiency here. into is fast and this is pretty idiomatic.
Just get more alternatives:
Instead of a map you can use a for
(into {}
(for [[key value] your-map]
[key (do-stuff value)]))
A faster way is reduce-kv
(reduce-kv
(fn [new-map key value]
(assoc new-map key (do-stuff value)))
{}
your-map))
Of course you can also use a simple reduce
(reduce (fn [m key]
(update m key do-stuff))
your-map
(keys your-map))
This is my caller
(resolveEntity [r entity-id]
(println "resolve" entity-id)
(recursive-get r entity-id)
(cache entity-id)
)
Called function is
(defn recursive-get [r entity-id]
(println "recursive" entity-id)
(let [e (f (merge {} (-> r :conns first d/db (d/entity entity-id))))]
(alter-var-root #'cache assoc entity-id e)
(for [[k v] e]
(if (:db/isComponent (k components))
(if (not= (class v) Long)
(map #(recursive-get r %) v)
(recursive-get r v)
)))))
The called function is called just once. If I remove the last line in the caller (cache entity-id), then it recurs every time that I want it to, but I need to return something else (cache entity-id).
I tested a similar but simpler code (a recursive function not called at the tail of a calling function) the REPL and it worked, so I am left crashing my head against the table.
You have been bitten by a Lazy-Bug!
If you remove the last line then the return value of the function is the result of (recursive-get r entity-id) which the repl then iterates through so it can print it. The act of printing each value causes each entry in the lazy collection to be evaluated. When you put another line after that, the result of the map is ignored: nothing reads the entries, and they remain in the unrealized lazy state forever and the computation never happens.
To fix this wrap it in a call to dorun:
(dorun (recursive-get r entity-id))
Or if you need to save the result then use doall instead.