I feel like I'm missing something, but after perusing the docs for net/url and poking around in general, I was unable to figure out a way to issue a GET request from the interactive prompt. Basically, I want to imitate my python workflow for poking around a website:
response = urlopen("http://www.someurl.com")
is this feasible in Racket?
Using call/input-url has a few advantages:
You don't need to close the port yourself.
The port is closed even if there's an exception.
Its third argument is (input-port? -> any/c) -- that is, a function that takes an input-port and returns anything. In addition to a function you write yourself, this could be an already-defined function like port->string, read-html-as-xml, and so on.
For example:
(call/input-url (string->url "http://www.google.com/")
get-pure-port
port->string)
Note: As I was typing this answer, I notice that Óscar edited his to do redirects. My similar edit would be:
(call/input-url (string->url "http://www.google.com/")
(curry get-pure-port #:redirections 4)
port->string)
Of course, either way is still fairly verbose to type frequently at the REPL. So Óscar's suggestion to define your own url-open function is a good one. Implementing it using call/input-url would be preferable, I think.
Try this:
(require net/url)
(define input (get-pure-port (string->url "http://www.someurl.com")))
(define response (port->string input))
(close-input-port input)
Now the response variable will contain the http response from the server. Even better, pack the above in a procedure, also notice that I added a maximum number of redirections allowed:
(define (urlopen url)
(let* ((input (get-pure-port (string->url url) #:redirections 5))
(response (port->string input)))
(close-input-port input)
response))
(urlopen "http://www.someurl.com") ; this will return the response
EDIT:
Following #GregHendershott's excellent advice (see his answer for details), here's another, more robust way to implement the desired functionality:
(define (urlopen url)
(call/input-url
(string->url url)
(curry get-pure-port #:redirections 5)
port->string))
Related
By accident, I recently came across a latent coding error in one of my functions, dealing with a when statement. A reduced paraphrase might look like:
(defparameter a 0)
(when (or (= a 0)
(= a 1)
(* a a)))
The problem is a misplaced parenthesis, so it actually is
(when (or (= a 0)
(= a 1)
(* a a)))
In a situation like this, wouldn't it be useful for the compiler to generate either a style warning or note? It seems to me that the meaning of a when statement normally implies a condition and a body, even though the body is strictly optional. Of course, a print pretty would have caught this in the editor, but I had copied it from elsewhere. Is there a reason that SBCL does not check for these kinds of mistakes?
a print pretty would have caught this in the editor
To discuss the options, I know about:
trivial-formatter will format the source code.
(trivial-formatter:fmt :your-system :supersede)
cl-indentify indents the source code. Has a command line utility. I tried it once and it was not bad, but different than Emacs' indentation, thus annoying for me.
$ cl-indentify bar.lisp
It links to lispindent but I was less happy with its result.
However, the best would be to not only format the code and re-read it ourselves, but to
run checks against a set of rules to warn against code smells
This is what proposes the lisp-critic. It can critique a function or a file. However:
(edit) it doesn't really have a Slime integration, we have to either critique a function or a whole file.
if you feel adventurous, see an utility of mine here. It could be an easier way to test snippets that you enter at the REPL.
it hasn't the rule about when without a body (we can easily add it)
And it would be best that the run failed with an error status code if it found a code smell. Again, a little project of mine in beta tries to do that, see here. It doesn't have much rules now, but I just pushed a check for this. You can call the script:
$colisper.sh tests/playground.lisp
it shows an error (but doesn't write it in-place by default):
|;; when with no body
|(when (or (= a 0)
| (= a 1)
!| (* a a))
!| (error "colisper found a 'when' with a missing body. (we should error the script without this rewrite!)"))
and returns with an exit code, so we can use it has a git hook or on a CI pipeline.
The problem is that if a human writes (when x) (or whatever that expands into, perhaps (if x (progn) nil)) this is probably a mistake, but when a program writes it it may well not be: it may be just some edge case that the program hasn't been smart enough to optimize completely away. And a huge amount of code that the compiler processes is written by programs, not humans.
I understand that Scheme uses ports to perform Input and Output. While trying to learn how to get console input and output, I have come across MIT-Scheme's console-i/o-port variable.
But, the guile interpreter says it is an Unbound Variable. I would like to know how we can use ports to get input from and output to the console (Terminal in Unix) in a Guile Scheme Script. I am still a rookie in Scheme and Linux, a clear step-by-step is appreciated.
Also, how does (display <object>) work? Does it use ports inherently or is there another way.
P.S. If there is another way without using ports please let me know how to use that too.
If you want to read and write SExps, in guile you have (read), (write), (display) etc., if you want to read characters only use (read-char) and (write-char) -- they all use the input/output ports resp. you picked, by default they are stdin and stdout. Everything is rather straightforward (https://www.gnu.org/software/guile/manual/html_node/Input-and-Output.html#Input-and-Output).
You might also be interested in guile-ncurses (https://www.gnu.org/software/guile-ncurses/).
Of some more goodies check out pretty-print module from ice-9 (on very long sexps it's slow but outputs them really nicely formatted, great for e.g. code generation):
(use-modules (ice-9 pretty-print))
(pretty-print `(super cool stuff (+ 2 3) => ,(+ 2 3)))
And if you need your own parser, check out the lalr module (system base lalr).
edit a small example which reads a number, multiplies by itself and prints out the result:
#!/usr/bin/guile -s
!#
(let ((x (read)))
(display (* x x))
(newline))
(remember to chmod +x this script).
edit changed the expression to let form as Chris suggested, indeed the fewer parentheses the better
In guile you have 2 functions: current-input-port and current-output-port (the docs)
to read and put text into string (if you don't want to read s-expressions) you can use this function:
(define (read-port port)
(let iter ((result '()) (chr (read-char port)))
(if (eof-object? chr)
(list->string result)
(iter (append result (list chr)) (read-char port)))))
reading from stdin will be:
(read-port (current-input-port))
to write to stdout you can use display it also accept second argument which is port relevant docs
I'm trying to build a lazy seq which pulls its data from aws S3 as needed, (via the amazonica library). I've got the following code which almost does what I want, but makes one more network call than needed. (If there is more data available, it always realizes one more recursive call)
Edit: Thanks Alex, for pointing out that my println was in a place where it would be called even if the network call wasn't realized. This code performs as desired now. So that just leaves the question is there a better way to do it?
(defn chunked-list-objects-seq
"Returns a listing of objects in a bucket, with given prefix. These
are lazily chunked, to avoid unneeded network calls.
opts are :bucket-name :prefix :next-marker"
[cred opts]
(lazy-seq
(let [response (s3/list-objects cred opts)
chunk-size (count (:object-summaries response))]
(println "pulling from network")
(chunk-cons
(let [buffer (chunk-buffer chunk-size)]
(dotimes [i chunk-size]
(chunk-append buffer (nth (:object-summaries response) i)))
(chunk buffer))
(if (:truncated? response)
(chunked-list-objects-seq cred (assoc opts :next-marker (:next-marker response)))
nil)))))
Above code was adapted from "Clojure High Performance Programming" pg. 28 (custom chunking)
Calling it looks like this:
user> (time (pprint (count (take 990 (chunked-list-objects-seq cred {:bucket-name "bucket-name" :prefix "path-prefix/"})))))
=> pulling from network
990
"Elapsed time: 2009.723 msecs"
(AWS seems to like returning 1k chunks, when there are more than 1k items in a bucket)
There are certainly other ways to do this, (an atom & future implementation comes to mind), but this seems to fit the interface of a seq the best.
So basically, can this code be fixed to not make unnecessary network calls, and is this a good way to do this?
I think that making a lazy sequence with chunking that fetches blocks of data over the network is a perfectly reasonable approach - with the only caveat being that extra care is needed if the S3 client code happens to rely on having any dynamic bindings set.
Your initial code had the println outside the call to set up the lazy-seq to fetch the next block of data, so you were seeing the message printed regardless of whether the next block was actually fetched. Putting the println closer to the call to list-objects will give you a better idea of when the network request is made.
Basically what I am trying to do with this is alter the reader so that it will accept vector notation like i+j+k. The goal of this is to convert things that look like vectors into strings while not affecting anything else. For instance, if someone typed in abc it would still throw an error message, but if someone typed in i+j+k would return "i+j+k".
This is what I have so far:
#lang racket/base
(require syntax/strip-context)
(provide (except-out (all-from-out racket/base)
#%top)
(rename-out (#%my-top #%top)))
(require (for-syntax racket/base))
(define-syntax (#%my-top stx)
(syntax-case stx ()
[(_ . quat)
#' (test (#%datum . quat))]
[(_ . other)
(syntax/loc stx
(#%datum . other))]))
(define (quat? testExp)
(regexp-match-exact? #rx"([+-]?([0-9]+[i-k]?|[0-9]*[i-k])([+-]([0-9]+[i-k]?|[0-9]*[i-k]))*)" testExp))
(define (test x)
(cond ((quat? (symbol->string x)) (symbol->string x))
(else (strip-context x))))
This code does work in converting vector notation into strings. i+j+k successfully becomes "i+j+k". But the problem is that it converts EVERYTHING into syntax first, and I cannot get it back to whatever it originally was. For example, abc will become 'abc and it wont throw an error like it normally should. I have no idea how to get around this. I originally thought all I needed was a simple change to that else statement on the final line, but I cannot get anything to work. I am beginning to think I need to try a completely new approach. Any help would be greatly appreciated.
Also, it should be noted that you have to have this code in one file, I called mine parse.rkt, and perform your tests in a separate file that only has
#lang racket
(require "parse.rkt")
I've been using noir in a web project and I came up to the point of restricting access to users, depending on their access level (and sublevel) to all possible routes defined by a defpage macro. So originally I had
(defpage [:post "/mysite"] {:as input}
(if-not (has-reqd-user-level? :levelX :sublevelY "/grantedRoute")
(noir.response/redirect "/insufficientRights")
...))
And then I thought this would get rid of boilerplate code:
(defmacro defpage-with-user-level [level sublevel granted-route route data expr]
`(defpage ~route ~data
(if-not (has-reqd-user-level? ~level ~sublevel ~granted-route)
(noir.response/redirect "/insufficientRights")
~expr)))
Finally, we use it as follows:
(defpage-with-user-level :levelX :sublevelY "/grantedRoute"
[:post "/mysite"] {:as input}
(html
[:body [:h1 (str "Hello " (:name input) "!")]]))
But as mentioned in this post made by Rich Hickey, https://groups.google.com/forum/#!msg/clojure/4II-HKr_Pu0/2IcKit99cagJ , it feels a little bit awkward because of positional binding, which is not idiomatic when there already exist maps.
However, I've been looking for some examples or discussions regarding the use of destructuring bindings in macros, and sadly, I hadn't found any clear use of them, because of its unevaluated expressions being passed through all along.
So, the following solution came to my mind:
(defmacro defpage-with-user-level [dts expr]
`(defpage (:route ~dts) (:data ~dts)
(if-not (has-reqd-user-level? (:level ~dts) (:sublevel ~dts) (:granted-route ~dts))
(noir.response/redirect "/insufficientRights")
~expr)))
But now, it's not clear how to pass the data map that maps locals from :get and :post into a local as in the examples above.
Am I doing right leaving my first attempt untampered or do I really need to use the second approach? I hope not. Is there any other option? Please, let me know.
Your first solution is fine. What Rich was talking about was using plain old maps to passing data around rather then creating new types/classes for each type of data. Ex: you can represnt a user information using a simple map rather than creating a class to represent user data.
As far as your second attempt is concerned, you can use map de-structuring in macro as:
(defmacro defpage-with-user-level [{:keys [route data level sublevel granted-route]} expr]
`(defpage ~route ~data
(if-not (has-reqd-user-level? ~level ~sublevel ~granted-route)
(noir.response/redirect "/insufficientRights")
~expr)))
(defpage-with-user-level {:level :levelX
:sublevel :sublevelY
:granted-route "/grantedRoute"
:route [:post "/mysite"]
:data {:as input}}
(html
[:body [:h1 (str "Hello " (:name input) "!")]]))