Is there an AES encryption library for clojure? should I use a java libray available through maven or clojars? Thank your for your time and consideration.
Here is a perhaps more idiomatic example using the available java crypto libraries. encrypt and decrypt here each simply take input text and the encryption key, both as Strings.
(import (javax.crypto Cipher KeyGenerator SecretKey)
(javax.crypto.spec SecretKeySpec)
(java.security SecureRandom)
(org.apache.commons.codec.binary Base64))
(defn bytes [s]
(.getBytes s "UTF-8"))
(defn base64 [b]
(Base64/encodeBase64String b))
(defn debase64 [s]
(Base64/decodeBase64 (bytes s)))
(defn get-raw-key [seed]
(let [keygen (KeyGenerator/getInstance "AES")
sr (SecureRandom/getInstance "SHA1PRNG")]
(.setSeed sr (bytes seed))
(.init keygen 128 sr)
(.. keygen generateKey getEncoded)))
(defn get-cipher [mode seed]
(let [key-spec (SecretKeySpec. (get-raw-key seed) "AES")
cipher (Cipher/getInstance "AES")]
(.init cipher mode key-spec)
cipher))
(defn encrypt [text key]
(let [bytes (bytes text)
cipher (get-cipher Cipher/ENCRYPT_MODE key)]
(base64 (.doFinal cipher bytes))))
(defn decrypt [text key]
(let [cipher (get-cipher Cipher/DECRYPT_MODE key)]
(String. (.doFinal cipher (debase64 text)))))
Used suchwise:
(def key "secret key")
(def encrypted (encrypt "My Secret" key)) ;; => "YsuYVJK+Q6E36WjNBeZZdg=="
(decrypt encrypted key) ;; => "My Secret"
Java's AES implementation is well-tested and included in the JDK…any Clojure library would likely use that impl itself.
See Java 256-bit AES Password-Based Encryption for a decent discussion of the Java API. Also, http://jyliao.blogspot.com/2010/08/exploring-java-aes-encryption-algorithm.html has an example of using the API from Clojure (though the code there isn't entirely idiomatic).
Tinklj is a great library that is wrapping Clojure around the Google Tink Java API. Google Tink supports all forms of encryption/decryption using streaming or deterministic. AES is supported AeadKeyTemplates.AES128_GCM.
It also offers MAC and Digital Signatures well worth checking out and getting involved.
For example:
(:require [tinklj.keys.keyset-handle :as keyset-handles])
(keyset-handles/generate-new :aes128-gcm)
(:require [tinklj.encryption.aead :refer [encrypt decrypt])
(encrypt aead (.getBytes data-to-encrypt) aad)
(decrypt aead encrypted aad)
The encryption library Clojure-AES is written entirely in Clojure. The only dependency is on Clojure 1.10 (as of Nov 2021), and it does not utilize any Java-based libraries. Feel free to try it out. It was coded directly from the original NIST specification.
Related
I am new to common lisp. Is there a CL library to pretty print collections, in my case, nested hash tables?
If you considering writing it yourself, here is a starting point using print-object. It is not implementation independent, but this works at least in LispWorks and SBCL.
(defmethod print-object ((object hash-table) stream)
(format stream "#HASH{~{~{(~a : ~a)~}~^ ~}}"
(loop for key being the hash-keys of object
using (hash-value value)
collect (list key value))))
First, CL does not have a "collection" type.
Second, some (most?) CL implementations will print hash tables with content if you set *print-array* to t.
Third, if your CL implementation does not do that, you can easily whip up your own, based on, say, hash-table->alist.
With the Serapeum library, we can use the dict constructor and enable pretty-printing with (toggle-pretty-print-hash-table):
(dict :a 1 :b 2 :c 3)
;; =>
(dict
:A 1
:B 2
:C 3
)
With the Rutils library:
if we enable pretty printing of hash-tables with (toggle-print-hash-table), they are printed like so:
rutils-user> #h(:foo 42)
#{
:FOO 42
}
It uses print-object under the hood, so its warning applies (not standard, but works in some implementations like SBCL).
The #h reader macro is a shortcut to create hash-tables.
I would like to make a single CGI program in Common Lisp, that reads a cookie.
It is clear to me that I can send a Cookie by sending HTTP headers. Yet, I would like to know how to read a cookie on a server, through CGI program, from the client who is accessing the program.
Based on Kaz answer, we need to get an environment variable. But I won't refer to the cl-cookbook[1], now we have this solution that comes with asdf:
(uiop:getenv)
Also the Osicat library (os, files and directories) has (environment-variable name) for posix systems, incl. windows.
[1] the cl-cookbook on sourceforge is old and sometimes outdated. There is a copy on github, maintained (well, a bit) and editable: https://lispcookbook.github.io/cl-cookbook/os.html (https://github.com/LispCookbook/cl-cookbook/).
edit: the cookbook on github has been edited accordingly and should be deployed soon on https://lispcookbook.github.io/cl-cookbook/os.html
The CGI mechanism passes the cookie using the HTTP_COOKIE environment variable. ANSI Common Lisp doesn't have any API for accessing POSIX and Windows system environment variables. Implementation-specific functions, or else an implementation's FFI, is used to achieve the equivalent of C's getenv.
The Common Lisp Cookbook suggests this:
* (defun my-getenv (name &optional default)
#+CMU
(let ((x (assoc name ext:*environment-list*
:test #'string=)))
(if x (cdr x) default))
#-CMU
(or
#+Allegro (sys:getenv name)
#+CLISP (ext:getenv name)
#+ECL (si:getenv name)
#+SBCL (sb-unix::posix-getenv name)
#+LISPWORKS (lispworks:environment-variable name)
default))
MY-GETENV
* (my-getenv "HOME")
"/home/edi"
* (my-getenv "HOM")
NIL
* (my-getenv "HOM" "huh?")
"huh?"
Before using, I'd slightly modify this to:
(or #+CMU (let ((x ...)) (if ...))
#+Allegro ...
#+CLISP
...
default)
Exactly as the question says. I want to use shared memory to communicate between two lisp processes. Any pointers on how to do that?
I can see some tutorials on doing this in clozure at :-
http://ccl.clozure.com/manual/chapter4.7.html
Can someone point me to a similar library to do this with sbcl?
For a portable implementation, you might want to use the osicat library, which provides a CFFI wrapper for many POSIX calls in the osicat-posix package.
There is a very nice and short article with code for using it at http://wandrian.net/2012-04-07-1352-mmap-files-in-lisp.html (by Nicolas Martyanoff).
To preserve that, I mostly cite from there:
Mapping a file is done by opening it with osicat-posix:open, reading its size with fstat, then calling mmap. Once the file has been mapped we can close the file descriptor, it’s not needed anymore.
(defun mmap-file (path)
(let ((fd (osicat-posix:open path (logior osicat-posix:o-rdonly))))
(unwind-protect
(let* ((size (osicat-posix:stat-size (osicat-posix:fstat fd)))
(addr (osicat-posix:mmap (cffi:null-pointer) size
(logior osicat-posix:prot-read)
(logior osicat-posix:map-private)
fd 0)))
(values addr size))
(osicat-posix:close fd))))
The mmap-file function returns two values: the address of the memory mapping and its size.
Unmapping this chunk of memory is done with osicat-posix:munmap.
Let’s add a macro to safely map and unmap files:
(defmacro with-mmapped-file ((file addr size) &body body)
(let ((original-addr (gensym "ADDR-"))
(original-size (gensym "SIZE-")))
`(multiple-value-bind (,addr ,size)
(mmap-file ,file)
(let ((,original-addr ,addr)
(,original-size ,size))
(unwind-protect
(progn ,#body)
(osicat-posix:munmap ,original-addr ,original-size))))))
This macro mmaps the given file and binds the two given variables to its address and and size. You can then calculate address pointers with cffi:inc-pointer and access the file contents with cffi:mem-aref. You might want to build your own wrappers around this to represent the format of your file (e. g. plain text in UTF-8).
(In comparison to the posting linked above, I removed the wrapping of osicat-posix:munmap into another function of exactly the same signature and effect, because it seemed superfluous to me.)
There is low-level mmap function bundled with sbcl:
CL-USER> (apropos "MMAP")
SB-POSIX:MMAP (fbound)
; No value
CL-USER> (describe 'sb-posix:mmap)
SB-POSIX:MMAP
[symbol]
MMAP names a compiled function:
Lambda-list: (ADDR LENGTH PROT FLAGS FD OFFSET)
Derived type: (FUNCTION (T T T T T T)
(VALUES SYSTEM-AREA-POINTER &OPTIONAL))
Inline proclamation: INLINE (inline expansion available)
Source file: SYS:CONTRIB;SB-POSIX;INTERFACE.LISP.NEWEST
; No value
You have to use explicit address arithmetics to use it, as in C.
I'm trying to use the Racket Crypto library to encrypt blocks of 16 bytes with a 16-byte key. I'm expecting to have a 16-bytes output block but I get a 32-byte one.
A 15-byte input block give a 16-bit output.
#lang racket
(require (planet vyzo/crypto))
(bytes-length (encrypt cipher:aes-128-ecb
(string->bytes/latin-1 "0123456789ABCDEF") ; 16-byte key
(make-bytes 16) ; IV
(string->bytes/latin-1 "0123456789ABCDEF"))) ; 16-byte data
; -> 32
(bytes-length (encrypt cipher:aes-128-ecb
(string->bytes/latin-1 "0123456789ABCDEF") ; 16-byte key
(make-bytes 16)
(string->bytes/latin-1 "0123456789ABCDE"))) ; 15-byte data
; -> 16
Am I wrong somewhere? Is this due to padding?
Note: I'm aware of the problems with ECB mode, My goal is to implement CBC mode.
You're right, it's because of the padding. Unfortunately the API of the vyzo/crypto lib doesn't allow you to easily disable padding (and rightly so, see Caveat below).
How to disable the padding
However, based on this Thread on the Racket users mailing list, you could disable the padding like this:
#lang racket
(require (planet vyzo/crypto) (planet vyzo/crypto/util))
(define (cipher-encrypt-unpadded type key iv)
(lambda (ptext)
(let ((octx (cipher-encrypt type key iv #:padding #f)))
(bytes-append (cipher-update! octx ptext)
(cipher-final! octx)))))
(define (cipher-decrypt-unpadded type key iv)
(lambda (ctext)
(let ((ictx (cipher-decrypt type key iv #:padding #f)))
(bytes-append (cipher-update! ictx ctext)
(cipher-final! ictx)))))
; bytes-> bytes
; convenience function for encryption
(define enc-aes-128-ecb-unpadded
(cipher-encrypt-unpadded cipher:aes-128-ecb
(string->bytes/latin-1 "0123456789ABCDEF"); 16-byte key
(make-bytes 16)))
; bytes -> bytes
; convenience function for decryption
(define dec-aes-128-ecb-unpadded
(cipher-decrypt-unpadded cipher:aes-128-ecb
(string->bytes/latin-1 "0123456789ABCDEF"); 16-byte key
(make-bytes 16)))
(define message (string->bytes/latin-1 "0123456789ABCDEF")) ; 16-byte data
(bytes-length (enc-aes-128-ecb-unpadded message))
; -> 16
(dec-aes-128-ecb-unpadded (enc-aes-128-ecb-unpadded message))
; -> #"0123456789ABCDEF"
This worked well on my machine. Also, switching to CBC mode is trivial.
Caveat
When you disable padding, your messages have to have a length that is a multiple of the block size. For AES128 that is an exact multiple of 16 Bytes. Otherwise the function will blow up in your face:
(enc-aes-128-ecb-unpadded (string->bytes/latin-1 "too short!"))
EVP_CipherFinal_ex: libcrypto error: data not multiple of block length [digital envelope routines:EVP_EncryptFinal_ex:101183626]
It looks like all input is being padded to the next block boundary. That means that a 16 byte input will be padded to the next boundary at 32 bytes. If all your input is going to be exact block sizes, then you could turn off padding. If the input can end in the middle of a block then you will have to leave padding switched on.
If you are going to be using CBC mode, then you might need to think about authentication as well. If you do need it, then HMAC is probably the easiest to get started with.
I'm attempting to learn a little more about handling sockets and network connections in SBCL; so I wrote a simple wrapper for HTTP. Thus far, it merely makes a stream and performs a request to ultimately get the header data and page content of a website.
Until now, it has worked at somewhat decently. Nothing to brag home about, but it at least worked.
I have come across a strange problem, however; I keep getting "400 Bad Request" errors.
At first, I was somewhat leery about how I was processing the HTTP requests (more or less passing a request string as a function argument), then I made a function that formats a query string with all the parts I need and returns it for use later... but I still get errors.
What's even more odd is that the errors don't happen every time. If I try the script on a page like Google, I get a "200 Ok" return value... but at other times on other sites, I'll get "400 Bad Request".
I'm certain its a problem with my code, but I'll be damned if I know exactly what is causing it.
Here is the code that I am working with:
(use-package :sb-bsd-sockets)
(defun read-buf-nonblock (buffer stream)
(let ((eof (gensym)))
(do ((i 0 (1+ i))
(c (read-char stream nil eof)
(read-char-no-hang stream nil eof)))
((or (>= i (length buffer)) (not c) (eq c eof)) i)
(setf (elt buffer i) c))))
(defun http-connect (host &optional (port 80))
"Create I/O stream to given host on a specified port"
(let ((socket (make-instance 'inet-socket
:type :stream
:protocol :tcp)))
(socket-connect
socket (car (host-ent-addresses (get-host-by-name host))) port)
(let ((stream (socket-make-stream socket
:input t
:output t
:buffering :none)))
stream)))
(defun http-request (stream request &optional (buffer 1024))
"Perform HTTP request on a specified stream"
(format stream "~a~%~%" request )
(let ((data (make-string buffer)))
(setf data (subseq data 0
(read-buf-nonblock data
stream)))
(princ data)
(> (length data) 0)))
(defun request (host request)
"formated HTTP request"
(format nil "~a HTTP/1.0 Host: ~a" request host))
(defun get-page (host &optional (request "GET /"))
"simple demo to get content of a page"
(let ((stream (http-connect host)))
(http-request stream (request host request)))
A few things. First, to your concern about the 400 errors you are getting back, a few possibilities come to mind:
"Host:" isn't actually a valid header field in HTTP/1.0, and depending on how fascist the web server you are contacting is about standards, it would reject this as a bad request based on the protocol you claim to be speaking.
You need a CRLF between your Request-line and each of the header lines.
It is possible that your (request) function is returning something for the Request-URI field -- you substitute in the value of request as the contents of this part of the Request-line -- that is bogus in one way or another (badly escaped characters, etc.). Seeing what it is outputting might help out some.
Some other more general pointer to help you along your way:
(read-buf-nonblock) is very confusing. Where is the symbol 'c' defined? Why is 'eof' (gensym)ed and then not assigned any value? It looks very much like a byte-by-byte copy taken straight out of an imperative program, and plopped into Lisp. It looks like what you have reimplemented here is (read-sequence). Go look here in the Common Lisp Hyperspec, and see if this is what you need. The other half of this is to set your socket you created to be non-blocking. This is pretty easy, even though the SBCL documentation is almost silent on the topic. Use this:
(socket-make-stream socket
:input t
:output t
:buffering :none
:timeout 0)
The last (let) form of (http-connect) isn't necessary. Just evaluate
(socket-make-stream socket
:input t
:output t
:buffering :none)
without the let, and http-connect should still return the right value.
In (http-request)...
Replace:
(format stream "~a~%~%" request )
(let ((data (make-string buffer)))
(setf data (subseq data 0
(read-buf-nonblock data
stream)))
(princ data)
(> (length data) 0)))
with
(format stream "~a~%~%" request )
(let ((data (read-buf-nonblock stream)))
(princ data)
(> (length data) 0)))
and make (read-buf-nonblock) return the string of data, rather that having it assign within the function. So where you have buffer being assigned, create a variable buffer within and then return it. What you are doing is called relying on "side-effects," and tends to produce more errors and harder to find errors. Use it only when you have to, especially in a language that makes it easy not to depend on them.
I mostly like the the way get-page is defined. It feels very much in the functional programming paradigm. However, you should either change the name of the (request) function, or the variable request. Having both in there is confusing.
Yikes, hands hurt. But hopefully this helps. Done typing. :-)
Here's a possibility:
HTTP/1.0 defines the sequence CR LF as the end-of-line marker.
The ~% format directive is generating a #\Newline (LF on most platforms, though see CLHS).
Some sites may be tolerant of the missing CR, others not so much.