I'm trying to get a good understanding of how math is built up in Isabelle. For whatever reason, all the tutorial/manuals hide a lot of the implementation details of basic types such as how the natural numbers, integers, rationals, and reals are constructed. When looking the src/HOL directory and examining the .thy files, I've encountered code blocks such as:
keywords
"print_quotmapsQ3" "print_quotientsQ3" "print_quotconsts" :: diag and
"quotient_type" :: thy_goal_defn and "/" and
"quotient_definition" :: thy_goal_defn
begin
in Quotient.thy. Here, keywords is being used so that later you can define a type as:
quotient_type rat = "int * int" / partial: "ratrel"
and other related definitions. I haven't been able to figure out how the "keywords" feature works. It's not particularly obvious from the code, and the only documentation I can find is in the Isabelle/Isar Reference Manual where the following is written:
"The keywords specification declares outer syntax (chapter 3) that is introduced in this theory later on (rare in end-user applications). Both minor keywords and major keywords of the Isar command lan- guage need to be specified, in order to make parsing of proof docu- ments work properly. Command keywords need to be classified ac- cording to their structural role in the formal text. Examples may be seen in Isabelle/HOL sources itself, such as keywords "typedef" :: thy_goal_defn or keywords "datatype" :: thy_defn for theory-level definitions with and without proof, respectively." (p. 91)
This raises the question what a theory-level definition, which I haven't been able to figure out.
Isabelle's surface language, Isar, is extensible in multiple dimensions. In particular, a significant chunk of the keywords you'd usually use in day-to-day formalizations are defined in userspace. This sets Isar apart from many other programming languages, where syntax is fixed.
Roughly speaking, a theory file in Isabelle consists of two parts:
The header, which can be parsed statically, i.e. without running any custom code.
The contents, where e.g. logical definitions (types, constants, ...) and proofs can be mode.
Parsing of the contents happens in two phases:
First, the command structure is being parsed. This can be done by looking at the table of all the keywords that exist (those are declared in the header). There are various different types of keywords (as the manual points out). Command keywords start a new atomic chunk in the theory. (Consequently, theory contents can be seen as a sequence of commands.)
Second, the commands themselves are parsed, by using custom parsing code specified by whoever declared the corresponding keyword. This will execute any action, e.g. actually defining a type in the theory when the keyword typedef is encountered.
Commands can be classified according to the context in which they can appear. Top-level commands may only appear – well – on the top-level of a theory. Other commands may be freely nested in local contexts. Yet other commands do not modify the theory, but only print diagnostic output (diag). Theory processing in Isabelle takes that into account when e.g. parallelizing execution of a theory.
The example you mentioned, thy_goal_defn, is a keyword that adds some definitions to the theory and also enters proof mode, because quotient_type requires some proofs about the wellformedness of the definition.
Related
The Isabelle implementation manual says:
Types ctyp and cterm represent certified types and terms, respectively. These are abstract datatypes that guarantee that its values have passed the full well-formedness (and well-typedness) checks, relative to the declarations of type constructors, constants etc. in the background theory.
My understanding is: when I write cterm t, Isabelle checks that the term is well-built according to the theory where it lives in.
The abstract types ctyp and cterm are part of the same inference kernel that is mainly responsible for thm. Thus syntactic operations on ctyp and cterm are located in the Thm module, even though theorems
are not yet involved at that stage.
My understanding is: if I want to modify a cterm at the ML level I will use operations of the Thm module (where can I find that module?)
Furthermore, it looks like cterm t is an entity that converts a term of the theory level to a term of the ML-level. So I inspect the code of cterm in the declaration:
ML_val ‹
some_simproc #{context} #{cterm "some_term"}
›
and get to ml_antiquotations.ML:
ML_Antiquotation.value \<^binding>‹cterm› (Args.term >> (fn t =>
"Thm.cterm_of ML_context " ^ ML_Syntax.atomic (ML_Syntax.print_term t))) #>
This line of code is unreadable to me with my current knowledge.
I wonder if someone could give a better low-level explanation of cterm. What is the meaning of the code below? Where are located the checks that cterm performs on theory terms? Where are located the manipulations that we can do on cterms (the module Thm above)?
The ‘c’ stands for ‘certified’ (Or ‘checked’? Not sure). A cterm is basically a term that has been undergone checking. The #{cterm …} antiquotation allows you to simply write down terms and directly get a cterm in various contexts (in this case probably the context of ML, i.e. you directly get a cterm value with the intended content). The same works for regular terms, i.e. #{term …}.
You can manipulate cterms directly using the functions from the Thm structure (which, incidentally, can be found in ~~/src/Pure/thm.ML; most of these basic ML files are in the Pure directory). However, in my experience, it is usually easier to just convert the cterm to a regular term (using Thm.term_of – unlike Thm.cterm_of, this is a very cheap operation) and then work with the term instead. Directly manipulating cterms only really makes sense if you need another cterm in the end, because re-certifying terms is fairly expensive (still, unless your code is called very often, it probably isn't really a performance problem).
In most cases, I would say the workflow is like this: If you get a cterm as an input, you turn it into a regular term. This you can easily inspect/take apart/whatever. At some point, you might have to turn it into a cterm again (e.g. because you want to instantiate some theorem with it or use it in some other way that involves the kernel) and then you just use Thm.cterm_of to do that.
I don't know exactly what the #{cterm …} antiquotation does internally, but I would imagine that at the end of the day, it just parses its parameter as an Isabelle term and then certifies it with the current context (i.e. #{context}) using something like Thm.cterm_of.
To gather my findings with cterms, I post an answer.
This is how a cterm looks like in Pure:
abstype cterm =
Cterm of {cert: Context.certificate,
t: term, T: typ,
maxidx: int,
sorts: sort Ord_List.T}
(To be continued)
I was reading up on the documentation of macros and ran into the following under the `Hold up: why macros' section. The reasoning given to use macros is as follows:
Macros are necessary because they execute when code is parsed,
therefore, macros allow the programmer to generate and include
fragments of customized code before the full program is run
This leads me to wonder why someone would want to use "generate and include fragments of customized code before the full program is run". Can someone provide context as to why this would be beneficial and/or other good use cases for macros?
Let me give you my view on macros.
A macro basically is a code -> code function. It takes code (a Julia expression) as input and spits out code (a different Julia expression).
Why is this useful? It has multiple purposes:
compile time copy-and-paste: You don't have to write the same piece of code multiple times but instead can define a short macro that writes it for you wherever you put it. (example)
domain specific language (DSL): You can create special syntax that after the macros code -> code transform is replaced by pure Julia constructs. This is used in many packages to define special syntax, for example here and here.
code generation: Imagine you want to write a really long piece of code which, although being long, is very simple because it has some kind of pattern that repeats itself rather trivially. Writing that code by hand can be a pain (or even practically impossible). A macro can programmatically generate the code for you. One example is for-loop unrolling (see here and here). But even the #time macro isn't doing much more than just putting a bunch of Base.time_ns() function calls around the provided Julia expression.
special string parsing: If you type the literal 3.2 in Julia it will be parsed and interpreted as a Float64. Now, imagine you want to supply a number literally that goes beyond Float64 precision but would fit into a BigFloat. Typing big(3.123124812498124812498) won't work, because the literal number is first interpreted as a Float64 and then handed to the big function. Instead you need a way to tell Julia at parse time that this should become a BigFloat. This is handled by a #big_str 3.2 macro which (for convenience) can also be written as big"3.2". The latter is just syntactic sugar.
There might be many more applications of macros, but those are the most important to me.
Let me end by referencing Steven G. Johnson's great talk at JuliaCon 2019:
Most of the time, don't do metaprogramming :)
In various web pages, I see references to jq functions with a slash and a number following them. For example:
walk/1
I found the above notation used on a stackoverflow page.
I could not find in the jq Manual page a definition as to what this notation means. I'm guessing it might indicate that the walk function that takes 1 argument. If so, I wonder why a more meaningful notation isn't used such as is used with signatures in C++, Java, and other languages:
<function>(type1, type2, ..., typeN)
Can anyone confirm what the notation <function>/<number> means? Are other variants used?
The notation name/arity gives the name and arity of the function. "arity" is the number of arguments (i.e., parameters), so for example explode/0 means you'd just write explode without any arguments, and map/1 means you'd write something like map(f).
The fact that 0-arity functions are invoked by name, without any parentheses, makes the notation especially handy. The fact that a function name can have multiple definitions at any one time (each definition having a distinct arity) makes it easy to distinguish between them.
This notation is not used in jq programs, but it is used in the output of the (new) built-in filter, builtins/0.
By contrast, in some other programming languages, it (or some close variant, e.g. module:name/arity in Erlang) is also part of the language.
Why?
There are various difficulties which typically arise when attempting to graft a notation that's suitable for languages in which method-dispatch is based on types onto ones in which dispatch is based solely on arity.
The first, as already noted, has to do with 0-arity functions. This is especially problematic for jq as 0-arity functions are invoked in jq without parentheses.
The second is that, in general, jq functions do not require their arguments to be any one jq type. Having to write something like nth(string+number) rather than just nth/1 would be tedious at best.
This is why the manual strenuously avoids using "name(type)"-style notation. Thus we see, for example, startswith(str), rather than startswith(string). That is, the parameter names in the documentation are clearly just names, though of course they often give strong type hints.
If you're wondering why the 'name/arity' convention isn't documented in the manual, it's probably largely because the documentation was mostly written before jq supported multi-arity functions.
In summary -- any notational scheme can be made to work, but name/arity is (1) concise; (2) precise in the jq context; (3) easy-to-learn; and (4) widely in use for arity-oriented languages, at least on this planet.
I am trying to create a function
import Language.Reflection
foo : Type -> TT
I tried it by using the reflect tactic:
foo = proof
{
intro t
reflect t
}
but this reflects on the variable t itself:
*SOQuestion> foo
\t => P Bound (UN "t") (TType (UVar 41)) : Type -> TT
Reflection in Idris is a purely syntactic, compile-time only feature. To predict how it will work, you need to know about how Idris converts your program to its core language. Importantly, you won't be able to get ahold of reflected terms at runtime and reconstruct them like you would with Lisp. Here's how your program is compiled:
Internally, Idris creates a hole that will expect something of type Type -> TT.
It runs the proof script for foo in this state. We start with no assumptions and a goal of type Type -> TT. That is, there's a term being constructed which looks like ?rhs : Type => TT . rhs. The ?foo : ty => body syntax shows that there's a hole called foo whose eventual value will be available inside of body.
The step intro t creates a function whose argument is t : Type - this means that we now have a term like ?foo_body : TT . \t : Type => foo_body.
The reflect t step then fills the current hole by taking the term on its right-hand side and converting it to a TT. That term is in fact just a reference to the argument of the function, so you get the variable t. reflect, like all other proof script steps, only has access to the information that is available directly at compile time. Thus, the result of filling in foo_body with the reflection of the term t is P Bound (UN "t") (TType (UVar (-1))).
If you could do what you are wanting here, it would have major consequences both for understanding Idris code and for running it efficiently.
The loss in understanding would come from the inability to use parametricity to reason about the behavior of functions based on their types. All functions would effectively become potentially ad-hoc polymorphic, because they could (say) run differently on lists of strings than on lists of ints.
The loss in performance would come from representing enough type information to do the reflection. After Idris code is compiled, there is no type information left in it (unlike in a system such as the JVM or .NET or a dynamically typed system such as Python, where types have a runtime representation that code can access). In Idris, types can be very large, because they can contain arbitrary programs - this means that far more information would have to be maintained, and computation occurring at the type level would also have to be preserved and repeated at runtime.
If you're wanting to reflect on the structure of a type for further proof automation at compile time, take a look at the applyTactic tactic. Its argument should be a function that takes a reflected context and goal and gives back a new reflected tactic script. An example can be seen in the Data.Vect source.
So I suppose the summary is that Idris can't do what you want, and it probably never will be able to, but you might be able to make progress another way.
I'm building on some old Common Lisp code written by others, which includes lines such as the following at the start of a few functions:
(declare (ftype (function (&rest float) float) + - * min max))
My understanding is that the purpose of this is to tell the compiler that the five functions listed at the end of the form will only be passed floats. The compiler may use this information to create more efficient code.
Some Lisps do not complain about this declaration (ABCL, CCL, ECL, LispWorks, CLISP), but SBCL will not accept this declaration in the default configuration. SBCL can be made to accept it by placing
(unlock-package 'common-lisp)
in the .sbclrc initialization file. That's what I've been doing for the last year or so. I assume that this is needed because +, -, etc. are in that package, and the code alters these functions' declarations.
My question is: Can declaring the function type of built-in functions such as + and min have a beneficial effect on compiled code in SBCL? (If it can, then why does SBCL complain about these declarations by default?) Would I be better off removing such ftype declarations, and then getting rid of the unlock-package line in .sbclrc?
Thanks.
My understanding is that the purpose of this is to tell the compiler that the five functions listed at the end of the form will only be passed floats. The compiler may use this information to create more efficient code.
Also, they will only return floats. With certain optimization settings, the a Common Lisp compiler does not generate runtime checks and may only generate code for float computations. Also SBCL may show compile-time warnings in certain cases, where it detects that code violates type declarations.
It's also a source for errors, since from now on (within the scope of the declaration) basic functions like +and - are declared not to work on other number types (integer, complex, ...).
So, what is the purpose for these declarations? Since it is portable code (and most implementations don't implement compile-time type checking), it can only be for optimization purposes. Some of that might not be necessary in SBCL, since it uses type inference.
Why does SBCL not allow to alter the built-in functionality by default? It is so to prevent shooting in your foot: you are altering the base language. Now basic numeric operations may lead to errors.
Ways to deal with that:
use only local declarations, don't alter the language globally. You indicate that these are only locally declared - that's good.
declare the values of variables instead
write special functions for the float case and declare them inline.
only unlock the package CL during compilation of these few functions. keep it locked later.
My question is: Can declaring the function type of built-in functions such as + and min have a beneficial effect on compiled code in SBCL?
You can check that by looking at the disassembled code and also by profiling. Make sure that you compile the function with the right optimization setting. In Common Lisp the function DISASSEMBLE should show you machine code in a readable way. The SBCL compiler should also tell you if it can't optimize the compiled code.