Related
Background
While playing around with dialyzer, typespecs and currying, I was able to create an example of a false positive in dialyzer.
For the purposes of this MWE, I am using diallyxir (versions included) because it makes my life easier. The author of dialyxir confirmed this was not a problem on their side, so that possibility is excluded for now.
Environment
$ elixir -v
Erlang/OTP 24 [erts-12.2.1] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [jit]
Elixir 1.13.2 (compiled with Erlang/OTP 24)
Which version of Dialyxir are you using? (cat mix.lock | grep dialyxir):
"dialyxir": {:hex, :dialyxir, "1.1.0", "c5aab0d6e71e5522e77beff7ba9e08f8e02bad90dfbeffae60eaf0cb47e29488", [:mix], [{:erlex, ">= 0.2.6", [hex: :erlex, repo: "hexpm", optional: false]}], "hexpm", "07ea8e49c45f15264ebe6d5b93799d4dd56a44036cf42d0ad9c960bc266c0b9a"},
"erlex": {:hex, :erlex, "0.2.6", "c7987d15e899c7a2f34f5420d2a2ea0d659682c06ac607572df55a43753aa12e", [:mix], [], "hexpm", "2ed2e25711feb44d52b17d2780eabf998452f6efda104877a3881c2f8c0c0c75"},
Current behavior
Given the following code sample:
defmodule PracticingCurrying do
#spec greater_than(integer()) :: (integer() -> String.t())
def greater_than(min) do
fn number -> number > min end
end
end
Which clearly has a wrong typing, I get a success message:
$ mix dialyzer
Compiling 1 file (.ex)
Generated grokking_fp app
Finding suitable PLTs
Checking PLT...
[:compiler, :currying, :elixir, :gradient, :gradualizer, :kernel, :logger, :stdlib, :syntax_tools]
Looking up modules in dialyxir_erlang-24.2.1_elixir-1.13.2_deps-dev.plt
Finding applications for dialyxir_erlang-24.2.1_elixir-1.13.2_deps-dev.plt
Finding modules for dialyxir_erlang-24.2.1_elixir-1.13.2_deps-dev.plt
Checking 518 modules in dialyxir_erlang-24.2.1_elixir-1.13.2_deps-dev.plt
Adding 44 modules to dialyxir_erlang-24.2.1_elixir-1.13.2_deps-dev.plt
done in 0m24.18s
No :ignore_warnings opt specified in mix.exs and default does not exist.
Starting Dialyzer
[
check_plt: false,
init_plt: '/home/user/Workplace/fl4m3/grokking_fp/_build/dev/dialyxir_erlang-24.2.1_elixir-1.13.2_deps-dev.plt',
files: ['/home/user/Workplace/fl4m3/grokking_fp/_build/dev/lib/grokking_fp/ebin/Elixir.ImmutableValues.beam',
'/home/user/Workplace/fl4m3/grokking_fp/_build/dev/lib/grokking_fp/ebin/Elixir.PracticingCurrying.beam',
'/home/user/Workplace/fl4m3/grokking_fp/_build/dev/lib/grokking_fp/ebin/Elixir.TipCalculator.beam'],
warnings: [:unknown]
]
Total errors: 0, Skipped: 0, Unnecessary Skips: 0
done in 0m1.02s
done (passed successfully)
Expected behavior
I expected dialyzer to tell me the correct spec is #spec greater_than(integer()) :: (integer() -> bool()).
As a side note (and comparison, if you will) gradient does pick up the error.
I know that comparing these tools is like comparing oranges and apples, but I think it is still worth mentioning.
Questions
Is dialyzer not intended to catch this type of error?
If it should catch the error, what can possibly be failing? (is it my example that is incorrect, or something inside dialyzer?)
I personally find it hard to believe this could be a bug in Dialyzer, the tool has been used rather extensively by a lot of people for me to be the first to discover this error. However, I cannot explain what is happening.
Help is appreciated.
Dialyzer is pretty optimistic in its analysis and ignores some categories of errors.
This article provides some advanced explanations about its approach and limitations.
In the particular case of anonymous functions, dialyzer seems to perform a very minimal check
when they are being declared: it will ignore both the types of its arguments and return type, e.g.
the following doesn't lead any error even if is clearly wrong:
# no error
#spec add(integer()) :: (String.t() -> String.t())
def add(x) do
fn y -> x + y end
end
It will however point out a mismatch in arity, e.g.
# invalid_contract
# The #spec for the function does not match the success typing of the function.
#spec add2(integer()) :: (integer(), integer() -> integer())
def add2(x) do
fn y -> x + y end
end
Dialyzer might be able to detect a type conflict when trying to use the anonymous function,
but this isn't guaranteed (see article above), and the error message might not be helpful:
# Function main/0 has no local return.
def main do
positive? = greater_than(0)
positive?.(2)
end
We don't know what is the problem exactly, not even the line causing the error. But at least we know there is one and can debug it.
In the following example, the error is a bit more informative (using :lists.map/2 instead of Enum.map/2 because
dialyzer doesn't understand the enumerable protocol):
# Function main2/0 has no local return.
def main2 do
positive? = greater_than(0)
# The function call will not succeed.
# :lists.map(_positive? :: (integer() -> none()), [-2 | 0 | 1, ...])
# will never return since the success typing arguments are
# ((_ -> any()), [any()])
:lists.map(positive?, [1, 0, -2])
end
This tells us that dialyzer inferred the return type of greater_than/1 to be (integer() -> none()).
none is described in the article above as:
This is a special type that means that no term or type is valid.
Usually, when Dialyzer boils down the possible return values of a function to none(), it means the function should crash.
It is synonymous with "this stuff won't work."
So dialyzer knows that this function cannot be called successfully, but doesn't consider it to be a type clash until actually called, so it will allow the declaration (in the same way you can perfectly create a function that just raises).
Disclaimer: I couldn't find an official explanation regarding how dialyzer handles anonymous
functions in detail, so the explanations above are based on my observations and interpretation
In Isabelle/HOL, how do I find where a given type was instantiated for a given class? For the sake of this post for example, where real was instantiated as a conditionally_complete_linorder. To justify the question: I might want to know this for inspiration for a similar instantiation, for showing it to someone(s), for Isabelle/HOL practice reading, for curiosity, and so on. My process at the moment:
First, check it actually is: type instantiation real :: conditionally_complete_linorder begin end and see if I get the error message "No parameters and no pending instance proof obligations in instantiation."
Next, ideally before where I'd need to know how i.e. whether it was direct, or implicit via classes C_1[, C_2, C_3, etc]. Then, I would need to find where those instantiations are, either an explicit instantiation real :: conditionally_complete_linorder or the implicit ones for the C_i (same process for either case ofc). I don't know how to find out how, so I have to check for an explicit instantiation, then all possible implicit instantiations.
For explicit, I can do a grep -Ern ~/.local/src/Isabelle2019 -e 'instantiation real :: conditionally_complete_linorder' (and hope the whitespace isn't weird, or do a more robust search :)). Repeat for AFP location. Alternatively, to stay within the jEdit window:
I can find where the class itself was defined by typing term "x::'a::conditionally_complete_linorder" then Ctrl-clicking the class name, and then check if real is directly instantiated in that file with Ctrl-f.
I could then check if it's instantiated where the type real is defined by typing term "x::real" and Ctrl-clicking real, then Ctrl-f for conditionally_complete_linorder in that file.
(If it is in either place it'll be whichever is further down in the import hierarchy, but I find just going through those two steps simpler.) However, if neither two places turn it up then either, for whatever reason, it is explicitly instantiated somewhere else or is implicitly instantiated. For that reason grep is more robust.
If explicit turns nothing up then I check implicit. Looking at the class_deps graph I can see that conditionally_complete_linorder can follow from either complete_linorder or linear_continuum. I can then continue the search by seeing if real is instantiated as either of them (disregarding any I happen to know real can't be instantiated as). I can also check to see if it's instantiated as both conditioanlly_complete_lattice and linorder, which is what I can see conditionally_complete_linorder is a simple (no additional assumptions) combination of*. Repeat for all of these classes recursively until the instantiations are found. In this case, I can see that linear_continuum_topology implies linear_continuum, so kill two birds with one stone with grep -Ern ~/.local/src/Isabelle2019 -e "instantiation.*real" | grep continuum and find /path/to/.local/src/Isabelle2019/src/HOL/Real.thy:897:instantiation real :: linear_continuum.
This process is quite tedious. Less but still quite tedious** would be to get the class_deps graph up and Ctrl-f for "instantiation real" in Real.thy and look for instantiations of: the original class, the superclasses of it, or the classes which imply it. Then in the files each those classes are defined search for "instantiation real". Do this recursively till done. In this case I would have found what I needed in Real.thy.
Is there an easier way? Hope I just missed something obvious.
* I can't Ctrl-click in Conditionally_Complete_Lattices.thy to jump to linorder directly, I guess because of something to do with it being pre-built, so I have to do the term "x::'a::linorder" thing again.
** And also less robust, as it is minus grep-ing which can turn up weirder instantiation locations, then again I'm not sure if this ever comes up in practice.
Thanks
You can import the theory in the code listing below and then use the command find_instantiations. I will leave the code without further explanation, but please feel free to ask further questions in the comments if you need further details or suspect that something is not quite right.
section ‹Auxiliary commands›
theory aux_cmd
imports Complex_Main
keywords "find_instantiations" :: thy_decl
begin
subsection ‹Commands›
ML ‹
fun find_instantiations ctxt c =
let
val {classes, ...} = ctxt |> Proof_Context.tsig_of |> Type.rep_tsig;
val algebra = classes |> #2
val arities = algebra |> Sorts.arities_of;
in
Symtab.lookup arities c
|> the
|> map #1
|> Sorts.minimize_sort algebra
end
fun find_instantiations_cmd tc st =
let
val ctxt = Toplevel.context_of st;
val _ = tc
|> Syntax.parse_typ ctxt
|> dest_Type
|> fst
|> find_instantiations ctxt
|> map Pretty.str
|> Pretty.writeln_chunks
in () end
val q = Outer_Syntax.command
\<^command_keyword>‹find_instantiations›
"find all instantiations of a given type constructor"
(Parse.type_const >> (fn tc => Toplevel.keep (find_instantiations_cmd tc)));
›
subsection ‹Examples›
find_instantiations filter
find_instantiations nat
find_instantiations real
end
Remarks
I would be happy to provide amendments if you find any problems with it, but do expect a reasonable delay in further replies.
The command finds both explicit and implicit instantiations, i.e. it also finds the ones that were achieved by means other than the use of the commands instance or instantiation, e.g. inside an ML function.
Unfortunately, the command does not give you the location of the file where the instantiation was performed - this is something that would be more difficult to achieve, especially, given that instantiations can also be performed programmatically. Nevertheless, given a list of all instantiations, I believe, it is nearly always easy to use the in-built search functionality on the imported theories to narrow down the exact place where the instantiation was performed.
This is a derivative question of Existing constants (e.g. constructors) in type class instantiations.
The short question is this: How can I prevent the error that occurs due to free_constructors, so that I can combine the two theories that I include below.
I've been sitting on this for months. The other question helped me move forward (it appears). Thanks to the person who deserves thanks.
The real issue here is about overloading notation, though it looks like I now just have a namespace problem.
At this point, it's not a necessity, just an inconvenience that two theories have to be used. If the system allows, all this will disappear, but I ask anyway to make it possible to get a little extra information.
The big explanation here comes in explaining the motivation, which may lead to getting some extra information. I explain some, then include S1.thy, make a few comments, and then include S2.thy.
Motivation: using syntactic type classes for overloading notation of multiple binary datatypes
The basic idea is that I might have 5 different forms of binary words that have been defined with datatype, and I want to define some binary and hexadecimal notation that's overloaded for all 5 types.
I don't know what all is possible, but the past tells me (by others telling me things) that if I want code generation, then I should use type classes, to get the magic that comes with type classes.
The first theory, S1
Next is the theory S1.thy. What I do is instantiate bool for the type classes zero and one, and then use free_constructors to set up the notation 0 and 1 for use as the bool constructors True and False. It seems to work. This in itself is something I specifically wanted, but didn't know how to do.
I then try to do the same thing with an example datatype, BitA. It doesn't work because constant case_BitA is created when BitA is defined with datatype. It causes a conflict.
Further comments of mine are in the THY.
theory S1
imports Complex_Main
begin
declare[[show_sorts]]
(*---EXAMPLE, NAT 0: IT CAN BE USED AS A CONSTRUCTOR.--------------------*)
fun foo_nat :: "nat => nat" where
"foo_nat 0 = 0"
(*---SETTING UP BOOL TRUE & FALSE AS 0 AND 1.----------------------------*)
(*
I guess it works, because 'free_constructors' was used for 'bool' in
Product_Type.thy, instead of in this theory, like I try to do with 'BitA'.
*)
instantiation bool :: "{zero,one}"
begin
definition "zero_bool = False"
definition "one_bool = True"
instance ..
end
(*Non-constructor pattern error at this point.*)
fun foo1_bool :: "bool => bool" where
"foo1_bool 0 = False"
find_consts name: "case_bool"
free_constructors case_bool for "0::bool" | "1::bool"
by(auto simp add: zero_bool_def one_bool_def)
find_consts name: "case_bool"
(*found 2 constant(s):
Product_Type.bool.case_bool :: "'a∷type => 'a∷type => bool => 'a∷type"
S1.bool.case_bool :: "'a∷type => 'a∷type => bool => 'a∷type" *)
fun foo2_bool :: "bool => bool" where
"foo2_bool 0 = False"
|"foo2_bool 1 = True"
thm foo2_bool.simps
(*---TRYING TO WORK A DATATYPE LIKE I DID WITH BOOL.---------------------*)
(*
There will be 'S1.BitA.case_BitA', so I can't do it here.
*)
datatype BitA = A0 | A1
instantiation BitA :: "{zero,one}"
begin
definition "0 = A0"
definition "1 = A1"
instance ..
end
find_consts name: "case_BitA"
(*---ERROR NEXT: because there's already S1.BitA.case_BitA.---*)
free_constructors case_BitA for "0::BitA" | "1::BitA"
(*ERROR: Duplicate constant declaration "S1.BitA.case_BitA" vs.
"S1.BitA.case_BitA" *)
end
The second theory, S2
It seems that case_BitA is necessary for free_constructors to set things up, and it occurred to me that maybe I could get it to work by using datatype in one theory, and use free_constructors in another theory.
It seems to work. Is there a way I can combine these two theories?
theory S2
imports S1
begin
(*---HERE'S THE WORKAROUND. IT WORKS BECAUSE BitA IS IN S1.THY.----------*)
(*
I end up with 'S1.BitA.case_BitA' and 'S2.BitA.case_BitA'.
*)
declare[[show_sorts]]
find_consts name: "BitA"
free_constructors case_BitA for "0::BitA" | "1::BitA"
unfolding zero_BitA_def one_BitA_def
using BitA.exhaust
by(auto)
find_consts name: "BitA"
fun foo_BitA :: "BitA => BitA" where
"foo_BitA 0 = A0"
|"foo_BitA 1 = A1"
thm foo_BitA.simps
end
The command free_constructors always creates a new constant of the given name for the case expression and names the generated theorems in the same way as datatype does, because datatype internaly calls free_constructors.
Thus, you have to issue the command free_constructors in a context that changes the name space. For example, use a locale:
locale BitA_locale begin
free_constructors case_BitA for "0::BitA" | "1::BitA" ...
end
interpretation BitA!: BitA_locale .
After that, you can use both A0 and A1 as constructors in pattern matching equations and 0 and 1, but you should not mix them in a single equation. Yet, A0 and 0 are still different constants to Isabelle. This means that you may have to manually convert the one into the other during proofs and code generation works only for one of them. You would have to set up the code generator to replace A0 with 0 and A1 with 1 (or vice versa) in the code equations. To that end, you want to declare the equations A0 = 0 and A1 = 1 as [code_unfold], but you also probably want to write your own preprocessor function in ML that replaces A0 and A1 in left-hand sides of code equations, see the code generator tutorial for details.
Note that if BitA was a polymorphic datatype, packages such as BNF and lifting would continue to use the old set of constructors.
Given these problems, I would really go for the manual definition of the type as described in my answer to another question. This saves you a lot of potential issues later on. Also, if you are really only interested in notation, you might want to consider adhoc_overloading. It works perfectly well with code generation and is more flexible than type classes. However, you cannot talk about the overloaded notation abstractly, i.e., every occurrence of the overloaded constant must be disambiguated to a single use case. In terms of proving, this should not be a restriction, as you assume nothing about the overloaded constant. In terms of definitions over the abstract notation, you would have to repeat the overloading there as well (or abstract over the overloaded definitions in a locale and interpret the locale several times).
fact(1,1):-!.
fact(N,F):-
N1=N-1,
fact(N1,F1),
F=F1*N.
It leads to the stackoverflow(not the site)! It shouldn't because of the cut(!). Does it work in SWI-Prolog?
Please note that both definitions (the OP's and pad's) do not terminate for a query like fact(0,N). But also fact(1,2) does not terminate. It should fail. And for fact(N, F) it gives only one correct answer, but there should be infinitely many. Using cuts for such purpose is very tricky. The cleanest way to fix this is to add the goal N > 0 to the rule and have a fact fact(0,1). There is an even better way, provided you use SWI-Prolog: Use library(clpfd). In the documentation, you find n_factorial/2 already defined. It can be used for queries like:
?- n_factorial(47, F).
F = 258623241511168180642964355153611979969197632389120000000000
; false.
?- n_factorial(N, 1).
N = 0
; N = 1
; false.
?- n_factorial(N, 3).
false.
I'm working on a p2p app that uses hash trees.
I am writing the hash tree construction functions (publ/4 and publ_top/4) but I can't see how to fix publ_top/4.
I try to build a tree with publ/1:
nivd:publ("file.txt").
prints hashes...
** exception error: no match of right hand side value [67324168]
in function nivd:publ_top/4
in call from nivd:publ/1
The code in question is here:
http://github.com/AndreasBWagner/nivoa/blob/886c624c116c33cc821b15d371d1090d3658f961/nivd.erl
Where do you think the problem is?
Thank You,
Andreas
Looking at your code I can see one issue that would generate that particular exception error
publ_top(_,[],Accumulated,Level) ->
%% Go through the accumulated list of hashes from the prior level
publ_top(string:len(Accumulated),Accumulated,[],Level+1);
publ_top(FullLevelLen,RestofLevel,Accumulated,Level) ->
case FullLevelLen =:= 1 of
false -> [F,S|T]=RestofLevel,
io:format("~w---~w~n",[F,S]),
publ_top(FullLevelLen,T,lists:append(Accumulated,[erlang:phash2(string:concat([F],[S]))]),Level);
true -> done
end.
In the first function declaration you match against the empty list. In the second declaration you match against a list of length (at least) 2 ([F,S|T]). What happens when FullLevelLen is different from 1 and RestOfLevel is a list of length 1? (Hint: You'll get the above error).
The error would be easier to spot if you would pattern match on the function arguments, perhaps something like:
publ_top(_,[],Accumulated,Level) ->
%% Go through the accumulated list of hashes from the prior level
publ_top(string:len(Accumulated),Accumulated,[],Level+1);
publ_top(1, _, _, _) ->
done;
publ_top(_, [F,S|T], Accumulated, Level) ->
io:format("~w---~w~n",[F,S]),
publ_top(FullLevelLen,T,lists:append(Accumulated,[erlang:phash2(string:concat([F],[S]))]),Level);
%% Missing case:
% publ_top(_, [H], Accumulated, Level) ->
% ...