Iām trying to write a simple code that continuously checks when files in a given folder are modified and, upon modification, triggers a function.
I've added a MWE below, it "works" but I was wondering if
there are libraries or utilities that I should look into that help for such a problem?
there was a cleaner way to have something running continuously until some event (here an interruption)
files_and_times = Dict{String, Int}()
for (root, _, files) ā walkdir(dirpath)
for f ā files
fpath = joinpath(root, f)
files_and_times[fpath] = stat(fpath).mtime
end
end
try
while true
for (f, t) ā files_and_times
cur_t = stat(f).mtime
if cur_t > t
files_and_times[f] = cur_t
println("file $f was modified")
end
end
sleep(0.5)
end
catch x
if isa(x, InterruptException)
println("Shutting down.")
else
throw(x)
end
end
The comments by #rickhg12hs and #Colin_T_Bowers effectively answer my question. Posting an answer to close the question.
FileWatching seems to be precisely what I was looking for,
The package Revise seems to have good examples showing how to use it and,
this discourse thread discusses limitations.
Related
Background
While playing around with dialyzer, typespecs and currying, I was able to create an example of a false positive in dialyzer.
For the purposes of this MWE, I am using diallyxir (versions included) because it makes my life easier. The author of dialyxir confirmed this was not a problem on their side, so that possibility is excluded for now.
Environment
$ elixir -v
Erlang/OTP 24 [erts-12.2.1] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [jit]
Elixir 1.13.2 (compiled with Erlang/OTP 24)
Which version of Dialyxir are you using? (cat mix.lock | grep dialyxir):
"dialyxir": {:hex, :dialyxir, "1.1.0", "c5aab0d6e71e5522e77beff7ba9e08f8e02bad90dfbeffae60eaf0cb47e29488", [:mix], [{:erlex, ">= 0.2.6", [hex: :erlex, repo: "hexpm", optional: false]}], "hexpm", "07ea8e49c45f15264ebe6d5b93799d4dd56a44036cf42d0ad9c960bc266c0b9a"},
"erlex": {:hex, :erlex, "0.2.6", "c7987d15e899c7a2f34f5420d2a2ea0d659682c06ac607572df55a43753aa12e", [:mix], [], "hexpm", "2ed2e25711feb44d52b17d2780eabf998452f6efda104877a3881c2f8c0c0c75"},
Current behavior
Given the following code sample:
defmodule PracticingCurrying do
#spec greater_than(integer()) :: (integer() -> String.t())
def greater_than(min) do
fn number -> number > min end
end
end
Which clearly has a wrong typing, I get a success message:
$ mix dialyzer
Compiling 1 file (.ex)
Generated grokking_fp app
Finding suitable PLTs
Checking PLT...
[:compiler, :currying, :elixir, :gradient, :gradualizer, :kernel, :logger, :stdlib, :syntax_tools]
Looking up modules in dialyxir_erlang-24.2.1_elixir-1.13.2_deps-dev.plt
Finding applications for dialyxir_erlang-24.2.1_elixir-1.13.2_deps-dev.plt
Finding modules for dialyxir_erlang-24.2.1_elixir-1.13.2_deps-dev.plt
Checking 518 modules in dialyxir_erlang-24.2.1_elixir-1.13.2_deps-dev.plt
Adding 44 modules to dialyxir_erlang-24.2.1_elixir-1.13.2_deps-dev.plt
done in 0m24.18s
No :ignore_warnings opt specified in mix.exs and default does not exist.
Starting Dialyzer
[
check_plt: false,
init_plt: '/home/user/Workplace/fl4m3/grokking_fp/_build/dev/dialyxir_erlang-24.2.1_elixir-1.13.2_deps-dev.plt',
files: ['/home/user/Workplace/fl4m3/grokking_fp/_build/dev/lib/grokking_fp/ebin/Elixir.ImmutableValues.beam',
'/home/user/Workplace/fl4m3/grokking_fp/_build/dev/lib/grokking_fp/ebin/Elixir.PracticingCurrying.beam',
'/home/user/Workplace/fl4m3/grokking_fp/_build/dev/lib/grokking_fp/ebin/Elixir.TipCalculator.beam'],
warnings: [:unknown]
]
Total errors: 0, Skipped: 0, Unnecessary Skips: 0
done in 0m1.02s
done (passed successfully)
Expected behavior
I expected dialyzer to tell me the correct spec is #spec greater_than(integer()) :: (integer() -> bool()).
As a side note (and comparison, if you will) gradient does pick up the error.
I know that comparing these tools is like comparing oranges and apples, but I think it is still worth mentioning.
Questions
Is dialyzer not intended to catch this type of error?
If it should catch the error, what can possibly be failing? (is it my example that is incorrect, or something inside dialyzer?)
I personally find it hard to believe this could be a bug in Dialyzer, the tool has been used rather extensively by a lot of people for me to be the first to discover this error. However, I cannot explain what is happening.
Help is appreciated.
Dialyzer is pretty optimistic in its analysis and ignores some categories of errors.
This article provides some advanced explanations about its approach and limitations.
In the particular case of anonymous functions, dialyzer seems to perform a very minimal check
when they are being declared: it will ignore both the types of its arguments and return type, e.g.
the following doesn't lead any error even if is clearly wrong:
# no error
#spec add(integer()) :: (String.t() -> String.t())
def add(x) do
fn y -> x + y end
end
It will however point out a mismatch in arity, e.g.
# invalid_contract
# The #spec for the function does not match the success typing of the function.
#spec add2(integer()) :: (integer(), integer() -> integer())
def add2(x) do
fn y -> x + y end
end
Dialyzer might be able to detect a type conflict when trying to use the anonymous function,
but this isn't guaranteed (see article above), and the error message might not be helpful:
# Function main/0 has no local return.
def main do
positive? = greater_than(0)
positive?.(2)
end
We don't know what is the problem exactly, not even the line causing the error. But at least we know there is one and can debug it.
In the following example, the error is a bit more informative (using :lists.map/2 instead of Enum.map/2 because
dialyzer doesn't understand the enumerable protocol):
# Function main2/0 has no local return.
def main2 do
positive? = greater_than(0)
# The function call will not succeed.
# :lists.map(_positive? :: (integer() -> none()), [-2 | 0 | 1, ...])
# will never return since the success typing arguments are
# ((_ -> any()), [any()])
:lists.map(positive?, [1, 0, -2])
end
This tells us that dialyzer inferred the return type of greater_than/1 to be (integer() -> none()).
none is described in the article above as:
This is a special type that means that no term or type is valid.
Usually, when Dialyzer boils down the possible return values of a function to none(), it means the function should crash.
It is synonymous with "this stuff won't work."
So dialyzer knows that this function cannot be called successfully, but doesn't consider it to be a type clash until actually called, so it will allow the declaration (in the same way you can perfectly create a function that just raises).
Disclaimer: I couldn't find an official explanation regarding how dialyzer handles anonymous
functions in detail, so the explanations above are based on my observations and interpretation
EDIT: The original question had unnecessary details
I have a source file which I do value analysis in Frama-C, some of the code is highlighted as dead code in the normalized window, no the original source code.
Can I obtain a slice of the original code that removes the dead code?
Short answer: there's nothing in the current Frama-C version that will let you do that directly. Moreover, if your original code contains macros, Frama-C will not even see the real original code, as it relies on an external preprocessor (e.g. cpp) to do macro expansion.
Longer answer: Each statement in the normalized (aka CIL) Abstract Syntax Tree (AST, the internal representation of C code within Frama-C) contains information about the location (start point and end point) of the original statement where it stems from, and this information is also available in the original AST (aka Cabs). It might thus be possible for someone with a good knowledge of Frama-C's inner workings (e.g. a reader of the developer's manual), to build a correspondance between both, and to use that to detect dead statement in Cabs. Going even further, one could bypass Cabs, and identify zones in the original text of the program which are dead code. Note however that it would be a tedious and quite error prone (notably because a single original statement can be expanded in several normalized ones) task.
Given your clarifications, I stand by #Virgile's answer; but for people interested in performing some simplistic dead code elimination within Frama-C, the script below, gifted by a colleague who has no SO account, could be helpful.
(* remove_dead_code.ml *)
let main () =
!Db.Value.compute ();
Slicing.Api.Project.reset_slicing ();
let selection = ref Slicing.Api.Select.empty_selects in
let o = object (self)
inherit Visitor.frama_c_inplace
method !vstmt_aux stmt =
if Db.Value.is_reachable_stmt stmt then
selection :=
Slicing.Api.Select.select_stmt ~spare:true
!selection
stmt
(Extlib.the self#current_kf);
Cil.DoChildren
end in
Visitor.visitFramacFileSameGlobals o (Ast.get ());
Slicing.Api.Request.add_persistent_selection !selection;
Slicing.Api.Request.apply_all_internal ();
Slicing.Api.Slice.remove_uncalled ();
ignore (Slicing.Api.Project.extract "no-dead")
let () = Db.Main.extend main
Usage:
frama-c -load-script remove_dead_code.ml file.c -then-last -print -ocode output.c
Note that this script does not work in all cases and could have further improvements (e.g. to handle initializers), but for some quick-and-dirty hacking, it can still be helpful.
This snippet of code is from JuliaBoxTutorials
myfriends = ["Ted", "Robyn", "Barney", "Lily", "Marshall"]
i = 1;
while i <= length(myfriends)
friend = myfriends[i]
println("Hi $friend, it's great to see you!")
i += 1
end
giving this error when run with Julia v1.0
UndefVarError: i not defined
Stacktrace:
[1] top-level scope at ./In[12]:5 [inlined]
[2] top-level scope at ./none:0
But when i += 1 is replaced with global i += 1 it works. I guess this was still working in v0.6 and the tutorial will be adapted once the new Intro to Julia is published this Friday.
I was just wondering, is it possible, to make a while loop without stating a global variable?
As #Michael Paul and #crstnbr already replied in the comments, the scoping rules have been changed (Scope of variables in Julia). The for and while loop introduce a new scope and have no access to the outside (global) variables. You can get scope access using the global keyword but the recommended workflow is wrapping your code in functions.
One of the benefits of the new design is that the user is forced to avoid such global constructs which directly affect the performance of functions - which cannot be type stable when they access global variables.
One downside is the confusion when experimenting in the REPL and seeing such errors.
In my opinion the new behaviour is the cleaner one with respect to predictability. It was however a very tough and long-running discussion within the whole Julia community ;)
There is currently a discussion if the REPL will be changed to behave like the old one by making use of let-wraps: https://github.com/JuliaLang/julia/issues/28789
This is something which is not practical to be done manually (much more complicated then using the global keyword), see the example by Stefan Karpinski: https://github.com/JuliaLang/julia/issues/28789#issuecomment-414666648
Anyways, for the sake of completeness (although I would not recommend doing this) here is a version using global:
myfriends = ["Ted", "Robyn", "Barney", "Lily", "Marshall"]
i = 1;
N = length(myfriends)
while i <= N # you cannot even call a function here
# with a global, like length(myfriends)
global i, myfriends
friend = myfriends[i]
println("Hi $friend, it's great to see you!")
i += 1
end
Note however that this is also completely valid:
myfriends = ["Ted", "Robyn", "Barney", "Lily", "Marshall"]
greet(friend) = println("Hi $friend, it's great to see you!")
for friend in myfriends
greet(friend)
end
I found that this works in Julia v1.0:
let
myfriends = ["Ted", "Robyn", "Barney", "Lily", "Marshall"]
i = 1;
while i <= length(myfriends) # this function here seems to work no problem
friend = myfriends[i]
println("Hi $friend, it's great to see you!")
i = i + 1 # gives syntax error when prefixed with global
end
end
In fact, it will give me a syntax error if I try to make i global :) within the while loop.
What is the sane way to go from a Module object to a path to the file in which it was declared?
To be precise, I am looking for the file where the keyword module occurs.
The indirect method is to find the location of the automatically defined eval method in each module.
moduleloc(mm::Module) = first(functionloc(mm.eval, (Symbol,)))
for example
moduleloc(mm::Module) = first(functionloc(mm.eval, (Symbol,)))
using DataStructures
moduleloc(DataStructures)
Outputs:
/home/oxinabox/.julia/v0.6/DataStructures/src/DataStructures.jl
This indirect method works, but it feels like a bit of a kludge.
Have I missed some inbuilt function to do this?
I will remind answered that Modules are not the same thing as packages.
Consider the existence of submodules, or even modules that are being loaded via includeing some abolute path that is outside the package directory or loadpath.
Modules simply do not store the file location where they were defined. You can see that for yourself in their definition in C. Your only hope is to look through the bindings they hold.
Methods, on the other hand, do store their file location. And eval is the one function that is defined in every single module (although not baremodules). Slightly more correct might be:
moduleloc(mm::Module) = first(functionloc(mm.eval, (Any,)))
as that more precisely mirrors the auto-defined eval method.
If you aren't looking for a programmatic way of doing it you can use the methods function.
using DataFrames
locations = methods(DataFrames.readtable).ms
It's for all methods but it's hardly difficult to find the right one unless you have an enormous number of methods that differ only in small ways.
There is now pathof:
using DataStructures
pathof(DataStructures)
"/home/ederag/.julia/packages/DataStructures/59MD0/src/DataStructures.jl"
See also: pkgdir.
pkgdir(DataStructures)
"/home/ederag/.julia/packages/DataStructures/59MD0"
Tested with julia-1.7.3
require obviously needs to perform that operation. Looking into loading.jl, I found that finding the module path has changed a bit recently: in v0.6.0, there is a function
load_hook(prefix::String, name::String, ::Void)
which you can call "manually":
julia> Base.load_hook(Pkg.dir(), "DataFrames", nothing)
"/home/philipp/.julia/v0.6/DataFrames/src/DataFrames.jl"
However, this has changed to the better in the current master; there's now a function find_package, which we can copy:
macro return_if_file(path)
quote
path = $(esc(path))
isfile(path) && return path
end
end
function find_package(name::String)
endswith(name, ".jl") && (name = chop(name, 0, 3))
for dir in [Pkg.dir(); LOAD_PATH]
dir = abspath(dir)
#return_if_file joinpath(dir, "$name.jl")
#return_if_file joinpath(dir, "$name.jl", "src", "$name.jl")
#return_if_file joinpath(dir, name, "src", "$name.jl")
end
return nothing
end
and add a little helper:
find_package(m::Module) = find_package(string(module_name(m)))
Basically, this takes Pkg.dir() and looks in the "usual locations".
Additionally, chop in v0.6.0 doesn't take these additional arguments, which we can fix by adding
chop(s::AbstractString, m, n) = SubString(s, m, endof(s)-n)
Also, if you're not on Unix, you might want to care about the definitions of isfile_casesensitive above the linked code.
And if you're not so concerned about corner cases, maybe this is enough or can serve as a basis:
function modulepath(m::Module)
name = string(module_name(m))
Pkg.dir(name, "src", "$name.jl")
end
julia> Pkg.dir("DataStructures")
"/home/liso/.julia/v0.7/DataStructures"
Edit: I now realized that you want to use Module object!
julia> m = DataStructures
julia> Pkg.dir(repr(m))
"/home/liso/.julia/v0.7/DataStructures"
Edit2: I am not sure if you are trying to find path to module or to object defined in module (I hope that parsing path from next result is easy):
julia> repr(which(DataStructures.eval, (String,)))
"eval(x) in DataStructures at /home/liso/.julia/v0.7/DataStructures/src/DataStructures.jl:3"
I just started developing a frama-c plugin that is doing some kind of alias analysis. I'm using the Dataflow.Backwards analysis and now I have to go through the different assignment statements and collect some stuff about the lvalues.
Does frama-c provide me with 3-address code? Do I have some guarantees about the shape of the lvalue (or any memory access)? I mean, sth like in soot or wala that there is at most one field access, s.t., for a->b->c, there would be a temp variable like tmp=a->b; tmp->c;? I checked the manuals, but I couldn't find anything related to this.
No, there is no such normalization in Frama-C. If you really need it, you can first use a visitor in order to normalize the code so that it suits the requirements of your plug-in. It'd go like that:
class normalize prj: Visitor.frama_c_visitor =
object
inherit Visitor.frama_c_copy prj
method vinstr i =
match i with
| Set (lv,e) -> ...
....
end
let analyze () = ...
let run () =
let my_prj = File.create_project_from_visitor "my_project" (fun prj -> new normalize prj) in
Project.on my_prj analyze ()
The following module from Cil does probably what you want:
http://www.cs.berkeley.edu/~necula/cil/ext.html#toc26. Be aware that the type of the resulting AST is the standard Cil one. You won't be getting any help from the OCaml compiler as to which constructs can be present in the simplified AST, and which ones cannot.
Note also that this module has not been ported to Frama-C so far. You will need some minor adaptation to make it work within Frama-C.