When running Frama-C value analysis with some benchmarks, e.g. susan in http://www.eecs.umich.edu/mibench/automotive.tar.gz, we noticed that a lot of blocks are considered dead code or unreachable. However, in practice, these code is executed as we printed out some debug information from these blocks. Is there anybody noticed this issue? How can we solve this?
Your code has a peculiarity which is not in Pascal's list, and which explains some parts of the dead code. Quite a few functions are declared as such
f(int x, int y);
The return type is entirely missing. The C standard indicates that such functions should return int, and Frama-C follows this convention. When parsing those function, it indicates that they never return anything on some of their paths
Body of function f falls-through. Adding a return statement.
On top on the return statement, Frama-C also adds an /*# assert \false; annotation, to indicate that the execution paths of the functions that return nothing should be dead code. In your code, this annotation is always false: those functions are supposed to return void, and not int. You should correct your code with the good return type.
Occurrences of dead code in the results of Frama-C's value analysis boil down to two aspects, and even these two aspects are only a question of human intentions and are indistinguishable from the point of view of the analyzer.
Real bugs that occur with certainty everytime a particular statement is reached. For instance the code after y = 0; x = 100 / y; is unreachable because the program stops at the division everytime. Some bugs that should be run-time errors do not always stop execution, for instance, writing to an invalid address. Consider yourself lucky that they do in Frama-C's value analysis, not the other way round.
Lack of configuration of the analysis context, including not having provided an informative main() function that sets up variation ranges of the program's inputs with such built-in functions as Frama_C_interval(), missing library functions for which neither specifications nor replacement code are provided, assembly code inside the C program, missing option -absolute-valid-range when one would be appropriate, ...
Related
function somefun()
x::Int = 1
x = 0.5
end
this compiles with no warning. of course calling it produces an InexactError: Int64(0.5). question: can you enforce a compile time check?
Julia is a dynamic language in this sense. So, no, it appears you cannot detect if the result of an assignment will result in such an error without running the function first, as this kind of type checking is done at runtime.
I wasn't sure myself, so I wrapped this function in a module to force (pre)compilation in the absence of running the function, and the result was that no error was thrown, which confirms this idea. (see here if you want to see what I mean by this).
Having said this, to answer the spirit of your question: is there a way to avoid such obscure runtime errors from creeping up in unexpected ways?
Yes there is. Consider the following two, almost equivalent functions:
function fun1(x ); y::Int = x; return y; end;
function fun2(x::Int); y::Int = x; return y; end;
fun1(0.5) # ERROR: InexactError: Int64(0.5)
fun2(0.5) # ERROR: MethodError: no method matching fun2(::Float64)
You may think, big deal, we exchanged one error for another. But this is not the case. In the first instance, you don't know that your input will cause a problem until the point where it gets used in the function. Whereas in the second case, you are effectively enforcing a type check at the point of calling the function.
This is a trivial example of programming "by contract", by making use of Julia's elegant type-checking system. See Design By Contract for details.
So the answer to your question is, yes, if you rethink your design and follow good programming practices, such that this kind of errors are caught early on, then you can avoid having them occuring later on in obscure scenarios where they are hard to fix or detect.
The Julia manual provides a style guide which may also be of help (the example I give above is right at the top!).
It's worth thinking through what "compile time" really is in Julia — because it's probably not what you're thinking.
When you define the function:
julia> function somefun()
x::Int = 1
x = 0.5
end
somefun (generic function with 1 method)
You are not compiling it. Julia won't compile it, in fact, until you call it. Julia's compiler can be thought of as Just-Barely-Ahead-Of-Time, standing in contrast to typical JIT or AOT designs.
Now, when you call the function it compiles it and then runs it which throws the error. You can see this compilation happening the very first time you call the function — it takes a bit more time and memory as it generates and caches the specialized code:
julia> #time try somefun() catch end
0.005828 seconds (6.76 k allocations: 400.791 KiB)
julia> #time try somefun() catch end
0.000107 seconds (6 allocations: 208 bytes)
So perhaps you can see that with Julia's compilation model it doesn't so much matter if it gets caught at compile time or not — even if Julia refused to compile (and cache) the code it'd behave exactly like what you currently see. It'd still allow you to define the function in the first place, and it'd still only throw its error upon calling the function.
The question you mean to ask is if Julia could (or should) catch this error at function definition time. And then the question is really — is it ok to define a method that always results in an error? What about a function like error itself? In Julia, it's totally fine to define a method that unconditionally errors like this one, and there can be good reasons to do so.
Now, there are ways to ask Julia if it is able to detect that this method will always unconditionally error:
julia> #code_typed somefun()
CodeInfo(
1 ─ invoke Base.convert(Main.Int::Type{Int64}, 0.5::Float64)::Union{}
└── $(Expr(:unreachable))::Union{}
) => Union{}
This is the very first step in Julia's process of compilation, and in this case it can see that everything beyond convert(Int, 0.5) is unreachable — that is, it errors. Further, it knows that since the function will never return, it's return type is Union{} (that is, no possible type can ever be returned!) So you can ask Julia to do this step with, for example, the #inferred macro as part of a test suite.
We've been using Frama-C for 'experimental' static analysis on a commercial project (integrated into our CI, with a few selective blocking checks, on a small section of the overall codebase).
One of the snags that comes up relates to satisfying the proof obligations that the wp plugin generates anytime it encounters a memcpy call. Specifically, the three obligations below:
From the 'goal' notes, it looks like Frama-C is trying to prove that the destination and source memory are valid, .
I've tried adding requires \valid() preconditions, but that doesn't seem to help. In these instances, the memcpy call within the function under test is copying data from an input parameter to the function, and placing that data into a local variable (scoped within the function).
To further complicate matters, the local variable where the data is being copied is an attribute within a packed struct.
Concretely, I'm hoping that someone out there is able to share some real examples of memcpy uses where the goals introduced by wp can be satisfied (e.g. what preconditions must I add to make it provable?)
If it matters, I'm running Frama-C Magnesium-20151002 (according to apt-get on Ubuntu 16, this is 'up to date'), and invoking with the following parameters:
frama-c -wp -wp-split -wp-dynamic -lib-entry -wp-proof alt-ergo -wp-report
Also related, but missing a clear working example: Frama-c : Trouble understanding WP memory models
As you mentioned in you comment, the proper solution is to use -wp-model "Typed+Cast" in order to let WP accept casts to/from void* (more precisely, it will consider that p and (void*)p are the same thing for any pointer, which will be sufficient for proving the requires of memcpy). Now, as mentioned in the answer to the question you linked to, the main issue of this memory model (and the reason why it is not the default) is that it is inherently unsafe: it relies on hypotheses that by definition cannot be assessed by WP itself. Here is a small example that highlights this issue:
int x;
char* c;
/*# assigns c;
ensures c == ((char *)&x);
*/
void g(void) {
c = &x;
}
/*# assigns \nothing;
ensures \separated(&x,c);
*/
void f() {
}
void main () {
g();
f();
//# assert \false;
}
Basically, the default Typed memory model ensures the separation between the location pointed to by c and x (i.e. the post-condition of f), because int and char are different, and you neither can prove the post-condition of g or use it as an hypothesis to derive \false in main, because the equality can't be expressed in the model at all.
Now, if you use Typed+Cast, the post-condition of g is now properly understood, and completely trivial to prove. WP won't let you prove at the same time that &x and c are separated, because they are involved together in an assignment. However, in f no such assignment exists, and the post-condition is also easily proved, leading to proving \false in main since we have two contradictory statements about &x and c. More generally, WP relies on a local alias analysis to track potential aliases between pointers of different types (a global analysis would defeat the purpose of having a modular analyzer). Passing option -wp-model +Cast can thus be seen as way to tell WP "Trust me, the program won't create miss-typed aliases". It is however possible to pass alias information by hand (or with the help of e.g. a yet-to-be-written global alias detection plug-in). For instance, with option -wp-alias-vars x,c the post-condition of f becomes Unknown (i.e. the separation between &x and c is not an assumption anymore, even for f).
I need to write a program, which calculates product of product in range:
I written the following code:
mult(N,N,R,R).
mult(N,Nt,R,Rt):-N1=Nt+1,R1=Rt*(1/(log(Nt))),mult(N,N1,R,R1).
This should implement basic product from Nt to N of 1/ln(j). As far as I understand it's got to be stopped when Nt and N are equal. However, I can't get it working due to:
?- mult(10,2,R,1), write(R).
ERROR: Out of global stack
The following error. Is there any other way to implement loop not using default libraries of SWI-Prolog?
Your program never terminates! To see this consider the following failure-slice of your program:
mult(N,N,R,R) :- false.
mult(N,Nt,R,Rt):-
N1=Nt+1,
R1=Rt*(1/(log(Nt))),
mult(N,N1,R,R1), false.
This new program does never terminate, and thus the original program doesn't terminate. To see that this never terminates, consider the two (=)/2 goals. In the first, the new variable N1 is unified with something. This will always succeed. Similarly, the second goal with always succeed. There will never be a possibility for failure prior to the recursive goal. And thus, this program never terminates.
You need to add some goal, or to replace existing goals. in the visible part. Maybe add
N > Nt.
Further, it might be a good idea to replace the two (=)/2 goals by (is)/2. But this is not required for termination, strictly speaking.
Out of global stack means you entered a too-long chain of recursion, possibly an infinite one.
The problem stems from using = instead of is in your assignments.
mult(N,N,R,R).
mult(N,Nt,R,Rt):-N1 is Nt+1, R1 is Rt*(1/(log(Nt))), mult(N,N1,R,R1).
You might want to insert a cut in your first clause to avoid going on after getting an answer.
If you have a graphical debugger (like the one in SWI) try setting 'trace' and 'debug' on and running. You'll soon realize that performing N1 = Nt+1 giving Ntas 2 yields the term 2+1. Since 2+1+1+1+(...) will never unify with with 10, that's the problem right there.
I have some questions on options of value analysis module and some extensions options.
I use the command : frama-c-gui -val -slevel 100 -plevel 300 -absolute-valid-range 0x00000000-0xffffffff -metrics -metrics-value-cover -scope-def-interproc -main MYMAIN CODE/*.c
On a single file, -metrics give me 3 goto on a function without, how goto is compute ?
What is "Coverage estimation = 100.0%" with -metrics-value-cover I get a value between 80 and 100%, at the beginning I thought get <100% when I had dead code, but I had dead code when I get 100%, so I think get 100% if all functions in sources files are analysed ?
I suppose so 157 stmts in analyzed functions, 148 stmts analyzed (94.3%) that means I have dead code on my projet, it's that ?
With option -scope-def-interproc I get me 32 warning (62 without) but on website, we can read (in scope documentation)
The alarms emitted by the value analysis should therefore be examined carefully by the user.
So I need to verify all of 62 warning or just 32 got by this options ?
On a single file, -metrics [introduces] 3 goto on a function [that doesn't have any]
The C constructs &&, || and break; can be “normalized” into goto constructs.
I suppose so 157 stmts in analyzed functions, 148 stmts analyzed (94.3%) that means I have dead code on my projet, it's that ?
Yes. For the inputs visible to the value analysis, only 148 out of 157 statements are reachable. Note that if, for instance, main() has arguments argc and argv, the abstract values built for these arguments may not encompass all the values that should be considered. The best way to determine whether these 9 statements are really unreachable is to look at them (they are displayed in red in Frama-C's GUI).
With option -scope-def-interproc I get me 32 warning (62 without) but on website, we can read (in scope documentation)
It is not very clear what you are asking. It would help if you provided an example with full information (source code, commandline(s)) so that one can reproduce your steps and clarify the meaning of the emitted messages for you. Produce a small, reduced example if it is impossible to get the complete example out: it is also (nearly) impossible to answer you with the information provided so far.
What is "Coverage estimation = 100.0%" with -metrics-value-cover I get
a value between 80 and 100%, at the beginning I thought get <100% when
I had dead code, but I had dead code when I get 100%, so I think get
100% if all functions in sources files are analysed ?
Let's take an example.
[metrics] Value coverage statistics
=========================
Syntactically reachable functions = 3 (out of 3)
Semantically reached functions = 1
Coverage estimation = 33.3%
Unseen functions (2) =
<tests/metrics/reach.c>: baz; foo;
The first line, Syntactically reachable functions is an over-approximation of the number of functions of your program that may be eventually be called, starting from main. For example, a function whose address is never taken and which is never called directly won't be inside this set.
Semantically reached functions is the number of functions of your program indeed analyzed by the Value analysis.
Coverage estimation is the ratio between those two numbers. For small programs, in which all functions are reachable, it is usually 100%.
Unseen functions is the list of functions that were expected to be reached (syntactically), but were never analyzed by Value.
Notice that none of these numbers talk about instructions, so you may still get 100% with dead code.
I've written an experimental function evaluator that allows me to bind simple functions together such that when the variables change, all functions that rely on those variables (and the functions that rely on those functions, etc.) are updated simultaneously. The way I do this is instead of evaluating the function immediately as it's entered in, I store the function. Only when an output value is requested to I evaluate the function, and I evaluate it each and every time an output value is requested.
For example:
pi = 3.14159
rad = 5
area = pi * rad * rad
perim = 2 * pi * rad
I define 'pi' and 'rad' as variables (well, functions that return a constant), and 'area' and 'perim' as functions. Any time either 'pi' or 'rad' change, I expect the results of 'area' and 'perim' to change in kind. Likewise, if there were any functions depending on 'area' or 'perim', the results of those would change as well.
This is all working as expected. The problem here is when the user introduces recursion - either accidental or intentional. There is no logic in my grammar - it's simply an evaluator - so I can't provide the user with a way to 'break out' of recursion. I'd like to prevent it from happening at all, which means I need a way to detect it and declare the offending input as invalid.
For example:
a = b
b = c
c = a
Right now evaluating the last line results in a StackOverflowException (while the first two lines evaluate to '0' - an undeclared variable/function is equal to 0). What I would like to do is detect the circular logic situation and forbid the user from inputing such a statement. I want to do this regardless of how deep the circular logic is hidden, but I have no idea how to go about doing so.
Behind the scenes, by the way, input strings are converted to tokens via a simple scanner, then to an abstract syntax tree via a hand-written recursive descent parser, then the AST is evaluated. The language is C#, but I'm not looking for a code solution - logic alone will be fine.
Note: this is a personal project I'm using to learn about how parsers and compilers work, so it's not mission critical - however the knowledge I take away from this I do plan to put to work in real life at some point. Any help you guys can provide would be appreciated greatly. =)
Edit: In case anyone's curious, this post on my blog describes why I'm trying to learn this, and what I'm getting out of it.
I've had a similar problem to this in the past.
My solution was to push variable names onto a stack as I recursed through the expressions to check syntax, and pop them as I exited a recursion level.
Before I pushed each variable name onto the stack, I would check if it was already there.
If it was, then this was a circular reference.
I was even able to display the names of the variables in the circular reference chain (as they would be on the stack and could be popped off in sequence until I reached the offending name).
EDIT: Of course, this was for single formulae... For your problem, a cyclic graph of variable assignments would be the better way to go.
A solution (probably not the best) is to create a dependency graph.
Each time a function is added or changed, the dependency graph is checked for cylces.
This can be cut short. Each time a function is added, or changed, flag it. If the evaluation results in a call to the function that is flagged, you have a cycle.
Example:
a = b
flag a
eval b (not found)
unflag a
b = c
flag b
eval c (not found)
unflag b
c = a
flag c
eval a
eval b
eval c (flagged) -> Cycle, discard change to c!
unflag c
In reply to the comment on answer two:
(Sorry, just messed up my openid creation so I'll have to get the old stuff linked later...)
If you switch "flag" for "push" and "unflag" for "pop", it's pretty much the same thing :)
The only advantage of using the stack is the ease of which you can provide detailed information on the cycle, no matter what the depth. (Useful for error messages :) )
Andrew