folks!
I pass a struct full of data to my kernel, and I run into the following difficulty using it (very stripped down):
[edit: mac osx / xcode 3.2 on mac book pro; this compile is obviously for cpu]
typedef struct
{
float xoom;
int sizex;
} varholder;
float zX, xd;
__kernel void Harlan( __global varholder * vh )
{
int X = get_global_id(0), Y = get_global_id(1);
zX = ( ( X - vh->sizex/2 ) / vh->xoom + vh->sizex/2 ); // (a)
xd = zX; // (b) BOOM!!
}
after executing line (a), the line marked (b), a simple assignment, gives "LLVM compiler failed to compile a function".
if, however, we do not execute line (a), then line (b) is fine.
So, through my fiddling around a LOT with this, it seems as if it is the assignment statement (a), which uses a passed-in parameter, that messes up the future access of the variable zX. However, of course I need to be able to use the results of calculations further down the line.
I have zX and xd declared at the file level because my helper functions need them.
Any thoughts?
Thanks!
David
p.s. I'm now registered so will be able to upvote and accept answers, which I am sadly unable to do for the last person who helped me (used same username to register, but can't seem to vote on the old post; sorry!).
No, say it ain't so!
I am sincerely hoping that this is not a "correct" answer to my own question. I found on another forum (though not the same question asked!) the following, and I am afraid that it refers to what I'm trying to do:
(quote)
You're doing something the standard prohibits. Section 6.5 says:
'All program scope variables must be declared in the __constant address space.'
In other words, program scope variables cannot be mutable.
(end quote)
... well, tcha!!!! What an astoundingly inconvenient restriction! I'm sure there's reasoning behind it.
[edit: Not At All inconvenient! it was in fact astonishingly easy to work around, given a fresh start the next morning. (And no alcohol.)]
You guys & dolls all knew this, right, and didn't have the heart to tell me?...
Related
EDIT: The original question had unnecessary details
I have a source file which I do value analysis in Frama-C, some of the code is highlighted as dead code in the normalized window, no the original source code.
Can I obtain a slice of the original code that removes the dead code?
Short answer: there's nothing in the current Frama-C version that will let you do that directly. Moreover, if your original code contains macros, Frama-C will not even see the real original code, as it relies on an external preprocessor (e.g. cpp) to do macro expansion.
Longer answer: Each statement in the normalized (aka CIL) Abstract Syntax Tree (AST, the internal representation of C code within Frama-C) contains information about the location (start point and end point) of the original statement where it stems from, and this information is also available in the original AST (aka Cabs). It might thus be possible for someone with a good knowledge of Frama-C's inner workings (e.g. a reader of the developer's manual), to build a correspondance between both, and to use that to detect dead statement in Cabs. Going even further, one could bypass Cabs, and identify zones in the original text of the program which are dead code. Note however that it would be a tedious and quite error prone (notably because a single original statement can be expanded in several normalized ones) task.
Given your clarifications, I stand by #Virgile's answer; but for people interested in performing some simplistic dead code elimination within Frama-C, the script below, gifted by a colleague who has no SO account, could be helpful.
(* remove_dead_code.ml *)
let main () =
!Db.Value.compute ();
Slicing.Api.Project.reset_slicing ();
let selection = ref Slicing.Api.Select.empty_selects in
let o = object (self)
inherit Visitor.frama_c_inplace
method !vstmt_aux stmt =
if Db.Value.is_reachable_stmt stmt then
selection :=
Slicing.Api.Select.select_stmt ~spare:true
!selection
stmt
(Extlib.the self#current_kf);
Cil.DoChildren
end in
Visitor.visitFramacFileSameGlobals o (Ast.get ());
Slicing.Api.Request.add_persistent_selection !selection;
Slicing.Api.Request.apply_all_internal ();
Slicing.Api.Slice.remove_uncalled ();
ignore (Slicing.Api.Project.extract "no-dead")
let () = Db.Main.extend main
Usage:
frama-c -load-script remove_dead_code.ml file.c -then-last -print -ocode output.c
Note that this script does not work in all cases and could have further improvements (e.g. to handle initializers), but for some quick-and-dirty hacking, it can still be helpful.
I'm looking for WP options/model that could allow me to prove basic C memory manipulations like :
memcpy : I've tried to prove this simple code :
struct header_src{
char t1;
char t2;
char t3;
char t4;
};
struct header_dest{
short t1;
short t2;
};
/*# requires 0<=n<=UINT_MAX;
# requires \valid(dest);
# requires \valid_read(src);
# assigns (dest)[0..n-1] \from (src)[0..n-1];
# assigns \result \from dest;
# ensures dest[0..n] == src[0..n];
# ensures \result == dest;
*/
void* Frama_C_memcpy(char *dest, const char *src, uint32_t n);
int main(void)
{
struct header_src p_header_src;
struct header_dest p_header_dest;
p_header_src.t1 = 'e';
p_header_src.t2 = 'b';
p_header_src.t3 = 'c';
p_header_src.t4 = 'd';
p_header_dest.t1 = 0x0000;
p_header_dest.t2 = 0x0000;
//# assert \valid(&p_header_dest);
Frama_C_memcpy((char*)&p_header_dest, (char*)&p_header_src, sizeof(struct header_src));
//# assert p_header_dest.t1 == 0x6265;
//# assert p_header_dest.t2 == 0x6463;
}
but the two last assert weren't verified by WP (with default prover Alt-Ergo). It can be proved thanks to Value analysis, but I mostly want to be able to prove the code not using abstract interpretation.
Cast pointer to int : Since I'm programming embedded code, I want to be able to specify something like:
#define MEMORY_ADDR 0x08000000
#define SOME_SIZE 10
struct some_struct {
uint8_t field1[SOME_SIZE];
uint32_t field2[SOME_SIZE];
}
// [...]
// some function body {
struct some_struct *p = (some_struct*)MEMORY_ADDR;
if(p == NULL) {
// Handle error
} else {
// Do something
}
// } body end
I've looked a little bit at WP's documentation and it seems that the version of frama-c that I use (Magnesium-20151002) has several memory model (Hoare, Typed , +cast, +ref, ...) but none of the given example were proved with any of the model above. It is explicitly said in the documentation that Typed model does not handle pointer-to-int casts. I've a lot of trouble to understand what's really going on under the hood with each wp-model. It would really help me if I was able to verify at least post-conditions of the memcpy function. Plus, I have seen this issue about void pointer that apparently are not very well handled by WP at least in the Magnesium version. I didn't tried another version of frama-c yet, but I think that newer version handle void pointer in a better way.
Thank you very much in advance for your suggestions !
memcpy
Reasoning about the result of memcpy (or Frama_C_memcpy) is out of range of the current WP plugin. The only memory model that would work in your case is Bytes (page 13 of the manual for Chlorine), but it is not implemented.
Independently, please note that your postcondition from Frama_C_memcpy is not what you want here. You are asserting the equality of the sets dest[0..n] and src[0..n]. First, you should stop at n-1. Second, and more importantly, this is far too weak, and is in fact not sufficient to prove the two assertions in the caller. What you want is a quantification on all bytes. See e.g. the predicate memcmp in Frama-C's stdlib, or the variant \forall int i; 0 <= i < n -> dest[i] == src[i];
By the way, this postcondition holds only if dest and src are properly separated, which your function does not require. Otherwise, you should write dest[i] == \at (src[i], Pre). And your requires are also too weak for another reason, as you only require the first character to be valid, not the n first ones.
Cast pointer to int
Basically, all current models implemented in WP are unable to reason on codes in which the memory is accessed with multiple incompatible types (through the use of unions or pointer casts). In some cases, such as Typed, the cast is detected syntactically, and a warning is issued to warn that the code cannot be analyzed. The Typed+Cast model is a variant of Typed in which some casts are accepted. The analysis is correct only if the pointer is re-cast into its original type before being used. The idea is to allow using some libc functions that require a cast to void*, but not much more.
Your example is again a bit different, because it is likely that MEMORY_ADDR is always addressed with type some_stuct. Would it be acceptable to change the code slightly, and change your function as taking a pointer to this type? This way, you would "hide" the cast to MEMORY_ADDR inside a function that would remain unproven.
I tried this example in the latest version of Frama-C (of course the format is modified a little bit).
for the memcpy case
Assertion 2 fails but assertion 3 is successfully proved (basically because the failure of assertion 2 leads to a False assumption, which proves everything).
So in fact both assertion cannot be proved, same as your problem.
This conclusion is sound because the memory models used in the wp plugin (as far as I know) has no assumption on the relation between fields in a struct, i.e. in header_src the first two fields are 8 bit chars, but they may not be nestedly organized in the physical memory like char[2]. Instead, there may be paddings between them (refer to wiki for detailed description). So when you try to copy bits in such a struct to another struct, Frama-C becomes completely confused and has no idea what you are doing.
As far as I am concerned, Frama-C does not support any approach to precisely control the memory layout, e.g. gcc's PACKED which forces the compiler to remove paddings.
I am facing the same problem, and the (not elegant at all) solution is, use arrays instead. Arrays are always nested, so if you try to copy a char[4] to a short[2], I think the assertion can be proved.
for the Cast pointer to int case
With memory model Typed+cast, the current version I am using (Chlorine-20180501) supports casting between pointers and uint64_t. You may want to try this version.
Moreover, it is strongly suggested to call Z3 and CVC4 through why3, whose performance is certainly better than Alt-Ergo.
What is a purpose in Pascal to declare variable that is pointer to a pointer? I have a code in Mac Think Pascal. Here is some parts from the code that I don't understand:
type
my_array = array[1..100] of integer;
my_array_pointer = ^my_array;
my_array_handle = ^my_array_pointer;
...
var
xx : my_array_handle;
...
begin
xx:= my_array_handle(NewHandle( sizeof(my_array)) );
As you see, the last line is an assignment of a type my_array_handle to variable xx. What does it mean? What does NewHandle function do? (This is an internal function of Think Pascal). Actually, I need to convert a Think Pascal program to Windows Pascal. However I cannot find the description of NewHandle function, and don't know how to implement this function using the standard (New(), GetMem() etc) pointer functions.
This is a classic macos feature, not typically something of Pascal.
I don't know exactly, but it had something to do with relocatability of the loaded program in a non PM environment.
Note that the indirect pointer is allocated via an OS function, which probably means that it is allocated in some table that is maintained by the OS. (so that the OS can move/relocate the program?)
In modern Mac (and other) programming this whole principle is alien. Just clean it up.
(added)
If you want to keep these redirections, you could try your luck with something like:
function newhandle( nrbytes:integer):ppointer;
var xx : ppointer;
begin
new(xx);
getmem(xx^,nrbytes);
newhandle:=xx;
end;
I didn't add this originally, but I recommend you simply clean up these anachronistic indirect references and use my_array_pointer based pointers and getmem or new directly.
The meaning of these indirect references has no use on non m68k Classic Mac OS systems, though afaik later PPC versions still somewhat support them. (PPC is always protected mode)
If you really want the gritty details, you probably want to subscribe to the mac-pascal list.
I am using the Kernel Density Estimator toolbox form http://www.ics.uci.edu/~ihler/code/kde.html . But I am getting the following error when I try to execute the demo files -
>> demo_kde_3
KDE Example #3 : Product sampling methods (single, anecdotal run)
Attempt to reference field of non-structure array.
Error in double (line 10)
if (npd.N > 0) d = 1; % return 1 if the density exists
Error in repmat (line 49)
nelems = prod(double(siz));
Error in kde (line 39)
if (size(ks,1) == 1) ks = repmat(ks,[size(points,1),1]); end;
Error in demo_kde_3 (line 8)
p = kde([.1,.45,.55,.8],.05); % create a mixture of 4 gaussians for
testing
Can anyone suggest what might be wrong? I am new to Matlab and having a hard time to figure out the problem.
Thank You,
Try changing your current directory away from the #kde folder; you may have to add the #kde folder to your path when you do this. For example run:
cd('c:\');
addpath('full\path\to\the\folder\#kde');
You may also need to add
addpath('full\path\to\the\folder\#kde\examples');
Then see if it works.
It looks like function repmat (a mathworks function) is picking up the #kde class's version of the double function, causing an error. Usually, only objects of the class #kde can invoke that functions which are in the #kde folder.
I rarely use the #folder form of class definitions, so I'm not completely sure of the semantics; I'm curious if this has any effect on the error.
In general, I would not recommend using the #folder class format for any development that you do. The mathworks overhauled their OO paradigm a few versions ago to a much more familiar (and useful) format. Use help classdef to see more. This #kde code seems to predate this upgrade.
MATLAB gives you the code line where the error occurs. As double and repmat belong to MATLAB, the bug probably is in kde.m line 39. Open that file in MATLAB debugger, set a breakpoint on that line (so the execution stops immediately before the execution of that specific line), and then when the code is stopped there, check the situation. Try the entire code line in console (copy-paste or type it, do not single-step, as causing an uncatched error while single-stepping ends the execution of code in debugger), it should give you an error (but doesn't stop execution). Then try pieces of the code of that code line, what works as it should and what not, eg. does the result of size(points, 1) make any sense.
However, debugging unfamiliar code is not an easy task, especially if you're a beginner in MATLAB. But if you learn and understand the essential datatypes of MATLAB (arrays, cell arrays and structs) and the different ways they can be addressed, and apply that knowledge to the situation on the line 39 of kde.m, hopefully you can fix the bug.
Repmat calls double and expects the built-in double to be called.
However I would guess that this is not part of that code:
if (npd.N > 0) d = 1; % return 1 if the density exists
So if all is correct this means that the buil-tin function double has been overloaded, and that this is the reason why the code crashes.
EDIT:
I see that #Pursuit has already addressed the issue but I will leave my answer in place as it describes the method of detection a bit more.
I see qCopy, and qCopybackward but neither seems to let me make a copy in reverse order. qCopybackward only copies it in reverse order, but keeps the darn elements in the same order! All I want to do is return a copy of the list in reverse order. There has to be a function for that, right?
If you don't like the QTL, just use the STL. They might not have a Qt-ish API, but the STL API is rock-stable :) That said, qCopyBackward is just std::copy_backward, so at least they're consistent.
Answering your question:
template <typename T>
QList<T> reversed( const QList<T> & in ) {
QList<T> result;
result.reserve( in.size() ); // reserve is new in Qt 4.7
std::reverse_copy( in.begin(), in.end(), std::back_inserter( result ) );
return result;
}
EDIT 2015-07-21: Obviously (or maybe not), if you want a one-liner (and people seem to prefer that, looking at the relative upvotes of different answers after five years) and you have a non-const list the above collapses to
std::reverse(list.begin(), list.end());
But I guess the index fiddling stuff is better for job security :)
Reverse your QList with a single line:
for(int k = 0; k < (list.size()/2); k++) list.swap(k,list.size()-(1+k));
[Rewrite from original]
It's not clear if OP wants to know "How [do I] reverse a QList?" or actually wants a reversed copy. User mmutz gave the correct answer for a reversed copy, but if you just want to reverse the QList in place, there's this:
#include <algorithm>
And then
std::reverse(list.begin(), list.end());
Or in C++11:
std::reverse(std::begin(list), std::end(list));
The beauty of the C++ standard library (and templates in general) is that the algorithms and containers are separate. At first it may seem annoying that the standard containers (and to a lesser extent the Qt containers) don't have convenience functions like list.reverse(), but consider the alternatives: Which is more elegant: Provide reverse() methods for all containers, or define a standard interface for all containers that allow bidirectional iteration and provide one reverse() implementation that works for all containers that support bidirectional iteration?
To illustrate why this is an elegant approach, consider the answers to some similar questions:
"How do you reverse a std::vector<int>?":
std::reverse(std::begin(vec), std::end(vec));
"How do you reverse a std::deque<int>?":
std::reverse(std::begin(deq), std::end(deq));
What about portions of the container?
"How do you reverse the first seven elements of a QList?": Even if the QList authors had given us a convenience .reverse() method, they probably wouldn't have given us this functionality, but here it is:
if (list.size() >= 7) {
std::reverse(std::begin(list), std::next(std::begin(list), 7));
}
But it gets better: Because the iterator interface is the same as C pointer syntax, and because C++11 added the free std::begin() and std::end functions, you can do these:
"How do you reverse an array float x[10]?":
std::reverse(std::begin(x), std::end(x));
or pre C++11:
std::reverse(x, x + sizeof(x) / sizeof(x[0]));
(That is the ugliness that std::end() hides for us.)
Let's go on:
"How do you reverse a buffer float* x of size n?":
std::reverse(x, x + n);
"How do you reverse a null-terminated string char* s?":
std::reverse(s, s + strlen(s));
"How do you reverse a not-necessarily-null-terminated string char* s in a buffer of size n?":
std::reverse(s, std::find(s, s + n, '\0'));
Note that std::reverse uses swap() so even this will perform pretty much as well as it possibly could:
QList<QList<int> > bigListOfBigLists;
....
std::reverse(std::begin(bigListOfBigLists), std::end(bigListOfBigLists));
Also note that these should all perform as well as a hand-written loop since, when possible, the compiler will turn these into pointer arithmetic. Also, you can't cleanly write a reusable, generic, high-performance reverse function like this C.
#Marc Jentsch's answer is good. And if you want to get an additional 30% performance boost you can change his one-liner to:
for(int k=0, s=list.size(), max=(s/2); k<max; k++) list.swap(k,s-(1+k));
One a ThinkPad W520 with a QList of 10 million QTimers I got these numbers:
reversing list stack overflow took 194 ms
reversing list stack overflow with max and size took 136 ms
The boost is a result of
the expression (list.size()/2) being calculated only once when initializing the loop and not after every step
the expression list.size() in swap() is called only once when initializing the loop and not after every step
You can use the Java style iterator. Complete example here (http://doc.qt.digia.com/3.2/collection.html). Look for the word "reverse".
QList<int> list; // initial list
list << 1;
list << 2;
list << 3;
QList<int> rlist; // reverse list+
QListIterator<int> it(list);
while (it.hasPrevious()) {
rlist << it.previous();
}
Reversing a QList is going to be O(n) however you do it, since QList isn't guaranteed to have its data stored contiguously in memory (unlike QVector). You might consider just traversing the list in backwards order where you need to, or use something like a QStack which lets you retrieve the elements in the opposite order they were added.
For standard library lists it would look like this
std::list<X> result;
std::copy(list.rbegin(), list.rend(), std::back_inserter(result));
Unfortunately, Qt doesn't have rbegin and rend functions that return reverse iterators (the ones that go from the end of the container to its begnning). You may write them, or you can just write copying function on your own -- reversing a list is a nice excersize. Or you can note that QList is actually an array, what makes writing such a function trivial. Or you can convert the list to std::list, and use rbegin and rend. Choose whatever you like.
As of Qt 5.14 (circa 2020), QList provides a constructor that takes iterators, so you can just construct a reversed copy of the list with the reverse iterators of the source:
QList<int> backwards(forwards.rbegin(), forwards.rend());
Or if you want to be able to inline it, more generically (replace QList<I> with just I if you want to be super duper generic):
template <typename I> QList<I> reversed (const QList<I> &forwards) {
return QList<I>(forwards.rbegin(), forwards.rend());
}
Which lets you do fun one-liners with temporaries like:
QString badDay = "reporter covers whale exploding";
QString worseDay = reversed(badDay.split(' ')).join(' ');