cffi function call hangs - common-lisp

I want to use stat(2) from Common Lisp.
I've defined the structs used by the stat function:
(cffi:defctype mode_t :unsigned-int)
(cffi:defctype ino_t :unsigned-int)
(cffi:defctype dev_t :int)
(cffi:defctype nlink_t :int)
(cffi:defctype uid_t :unsigned-int)
(cffi:defctype gid_t :unsigned-int)
(cffi:defctype off_t :int)
(cffi:defctype time_t :long)
(cffi:defctype blksize_t :unsigned-int)
(cffi:defctype blkcnt_t :int)
(cffi:defcstruct stat
(st_dev dev_t)
(st_ino ino_t)
(st_mode mode_t)
(st_nlink nlink_t)
(st_uid uid_t)
(st_gid gid_t)
(st_rdev dev_t)
(st_size off_t)
(st_atime time_t)
(st_mtime time_t)
(st_ctime time_t)
(st_blksize blksize_t)
(st_blocks blkcnt_t))
And the function itself:
(cffi:defcfun "stat" :int
(path :string)
(buf (:pointer (:struct stat))))
I'm trying to call it quite simply like this:
(cffi:with-foreign-object (buf '(:pointer (:struct stat)))
(stat "/home/florian/tmp/msg.txt" buf)
(cffi:with-foreign-slots ((st_mode) buf (:struct stat))
st_mode))
And Slime just hangs. No error, and the REPL input doesn't come back.

If you look into your *inferior-lisp* buffer, you will see that SBCL has dropped into its low-level debugger due to some severe memory corruption.
The specific layout of struct stat depends quite heavily on the architecture you have. On my 64-bit Ubuntu 14.04, the following seems to be 144 bytes in length, while your definition is only 52 bytes long - no wonder that trying to write into it causes memory corruption.
It is probably a Bad Idea to try writing defcstruct forms for structs that the operating system is free to define in any way it wants. The code may run on your computer, but probably won't run on everybody else's system, if they are using a different OS or processor architecture - but if you really need to get it running on your system only, the easiest way is probably to write a short C program that dumps the sizes and offsets of all the fields on screen, then build up from there.
If you need to write code that runs on multiple different systems and that uses stat, a good option is to write a simple proxy module in C yourself, with a well-defined, constant interface, that calls stat on your behalf. This is the approach I have used on several occasions, keeping foreign stuff safe on the C side and only passing the really required data over the FFI.
For more robust CFFI definitions, there is also SWIG.

#jlahd's answer helped me to find the right solution.
After finding the correct structure, my calling code was still wrong. Here is the final code:
(cffi:with-foreign-object (buf '(:struct stat))
(stat "/home/florian/tmp/msg.txt" buf)
(cffi:with-foreign-slots ((st_mode) buf (:struct stat))
st_mode))
Note that it uses '(:struct stat) instead of '(:pointer (:struct stat)) for buf foreign type.
How did I get the correct structure though? I wanted to document it.
Here is the final cffi structure that I've used:
(cffi:defcstruct (stat :size 144)
(st_mode :unsigned-int :offset 24))
Here is the C program I used to find out the sizes and offset of the stat struct and its members:
#include <stddef.h>
#include <sys/stat.h>
#include <stdio.h>
#include <stdlib.h>
int main (int argc, char* argv[])
{
struct stat foo;
printf("stat sizeof: %zu\n", sizeof(struct stat));
printf("st_mode sizeof: %zu\n", sizeof(foo.st_mode));
printf("st_mode offset: %zu\n", offsetof(struct stat, st_mode));
}
On my computer, this gives me this:
$ gcc stat.c
$ ./a.out
stat sizeof: 144
st_mode sizeof: 4
st_mode offset: 24
I could then check that the size of :unsigned-int was 4 bytes:
CL-USER> (cffi:foreign-type-size :unsigned-int)
4
And check the size of my cffi struct:
CL-USER> (cffi:foreign-type-size '(:struct stat))
144
Which matches sizeof(struct stat) in C.

Related

How to validate code that read/write to hardware memory mapped registers (mmio) with frama-c Eva plugin or WP-RTE?

The closest answer I found maybe related to -absolute-valid-range for the Eva plugin but is that it? Do I have to come up with read/write ACSL predicates to do dummy read/write?
Sample code:
#include <stdint.h>
#define BASE_ADDR 0x0e000000
#define BASE_LIMIT 0x0e001000
#define TEST_REG 0x10
/*# requires BASE_ADDR <= addr < BASE_LIMIT;
# assigns \nothing;
*/
static inline uint32_t mmio_read32(volatile uintptr_t addr)
{
volatile uint32_t *ptr = (volatile uint32_t *)addr;
return *ptr;
}
/*#
# requires 0 <= offset <= 0x1000;
# assigns \nothing;
*/
static inline uint32_t read32(uintptr_t offset)
{
return mmio_read32((uintptr_t)BASE_ADDR + offset);
}
void main(){
uint32_t test;
test = read32(TEST_REG);
return;
}
Frama-c command and output:
[frama -absolute-valid-range 0x0e000000-0x0e001000 -wp mmio2.c
[kernel] Parsing mmio2.c (with preprocessing)
[wp] Warning: Missing RTE guards
[wp] 6 goals scheduled
[wp] [Alt-Ergo] Goal typed_read32_call_mmio_read32_pre : Unknown (Qed:4ms) (51ms)
[wp] Proved goals: 5 / 6
Qed: 5
Alt-Ergo: 0 (unknown: 1)][1]
How to discharge goal "typed_read32_call_mmio_read32_pre" or is this expected?
The fact that the proof fails is related to two independent issues, but none of them is related to using absolute addresses.
First, as the argument of mmio_read32 is marked as volatile, WP consider that its value can be anything. In particular, none of the hypotheses that are made on offset are known to hold when evaluating addr. You can see that in the GUI by looking at the generated goal (go in the WP Goals tab at the bottom and double click in the Script colon and the line of the failed proof attempt):
Goal Instance of 'Pre-condition'
(call 'mmio_read32'):
Assume {
Type: is_uint32(offset_0).
(* Pre-condition *)
Have: (0 <= offset_0) /\ (offset_0 <= 4095).
}
Prove: (234881024 <= w) /\ (w_1 <= 234885119).
w and w_1 correspond to two read accesses to the volatile content of addr. I'm not sure if you really intended the addr parameter to be volatile (as opposed to a non-volatile pointer to a volatile location), as this would require a pretty weird execution environment.
Assuming that the volatile qualifier should not present in the declaration of addr, a remaining issue is that the constraint on offset in read32 is too weak: it should read offset < 0x1000 with a strict inequality (or the precondition of mmio_read32 be made non-strict, but again this would be quite uncommon).
Regarding the initial question about physical adressing and volatile, in ACSL (see section 2.12.1 of the manual), you have a specific volatile clause that allows you to specify (C, usually ghost) functions to represent read and write accesses to volatile locations. Unfortunately, support for these clauses is currently only accessible through a non-publicly distributed plug-in.
I'm afraid that if you want to use WP on code with physical adressing, you need indeed to use an encoding (e.g. with a ghost array of appropriate size), and/or model volatile accesses with appropriate functions.

What could be the reason to not be able to use Math built-in functions in OpenCL? Should I use some directive to active?

The build returns -11 error. Removing pow function compiles fine. I'm not using embedded profile.
__kernel void VectorAdd(__global int* a)
{
unsigned int n = get_global_id(0);
a[n] = pow(2, 2);
}
Im catching the error but the string is empty
int err = clBuildProgram(OpenCLProgram, 0, NULL, NULL, NULL, NULL);
if (err != CL_SUCCESS)
{
size_t len;
char buffer[2048];
printf("Error: Failed to build program executable!\n");
clGetProgramBuildInfo(OpenCLProgram, cdDevice, CL_PROGRAM_BUILD_LOG, sizeof(buffer), buffer, &len);
printf("%s\n", buffer);
exit(1);
}
Some useful info:
CL_DEVICE_NAME: AMD Radeon HD - FirePro D300 Compute Engine
CL_DRIVER_VERSION: 1.2 (Jan 10 2017 22:25:08)
If you look at the OpenCL documentation for pow you will notice that it is defined as gentype pow(gentype x, gentype y). The document also states that
The generic type name gentype is used to indicate that the function can take float, float2, float3, float4, float8, float16, double, double2, double3, double4, double8, or double16 as the type for the arguments.
So pow() takes two float or two double values or vectors thereof and returns a value of the same type. Since the compiler cannot determine wether you wanted to call pow(2.0, 2.0) (double precision) or pow(2.0f, 2.0f) (single precision), you get an error instead.
Note that there is also the similar-named function float pown(float x, int y) which takes an integer value for the exponent (e.g. pown(2.0f, 2)) and may provide an optimized implementation of this case.
What does clGetProgramBuildInfo() with param_name=CL_PROGRAM_BUILD_LOG say? This should give you a much more detailed error message. Update the question with this and I might be able to expand this answer.
What version of OpenCL is this? Note that prior to 1.2, the pow() function was only defined for floating-point types; you're expecting it to work with integers.

Error compiling VexCL program: raw_ptr not a member of device_vector

I am trying to compile my first VexCL program using the thrust example and I get the following error message:
raw_ptr is not a member of 'vex::backend::opencl::device_vector'
Here is the code
vex::Context ctx(vex::Filter::Env && vex::Filter::Count(1));
std::cout << ctx << std::endl;
vex::profiler<> prof(ctx);
typedef int T;
const size_t n = 16 * 1024 * 1024;
vex::vector<T> x(ctx, n);
vex::Random<T> rnd;
// Get raw pointers to the device memory.
T *x_begin = x(0).raw_ptr(); // Here is where the error is occurring.
T *x_end = x_begin + x.size();
I do not understand the language well enough. I appreciate any help in this matter.
Thanks
Chris
The thrust example is not the best to start with, as it deals with interfacing VexCL and Thrust (another high-level library that is targeted on CUDA).
So in order to compile the example, you need to use the CUDA backend in VexCL. That is, you need to define VEXCL_BACKEND_CUDA preprocessor macro
and to link against libcuda.so (or cuda.lib if on Windows) instead of libOpenCL.so/OpenCL.lib.
The error you got is because the device_vector class only exposes raw_ptr() method when on CUDA backend.

Using async_work_group_copy with a custom data type

I need to copy some data from __global to __local in openCL using async_work_group_copy. The issue is, I'm not using a built-in data type.
The code snip of what I have tried is as follows:
typedef struct Y
{
...
} Y;
typedef struct X
{
Y y[MAXSIZE];
} X;
kernel void krnl(global X* restrict x){
global const Y* l = x[a].y;
local Y* l2;
size_t sol2 = sizeof(l);
async_work_group_copy(l2, l, sol2, 0);
}
where 'a' is just a vector of int. This code does not work, specifically because the gen_type is not a built-in one. The specs (1.2) says:
We use the generic type name gentype to indicate the built-in data
types ... as the type for the arguments unless otherwise stated.
so how do I otherwisely state this data type?
OpenCL async_work_group_copy() is designed to copy N elements of a basic data type. However it does not really know what is being copied. So you can tell it to copy N bytes, containing any type inside (including structs). Similar to memcpy().
You can do:
kernel void krnl(global X* restrict x){
global const Y* l = x[a].y;
local Y l2;
size_t sol2 = sizeof(Y);
async_work_group_copy((local char *)&l2, (global char *)l, sol2, 0);
}
However, remember that you need to declare local memory explicitly in side the kernel, or dinamically from the API side and pass a pointer to local memory.
You can't just create a local pointer without any initialization and copy there. (see my code)

forcing stack w/i 32bit when -m64 -mcmodel=small

have C sources that must compile in 32bit and 64bit for multiple platforms.
structure that takes the address of a buffer - need to fit address in a 32bit value.
obviously where possible these structures will use natural sized void * or char * pointers.
however for some parts an api specifies the size of these pointers as 32bit.
on x86_64 linux with -m64 -mcmodel=small tboth static data and malloc()'d data fit within the 2Gb range. data on the stack, however, still starts in high memory.
so given a small utility _to_32() such as:
int _to_32( long l ) {
int i = l & 0xffffffff;
assert( i == l );
return i;
}
then:
char *cp = malloc( 100 );
int a = _to_32( cp );
will work reliably, as would:
static char buff[ 100 ];
int a = _to_32( buff );
but:
char buff[ 100 ];
int a = _to_32( buff );
will fail the assert().
anyone have a solution for this without writing custom linker scripts?
or any ideas how to arrange the linker section for stack data, would appear it is being put in this section in the linker script:
.lbss :
{
*(.dynlbss)
*(.lbss .lbss.* .gnu.linkonce.lb.*)
*(LARGE_COMMON)
}
thanks!
The stack location is most likely specified by the operating system and has nothing to do with the linker.
I can't imagine why you are trying to force a pointer on a 64 bit machine into 32 bits. The memory layout of structures is mainly important when you are sharing the data with something which may run on another architecture and saving to a file or sending across a network, but there are almost no valid reasons that you would send a pointer from one computer to another. Debugging is the only valid reason that comes to mind.
Even storing a pointer to be used later by another run of your program on the same machine would almost certainly be wrong since where your program is loaded can differ. Making any use of such a pointer would be undefined abd unpredictable.
the short answer appears to be there is no easy answer. at least no easy way to reassign range/location of the stack pointer.
the loader 'ld-linux.so' at a very early stage in process activation gets the address in the hurd loader - in the glibc sources, elf/ and sysdeps/x86_64/ search out elf_machine_load_address() and elf_machine_runtime_setup().
this happens in the preamble of calling your _start() entry and related setup to call your main(), is not for the faint hearted, even i couldn't convince myself this was a safe route.
as it happens - the resolution presents itself in some other old school tricks... pointer deflations/inflation...
with -mcmodel=small then automatic variables, alloca() addresses, and things like argv[], and envp are assigned from high memory from where the stack will grow down. those addresses are verified in this example code:
#include <stdlib.h>
#include <stdio.h>
#include <alloca.h>
extern char etext, edata, end;
char global_buffer[128];
int main( int argc, const char *argv[], const char *envp )
{
char stack_buffer[128];
static char static_buffer[128];
char *cp = malloc( 128 );
char *ap = alloca( 128 );
char *xp = "STRING CONSTANT";
printf("argv[0] %p\n",argv[0]);
printf("envp %p\n",envp);
printf("stack %p\n",stack_buffer);
printf("global %p\n",global_buffer);
printf("static %p\n",static_buffer);
printf("malloc %p\n",cp);
printf("alloca %p\n",ap);
printf("const %p\n",xp);
printf("printf %p\n",printf);
printf("First address past:\n");
printf(" program text (etext) %p\n", &etext);
printf(" initialized data (edata) %p\n", &edata);
printf(" uninitialized data (end) %p\n", &end);
}
produces this output:
argv[0] 0x7fff1e5e7d99
envp 0x7fff1e5e6c18
stack 0x7fff1e5e6a80
global 0x6010e0
static 0x601060
malloc 0x602010
alloca 0x7fff1e5e69d0
const 0x400850
printf 0x4004b0
First address past:
program text (etext) 0x400846
initialized data (edata) 0x601030
uninitialized data (end) 0x601160
all access to/from the 32bit parts of structures must be wrapped with inflate() and deflate() routines, e.g.:
void *inflate( unsigned long );
unsigned int deflate( void *);
deflate() tests for bits set in the range 0x7fff00000000 and marks the pointer so that inflate() will recognize how to reconstitute the actual pointer.
hope that helps if anyone similarly must support structures with 32bit storage for 64bit pointers.

Resources