to ccall a custom made library, I need to write down the library full path on the system:
j = ccall((:add3, "[FULL_PATH]/libmylib.so"), Float32, (Float32,), 2)
I am trying to use instead a relative path with:
j = ccall((:add3, "$(pwd())/libmylib.so"), Float32, (Float32,), 2)
but, while "$(pwd())/libmylib.so" returns the right path for the library, ccall with pwd returns a TypeError: in ccall: first argument not a pointer or valid constant expression, expected Ptr, got Tuple{Symbol,String}.
So, how to ccall a library that is in the same folder than the Julia script/current working directory ?
I am puzzled, as according to this thread in Windows seems to work, even if the doc for ccall specify:
Note that the argument type tuple must be a literal tuple, and not a tuple-valued variable or expression.
For info, I'm in Ubuntu 18.04 and the library has been implemented with
mylib.c:
float add3 (float i){
return i+3;
}
mylib.h:
#ifndef _MYLIB_H_
#define _MYLIB_H_
extern float get2 ();
extern float add3 (float i);
Compilation (gcc):
gcc -o mylib.o -c mylib.c
gcc -shared -o libmylib.so mylib.o -lm -fPIC
As far as I know this is the most used pattern:
const mylib = joinpath(pwd(), "libmylib.so")
j = ccall((:add3, mylib), Cfloat, (Cfloat,), 2)
Note that pwd can be a bit more in "flux" than you want for a library path, it is probably better to relate it to the file, e.g.
const mylib = joinpath(#__DIR__, "libmylib.so")
where #__DIR__ expands to the directory of the file itself.
As often, I found the solution only after posting on SO.. it seems setting the question helps defining better the problem..
Any how, the solution is to first get the function pointer with cglobal - where I can use pwd() - and then use the ccall method with the function pointer:
f = cglobal((:add3, "$(pwd())/libmylib.so"))
j = ccall(f, Float32, (Float32,), i)
Related
I am currently learning C and am trying to understand the possibilities of dynamic libraries.
My current question is, if I have a simple "Hello World" application in C called "ProgA", and this program dynamically loads a shared library with some example code called "LibB", can LibB access a global variable in ProgA, which was declared as external?
Given is the following example code for demonstration of the problem:
file header.h
#ifndef TEST_H
#define TEST_H
typedef struct test_import_s {
int some_field;
} test_import_t;
extern test_import_t newtestimport;
#endif
file prog_a.c
#include <stdio.h>
#include <windows.h>
#include "header.h"
test_import_t newtestimport = {
.some_field = 42
};
int main()
{
HINSTANCE hinstLib;
typedef void (*FunctionPointer)();
newtestimport.some_field = 42;
hinstLib = LoadLibrary("lib_b.dll");
if (hinstLib != NULL)
{
FunctionPointer initialize_lib_b;
initialize_lib_b = (FunctionPointer)GetProcAddress(hinstLib, "initialize_lib_b");
if (initialize_lib_b != NULL)
{
initialize_lib_b();
}
FreeLibrary(hinstLib);
}
return 0;
}
file lib_b.c
#include <stdio.h>
#include "header.h"
test_import_t *timp;
void initialize_lib_b() {
timp = &newtestimport;
int some_field = timp->some_field;
printf("Result from function: %d\n", some_field);
}
file CMakeLists.txt
cmake_minimum_required(VERSION 3.24)
project(dynamic-library-2 C)
set(CMAKE_C_STANDARD 23)
add_library(lib_b SHARED lib_b.c)
set_target_properties(lib_b PROPERTIES PREFIX "" OUTPUT_NAME "lib_b")
add_executable(prog_a prog_a.c)
target_link_libraries(prog_a lib_b)
In the above example, the headerfile header.h defines the struct test_import_t and an external variable newtestimport using this struct. In the C file of the main program prog_a.c one property of this struct is assigned the value 42. It then dynamically loads the library lib_b.c using the Windows API and executes a function in it. The function then should access the variable newtestimport of the main program and print out the value of the variable (42).
This example does not work. The compiler throws the following error:
====================[ Build | prog_a | Debug ]==================================
C:\Users\user1\AppData\Local\JetBrains\Toolbox\apps\CLion\ch-0\223.8617.54\bin\cmake\win\x64\bin\cmake.exe --build C:\Users\user1\projects\learning-c\cmake-build-debug --target prog_a -j 9
[1/2] Linking C shared library dynamic-library-2\lib_b.dll
FAILED: dynamic-library-2/lib_b.dll dynamic-library-2/liblib_b.dll.a
cmd.exe /C "cd . && C:\Users\user1\AppData\Local\JetBrains\Toolbox\apps\CLion\ch-0\223.8617.54\bin\mingw\bin\gcc.exe -fPIC -g -Wl,--export-all-symbols -shared -o dynamic-library-2\lib_b.dll -Wl,--out-implib,dynamic-library-2\liblib_b.dll.a -Wl,--major-image-version,0,--minor-image-version,0 dynamic-library-2/CMakeFiles/lib_b.dir/lib_b.c.obj -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 && cd ."
C:\Users\user1\AppData\Local\JetBrains\Toolbox\apps\CLion\ch-0\223.8617.54\bin\mingw\bin/ld.exe: dynamic-library-2/CMakeFiles/lib_b.dir/lib_b.c.obj:lib_b.c:(.rdata$.refptr.newtestimport[.refptr.newtestimport]+0x0): undefined reference to `newtestimport'
collect2.exe: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
How can the example be fixed to accomplish the described goal?
Windows DLLs are self-contained, and can not have undefined references similar to newtestimport, unless these references are satisfied by another DLL.
How can the example be fixed to accomplish the described goal?
The best fix is to pass the address of newtestimport into the function that needs it (initialize_lib_b() here).
If for some reason you can't do that, your next best option is to define the newtestimport as a dllexport variable in another DLL, e.g. lib_c.dll.
Then both the main executable and lib_b.dll would be linked against lib_c.lib, and would both use that variable from lib_c.dll.
P.S. Global variables are a "code smell" and a significant source of bugs. You should avoid them whenever possible, and in your example there doesn't seem to be any good reason to use them.
I have dynamic library created as follows
cat myfile.cc
struct Tcl_Interp;
extern "C" int My_Init(Tcl_Interp *) { return 0; }
1) complile the cc file
g++ -fPIC -c myfile.cc
2) Creating a shared library
g++ -static-libstdc++ -static-libgcc -shared -o libmy.so myfile.o -L/tools/linux64/qt-4.6.0/lib -lQtCore -lQtGui
3) load the library from a TCL proc
then I give command
tclsh
and given command
% load libmy.so
is there any C++ function/ Qt equivalent to load that can load the shared library on demand from another C++ function.
My requirement is to load the dynamic library on run time inside the function and then use the qt functions directly
1) load the qt shared libraries (for lib1.so)
2) call directly the functions without any call for resolve
For example we have dopen, but for that for each function call we have to call dsym. My requirement is only call for shared library then directly call those functions.
You want boilerplate-less delay loading. On Windows, MSVC implements delay loading by emitting a stub that resolves the function through a function pointer. You can do the same. First, let's observe that function pointers and functions are interchangeable if all you do is call them. The syntax for invoking a function or a function pointer is the same:
void foo_impl() {}
void (*foo)() = foo_impl;
int main() {
foo_impl();
foo();
}
The idea is to set the function pointer initially to a thunk that will resolve the real function at runtime:
extern void (*foo)();
void foo_thunk() {
foo = QLibrary::resolve("libmylib", "foo");
if (!foo) abort();
return foo();
}
void (*foo)() = foo_thunk;
int main() {
foo(); // calls foo_thunk to resolve foo and calls foo from libmylib
foo(); // calls foo from libmylib
}
When you first call foo, it will really call foo_thunk, resolve the function address, and call real foo implementation.
To do this, you can split the library into two libraries:
The library implementation. It is unaware of demand-loading.
A demand-load stub.
The executable will link to the demand-load stub library; that is either static or dynamic. The demand-load stub will automatically resolve the symbols at runtime and call into the implementation.
If you're clever, you can design the header for the implementation such that the header itself can be used to generate all the stubs without having to enter their details twice.
Complete Example
Everything follows, it's also available from https://github.com/KubaO/stackoverflown/tree/master/questions/demand-load-39291032
The top-level project consists of:
lib1 - the dynamic library
lib1_demand - the static demand-load thunk for lib1
main - the application that uses lib1_demand
demand-load-39291032.pro
TEMPLATE = subdirs
SUBDIRS = lib1 lib1_demand main
main.depends = lib1_demand
lib1_demand.depends = lib1
We can factor out the cleverness into a separate header. This header allows us to define the library interface so that the thunks can be automatically generated.
The heavy use of preprocessor and a somewhat redundant syntax is needed due to limitations of C. If you wanted to implement this for C++ only, there'd be no need to repeat the argument list.
demand_load.h
// Configuration macros:
// DEMAND_NAME - must be set to a unique identifier of the library
// DEMAND_LOAD - if defined, the functions are declared as function pointers, **or**
// DEMAND_BUILD - if defined, the thunks and function pointers are defined
#if defined(DEMAND_FUN)
#error Multiple inclusion of demand_load.h without undefining DEMAND_FUN first.
#endif
#if !defined(DEMAND_NAME)
#error DEMAND_NAME must be defined
#endif
#if defined(DEMAND_LOAD)
// Interface via a function pointer
#define DEMAND_FUN(ret,name,args,arg_call) \
extern ret (*name)args;
#elif defined(DEMAND_BUILD)
// Implementation of the demand loader stub
#ifndef DEMAND_CAT
#define DEMAND_CAT_(x,y) x##y
#define DEMAND_CAT(x,y) DEMAND_CAT_(x,y)
#endif
void (* DEMAND_CAT(resolve_,DEMAND_NAME)(const char *))();
#if defined(__cplusplus)
#define DEMAND_FUN(ret,name,args,arg_call) \
extern ret (*name)args; \
ret name##_thunk args { \
name = reinterpret_cast<decltype(name)>(DEMAND_CAT(resolve_,DEMAND_NAME)(#name)); \
return name arg_call; \
}\
ret (*name)args = name##_thunk;
#else
#define DEMAND_FUN(ret,name,args,arg_call) \
extern ret (*name)args; \
ret name##_impl args { \
name = (void*)DEMAND_CAT(resolve_,DEMAND_NAME)(#name); \
name arg_call; \
}\
ret (*name)args = name##_impl;
#endif // __cplusplus
#else
// Interface via a function
#define DEMAND_FUN(ret,name,args,arg_call) \
ret name args;
#endif
Then, the dynamic library itself:
lib1/lib1.pro
TEMPLATE = lib
SOURCES = lib1.c
HEADERS = lib1.h
INCLUDEPATH += ..
DEPENDPATH += ..
Instead of declaring the functions directly, we'll use DEMAND_FUN from demand_load.h. If DEMAND_LOAD_LIB1 is defined when the header is included, it will offer a demand-load interface to the library. If DEMAND_BUILD is defined, it'll define the demand-load thunks. If neither is defined, it will offer a normal interface.
We take care to undefine the implementation-specific macros so that the global namespace is not polluted. We can then include multiple libraries the project, each one individually selectable between demand- and non-demand loading.
lib1/lib1.h
#ifndef LIB_H
#define LIB_H
#ifdef __cplusplus
extern "C" {
#endif
#define DEMAND_NAME LIB1
#ifdef DEMAND_LOAD_LIB1
#define DEMAND_LOAD
#endif
#include "demand_load.h"
#undef DEMAND_LOAD
DEMAND_FUN(int, My_Add, (int i, int j), (i,j))
DEMAND_FUN(int, My_Subtract, (int i, int j), (i,j))
#undef DEMAND_FUN
#undef DEMAND_NAME
#ifdef __cplusplus
}
#endif
#endif
The implementation is uncontroversial:
lib1/lib1.c
#include "lib1.h"
int My_Add(int i, int j) {
return i+j;
}
int My_Subtract(int i, int j) {
return i-j;
}
For the user of such a library, demand loading is reduced to defining one macro and using the thunk library lib1_demand instead of the dynamic library lib1.
main/main.pro
if (true) {
# Use demand-loaded lib1
DEFINES += DEMAND_LOAD_LIB1
LIBS += -L../lib1_demand -llib1_demand
} else {
# Use direct-loaded lib1
LIBS += -L../lib1 -llib1
}
QT = core
CONFIG += console c++11
CONFIG -= app_bundle
TARGET = demand-load-39291032
TEMPLATE = app
INCLUDEPATH += ..
DEPENDPATH += ..
SOURCES = main.cpp
main/main.cpp
#include "lib1/lib1.h"
#include <QtCore>
int main() {
auto a = My_Add(1, 2);
Q_ASSERT(a == 3);
auto b = My_Add(3, 4);
Q_ASSERT(b == 7);
auto c = My_Subtract(5, 7);
Q_ASSERT(c == -2);
}
Finally, the implementation of the thunk. Here we have a choice between using dlopen+dlsym or QLibrary. For simplicity, I opted for the latter:
lib1_demand/lib1_demand.pro
QT = core
TEMPLATE = lib
CONFIG += staticlib
INCLUDEPATH += ..
DEPENDPATH += ..
SOURCES = lib1_demand.cpp
HEADERS = ../demand_load.h
lib1_demand/lib1_demand.cpp
#define DEMAND_BUILD
#include "lib1/lib1.h"
#include <QLibrary>
void (* resolve_LIB1(const char * name))() {
auto f = QLibrary::resolve("../lib1/liblib1", name);
return f;
}
Quite apart from the process of loading a library into your C++ code (which Kuber Ober's answer covers just fine) the code that you are loading is wrong; even if you manage to load it, your code will crash! This is because you have a variable of type Tcl_Interp at file scope; that's wrong use of the Tcl library. Instead, the library provides only one way to obtain a handle to an interpreter context, Tcl_CreateInterp() (and a few other functions that are wrappers round it), and that returns a Tcl_Interp* that has already been initialised correctly. (Strictly, it actually returns a handle to what is effectively an internal subclass of Tcl_Interp, so you really can't usefully allocate one yourself.)
The correct usage of the library is this:
Tcl_FindExecutable(NULL); // Or argv[0] if you have it
Tcl_Interp *interp = Tcl_CreateInterp();
// And now, you can use the rest of the API as you see fit
That's for putting a Tcl interpreter inside your code. To do it the other way round, you create an int My_Init(Tcl_Interp*) function as you describe and it is used to tell you where the interpreter is, but then you wouldn't be asking how to load the code, as Tcl has reasonable support for that already.
I'm trying to compile the sitmo prng under C++11 within an R package. The problematic code has been packaged and is available here. The objective of this R package is to make available the sitmo header file so that other packages are able to use the LinkTo field within description. As an added bonus, the package is scheduled to ship with an Armadillo + OpenMP example. There is one other package, mvnfast, that uses sitmo, but only under c++98 and boost headers.
I believe that the error which I am receiving is specific to OS X and clang. I haven't been able to replicate it on Windows via win-build. With that being said, the error is:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/random:3641:44: error: non-type template argument is not a constant expression
const size_t __logR = __log2<uint64_t, _URNG::max() - _URNG::min() + uint64_t(1)>::value;
The error has only popped up on the Rcpp dev list. The resolution in this case was to compile under C++98 and use boost.
The above error is followed by the following notes:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/random:3773:18: note: in instantiation of function template specialization 'std::__1::generate_canonical<double, 53, sitmo::prng_engine>' requested here
* _VSTD::generate_canonical<_RealType, numeric_limits<_RealType>::digits>(__g)
^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/random:3737:17: note: in instantiation of function template specialization 'std::__1::uniform_real_distribution<double>::operator()<sitmo::prng_engine>' requested here
{return (*this)(__g, __p_);}
^
sitmo_test.cpp:77:26: note: in instantiation of function template specialization 'std::__1::uniform_real_distribution<double>::operator()<sitmo::prng_engine>' requested here
double u = distunif(engine);
^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/random:3641:44: note: non-constexpr function 'max' cannot be used in a constant expression
const size_t __logR = __log2<uint64_t, _URNG::max() - _URNG::min() + uint64_t(1)>::value;
^
../inst/include/prng_engine.hpp:100:23: note: declared here
static result_type (max)() { return 0xFFFFFFFF; }
The version of clang being used is:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.3.0
Thread model: posix
Looking into the code, there is a bug in the sitmo prng_engine.h. min() and max() were declared as
static result_type (min)() { return 0; }
static result_type (max)() { return 0xFFFFFFFF; }
If you take a look at, say, standard LCG max from here, you could see that it is declared constexpr, ditto for min.
As soon as you make those methods constexpr in the sitmo header file, I believe you could use them in template expression.
UPDATE
I've looked into GCC 5 headers, methods indeed are declared constexpr
I know that g++ support constexpr math function. I want to do that on clang++. So I write a simple code.
#include<iostream>
#include<cmath>
int main()
{
constexpr auto a(std::floor(4.3));
std::cout<<a<<std::endl;
}
and then use clang++-libc++ -std=c++1y to compile it and the get the following error:
error: constexpr variable 'a' must be initialized by a constant expression
constexpr auto a(std::floor(4.3));
^ ~~~~~~~~~~~~~~~
note: non-constexpr function 'floor' cannot be used in a constant expression
constexpr auto a(std::floor(4.3));
^
/usr/include/x86_64-linux-gnu/bits/mathcalls.h:184:14: note: declared here
__MATHCALLX (floor,, (_Mdouble_ __x), (__const__));
^
/usr/include/math.h:58:26: note: expanded from macro '__MATHCALLX'
__MATHDECLX (_Mdouble_,function,suffix, args, attrib)
^
/usr/include/math.h:60:22: note: expanded from macro '__MATHDECLX'
__MATHDECL_1(type, function,suffix, args) __attribute__ (attrib); \
^
/usr/include/math.h:63:31: note: expanded from macro '__MATHDECL_1'
extern type __MATH_PRECNAME(function,suffix) args __THROW
^
/usr/include/math.h:66:42: note: expanded from macro '__MATH_PRECNAME'
#define __MATH_PRECNAME(name,r) __CONCAT(name,r)
^
/usr/include/x86_64-linux-gnu/sys/cdefs.h:88:23: note: expanded from macro '__CONCAT'
#define __CONCAT(x,y) x ## y
I use clang-3.5. So I want to ask whether clang++ support constexpr math function. If so, what compiler flag I need to pass to clang?
cppreference doesn't declare std::floor as constexpr. Not sure whether any standard does. I guess compilers might want to avoid implementing this unless it's in some standard, to avoid incompatible behavior. According to the manual, clang aims for support of C++11 and C++1y (likely C++14), with no extensions of C++ features mentioned.
I have a newb problem with compiling and linking the kissfft library 'out of the box'. I've downloaded the kissfft library and extracted it to a test directory. Upon entering the directory and running 'make testall' I get the following errors, which look like the std c math library is not being linked to properly.
sharkllama#quaaludes:~/KISSFFT/kiss_fft129$ make testall
# The simd and int32_t types may or may not work on your machine
make -C test DATATYPE=simd CFLAGADD="" test
make[1]: Entering directory `/home/sharkllama/KISSFFT/kiss_fft129/test'
cd ../tools && make all
make[2]: Entering directory `/home/sharkllama/KISSFFT/kiss_fft129/tools'
cc -o fft_simd -Wall -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Waggregate-return -Wcast-align -Wcast-qual -Wnested-externs -Wshadow -Wbad-function-cast -Wwrite-strings -I.. -DUSE_SIMD=1 -msse -lm ../kiss_fft.c fftutil.c kiss_fftnd.c kiss_fftr.c kiss_fftndr.c
/tmp/ccFbS0yK.o: In function `kiss_fft_alloc':
kiss_fft.c:(.text+0xd17): undefined reference to `sincos'
kiss_fft.c:(.text+0xd6b): undefined reference to `floor'
kiss_fft.c:(.text+0xe07): undefined reference to `sincos'
kiss_fft.c:(.text+0xeba): undefined reference to `sqrt'
/tmp/ccbYqDcf.o: In function `kiss_fftr_alloc':
kiss_fftr.c:(.text+0x118): undefined reference to `sincos'
kiss_fftr.c:(.text+0x188): undefined reference to `sincos'
collect2: ld returned 1 exit status
make[2]: *** [fft_simd] Error 1
make[2]: Leaving directory `/home/sharkllama/KISSFFT/kiss_fft129/tools'
make[1]: *** [tools] Error 2
make[1]: Leaving directory `/home/sharkllama/KISSFFT/kiss_fft129/test'
make: *** [testall] Error 2
sharkllama#quaaludes:~/KISSFFT/kiss_fft129$
Clearly, the makefile is trying to link to the math library as the -lm option has been included. Can't make any sense of this. I've compiled numerous programs that properly link to the math library before. Any help would be appreciated.
Thanks,
-B
Kissfft is not really something you need to make and install like other libraries. If you need complex ffts, then all you need to do is compile the kiss_fft.c in your project. If you need something more specialized like multidimensional or real ffts, then you should also compile the apropriate file(s) from the tools dir.
The make targets are largely for development testing of kissfft. There are a lot of system requirements to do that testing. Unless you are changing the internals of kissfft, you won't need to use those testing targets.
Just wanted to share a practical example on how to build a simple application using 1D FFT/IFFT from kissfft:
g++ example.cpp -o example -I kissfft kissfft/kiss_fft.c
example.cpp:
#include "kissfft/kiss_fft.h"
int main()
{
// initialize input data for FFT
float input[] = { 11.0f, 3.0f, 4.05f, 9.0f, 10.3f, 8.0f, 4.934f, 5.11f };
int nfft = sizeof(input) / sizeof(float); // nfft = 8
// allocate input/output 1D arrays
kiss_fft_cpx* cin = new kiss_fft_cpx[nfft];
kiss_fft_cpx* cout = new kiss_fft_cpx[nfft];
// initialize data storage
memset(cin, 0, nfft * sizeof(kiss_fft_cpx));
memset(cout, 0, nfft * sizeof(kiss_fft_cpx));
// copy the input array to cin
for (int i = 0; i < nfft; ++i)
{
cin[i].r = input[i];
}
// setup the size and type of FFT: forward
bool is_inverse_fft = false;
kiss_fft_cfg cfg_f = kiss_fft_alloc(nfft, is_inverse_fft, 0, 0); // typedef: struct kiss_fft_state*
// execute transform for 1D
kiss_fft(cfg_f, cin , cout);
// transformed: DC is stored in cout[0].r and cout[0].i
printf("\nForward Transform:\n");
for (int i = 0; i < nfft; ++i)
{
printf("#%d %f %fj\n", i, cout[i].r, cout[i].i);
}
// setup the size and type of FFT: backward
is_inverse_fft = true;
kiss_fft_cfg cfg_i = kiss_fft_alloc(nfft, is_inverse_fft, 0, 0);
// execute the inverse transform for 1D
kiss_fft(cfg_i, cout, cin);
// original input data
printf("\nInverse Transform:\n");
for (int i = 0; i < nfft; ++i)
{
printf("#%d %f\n", i, cin[i].r / nfft); // div by N to scale data back to the original range
}
// release resources
kiss_fft_free(cfg_f);
kiss_fft_free(cfg_i);
delete[] cin;
delete[] cout;
return 0;
}
To use the 2D transforms, include the appropriate header "kissfft/tools/kiss_fftnd.h" and adjust the build command to:
g++ example.cpp -o example -I kissfft kissfft/kiss_fft.c kissfft/tools/kiss_fftnd.c
Simple enough!