Unable to install pyrocksdb against RocksDB master - rocksdb

I followed the instructions from here: http://pyrocksdb.readthedocs.org/en/latest/installation.html
When I try pip install git+git://github.com/stephan-hof/pyrocksdb.git I get the following output with error:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.4
creating build/lib.linux-x86_64-3.4/rocksdb
copying rocksdb/__init__.py -> build/lib.linux-x86_64-3.4/rocksdb
copying rocksdb/interfaces.py -> build/lib.linux-x86_64-3.4/rocksdb
copying rocksdb/errors.py -> build/lib.linux-x86_64-3.4/rocksdb
creating build/lib.linux-x86_64-3.4/rocksdb/tests
copying rocksdb/tests/__init__.py -> build/lib.linux-x86_64-3.4/rocksdb/tests
copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-3.4/rocksdb/tests
copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-3.4/rocksdb/tests
running egg_info
creating pyrocksdb.egg-info
writing dependency_links to pyrocksdb.egg-info/dependency_links.txt
writing top-level names to pyrocksdb.egg-info/top_level.txt
writing requirements to pyrocksdb.egg-info/requires.txt
writing pyrocksdb.egg-info/PKG-INFO
writing manifest file 'pyrocksdb.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
reading manifest file 'pyrocksdb.egg-info/SOURCES.txt'
writing manifest file 'pyrocksdb.egg-info/SOURCES.txt'
copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-3.4/rocksdb
running build_ext
building 'rocksdb._rocksdb' extension
creating build/temp.linux-x86_64-3.4
creating build/temp.linux-x86_64-3.4/rocksdb
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/home/iulian/py3env/include -I/usr/include/python3.4m -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-3.4/rocksdb/_rocksdb.o -std=gnu++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
rocksdb/_rocksdb.cpp:289:27: fatal error: rocksdb/slice.h: No such file or directory
#include "rocksdb/slice.h"
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Command "/home/iulian/py3env/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-mndoekm3-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-3fwa2_ff-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/iulian/py3env/include/site/python3.4" failed with error code 1 in /tmp/pip-mndoekm3-build
I tried to checkout various versions of RocksDB (3.5, 3.6, 3.8, 3.9) and still no luck.
I am using Python 3.4 on a 64 bit Ubuntu 14.04.

You have to check that Cython==0.20 is installed! I had the same issue and that solved it!

Related

Compiling C++ with R using sourceCPP [duplicate]

I can’t figure out how to give my R package’s shared library’s debug symbols source line information. What am I missing?
I create the following src/Makevars file:
PKG_CXXFLAGS=-O0 -ggdb
PKG_LIBS=-O0 -ggdb
I compile the package using R CMD INSTALL --no-multiarch --with-keep.source:
* installing to library ‘~/.local/lib/R/3.6’
* installing *source* package ‘reticulate’ ...
** using staged installation
g++ -std=gnu++11 -I"/usr/include/R/" -DNDEBUG -I"$HOME/.local/lib/R/3.6/Rcpp/include" -D_FORTIFY_SOURCE=2 -O0 -ggdb -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c RcppExports.cpp -o RcppExports.o
** libs
g++ -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -o reticulate.so RcppExports.o event_loop.o libpython.o output.o python.o readline.o -O0 -ggdb -L/usr/lib64/R/lib -lR
installing to ~/.local/lib/R/3.6/00LOCK-reticulate/00new/reticulate/libs
I debug like this:
R -d gdb --slave -e 'reticulate::py_eval("print")()'
GNU gdb (GDB) 8.3
[...]
(No debugging symbols found in /usr/lib64/R/bin/exec/R)
(gdb) break py_get_formals
Function "py_get_formals" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (py_get_formals) pending.
(gdb) run
Starting program: /usr/lib/R/bin/exec/R --slave -e reticulate::py_eval\(\"print\"\)\(\)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[...]
Thread 1 "R" hit Breakpoint 1, 0x00007fffeb6b79a0 in py_get_formals(PyObjectRef, bool) () from /home/angerer/.local/lib/R/3.6/reticulate/libs/reticulate.so
(gdb) step
Single stepping until exit from function _Z14py_get_formals11PyObjectRefb,
which has no line number information.
[...]
Why does my function not have line numbers even though I specified -ggdb in both compilation? I see that only RcppExports.cpp is mentioned in the command line, is that the problem? If so, how can I change this?
Changing the Makevars doesn’t prompt recompilation.
I needed to rm -f src/*.o src/*.so before the object files get recompiled.
This is specifically for Windows. The simplest way to do it is to set the R_MAKEVARS_USER environment to point to the Makevars.win file. That seems to work. However, debug break points have stopped working!!!!

Compile a Rcpp package with line information in the debugging symbols

I can’t figure out how to give my R package’s shared library’s debug symbols source line information. What am I missing?
I create the following src/Makevars file:
PKG_CXXFLAGS=-O0 -ggdb
PKG_LIBS=-O0 -ggdb
I compile the package using R CMD INSTALL --no-multiarch --with-keep.source:
* installing to library ‘~/.local/lib/R/3.6’
* installing *source* package ‘reticulate’ ...
** using staged installation
g++ -std=gnu++11 -I"/usr/include/R/" -DNDEBUG -I"$HOME/.local/lib/R/3.6/Rcpp/include" -D_FORTIFY_SOURCE=2 -O0 -ggdb -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c RcppExports.cpp -o RcppExports.o
** libs
g++ -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -o reticulate.so RcppExports.o event_loop.o libpython.o output.o python.o readline.o -O0 -ggdb -L/usr/lib64/R/lib -lR
installing to ~/.local/lib/R/3.6/00LOCK-reticulate/00new/reticulate/libs
I debug like this:
R -d gdb --slave -e 'reticulate::py_eval("print")()'
GNU gdb (GDB) 8.3
[...]
(No debugging symbols found in /usr/lib64/R/bin/exec/R)
(gdb) break py_get_formals
Function "py_get_formals" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (py_get_formals) pending.
(gdb) run
Starting program: /usr/lib/R/bin/exec/R --slave -e reticulate::py_eval\(\"print\"\)\(\)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[...]
Thread 1 "R" hit Breakpoint 1, 0x00007fffeb6b79a0 in py_get_formals(PyObjectRef, bool) () from /home/angerer/.local/lib/R/3.6/reticulate/libs/reticulate.so
(gdb) step
Single stepping until exit from function _Z14py_get_formals11PyObjectRefb,
which has no line number information.
[...]
Why does my function not have line numbers even though I specified -ggdb in both compilation? I see that only RcppExports.cpp is mentioned in the command line, is that the problem? If so, how can I change this?
Changing the Makevars doesn’t prompt recompilation.
I needed to rm -f src/*.o src/*.so before the object files get recompiled.
This is specifically for Windows. The simplest way to do it is to set the R_MAKEVARS_USER environment to point to the Makevars.win file. That seems to work. However, debug break points have stopped working!!!!

Torch '!./install.sh' returns error in Colab when installing Torch

I followed http://torch.ch/docs/getting-started.html#_ to install Torch.
I always get the following error when running
!./install.sh
in google colab
error:
...
-- Installing: /root/torch/install/include/luaT.h
-- Installing: /root/torch/install/share/cmake/torch/luaTConfig.cmake
Updating manifest for /root/torch/install/lib/luarocks/rocks
torch scm-1 is now built and installed in /root/torch/install/ (license: BSD)
Updating manifest for /root/torch/install/lib/luarocks/rocks
dok scm-1 is now built and installed in /root/torch/install/ (license: BSD)
gcc -O2 -fPIC -I/root/torch/install/include -c utils.c -o utils.o
gcc -shared -o treplutils.so -L/root/torch/install/lib utils.o
gcc -O2 -fPIC -I/root/torch/install/include -c readline.c -o readline.o
readline.c:8:10: fatal error: readline/readline.h: No such file or directory
#include <readline/readline.h>
^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Error: Build error: Failed compiling object readline.o
.
Please I need help fixing this error.
Probably you need something like
!apt-get install libreadline-dev
before attempting to run the install.sh script.

Linux issue happens on Windows? "fatal error: fftw3.h: No such file or directory"

I am running MRO 3.5.0 on Win10, and trying to install a package image.CannyImage from bnosac/image on the github. But it keeps reporting fatal errors as following.
* installing *source* package 'image.CannyEdges' ...
** libs
c:/Rtools/mingw_64/bin/g++ -m64 -I"C:/PROGRA~1/MICROS~1/ROPEN~1/R-35~1.0/include" -DNDEBUG -I"C:/Users/eric1/Documents/R/win-library/3.5/Rcpp/include" -I"C:/swarm/workspace/External-R-3.5.0/vendor/extsoft/include" -O2 -Wall -mtune=core2 -c RcppExports.cpp -o RcppExports.o
c:/Rtools/mingw_64/bin/gcc -m64 -I"C:/PROGRA~1/MICROS~1/ROPEN~1/R-35~1.0/include" -DNDEBUG -I"C:/Users/eric1/Documents/R/win-library/3.5/Rcpp/include" -I"C:/swarm/workspace/External-R-3.5.0/vendor/extsoft/include" -O2 -Wall -std=gnu99 -mtune=core2 -c adsf.c -o adsf.o
c:/Rtools/mingw_64/bin/g++ -m64 -I"C:/PROGRA~1/MICROS~1/ROPEN~1/R-35~1.0/include" -DNDEBUG -I"C:/Users/eric1/Documents/R/win-library/3.5/Rcpp/include" -I"C:/swarm/workspace/External-R-3.5.0/vendor/extsoft/include" -O2 -Wall -mtune=core2 -c rcpp_canny.cpp -o rcpp_canny.o
In file included from rcpp_canny.cpp:13:0:
canny.h:8:19: fatal error: fftw3.h: No such file or directory
#include <fftw3.h>
^
compilation terminated.
make: *** [C:/PROGRA~1/MICROS~1/ROPEN~1/R-35~1.0/etc/x64/Makeconf:215: rcpp_canny.o] Error 1
ERROR: compilation failed for package 'image.CannyEdges'
* removing 'C:/Users/eric1/Documents/R/win-library/3.5/image.CannyEdges'
In R CMD INSTALL
Installation failed: Command failed (1)
^
compilation terminated.
Google says I can solve the issue by sudo apt-get remove libfftw3-dev. Unfortunately, I am on Windows, and some advised, apply lib /machine:i386 /def:libfftw3-3.def on cmd mode. Ouch, there is no such command or file called lib.exe on Win10.
Please advise, how I can solve this issue on my Win10+MRO system. Thanks.
Even though this question is already a bit old:
By now (Feb 2020) there are precompiled packages available that work also under Windows:
Canny Edges - Package
and more general:
List of all available packages
I have tried installing them and (at least for me) it worked.

building a mapnik 2.2.0 RPM on RHEL 7

I am trying to build a mapnik 2.2.0 RPM file on my RHEL/CENTOS 7 machine.
so I took the RPM SRC file from Fedora 22 which is mapnik-2.2.0-11.fc21.src.rpm ( I have the same result with mapnik-2.2.0-5.fc20.src.rpm )
when I am running the rpmbuild I receive the following error message:
Welcome to Mapnik...
scons: warning: Ignoring missing SConscript 'deps/agg/build.py'
File "/root/rpmbuild/BUILD/mapnik-v2.2.0/SConstruct", line 1799, in <module>
scons: warning: Ignoring missing SConscript 'deps/mapnik/build.py'
File "/root/rpmbuild/BUILD/mapnik-v2.2.0/SConstruct", line 1808, in <module>
scons: done reading SConscript files.
scons: Building targets ...
g++ -o bindings/python/mapnik_building_symbolizer.os -c -ansi -Wall -pthread -ftemplate-depth-300 -O3 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fno-strict-aliasing -finline-functions -Wno-inline -Wno-parentheses -Wno-char-subscripts -fPIC -DHAVE_JPEG -DMAPNIK_USE_PROJ4 -DHAVE_PNG -DHAVE_TIFF -DBIGINT -DBOOST_REGEX_HAS_ICU -DLINUX -DMAPNIK_THREADSAFE -DNDEBUG -DHAVE_CAIRO -DHAVE_PYCAIRO -I. -Iinclude -I/usr/include/polyclipping -I/usr/include/agg2 -I/usr/include -I/usr/include/freetype2 -I/usr/include/libxml2 -I/usr/include/gdal -I/usr/include/python2.7 -I/usr/include/cairo -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/libpng15 -I/usr/include/libdrm -I/usr/include/pycairo bindings/python/mapnik_building_symbolizer.cpp
In file included from include/mapnik/symbolizer.hpp:29:0,
from include/mapnik/building_symbolizer.hpp:29,
from bindings/python/mapnik_building_symbolizer.cpp:26:
include/mapnik/image_compositing.hpp:74:19: error: 'comp_op_grain_merge' is not a member of 'agg'
grain_merge = agg::comp_op_grain_merge,
^
include/mapnik/image_compositing.hpp:75:21: error: 'comp_op_grain_extract' is not a member of 'agg'
grain_extract = agg::comp_op_grain_extract,
^
scons: *** [bindings/python/mapnik_building_symbolizer.os] Error 1
scons: building terminated because of errors.
error: Bad exit status from /var/tmp/rpm-tmp.kuI6KW (%build)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.kuI6KW (%build)
Mapnik normally comes with a heavily patched version of the AGG library right there in its source tree. The Fedora build however disables that in favour of having the Mapnik-AGG patches directly in its own AGG package (because they don't like duplicating system libraries elsewhere).
You either need an AGG library for CentOS that is feature-compatible with the patched Fedora version, or you have to modify your Mapnik build to disable the "system_agg" patch (and drop the "rm -rf boost deps" from the spec file).

Resources