windows version of R package much slower than linux version - r

I'm running ubuntu, 64bit. I have this minimal test package that i made to learn how to do these things (I'm following this tutorial, except i also have some c code in the package).
The package build/runs in linux so i set about making it run in windows too.
I followed this answer and used the online windows package builder maintained by Uwe Ligges to get a (working) zip version of my package.
Now, when i install that .zip package on windows (7-64) the small demo code runs slower than the linux version. As in 30 times slower. I doubt the difference is always so large.
I'm wondering what i'm doing wrong and how i can fix this gap.
EDIT:
this is the source code (it's a minimal working example):
#include <algorithm>
#include <cstdlib>
#include <ctime>
#include <functional>
#include <fstream>
#include <iostream>
#include <math.h>
#include <time.h>
#include <stdio.h>
#include <stdlib.h>
#include <sstream>
#include <vector>
#include <Eigen/Dense>
#include <Eigen/LU>
#include <Eigen/SVD>
using namespace std;
using namespace Eigen;
using Eigen::MatrixXf;
using Eigen::VectorXf;
float median(VectorXf& x) {
int n=x.rows();
int half=(n+1)/2;
half--;
float med;
nth_element(x.data(),x.data()+half,x.data()+x.size());
if((n%2)==1){
med=x(half);
} else {
float tmp0=x(half);
float tmp1=x.segment(half+1,half-1).minCoeff();
med=0.5*(tmp0+tmp1);
}
return med;
}
VectorXf fx01(MatrixXf& x){
int p=x.cols();
int n=x.rows();
VectorXf Recept(n);
VectorXf Result(p);
for(int i=0;i<p;i++){
Recept=x.col(i);
Result(i)=median(Recept);
}
return Result;
}
extern "C"{
void mse(int* n,int* p,float* x,float* medsout){
MatrixXf x_cen=Map<MatrixXf>(x,*n,*p);
VectorXf MedsOut=fx01(x_cen);
Map<VectorXf>(medsout,*p)=MedsOut.array();
}
}
EDIT2:
Following cbeleites suggestion i ran the code multiple times. Doing this I found
a strange thing: the function's timing are actually the same as linux except when
i call apply() before calling my function --I was always comparing the timing
of the colwise median my pack computes to the timing of doing apply(X,2,median)--
Ok, problem solved. For now. Still i'm curious now: why would a good old fashioned
call to apply() (on a huge matrix X) wreck things so badly (system.time went from
90sec to 3sec)?

One possibility that comes to my mind where calculations on different machines can differ about is the BLAS (if there are linear algebra calculations in the example).
Do you have an optimized BLAS installed on Ubuntu (e.g. libopenblas) but not on Windows?

Related

Error in including boost tokenizer.hpp file when building R package on MacOS/CRAN

I'm running into issues publishing my R package to CRAN, because of a specific error when including boost libraries. The top of my one .cpp file in the package is
#include <Rcpp.h>
#include <boost/tokenizer.hpp>
#include <boost/algorithm/string.hpp>
#include <algorithm>
#include <string>
#include <unordered_map>
#include <omp.h>
#include <vector>
// [[Rcpp::depends(BH)]]
// [[Rcpp::plugins(openmp)]]
When running through the check on MacOS (via rhub::check(platform = "macos-highsierra-release-cran"), I get the following error:
In file included from wgt_jaccard.cpp:6:
/Users/user2suimGYX/R/BH/include/boost/tokenizer.hpp:63:9: error: field of type 'std::_1::wrap_iter<const char *>' has private constructor
: first(c.begin()), last(c.end()), f(f) { }
^
wgt_jaccard.cpp:117:19: note: in instantiation of function template specialization 'boost::tokenizer<boost::char_separator<char, std::__1::char_traits >, std::__1::__wrap_iter<const char *>, std::__1::basic_string ::tokenizer<Rcpp::internal::string_proxy<16, PreserveStorage> >' requested here
tokenizer tokens(y(i), sep);
^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/iterator:1420:31: note: declared private here
_LIBCPP_INLINE_VISIBILITY __wrap_iter(iterator_type __x) _NOEXCEPT_DEBUG : __i(__x) {}
^
My Makevars file's contents are
PKG_CXXFLAGS = $(SHLIB_OPENMP_CXXFLAGS) -DBOOST_NO_AUTO_PTR
PKG_LIBS = $(SHLIB_OPENMP_CXXFLAGS)
CXX_STD = CXX11
I've tried searching around, but can't find much on this error. The full package is located here. Any help is much appreciated.
Relative to the repo you kindly supplied, we found a need for two changes:
First, to no (unconditionally) have an #include <omp.h> as OpenMP can be optional, esp. on macOS. A simpled #ifdef OPENMP does the job.
Second, the (arguably near-incomprehensible) compiler message had to do with the fact that the Boost type / class for tokenizer was puzzled by the Rcpp object you gave it by directly indexing from an Rcpp::CharacterVector. Been there, done that -- a more conservative approach is to first assign to std::string and to then pass that on.
With those two changes, it's all roses and it compiles on macOS under clang++ as well.
The (by now merged, thanks) PR #2 has the gory details, but is short.

How to make a c++ code can be used in both Qt project (QTL style) and C++ project (STL style)

Is there any way to convert C++(STL) code to QT (QTL)?
I have a code written using STL (string, vector, iostream, fstream) and
I want to add it to my Qt Project (QString, QVector, ... when used).
My ideal way is to make it QTL-STL-compatible by adding some defines
in .h file like following
// STL to QTL
#ifdef STL_CPP
#include <string>
#include<vector>
#else // QTL
#include <QString>
#define string QString
#include <QVector>
#define vector QVector
#endif
Is there any existing work can convert some most commonly used STL code without
changing the source code ?
Note that I don't want to convert std datatype to qt datatype, what I want is to
make a code can used in both Qt project(ifndef STL_CPP) and C++ project (ifdef STL_CPP).
Using Macro Substitution to convert STL-style code to QTL-style code is dangerous
and may cause many unexpected errors.
To avoid that, you can use the STL in QT code and convert when interacting with the Qt framework (for a price paid in performance),
convert STL datatype to QTL datatype is quite simple, just like QVector::fromStdVector(const std::vector<T> &vector) and QString::fromStdString(const std::string &str) mentioned by drescherjm

PCL1.7.2 in QT5.4.1 --- error: C2589, C2059, C2181 in PCL header file

I tried to include my friend's code into my project.
The original code highly uses PCL 1.7.2 so I,
1.Installed PCL 1.7.2 package(not built with source code)
2.Edit INCLUDEPATH and LIBS in myproject.pro file, according to my friend's settings in Visual Studio 2013(yes, he wrote the code under VS2013). e.g
and more lines of path like above.
3.Include the header file in mainwindow.h
#include <opencv2/opencv.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv/cv.h>
#include <opencv/highgui.h>
#include <boost/shared_ptr.hpp>
#include <pcl/point_types.h>
#include <pcl/point_cloud.h>
#include <pcl/io/ply_io.h>
#include <pcl/io/pcd_io.h>
#include <pcl/kdtree/kdtree_flann.h>
#include <pcl/surface/mls.h>
#include <pcl/filters/statistical_outlier_removal.h>
#include <pcl/filters/voxel_grid.h>
#include <pcl/registration/icp.h>
#include <pcl/filters/radius_outlier_removal.h>
#include <pcl/features/normal_3d.h>
#include <pcl/surface/gp3.h>
#include <cv.h>
#include <cxcore.h>
using namespace cv;
using namespace std;
using namespace pcl;
4.Build(Compile) the project, and bunch of errors showed up
The errors are C2589, C2059 and C2181, basically lie in PCL headers like io_operators.h, pcl_io.h, correspondence.h, cloud_iterator.h. I have totally no clue of these, does this means I have to edit PCL's header files?
Those error codes are quite often related to the Windows issue warning C4003 and errors C2589 and C2059 on: x = std::numeric_limits<int>::max();
Indeed there is std::numeric_limits<>::max() in io_operators.h:66 and pcl_io.h:281.
So, if it is the only problem it can be fixed by changing order of headers by moving pcl headers on top to avoid including windows.h (that may be included from some other header) before or by defining NOMINMAX before all inclusions.

Arduino: error: 'abs' was not declared in this scope

I'm working on an arduino library needing the abs() function:
#include <math.h>
normTransFreq1 = abs(1.0);
Error: 'abs' was not declared in this scope
As math.h is already included in the cpp I'm entirely unsure how to fix this problem. A new install of arduino 1.5.2 didn't help.
Just found the solution:
Including math.h is not needed for the library. Instead, Arduino.h should be included by adding the following to the header file:
#if ARDUINO >= 100
#include "Arduino.h"
#else
#include "WProgram.h"
#endif

Is there a way for a C binary code to figure out where it is stored? [duplicate]

Is there a platform-agnostic and filesystem-agnostic method to obtain the full path of the directory from where a program is running using C/C++? Not to be confused with the current working directory. (Please don't suggest libraries unless they're standard ones like clib or STL.)
(If there's no platform/filesystem-agnostic method, suggestions that work in Windows and Linux for specific filesystems are welcome too.)
Here's code to get the full path to the executing app:
Variable declarations:
char pBuf[256];
size_t len = sizeof(pBuf);
Windows:
int bytes = GetModuleFileName(NULL, pBuf, len);
return bytes ? bytes : -1;
Linux:
int bytes = MIN(readlink("/proc/self/exe", pBuf, len), len - 1);
if(bytes >= 0)
pBuf[bytes] = '\0';
return bytes;
If you fetch the current directory when your program first starts, then you effectively have the directory your program was started from. Store the value in a variable and refer to it later in your program. This is distinct from the directory that holds the current executable program file. It isn't necessarily the same directory; if someone runs the program from a command prompt, then the program is being run from the command prompt's current working directory even though the program file lives elsewhere.
getcwd is a POSIX function and supported out of the box by all POSIX compliant platforms. You would not have to do anything special (apart from incliding the right headers unistd.h on Unix and direct.h on windows).
Since you are creating a C program it will link with the default c run time library which is linked to by ALL processes in the system (specially crafted exceptions avoided) and it will include this function by default. The CRT is never considered an external library because that provides the basic standard compliant interface to the OS.
On windows getcwd function has been deprecated in favour of _getcwd. I think you could use it in this fashion.
#include <stdio.h> /* defines FILENAME_MAX */
#ifdef WINDOWS
#include <direct.h>
#define GetCurrentDir _getcwd
#else
#include <unistd.h>
#define GetCurrentDir getcwd
#endif
char cCurrentPath[FILENAME_MAX];
if (!GetCurrentDir(cCurrentPath, sizeof(cCurrentPath)))
{
return errno;
}
cCurrentPath[sizeof(cCurrentPath) - 1] = '\0'; /* not really required */
printf ("The current working directory is %s", cCurrentPath);
This is from the cplusplus forum
On windows:
#include <string>
#include <windows.h>
std::string getexepath()
{
char result[ MAX_PATH ];
return std::string( result, GetModuleFileName( NULL, result, MAX_PATH ) );
}
On Linux:
#include <string>
#include <limits.h>
#include <unistd.h>
std::string getexepath()
{
char result[ PATH_MAX ];
ssize_t count = readlink( "/proc/self/exe", result, PATH_MAX );
return std::string( result, (count > 0) ? count : 0 );
}
On HP-UX:
#include <string>
#include <limits.h>
#define _PSTAT64
#include <sys/pstat.h>
#include <sys/types.h>
#include <unistd.h>
std::string getexepath()
{
char result[ PATH_MAX ];
struct pst_status ps;
if (pstat_getproc( &ps, sizeof( ps ), 0, getpid() ) < 0)
return std::string();
if (pstat_getpathname( result, PATH_MAX, &ps.pst_fid_text ) < 0)
return std::string();
return std::string( result );
}
If you want a standard way without libraries: No. The whole concept of a directory is not included in the standard.
If you agree that some (portable) dependency on a near-standard lib is okay: Use Boost's filesystem library and ask for the initial_path().
IMHO that's as close as you can get, with good karma (Boost is a well-established high quality set of libraries)
I know it is very late at the day to throw an answer at this one but I found that none of the answers were as useful to me as my own solution. A very simple way to get the path from your CWD to your bin folder is like this:
int main(int argc, char* argv[])
{
std::string argv_str(argv[0]);
std::string base = argv_str.substr(0, argv_str.find_last_of("/"));
}
You can now just use this as a base for your relative path. So for example I have this directory structure:
main
----> test
----> src
----> bin
and I want to compile my source code to bin and write a log to test I can just add this line to my code.
std::string pathToWrite = base + "/../test/test.log";
I have tried this approach on Linux using full path, alias etc. and it works just fine.
NOTE:
If you are on windows you should use a '\' as the file separator not '/'. You will have to escape this too for example:
std::string base = argv[0].substr(0, argv[0].find_last_of("\\"));
I think this should work but haven't tested, so comment would be appreciated if it works or a fix if not.
Filesystem TS is now a standard ( and supported by gcc 5.3+ and clang 3.9+ ), so you can use current_path() function from it:
std::string path = std::experimental::filesystem::current_path();
In gcc (5.3+) to include Filesystem you need to use:
#include <experimental/filesystem>
and link your code with -lstdc++fs flag.
If you want to use Filesystem with Microsoft Visual Studio, then read this.
No, there's no standard way. I believe that the C/C++ standards don't even consider the existence of directories (or other file system organizations).
On Windows the GetModuleFileName() will return the full path to the executable file of the current process when the hModule parameter is set to NULL. I can't help with Linux.
Also you should clarify whether you want the current directory or the directory that the program image/executable resides. As it stands your question is a little ambiguous on this point.
On Windows the simplest way is to use the _get_pgmptr function in stdlib.h to get a pointer to a string which represents the absolute path to the executable, including the executables name.
char* path;
_get_pgmptr(&path);
printf(path); // Example output: C:/Projects/Hello/World.exe
Maybe concatenate the current working directory with argv[0]? I'm not sure if that would work in Windows but it works in linux.
For example:
#include <stdio.h>
#include <unistd.h>
#include <string.h>
int main(int argc, char **argv) {
char the_path[256];
getcwd(the_path, 255);
strcat(the_path, "/");
strcat(the_path, argv[0]);
printf("%s\n", the_path);
return 0;
}
When run, it outputs:
jeremy#jeremy-desktop:~/Desktop$ ./test
/home/jeremy/Desktop/./test
For Win32 GetCurrentDirectory should do the trick.
You can not use argv[0] for that purpose, usually it does contain full path to the executable, but not nessesarily - process could be created with arbitrary value in the field.
Also mind you, the current directory and the directory with the executable are two different things, so getcwd() won't help you either.
On Windows use GetModuleFileName(), on Linux read /dev/proc/procID/.. files.
Just my two cents, but doesn't the following code portably work in C++17?
#include <iostream>
#include <filesystem>
namespace fs = std::filesystem;
int main(int argc, char* argv[])
{
std::cout << "Path is " << fs::path(argv[0]).parent_path() << '\n';
}
Seems to work for me on Linux at least.
Based on the previous idea, I now have:
std::filesystem::path prepend_exe_path(const std::string& filename, const std::string& exe_path = "");
With implementation:
fs::path prepend_exe_path(const std::string& filename, const std::string& exe_path)
{
static auto exe_parent_path = fs::path(exe_path).parent_path();
return exe_parent_path / filename;
}
And initialization trick in main():
(void) prepend_exe_path("", argv[0]);
Thanks #Sam Redway for the argv[0] idea. And of course, I understand that C++17 was not around for many years when the OP asked the question.
Just to belatedly pile on here,...
there is no standard solution, because the languages are agnostic of underlying file systems, so as others have said, the concept of a directory based file system is outside the scope of the c / c++ languages.
on top of that, you want not the current working directory, but the directory the program is running in, which must take into account how the program got to where it is - ie was it spawned as a new process via a fork, etc. To get the directory a program is running in, as the solutions have demonstrated, requires that you get that information from the process control structures of the operating system in question, which is the only authority on this question. Thus, by definition, its an OS specific solution.
#include <windows.h>
using namespace std;
// The directory path returned by native GetCurrentDirectory() no end backslash
string getCurrentDirectoryOnWindows()
{
const unsigned long maxDir = 260;
char currentDir[maxDir];
GetCurrentDirectory(maxDir, currentDir);
return string(currentDir);
}
For Windows system at console you can use system(dir) command. And console gives you information about directory and etc. Read about the dir command at cmd. But for Unix-like systems, I don't know... If this command is run, read bash command. ls does not display directory...
Example:
int main()
{
system("dir");
system("pause"); //this wait for Enter-key-press;
return 0;
}
Works with starting from C++11, using experimental filesystem, and C++14-C++17 as well using official filesystem.
application.h:
#pragma once
//
// https://en.cppreference.com/w/User:D41D8CD98F/feature_testing_macros
//
#ifdef __cpp_lib_filesystem
#include <filesystem>
#else
#include <experimental/filesystem>
namespace std {
namespace filesystem = experimental::filesystem;
}
#endif
std::filesystem::path getexepath();
application.cpp:
#include "application.h"
#ifdef _WIN32
#include <windows.h> //GetModuleFileNameW
#else
#include <limits.h>
#include <unistd.h> //readlink
#endif
std::filesystem::path getexepath()
{
#ifdef _WIN32
wchar_t path[MAX_PATH] = { 0 };
GetModuleFileNameW(NULL, path, MAX_PATH);
return path;
#else
char result[PATH_MAX];
ssize_t count = readlink("/proc/self/exe", result, PATH_MAX);
return std::string(result, (count > 0) ? count : 0);
#endif
}
For relative paths, here's what I did. I am aware of the age of this question, I simply want to contribute a simpler answer that works in the majority of cases:
Say you have a path like this:
"path/to/file/folder"
For some reason, Linux-built executables made in eclipse work fine with this. However, windows gets very confused if given a path like this to work with!
As stated above there are several ways to get the current path to the executable, but the easiest way I find works a charm in the majority of cases is appending this to the FRONT of your path:
"./path/to/file/folder"
Just adding "./" should get you sorted! :) Then you can start loading from whatever directory you wish, so long as it is with the executable itself.
EDIT: This won't work if you try to launch the executable from code::blocks if that's the development environment being used, as for some reason, code::blocks doesn't load stuff right... :D
EDIT2: Some new things I have found is that if you specify a static path like this one in your code (Assuming Example.data is something you need to load):
"resources/Example.data"
If you then launch your app from the actual directory (or in Windows, you make a shortcut, and set the working dir to your app dir) then it will work like that.
Keep this in mind when debugging issues related to missing resource/file paths. (Especially in IDEs that set the wrong working dir when launching a build exe from the IDE)
A library solution (although I know this was not asked for).
If you happen to use Qt:
QCoreApplication::applicationDirPath()
Path to the current .exe
#include <Windows.h>
std::wstring getexepathW()
{
wchar_t result[MAX_PATH];
return std::wstring(result, GetModuleFileNameW(NULL, result, MAX_PATH));
}
std::wcout << getexepathW() << std::endl;
// -------- OR --------
std::string getexepathA()
{
char result[MAX_PATH];
return std::string(result, GetModuleFileNameA(NULL, result, MAX_PATH));
}
std::cout << getexepathA() << std::endl;
This question was asked 15 years ago, so the existing answers are now incorrect. If you're using C++17 or greater, the solution is very straightforward today:
#include <filesystem>
std::cout << std::filesystem::current_path();
See cppreference.com for more information.
On POSIX platforms, you can use getcwd().
On Windows, you may use _getcwd(), as use of getcwd() has been deprecated.
For standard libraries, if Boost were standard enough for you, I would have suggested Boost::filesystem, but they seem to have removed path normalization from the proposal. You may have to wait until TR2 becomes readily available for a fully standard solution.
Boost Filesystem's initial_path() behaves like POSIX's getcwd(), and neither does what you want by itself, but appending argv[0] to either of them should do it.
You may note that the result is not always pretty--you may get things like /foo/bar/../../baz/a.out or /foo/bar//baz/a.out, but I believe that it always results in a valid path which names the executable (note that consecutive slashes in a path are collapsed to one).
I previously wrote a solution using envp (the third argument to main() which worked on Linux but didn't seem workable on Windows, so I'm essentially recommending the same solution as someone else did previously, but with the additional explanation of why it is actually correct even if the results are not pretty.
As Minok mentioned, there is no such functionality specified ini C standard or C++ standard. This is considered to be purely OS-specific feature and it is specified in POSIX standard, for example.
Thorsten79 has given good suggestion, it is Boost.Filesystem library. However, it may be inconvenient in case you don't want to have any link-time dependencies in binary form for your program.
A good alternative I would recommend is collection of 100% headers-only STLSoft C++ Libraries Matthew Wilson (author of must-read books about C++). There is portable facade PlatformSTL gives access to system-specific API: WinSTL for Windows and UnixSTL on Unix, so it is portable solution. All the system-specific elements are specified with use of traits and policies, so it is extensible framework. There is filesystem library provided, of course.
The linux bash command
which progname will report a path to program.
Even if one could issue the which command from within your program and direct the output to a tmp file and the program
subsequently reads that tmp file, it will not tell you if that program is the one executing. It only tells you where a program having that name is located.
What is required is to obtain your process id number, and to parse out the path to the name
In my program I want to know if the program was
executed from the user's bin directory or from another in the path
or from /usr/bin. /usr/bin would contain the supported version.
My feeling is that in Linux there is the one solution that is portable.
Use realpath() in stdlib.h like this:
char *working_dir_path = realpath(".", NULL);
The following worked well for me on macOS 10.15.7
brew install boost
main.cpp
#include <iostream>
#include <boost/filesystem.hpp>
int main(int argc, char* argv[]){
boost::filesystem::path p{argv[0]};
p = absolute(p).parent_path();
std::cout << p << std::endl;
return 0;
}
Compiling
g++ -Wall -std=c++11 -l boost_filesystem main.cpp

Resources