I'm trying to implement some draws using a polya urn scheme using Rcpp. Basically, I have a matrix I'm drawing from, and a 2nd matrix with weights proportional to the probabilities. After each draw, I need to increase the weight of whichever cell I drew.
I was running into some indexing errors which lead me to examine the sampling more generally, and I found that my weight matrix was getting modified by RcppArmadillo::sample. Two questions (1) is this behavior that I should have expected or is this a bug which I should report somewhere? (2) Any ideas on current work-around? Here's a reproducible example:
#include <RcppArmadilloExtensions/sample.h>
// [[Rcpp::depends(RcppArmadillo)]]
using namespace Rcpp ;
// [[Rcpp::export]]
void sampler(int N, int inc, NumericMatrix& weight_matrix, int reps) {
IntegerVector wm_tmp = seq_along(weight_matrix);
Rcout << "Initial weight_matrix:\n" << weight_matrix << "\n";
int x_ind;
for(int i = 0; i < reps; ++i) {
x_ind = RcppArmadillo::sample(wm_tmp, 1, true, weight_matrix)(0) - 1;
Rcout << "Weight matrix after sample: (rep = " << i << ")\n" << weight_matrix << "\n";
Rcout << "x_ind: " << x_ind << "\n";
// get indices
weight_matrix[x_ind] = weight_matrix[x_ind] + inc;
Rcout << "Add increment of " << inc << " to weight_matrix:\n" << weight_matrix << "\n";
}
}
//
// // [[Rcpp::export]]
// IntegerVector seq_cpp(IntegerMatrix x) {
// IntegerVector tmp = seq_along(x);
// IntegerVector ret = RcppArmadillo::sample(tmp, 2, true);
// return ret;
// }
/*** R
weight_matrix <- matrix(1, 5, 2)
sampler(5, 1, weight_matrix, 3)
weight_matrix <- matrix(1, 5, 2)
sampler(5, 0, weight_matrix, 3)
*/
Thanks!
That is known and documented behaviour.
You could do
i) Use Rcpp::clone() to create a distinct copy of your SEXP (ie NumericMatrix).
ii) Use an Armadillo matrix instead and pass as const arma::mat & m.
There are architectural reasons having to do with the way R organizes its data structure which mean that we cannot give you fast access (no copies!) and also protect against writes.
Related
in an R script I source a cpp file to make some calculations. In that R script, a function defined in the cpp file is called and a matrix and an integer is provided. After a few rounds through the loop it gives the error "Not a matrix" (in line of code resid = (x(_,j) - x(_,i))*(x(_,j) - x(_,i));), even though for the rounds before it worked.
R script:
## all together
# rm(list=ls())
library(RcppArmadillo)
library(Rcpp)
sourceCpp("~/test.cpp",verbose = FALSE)
cat("start loop")
for(n in c(45:46)){
cat("\n", n, "\n")
p_m <- matrix(data=rnorm(n^2,1,1),nrow = n, ncol=n)
print(class(p_m))
print(some_function(p_m,nosamples=10))
}
cat("finished")
I start this R script via the command line. R version R-4.1.0. In R-Studio it crashes with a fatal error.
The cpp file:
// [[Rcpp::depends(RcppArmadillo)]]
#include <RcppArmadillo.h>
using namespace Rcpp;
// [[Rcpp::export]]
NumericVector some_function(NumericMatrix x,int nosamples) {
int ncol = x.ncol();
NumericVector out2(nosamples);
int loops;
int loops2;
double result=0;
NumericVector::iterator it;
double acc = 0;
NumericVector resid(ncol);
NumericVector out(ncol*(ncol-1)/2);
loops2=0;
std::cout << nosamples << std::endl;
std::cout << (ncol-1) << std::endl;
std::cout << ncol*(ncol-1)/2 << std::endl;
while(loops2 < (nosamples)){
std::cout << "loops2:" << std::endl;
std::cout << loops2 << std::endl;
loops=0;
int i;
int j;
for(j=0;j<(ncol-1);++j){
std::cout << " j: " << j << std::endl;
for (i = (j+1); i < (ncol); ++i) {
std::cout << " i: " << i << std::endl;
resid = (x(_,j) - x(_,i))*(x(_,j) - x(_,i)); //here it stops
std::cout << " i: " << i << std::endl;
for(int ii=0; ii<ncol;++ii){
acc += resid[i];
}
result=sqrt(acc);
loops += 1;
out[loops] = result;
std::cout << " i: " << i << std::endl;
}
}
std::cout << "loops:" << std::endl;
std::cout << loops << std::endl;
out = out[out > 0];
it = std::min_element(out.begin(), out.end());
out= *it;
std::cout << out << std::endl;
loops2 += 1;
out2[loops2]=out[0];
}
std::cout << "cpp finished" << std::endl;
return(out2);
}
Can someone explain what the problem is about?
Thanks and kind regards
Edit
I adapted some things in the cpp file (shown below) and the error disappeared. First I thought, everything is fine. But when I increase the number of loops, another problem occurs: the function breaks, but no error is shown. It breaks after loop number 543 ("loop2: 543"). At least it does the same in each round of the while loop with the same data.
I adapted the R-script and the ccp file to make this problem (at least on my machine) reproducible.
I know this code seems to be somehow meaningless, but it is part of a bigger program and I wanted to give here a minimum example.
The R script:
## all together
# rm(list=ls())
library(RcppArmadillo)
library(Rcpp)
sourceCpp("~/test.cpp",verbose = FALSE)
cat("start loop")
for(n in c(100:101)){
cat("\n", n, "\n")
p_m <- matrix(data=rnorm(n^2,1,1),nrow = n, ncol=n)
print(class(p_m))
print(some_function(p_m,nosamples=800))
}
cat("finished")
The cpp file:
// [[Rcpp::depends(RcppArmadillo)]]
// [[Rcpp::depends(RcppEigen)]]
#include <RcppArmadillo.h>
#include <RcppEigen.h>
using namespace Rcpp;
using Eigen::Map;
using Eigen::VectorXd;
typedef Map<VectorXd> MapVecd;
// [[Rcpp::export]]
NumericVector some_function(NumericMatrix x,int nosamples) {
int ncol = x.ncol();
NumericVector out(ncol*(ncol-1)/2);
NumericVector out2(nosamples);
NumericVector out3(ncol*(ncol-1)/2);
NumericVector resid(ncol);
int loops;
int loops2;
double result=0;
double acc = 0;
int show_cout=0;
loops2=0;
std::cout << nosamples << std::endl;
std::cout << (ncol-1) << std::endl;
std::cout << ncol*(ncol-1)/2 << std::endl;
while(loops2 < (nosamples)){
std::cout << "loops2:" << loops2 << std::endl;
loops=0;
int i;
int j;
for(j=0;j<(ncol-1);++j){
// std::cout << " j: " << j << std::endl;
for (i = (j+1); i < (ncol); ++i) {
if(show_cout==1){
std::cout << " i: " << i << std::endl;
}
resid = (x(_,j) - x(_,i))*(x(_,j) - x(_,i));
if(show_cout==1){
std::cout << " i: " << i << std::endl;
}
for(int ii=0; ii<ncol;++ii){
acc += resid[ii];
}
result=sqrt(acc);
loops += 1;
out[loops] = result;
if(show_cout==1){
std::cout << " i: " << i << std::endl;
}
}
}
// std::cout << "loops:" << loops << std::endl;
//
out = out[out > 0];
const MapVecd xy(as<MapVecd>(out));
out3=xy.minCoeff();
out2[loops2]=out3[0];
loops2 += 1;
}
std::cout << "cpp finished" << std::endl;
return(out2);
}
Two things here:
Use out[loops++] = result; instead of loops += 1; out[loops] = result; because you were starting at 1, and probably accessing the last element outside of the range of this vector.
Use
for(int ii=0; ii<ncol;++ii){ double eps = x(ii, j) - x(ii, i); acc += eps * eps; }
instead of relying on this resid vector.
To generate standard uniform random numbers with Rcpp, I always used
Rcpp::runif(1, 0, 1)[0]
The [0] is due to Rcpp::runif returning vectors. I recently found that you can also use the R API and use R::runif() instead if you only want a scalar so I can avoid using this [0]. I tried this out, but I always get nan's. Here is a small example, the .cpp file:
#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
double unif_R() {
const double u = R::runif(1, 0);
Rcout << "u: " << u << "\n";
return u;
}
// [[Rcpp::export]]
double unif_Rcpp() {
const double u = Rcpp::runif(1, 0, 1)[0];
Rcout << "u: " << u << "\n";
return u;
}
The unif_R() function uses R::runif(a,b) that returns a scalar, while unif_Rcpp() uses Rcpp::runif(n, a, b), which returns a vector. However, when I call these in R using:
sourceCpp('runif_test.cpp')
set.seed(21)
unif_R()
set.seed(21)
unif_Rcpp()
set.seed(21)
runif(1)
I get the following output:
> set.seed(21)
> unif_R()
u: nan
[1] NaN
> set.seed(21)
> unif_Rcpp()
u: 0.786115
[1] 0.7861149
> set.seed(21)
> runif(1)
[1] 0.7861149
Clearly unif_Rcpp\ works, but why does unif_R\ give me nan's?
As noted by #user20650, I made a silly mistake and that I need b >= a. So the correct code should've been:
#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
double unif_R() {
const double u = R::runif(0, 1);
Rcout << "u: " << u << "\n";
return u;
}
// [[Rcpp::export]]
double unif_Rcpp() {
const double u = Rcpp::runif(1, 0, 1)[0];
Rcout << "u: " << u << "\n";
return u;
}
And this is fine.
I am trying to extract NARF keypoints and descriptors for raw pointcloud data using pcl::NarfKeypoint , pcl::NarfDescriptor.
In visualization process, I can simply plot my keypoints along with the range image generated from the original ponitcloud.
The problem, however, deals with visualizing descriptors along with keypoints.
As far as I have understood, Narf computes all indices for keypoints and using getVector3fMap(), one can simply visualize them with pcl::visualization.
When it comes to descriptors, the output would be x, y, z, roll, pitch, yaw and more importantly descriptors[36].
Does anyone know how to visualize the descriptors with keypoints in PCL?
Do we really need to utilize those 36 points in descriptors[36] to address this problem?
My sample code:
// --------------------------------
// -----Extract NARF keypoints-----
// --------------------------------
clock_t begin = clock();
pcl::RangeImageBorderExtractor range_image_border_extractor;
pcl::NarfKeypoint narfKp (&range_image_border_extractor);
narfKp.setRangeImage (&range_image);
narfKp.getParameters().support_size = support_size;
narfKp.getParameters().calculate_sparse_interest_image = true;
narfKp.getParameters().use_recursive_scale_reduction = true;
pcl::PointCloud<int> keyPoIdx;
narfKp.compute (keyPoIdx);
cout << "range image = " << range_image << "\n \n";
cout << "keypoint = "<< keyPoIdx <<"\n";
cout << "time to compute NARF keyPoints = " << (float)(clock() - begin) / CLOCKS_PER_SEC << " [sec] \n";
// --------------------------------
// ----Extract NARF descriptors----
// --------------------------------
vector<int> desIdx;
desIdx.resize(keyPoIdx.points.size());
for (unsigned int i = 0; i < desIdx.size(); i++)
{
desIdx[i] = keyPoIdx.points[i];
}
pcl::NarfDescriptor narfDes (&range_image, &desIdx);
narfDes.getParameters().support_size = support_size;
narfDes.getParameters().rotation_invariant = true; // cause more descriptors than keypoints
pcl::PointCloud<pcl::Narf36> outputNarfDes;
narfDes.compute(outputNarfDes);
cout << "Extracted "<< outputNarfDes.size() <<" descriptors for " << keyPoIdx.points.size() << " keypoints.\n";
//------------------------------------------------------------------ //
//-----------------------Visualization-------------------------------//
// ----------------------------------------------------------------- //
// ----------------------------------------------
// -----Show keypoints in range image widget-----
// ----------------------------------------------
//for (size_t i=0; i<keyPoIdx.points.size (); ++i)
//range_image_widget.markPoint (keyPoIdx.points[i]%range_image.width,
//keyPoIdx.points[i]/range_image.width);
// ---------------------------------------
// -----Show Descriptors in 3D viewer-----
// ---------------------------------------
pcl::PointCloud<pcl::PointXYZ>::Ptr descriptors_ptr (new pcl::PointCloud<pcl::PointXYZ>);
pcl::PointCloud<pcl::PointXYZ>& desVIZ = *descriptors_ptr;
desVIZ.points.resize(outputNarfDes.size());
cout << "descriptor index size = " << desVIZ.points.size() << "\n";
for (size_t i=0; i < desVIZ.points.size(); ++i)
//for (size_t i=0; i<desIdx.size(); ++i)
{
// ??????????????? MY PROBLEM ???????????????????
desVIZ.points[i].getVector3fMap () = range_image.points[outputNarfDes.points[i]].getVector3fMap ();
// ??????????????? MY PROBLEM ???????????????????
}
pcl::visualization::PointCloudColorHandlerCustom<pcl::PointXYZ> des_color_handler (descriptors_ptr, 200, 0, 50);
viewer.addPointCloud<pcl::PointXYZ> (descriptors_ptr, des_color_handler, "descriptors");
viewer.setPointCloudRenderingProperties (pcl::visualization::PCL_VISUALIZER_POINT_SIZE, 10, "descriptors");
// -------------------------------------
// -----Show keypoints in 3D viewer-----
// -------------------------------------
pcl::PointCloud<pcl::PointXYZ>::Ptr keypoints_ptr (new pcl::PointCloud<pcl::PointXYZ>);
pcl::PointCloud<pcl::PointXYZ>& keyPo = *keypoints_ptr;
keyPo.points.resize(keyPoIdx.points.size());
for (size_t i=0; i<keyPoIdx.points.size(); ++i)
{
keyPo.points[i].getVector3fMap () = range_image.points[keyPoIdx.points[i]].getVector3fMap ();
}
pcl::visualization::PointCloudColorHandlerCustom<pcl::PointXYZ> keypoints_color_handler (keypoints_ptr, 0, 200, 0);
viewer.addPointCloud<pcl::PointXYZ> (keypoints_ptr, keypoints_color_handler, "keypoints");
viewer.setPointCloudRenderingProperties (pcl::visualization::PCL_VISUALIZER_POINT_SIZE, 5, "keypoints");
//--------------------
// -----Main loop-----
//--------------------
while (!viewer.wasStopped ())
{
range_image_widget.spinOnce (); // process GUI events
viewer.spinOnce ();
pcl_sleep(0.01);
}
You can visualize the roll/pitch/yaw, by converting it to a vector. Refer to this answer for details. This vector can either be used as the normal of each points - that way, for each view, only keypoints sharing the same orientation will have a color. Alternatively you may try to draw arrows in the position of the keypoints.
To visualize the descriptor, you can project it to a three dimensional space using PCA. Than, it can be used to set the color of your keypoints.
I am trying to generate a random binary matrix and its inverse mod q where q is a power of 2. Sometimes when the determinant of my matrix is invertible modulo q (so the matrix over Z_q is invertible), I am getting the error "InvMod:inverse undefined Aborted (core dumped)" and other times the inverse is computed. What am I doing incorrectly?
#include <iostream>
//NTL files
#include <NTL/ZZ_p.h>
#include <NTL/vec_vec_ZZ_p.h>
#include <NTL/LLL.h>
#include <NTL/matrix.h>
#include <NTL/vector.h>
#include <NTL/tools.h>
#include <NTL/ZZ.h>
#include <NTL/vec_vec_ZZ.h>
using namespace std;
using namespace NTL;
int main(){//task generate a random matrix S with 0/1 entries stored as a ZZ_p matrix, then generate a random, invertible S
int nn = 8;
ZZ n = ZZ(nn);
ZZ N = ZZ(0);
ZZ q; power2(q, 4);
ZZ_p::init(q);
mat_ZZ S; S.SetDims(nn,nn);
for(int i = 0; i<nn; i++){
for(int j = 0; j<nn; j++){
S[i][j] = RandomBits_ZZ(1);
}
}
mat_ZZ_p S1; S1.SetDims(nn,nn);//copy to ZZ_P
mat_ZZ_p R; R.SetDims(nn,nn);//will set to inverse if
cout<<"The random matrix is S = "<<endl; //print S
for(int i = 0; i<nn; i++){
for(int j=0; j<n;j++){
cout<<S[i][j]<<", ";
} cout<<endl;
}
ZZ d; determinant(d,S); ZZ_p d1; conv(d1, d % q);
if(GCD(q,d) == 1){//convert to mod q datatype
for(int i = 0; i<nn; i++){
for(int j = 0; j<nn; j++){
conv(S1[i][j], S[i][j]);
}
}
//let's invert the matrix and print it!
cout<<"The random matrix is R = "<<endl; //print R
R = inv(S1); //mul(R,R,S1);
for(int i = 0; i<nn; i++){
for(int j=0; j<n;j++){
cout<<R[i][j]<<", ";
} cout<<endl;
}
}
cout<<endl<<"det of S is "<<d<<" and this mod q is "<<d1<<endl;
cout<<"Our modulus is "<< q <<endl;
return 0;
}
If the determinant is invertable mod q this only means that there exists an inverse matrix. But the algorithm that computes this matrix can still come to a point where it would need to calculate the inverse of an element that don't has one.
You don't have this problem if q is prime.
By the way, here is a simplified version of your code.
#include <iostream>
//NTL files
#include <NTL/mat_ZZ_p.h>
using namespace std;
using namespace NTL;
int main()
{//task generate a random matrix S with 0/1 entries stored as a ZZ_p matrix, then generate a random, invertible S
int nn = 8;
ZZ q;
power2(q, 4);
ZZ_p::init(q);
mat_ZZ_p S;
S.SetDims(nn, nn);
for(int i = 0; i < nn; i++)
{
for(int j = 0; j < nn; j++)
{
S[i][j] = conv<ZZ_p>(RandomBits_ZZ(1));
}
}
mat_ZZ_p R;
R.SetDims(nn, nn);//will set to inverse if
cout << "The random matrix is S = " << endl << S;
ZZ_p d;
determinant(d, S);
cout << endl << "det(S) = " << d << endl;
cout << "q = " << q << endl;
if(GCD(conv<ZZ>(d), q) == 1)
{
// let's invert the matrix and print it!
R = inv(S);
cout << "The random matrix is R = " << R << endl;
}
return 0;
}
I made a small reproducible example:
#include <Rcpp.h>
using namespace Rcpp;
class Index {
public:
Index(int i_) : i(i_) {}
int getI() { return i; }
private:
int i;
};
// [[Rcpp::export]]
SEXP getXPtrIndex(int i) {
Rcout << "getXPtrIndex: i = " << i << std::endl;
Index ind(i);
Rcout << "getXPtrIndex: ind.i = " << ind.getI() << std::endl;
return XPtr<Index>(&ind, true);
}
// [[Rcpp::export]]
void getXPtrIndexValue(SEXP ptr) {
XPtr<Index> ind_ptr(ptr);
Rcout << "getXPtrIndexValue: ind_ptr->i = " << ind_ptr->getI() << std::endl;
Index ind = *ind_ptr;
Rcout << "getXPtrIndexValue: ind.i = " << ind.getI() << std::endl;
}
Basically, I define a small class, along with a function to get an external pointer of an element of this class. The last function is used to print the weird accessor when returning the class element back to C++.
Results in R:
> (extptr <- getXPtrIndex(10))
getXPtrIndex: i = 10
getXPtrIndex: ind.i = 10
<pointer: 0x7ffeeec31b00>
> getXPtrIndexValue(extptr)
getXPtrIndexValue: ind_ptr->i = 33696400
getXPtrIndexValue: ind.i = 0
Why can't I access 10?
I'm using Rcpp version 0.12.12 (the latest I think).
It seems to have something to do with the temporary object---by the time your second function runs the "content" of the first is already gone.
So either just make
Index ind(10);
a global, and comment out the line in your first function. Then all is peachy (I changed the R invocation slightly):
R> extptr <- getXPtrIndex(10)
getXPtrIndex: i = 10
getXPtrIndex: ind.i = 10
R> getXPtrIndexValue(extptr)
getXPtrIndexValue: ind_ptr->i = 10
getXPtrIndexValue: ind.i = 10
R>
Or it also works the same way when you make you Index object static to ensure persistence. Corrected example below.
#include <Rcpp.h>
using namespace Rcpp;
class Index {
public:
Index(int i_) : i(i_) {}
int getI() { return i; }
private:
int i;
};
// [[Rcpp::export]]
SEXP getXPtrIndex(int i) {
Rcout << "getXPtrIndex: i = " << i << std::endl;
static Index ind(i);
Rcout << "getXPtrIndex: ind.i = " << ind.getI() << std::endl;
return XPtr<Index>(&ind, true);
}
// [[Rcpp::export]]
void getXPtrIndexValue(SEXP ptr) {
XPtr<Index> ind_ptr(ptr);
Rcout << "getXPtrIndexValue: ind_ptr->i = " << ind_ptr->getI() << std::endl;
Index ind = *ind_ptr;
Rcout << "getXPtrIndexValue: ind.i = " << ind.getI() << std::endl;
}
/*** R
extptr <- getXPtrIndex(10)
getXPtrIndexValue(extptr)
*/