I am trying to write a function r(d, n) in rcpp. The function returns n random draws from normal distribution N(0, d). This function should be well defined, therefore the function should return the same draws whenever the d and n do not change their value.
This won't be a problem if d is restricted to be integer, in which case I can set seed and do the job
// set seed
// [[Rcpp::export]]
void set_seed(unsigned int seed) {
Rcpp::Environment base_env("package:base");
Rcpp::Function set_seed_r = base_env["set.seed"];
set_seed_r(seed);
}
// function r(d, n)
// [[Rcpp::export]]
vec randdraw(int d, int n){
set_seed(d);
vec out = randn(n);
return out;
}
But clearly I don't want to restrict d to be integer. Ideally d should be double. Any thoughts? Thank you!
The issue that I think is happening is you are trying to disperse the randn offered by Armadillo that is restricted to being a standard normal, e.g. N(0,1), such that it matches N(0, d). There are two ways to go about this since it is a standard normal.
Option 1: Using Statistical Properties
The first way involves just multiplying the sample by the square root of d, e.g. sqrt(d)*sample. This is possible due to the random variable properties of variance and expectation giving sqrt(d)*N(0, 1) ~ N(0, sqrt(d)^2) ~ N(0, d).
One of the more important things to note here is that the set_seed() function will work since the Armadillo configuration of RcppArmadillo hooks into R's RNG library to access the ::Rf_runif function to generate random values. The only area of concern is you cannot use arma::arma_rng::set_seed() to set the seed due to limitations of the R/C++ interaction detailed in Section 6.3 of Writing R Extensions. If you do use this, then you would get warned with :
When called from R, the RNG seed has to be set at the R level via set.seed()
on the first detected call.
With this being said, here is a short code example where we multiple by sqrt(d).
Code:
#include <RcppArmadillo.h>
// [[Rcpp::depends(RcppArmadillo)]]
// set seed
// [[Rcpp::export]]
void set_seed(double seed) {
Rcpp::Environment base_env("package:base");
Rcpp::Function set_seed_r = base_env["set.seed"];
set_seed_r(std::floor(std::fabs(seed)));
}
// function r(d, n)
// [[Rcpp::export]]
arma::vec randdraw(double d, int n){
set_seed(d); // Set a seed for R's RNG library
// Call Armadillo's RNG procedure that references R's RNG capabilities
// and change dispersion slightly.
arma::vec out = std::sqrt(std::fabs(d))*arma::randn(n);
return out;
}
Output:
> randdraw(3.5, 5L)
[,1]
[1,] -0.8671559
[2,] -1.9507540
[3,] 2.9025090
[4,] -1.2953745
[5,] 2.0799176
Note: There is no direct equivalent as the rnorm procedure differs from the arma::randn generation.
Option 2: Rely upon R's RNG Functions
The second, and significantly better solution, is to explicitly rely upon R's RNG functions. Previously, we made an implicit use of R's RNG library due to RcppArmadillo's configuration. I tend to prefer this approach as you have already made an assumption that the code is specific to R when using the set_seed() function (Disclaimer: I wrote the post). If you are worried about the restriction of d being an integer, a slight coercion from double to int is possible with std::floor(std::fabs(seed)). Once the values are generated using either Rcpp::r*() or R::r*() , an armadillo vector is created using an advanced ctor that reuses the existing memory allocation.
Code:
#include <RcppArmadillo.h>
// [[Rcpp::depends(RcppArmadillo)]]
// set seed
// [[Rcpp::export]]
void set_seed(double seed) {
Rcpp::Environment base_env("package:base");
Rcpp::Function set_seed_r = base_env["set.seed"];
set_seed_r(std::floor(std::fabs(seed)));
}
// function r(d, n)
// [[Rcpp::export]]
arma::vec randdraw(double d, int n){
set_seed(d); // Set a seed for R's RNG library
Rcpp::NumericVector draws = Rcpp::rnorm(n, 0.0, d); // Hook into R's Library
// Use Armadillo's advanced CTOR to re-use memory and cast as an armadillo object.
arma::vec out = arma::vec(draws.begin(), n, false, true);
return out;
}
Output:
> randdraw(3.21,10)
[,1]
[1,] -3.08780627
[2,] -0.93900757
[3,] 0.83071017
[4,] -3.69834335
[5,] 0.62846287
[6,] 0.09669786
[7,] 0.27419092
[8,] 3.58431878
[9,] -3.91253230
[10,] 4.06825360
> set.seed(3)
> rnorm(10, 0, 3.21)
[1] -3.08780627 -0.93900757 0.83071017 -3.69834335 0.62846287 0.09669786 0.27419092 3.58431878 -3.91253230 4.06825360
Related
I'm searching for a way to solve a system of linear equations. Specifically 8 equations with a total of 16 unknown values.
Each unknown value (w[0...15]) is a 32-bit binary value which corresponds to 4 ascii characters written over 8 bits. For example:
For :
I've tried writing this system of linear equations as a single matrix equation. Which gives:
Right now, using the Eigen linear algebra library, I get my 16 solutions (w[0...15]) but all of them are either decimal or null values, which is not what I need. All 16 solutions need to be the equivalent of 4 hexadecimal characters under their binary representation. Meaning integers between 48 and 56 (ascii for '0' to '9'), 65 and 90 (ascii for 'A' to 'Z'), or 97 and 122 (ascii for 'a' to 'z').
Current 16 solutions:
I've found a solution to this problem using something called box-constraints. An example is shown here using python's lsq_linear function which allows the user to specify bounds. It seems Eigen does not let the user specify bounds in its decomposition methods.
Therefore, my question is, how do you get a similar result in C++ using a linear algebra library? Or is there a better way to solve such systems of equations without writing it under a single matrix equation?
Thanks in advance.
Since you're working with linear equations over Z/232Z, integer linear programming (as you tagged the question) may be a solution, and algorithms that are inherently floating point are not appropriate. Box constraints are not enough, they won't force the variables to take on integer values. Also, the model shown in the question does not taken into account that multiplying and adding in Z/232Z can wrap, which excludes many potential solutions (or perhaps that is intended?) and may make the instance accidentally infeasible when it was intended to be solvable.
ILP can model equations over Z/232Z relatively directly (using integer variables between 0 and 232 and some unconstrained additional variables scaled by 232 to "absorb" the wraparound), but it tends really struggle with that kind of formulation - I would say it's one of the worst cases for an ILP solver without getting into the "intentionally difficult" cases. A more indirect model with 32x boolean variables is also possible, but this leads to constraints with very large constants and ILP solvers tend to struggle with them too. Overall I do not recommend using ILP for this problem.
What I would recommend for this is an SMT solver that offers the bitvector theory, or as alternatively, a pseudo-boolean solver or plain SAT solver (which would leave the grunt work of implementing boolean circuits and converting them to CNF to you instead of having them builtin in the solver).
If you have more unknowns than equations for sure your system will be indeterminate, the rank of a 8 x 16 matrix is at most 8, thus you have at least 16 degrees of freedom.
Further more if you have bounds to your variables i.e. mixed equalities and inequalities, then your problem is better posed as a linear programming. You can set a dummy objective function c[i] = 0, you could use GLPK but that is a very generic solution. If you want a small code snipped you probably can find a toy implementation of the Simplex method that will satisfy your needs.
I went for an SMT solver as suggested by #harold. Specifically the CVC4 SMT Solver. Here is the code I've written in C++ answering my question about finding the 16 solutions (w[0...15]) for a system of 8 equations, constrained to be ascii characters. I have one last question though. What are pushing and popping for? (slv.push() and slv.pop())
#include <iostream>
#include <cvc4/cvc4.h>
using namespace std;
using namespace CVC4;
int main() {
// 1. initialize a CVC4 BitVector SMT solver
ExprManager em;
SmtEngine slv(&em);
slv.setOption("incremental", true); // enable incremental solving
slv.setOption("produce-models", true); // enable models
slv.setLogic("QF_BV"); // set the bitvector theory logic
Type bitvector8 = em.mkBitVectorType(size_8); // create a 8-bit wide bit-vector type (4 x 8-bit = 32-bit)
// 2. create the SMT solver variables
Expr w[16][4]; // w[0...15] where each w corresponds to 4 ascii characters
for (int i = 0; i < 16; ++i) {
for (int j = 0; j < 4; ++j) {
// a. define w[i] (four ascii characters per w[i])
w[i][j] = em.mkVar("w" + to_string(i) + to_string(j), bitvector8);
// b. constraint w[i][0...3] to be an ascii character
// - digit (0-9) constraint
// ascii lower bound digit constraint (bit-vector unsigned greater than or equal)
Expr digit_lower = em.mkExpr(kind::BITVECTOR_UGE, w[i][j], em.mkConst(BitVector(size_8, Integer(48))));
// ascii upper bound digit constraint (bit-vector unsigned less than or equal)
Expr digit_upper = em.mkExpr(kind::BITVECTOR_ULE, w[i][j], em.mkConst(BitVector(size_8, Integer(56))));
Expr digit_constraint = em.mkExpr(kind::AND, digit_lower, digit_upper);
// - lower alphanumeric character (a-z) constraint
// ascii lower bound alpha constraint (bit-vector unsigned greater than or equal)
Expr alpha_lower = em.mkExpr(kind::BITVECTOR_UGE, w[i][j], em.mkConst(BitVector(size_8, Integer(97))));
// ascii upper bound alpha constraint (bit-vector unsigned less than or equal)
Expr alpha_upper = em.mkExpr(kind::BITVECTOR_ULE, w[i][j], em.mkConst(BitVector(size_8, Integer(122))));
Expr alpha_constraint = em.mkExpr(kind::AND, alpha_lower, alpha_upper);
Expr ascii_constraint = em.mkExpr(kind::OR, digit_constraint, alpha_constraint);
slv.assertFormula(ascii_constraint);
}
}
// 3. encode the 8 equations
for (int i = 0; i < 8; ++i) {
// a. build the multiplication part (index * w[i])
vector<Expr> left_mult_hand;
for (int j = 0; j < 16; ++j) {
vector <Expr> inner_wj;
for (int k = 0; k < 4; ++k) inner_wj.push_back(w[j][k]);
Expr wj = em.mkExpr(kind::BITVECTOR_CONCAT, inner_wj);
Expr index = em.mkConst(BitVector(size_32, Integer(m_unknowns[j])));
left_mult_hand.push_back(em.mkExpr(kind::BITVECTOR_MULT, index, wj));
}
// b. sum each index * w[i]
slv.push();
Expr left_hand = em.mkExpr(kind::BITVECTOR_PLUS, left_mult_hand);
Expr result = em.mkConst(BitVector(size_32, Integer(globalSums.to_ulong())));
Expr assumption = em.mkExpr(kind::EQUAL, left_hand, result);
slv.assertFormula(assumption);
// c. check for satisfiability
cout << "Result from CVC4 is: " << slv.checkSat(em.mkConst(true)) << endl << endl;
slv.pop();
}
return 0;
}
I have a slow R function I am converting to RcppArmadillo. I am very new to Rcpp and RcppArmadillo. I managed to code up something that works, but it seems clunky and not as fast at it probably could be. Mainly, I am curious how to perform matrix-vector element-wise multiplication or division of an output without needing to assign it to a new object first. For example here is my RcppArmadillo function which works:
#include <RcppArmadillo.h>
// [[Rcpp::depends(RcppArmadillo)]]
using namespace Rcpp;
using namespace arma;
// [[Rcpp::export]]
arma::mat TTD(arma::mat X, arma::vec d, arma::vec taubar){
arma::vec top = d % sqrt(taubar);
arma::mat cubed = pow(X,3) * 4 * datum::pi;
arma::mat out = repmat(top,1,X.n_cols)/sqrt(cubed.each_col() % d);
return out;
}
But, can I do something like (and I know this is wrong): pow(X,3).n_cols() % d * 4 * datum::pi? And also how can I divide the vector top by the matrix without using repmat or assigning the product of d % sqrt(taubar) to the new object first? I don't want to change any of my inputs in place because I need them later in R for other functions.
Below is the R code which tests the function:
library(RcppArmadillo)
library(Rcpp)
sourceCpp("TTD.cpp")
d <- c(53.638145, 9.617256, 1.450767)
ttau <- matrix(c(36.405117983, 1.707159588, 0.000010000, 36.406568363, 1.707759972, 0.001141475,
36.408022369, 1.708361856, 0.002275780, 36.409480010, 1.708965246, 0.003412920),
nrow = 3, ncol = 4)
tau_bar <- c(66.48201, 9.90116, 6.08173)
TTD(X = ttau, d = d,taubar = tau_bar)
My actual matrices and vectors will be much larger, thus the need for the speed increase.
[EDIT: I understand that it is faster also because the function is written in C, but I want to know if It does a brute force search on all the training instances or something more sophisticated ]
I'm implementing in R, for studying purpose, the KNN algorithm.
I'm also checking the code correctness by comparison with the caret implementation.
The problem lies on the execution time of the two versions. My version seems to take a lot of time, instead the caret implementation is very fast (even with crossvalidation with 10 folds).
Why? I'm calculating every euclidean distance of my test instances from the training ones. Which means that I'm doing NxM distance calculation (where N are my test instances, and M my training instances):
for (i in 1:nrow(test)){
distances <- c()
classes <- c()
for(j in 1:nrow(training)){
d = calculateDistance(test[i,], training[j,])
distances <- c(distances, d)
classes <- c(classes, training[j,][[15]])
}
}
Is the caret implementation using some approximate search? Or an exact search, for example with the kd-tree? How can I speed up the search? I got 14 features for the problem, but I've been reading that the kd-tree is suggested for problem with 1 to 5 features.
EDIT:
I've found the C function called by R (VR_knn), which is pretty complex for me to understand, maybe someone can help.
Anyway I've written on the fly a brute force search in cpp, which seems to go faster than my previous R version, (but not fast as the caret C version) :
#include <Rcpp.h>
using namespace Rcpp;
double distance(NumericVector x1, NumericVector x2){
int vectorLen = x1.size();
double sum = 0;
for(int i=0;i<vectorLen-1;i++){
sum = sum + pow((x1.operator()(i)-x2.operator()(i)),2);
}
return sqrt(sum);
}
// [[Rcpp::export]]
void searchCpp(NumericMatrix training, NumericMatrix test) {
int numRowTr = training.rows();
int numColTr = training.cols();
int numRowTe = test.rows();
int numColTe = test.cols();
for (int i=0;i<numRowTe;i++)
{
NumericVector test_i = test.row(i);
NumericVector distances = NumericVector(numRowTe);
for (int j=0;j<numRowTr;j++){
NumericVector train_j = training.row(j);
double dist = distance(test_i, train_j);
distances.insert(i,dist);
}
}
}
My question is a follow-up of http://rcpp-devel.r-forge.r-project.narkive.com/qJMEsvOK/setting-the-r-random-seed-from-rcpp.
I want to be able to set the RNG state to a former state from within C++.
For example, I would like the following code to produce a matrix where each column contains the same realizations of Gamma random variables.
cppFunction('NumericMatrix rgamma_reset_seed(int n, double shape, double rate){
RNGScope rngscope;
Environment g = Environment::global_env();
Environment::Binding RandomSeed = g[".Random.seed"];
IntegerVector someVariable = RandomSeed;
NumericMatrix results(n, 2);
results(_,0) = rgamma(n, shape, 1/rate);
RandomSeed = someVariable;
results(_,1) = rgamma(n, shape, 1/rate);
return results;
}')
m <- rgamma_reset_seed(1000, 1.2, 0.8)
par(mfrow = c(2, 1))
plot(m[,1])
plot(m[,2])
But it does not seem to work. In R, I can achieve the result by lines such as
.Random.seed <- x # reset the state to x
x <- .Random.seed # store the current state
Am I missing something obvious? Any help would be much appreciated!
This may not work (easily) -- there is some language in Writing R Extension which states that you cannot set the seed from the C level API.
Now, you could cheat:
Init RNG from R
Do some work, make sure this is wrapped by RNGScope as our code does anyway.
Now cheat and use Rcpp::Function() to invoke set.seed().
Consider whether to go back to step 2 or to finish.
Is there a numerical library which can use a paralleled algorithm to do one dimensional integration (global adaptive method)? The infrastructure of my code decides that I cannot do multiple numerical integrations in parallel, but I have to use a paralleled algorithm to speed up.
Thanks!
Nag C numerical library does have a parallel version of adaptive quadrature (link here). Their trick is to request the user the following function
void (*f)(const double x[], Integer nx, double fv[], Integer *iflag, Nag_Comm *comm)
Here the function "f" evaluates the integrand at nx abscise points given by the vector x[]. This is where parallelization comes along, because you can use parallel_for (implemented in openmp for example) to evaluate f at those points concurrently. The integrator itself is single threaded.
Nag is a very expensive library, but if you code the integrator yourself using, for example, numerical recipes, it is not difficult to modify serial implementations to create parallel adaptive integrators using NAG idea.
I can't reproduce numerical recipes book to show where modifications are necessary due to license restriction. So let's take the simplest example of trapezoidal rule, where the implementation is quite simple and well known. The simplest way to create adaptive method using trapezoidal rule is to calculate the integral at a grid of points, then double the number of abscise points and compare the results. If the result changes by less than the requested accuracy, then there is convergence.
At each step, the trapezoidal rule can be computed using the following generic implementation
double trapezoidal( double (*f)(double x), double a, double b, int n)
{
double h = (b - a)/n;
double s = 0.5 * h * (f(a) + f(b));
for( int i = 1; i < n; ++i ) s += h * f(a + i*h);
return s;
}
Now you can make the following changes to implement NAG idea
double trapezoidal( void (*f)( double x[], int nx, double fv[] ), double a, double b, int n)
{
double h = (b - a)/n;
double x[n+1];
double fv[n+1];
for( int i = 0; i < n; ++i ) x[i+1] = (a + i * h);
x[n] = b;
f(x, n, fv); // inside f, use parallel_for to evaluate the integrand at x[i], i=0..n
double s = 0.5 * h * ( fv[0] + fv[n] );
for( int i = 1; i < n; ++i ) s += h * fv[i];
return s;
}
This procedure, however, will only speed-up your code if the integrand is very expensive to compute. Otherwise, you should parallelize your code at higher loops and not inside the integrator.
Why not simply implement a wrapper around a single threaded algorithm that dispatches integrals of subdivisions of the bounds to different threads and then adds them together at the end? e.g.
thread 0: i0 = integral(x0, (x0+x1)/2)
thread 1: i1 = integral((x0+x1)/2, x1)
i = i0 + i1