Sampling a log-concave distribution using the adaptive rejection sampling method (R) - r

I am not very familiar with R. I have been trying to use the implementation of the adaptive rejection sampling method in R, in order to sample from the following distribution:
here is my R code:
library(ars)
g1 <- function(x,r){(1./r)*((1-x)^r)}
f1 <- function(x,a,k) {
add<-0
for(i in 1:k) {
add<- add+g1(x,i)
}
res <- (a* add)+(a-1)*log(x)+k*log(1-x)
return(res)
}
g2 <- function(x,r){(1-x)^(r-1)}
f1prima <- function(x,a,k) {
add<-0
for(i in 1:k) {
add<- add-g2(x,i)
}
res <- (a* add)+(a-1)/x-k/(1-x)
return(res)
}
mysample1<-ars(20,f1,f1prima,x=c(0.001,0.09),m=2,emax=128,lb=TRUE,xlb=0.0, ub=TRUE, xub=1,a=0.5,k=100)
The function is a log-concave, but I get different error messages when I run ars and fiddling around with the input parameters won't help here. Any suggestion would be appreciated.

First thing, which you already noticed is that your log-concave function is not very well defined at x=0 and x=1.0. So useful interval would be something like 0.01...0.99, not 0.0...1.0
Second, I don't like the idea to compute hundreds of terms in your summation term.
So, good idea might be to express it in following way, starting with derivative
S1N-1 qi is obviously geometric series and could be replaced with
(1-qN)/(1-q), where q=1-x.
This is derivative, so to get to similar term in function itself, just integrate it.
http://www.wolframalpha.com/input/?i=integrate+(1-q%5EN)%2F(1-q)+dq will return Gauss Hypergeometric function 2F1 plus logarithm
-qN+1 2F1(1, N+1; N+2; q)/(N+1) - log(1-q)
NB: It is the same integral as Beta before, but dealing with it was a bit more cumbersome
So, code to compute those terms:
library(gsl)
library(ars)
library(ggplot2)
Gauss2F1 <- function(a, b, c, x) {
ifelse(x >= 0.0 & x < 1.0, hyperg_2F1(a, b, c, x), hyperg_2F1(c - a, b, c, 1.0 - 1.0/(1.0 - x))/(1.0 - x)^b)
}
f1sum <- function(x, N) {
q <- 1.0 - x
- q^(N+1) * Gauss2F1(1, N+1, N+2, q)/(N+1) - log(1.0 - q)
}
f1sum.1 <- function(x, N) {
q <- 1.0 - x
res <- rep(0.0, length.out = length(x))
s <- rep(1.0, length.out = length(x))
for(k in 1:N) {
s <- s * q / as.numeric(k)
res <- res + s
}
res
}
f1 <- function(x, a, N) {
a * f1sum(x, N) + (a - 1.0)*log(x) + N*log(1.0 - x)
}
f1.1 <- function(x, a, N) {
a * f1sum.1(x, N) + (a - 1.0)*log(x) + N*log(1.0 - x)
}
f1primesum <- function(x, N) {
q <- 1.0 - x
(1.0 - q^N)/(1.0 - q)
}
f1primesum.1 <- function(x, N) {
res <- rep(0.0, length.out = length(x))
s <- rep(1.0, length.out = length(x))
for(k in 1:N) {
res <- res + s
s <- s * q
}
-res
}
f1prime <- function(x, a, N) {
a* f1primesum(x, N) + (a - 1.0)/x - N/(1.0 - x)
}
f1prime.1 <- function(x, a, N) {
a* f1primesum.1(x, N) + (a - 1.0)/x - N/(1.0 - x)
}
p <- ggplot(data.frame(x = c(0, 1)), aes(x = x)) +
stat_function(fun = f1, args = list(0.5, 100), colour = "#4271AE") +
stat_function(fun = f1.1, args = list(0.5, 100), colour = "#1F3552") +
scale_x_continuous(name = "X", breaks = seq(0, 1, 0.2), limits=c(0.001, 0.5)) +
scale_y_continuous(name = "F") +
ggtitle("Log-concave function")
p
As you can see, I've implemented both versions - one using summation and another using analytical form of sums. Computed data for a=0.5, N=100.
First, there is a bit of a difference between direct sum and 2F1 - I attribute it to precision loss in summation.
Second, more important result - function is NOT log-concave. No questions why ars() if failing left and right. See graph below

Related

Is there a simple way to calculate the maximum likelihood estimate of a parameter in R?

I am trying to calculate the MLE of a poisson distribution in R. Is there a function in R that allows us to do this (eg. I know Stata has a mlexp function that allows us to make this calculation quite easily). I see that there is a mlexp function in the univariateML package for the exponential distribution. That being said, is there a command that allows this for more than just exponential distributions?
The fitdistrplus package might help with what you want, though it can't handle more complicated distributions. The mle() function of the stats4
package might also help. I believe the bbmle package is more general than the other options, but I haven't really used it myself.
Here's a function I've used for custom distributions. Similar to stats4::mle. It does assume that the PDF argument is named x (as with the R implementaion of all the classic distributions).
mle <- function(data, fun, init, logarg = TRUE, lower = -Inf, upper = Inf) {
if (logarg) {
fnll <- function(...) {
-sum(do.call(fun, l <- c(x = list(data), log = TRUE, as.list(...))))
}
} else {
fnll <- function(...) {
return(-sum(log(do.call(fun, c(x = list(data), as.list(...))))))
}
}
params <- optim(init, fnll, method = "L-BFGS-B", lower = lower, upper = upper)$par
names(params) <- formalArgs(fun)[-1][seq_along(init)]
return(params)
}
# optim will check the boundaries
# set lim0 and lim1 for help in setting bounds
lim0 <- .Machine$double.eps
lim1 <- 1 + lim0
# Poisson distribution
data <- rpois(1e5, 14)
rbind(trueML = c(lambda = mean(data)),
mle = mle(data, dpois, 1, lower = lim0))
#> lambda
#> trueML 13.99387
#> mle 13.99387
# normal distribution
data <- rnorm(1e5, -2, 3)
rbind(trueML = c(mean = mean(data), sd = sd(data)*sqrt((length(data) - 1)/length(data))),
mle = mle(data, dnorm, 0:1, lower = c(-Inf, lim0)))
#> mean sd
#> trueML -2.002658 2.993441
#> mle -2.002657 2.993442
# gamma distribution
data <- rgamma(1e5, 0.5, 0.1)
c(mle = mle(data, dgamma,
init = c(mean(data)^2/var(data), mean(data)/var(data)),
lower = rep(lim0, 2)))
#> mle.shape mle.rate
#> 0.5007139 0.1003400
# triangular distribution
dtri <- function(x, a, b, c) {
if (a > b) {a <- (b - a) + (b <- a)}
if (b > c) {c <- (b - c) + (b <- c)}
blna <- x < b
p <- numeric(length(x))
p[blna] <- 2*(x[blna] - a)/(c - a)/(b - a)
p[!blna] <- 2*(c - x[!blna])/(c - a)/(c - b)
return(p)
}
rtri <- function(n, a, b, c) {
if (a > b) {a <- (b - a) + (b <- a)}
if (b > c) {c <- (b - c) + (b <- c)}
fb <- (b - a)/(c - a)
U <- runif(n)
blna <- U < fb
r <-numeric(n)
r[blna] <- a + sqrt(U[blna]*(c - a)*(b - a))
r[!blna] <- c - sqrt((1 - U[!blna])*(c - a)*(c - b))
return(r)
}
data <- rtri(1e5, -6, -3, 3)
mind <- min(data); maxd <- max(data)
# set logarg to FALSE because dtri doesn't have a log argument
c(mle = mle(data, dtri, logarg = FALSE,
init = c(mind - abs(mind)*lim0, median(data), maxd + abs(maxd)*lim0),
lower = c(-Inf, min(data), maxd + abs(maxd)*lim0),
upper = c(mind - abs(mind)*lim0, max(data), Inf)))
#> mle.a mle.b mle.c
#> -6.003666 -3.000262 2.994473
Created on 2021-11-04 by the reprex package (v2.0.1)

Efficient implementation of double for-loop

I am new to R and I was wondering if there is any more efficient implementation of the following setting? Time series length (x,y) is around 5000 and h != nrow(q).
set.seed(1)
h = 21
x <- rnorm(5e3, 1)
y <- rnorm(5e3, 2)
q <- c(0.1, 0.3, 0.5, 0.7, 0.9)
qx <- quantile(x, probs = q)
qx <- expand.grid(qx, qx)
qy <- quantile(y, probs = q)
qy <- expand.grid(qy, qy)
q <- expand.grid(q, q)
f <- function(z, l, qz) {
n <- length(z)
1/(n - l) * sum((z[1:(n-l)] <= qz[[1]]) * (z[(1+l):n] <= qz[[2]])) - prod(q[i,])
}
sum = 0
for (i in 1:h) {
for (j in 1:nrow(q)) {
sum = sum + (f(x, l = i, qx[j,]) - f(y, l = i, qy[j,]))^2
}
}
sum
# 0.0008698279
Thank you very much!
One faster alternative to loops might be under some circumstances a sapply function.
This function works as follows: for each element of a vector perform some function.
Aternatively, you could take a look at a foreach package which offers some fast looping.
Here is an example using sapply: depending on what exactly you need, you might want to use either of the functions. Also, sapply is just one of the faster ways of doing this, not necessarily the fastest.
# setup from the question
set.seed(1)
h = 1
x <- rnorm(5e3, 1)
y <- rnorm(5e3, 2)
q <- c(0.1, 0.3, 0.5, 0.7, 0.9)
qx <- quantile(x, probs = q)
qx <- expand.grid(qx, qx)
qy <- quantile(y, probs = q)
qy <- expand.grid(qy, qy)
q <- expand.grid(q, q)
f <- function(z, l, qz) {
n <- length(z)
1/(n - l) * sum((z[1:(n-l)] <= qz[[1]]) * (z[(1+l):n] <= qz[[2]])) - prod(q[i,])
}
# load microbenchmark library for comparison of execution times
library(microbenchmark)
microbenchmark({
# the version from question with for loop
sum = 0
for (i in 1:h) {
for (j in 1:nrow(q)) {
sum = sum + (f(x, l = i, qx[j,]) - f(y, l = i, qy[j,]))^2
}
}
},
{
# using sapply and storing to object. this will give you h*j matrix as well as the sum
sum = 0
sapply(1:h, function(i) sapply(1:nrow(q), function(j) {sum <<- sum + (f(x, l = i, qx[j,]) - f(y, l = i, qy[j,]))^2}))
},
{
# use sapply and sum the output
sum(sapply(1:h, function(i) sapply(1:nrow(q), function(j) {(f(x, l = i, qx[j,]) - f(y, l = i, qy[j,]))^2})))},
# run each code 200 times to get the time comparison
times = 200
)

How to find the second derivative in R and while using newton's method with numerical derivation

The log-likelihood of the gamma distribution with scale parameter 1 can be written as:
(α−1)s−nlogΓ(α)
where alpha is the shape parameter and s=∑logXi is the sufficient statistic.
Randomly draw a sample of n = 30 with a shape parameter of alpha = 4.5. Using newton_search and make_derivative, find the maximum likelihood estimate of alpha. Use the moment estimator of alpha, i.e., mean of x as the initial guess. The log-likelihood function in R is:
x <- rgamma(n=30, shape=4.5)
gllik <- function() {
s <- sum(log(x))
n <- length(x)
function(a) {
(a - 1) * s - n * lgamma(a)
}
}
I have created the make_derivative function as follows:
make_derivative <- function(f, h) {
(f(x + h) - f(x - h)) / (2*h)
}
I also have created a newton_search function that incorporates the make_derivative function; However, I need to use newton_search on the second derivative of the log-likelihood function and I'm not sure how to fix the following code in order for it to do that:
newton_search2 <- function(f, h, guess, conv=0.001) {
set.seed(2)
y0 <- guess
N = 1000
i <- 1; y1 <- y0
p <- numeric(N)
while (i <= N) {
make_derivative <- function(f, h) {
(f(y0 + h) - f(y0 - h)) / (2*h)
}
y1 <- (y0 - (f(y0)/make_derivative(f, h)))
p[i] <- y1
i <- i + 1
if (abs(y1 - y0) < conv) break
y0 <- y1
}
return (p[(i-1)])
}
Hint: You must apply newton_search to the first and second derivatives (derived numerically using make_derivative) of the log-likelihood. Your answer should be near 4.5.
when I run newton_search2(gllik(), 0.0001, mean(x), conv = 0.001), I get double what the answer should be.
I re-wrote the code and it works perfectly now (even better than what I had originally wrote). Thanks to all who helped. :-)
newton_search <- function(f, df, guess, conv=0.001) {
set.seed(1)
y0 <- guess
N = 100
i <- 1; y1 <- y0
p <- numeric(N)
while (i <= N) {
y1 <- (y0 - (f(y0)/df(y0)))
p[i] <- y1
i <- i + 1
if (abs(y1 - y0) < conv) break
y0 <- y1
}
return (p[(i-1)])
}
make_derivative <- function(f, h) {
function(x){(f(x + h) - f(x - h)) / (2*h)
}
}
df1 <- make_derivative(gllik(), 0.0001)
df2 <- make_derivative(df1, 0.0001)
newton_search(df1, df2, mean(x), conv = 0.001)

Unused arguments within a function in R

Below is the code I have. It works for primitive functions, such as sin. However, when using a function called gllik, it returns an error in f(y0): unused argument (y0). I'm not sure how to correct this.
newton_search2 <- function(f, h, guess, conv=0.001) {
y0 <- guess
N = 100
i <- 1; y1 <- y0
p <- numeric(N)
while (i <= N) {
make_derivative <- function(f, h) {
(f(y0 + h) - f(y0 - h)) / (2*h)
}
y1 <- (y0 - (f(y0)/make_derivative(f, h)))
p[i] <- y1
i <- i + 1
if (abs(y1 - y0) < conv) break
y0 <- y1
}
return (p[(i-1)])
}
The gllik function is as follows:
x <- rgamma(n=30, shape=4.5)
gllik <- function() {
s <- sum(log(x))
n <- length(x)
function(a) {
(a - 1) * s - n * lgamma(a)
}
}
The code I used was:
newton_search2(gllik, 0.001, mean(x), conv = 0.001)
I'm not sure how to fix the error or get the correct answer which is supposed to be 4.5 (the maximum liklihood estimate of a).
The problem is that gllik does not take any arguments. Furthermore, it returns a function and not a value.
Perhaps what you want to to is the following?
gllik <- function(a) {
s <- sum(log(x))
n <- length(x)
return((a - 1) * s - n * lgamma(a))
}
EDIT: An alternative solution is to just use the returned function. While this type of construction is often elegant, it does seem like overkill in this case:
newton_search2(gllik(), 0.001, mean(x), conv = 0.001)

A bug in creating dynamic functions in R

I have found a very subtle bug in my R code just now. The following code takes a list of objects as input and create new fields for each of the objects.
Each object originally has two fields (w, p, s, u), and then I create more, beta, phi, etc.. The normal variables are OK. However the dynamic functions (Q, K, K1, K2) are not correct. Suppose I have two nigs, nigs[[1]] and nigs[[2]], the functions Q, K, K1 and K2 for nigs[[1]] would be the same as nigs[[2]]!
I just found this bug and would consult on how to get this code correct (while keeping its elegance:) Thanks!
D <- length(nigs)
for (i in 1:D) {
w <- nigs[[i]]$w
p <- nigs[[i]]$p
s <- nigs[[i]]$s
u <- nigs[[i]]$u
nigs[[i]]$beta <- beta <- w / s * p * (1-p^2)^(-1/2);
nigs[[i]]$phi <- phi <- w^2 / s^2;
nigs[[i]]$z <- z <- (x-u)/s;
nigs[[i]]$alpha_bar <- alpha_bar <- w * (1-p^2)^(-1/2);
nigs[[i]]$y_bar <- y_bar <- sqrt(1+z^2);
nigs[[i]]$Q <- Q <- function(t) { sqrt(1 - (2*beta*t+t^2)/phi) }
nigs[[i]]$K <- K <- function(t) { u*t - w*Q(t) + w }
nigs[[i]]$K1 <- K1 <- function(t) { (u + w * (beta+t) / (Q(t)*phi)) }
nigs[[i]]$K2 <- K2 <- function(t) { qt = Q(t); (w/(qt * phi) + w * (beta+t)^2 / (qt^3 * phi^2)); }
}
EDIT
The primary error I made is that I assumed that for { } introduced new scopes, in that case, w,p,s,u are different w,p,s,u every time, actually not. Only functions in R introduce new scopes. And this scoping rule is different from C/Java.
That is normal behavior of the lexical scope.
You can use closure instead.
f <- list()
g <- list()
for (i in 1:2) {
j <- i * 2
f[[i]] <- function() print(j)
g[[i]] <- (function() {j <- j; function() print(j)}) ()
}
then,
> for (i in 1:2) f[[i]]()
[1] 4
[1] 4
> for (i in 1:2) g[[i]]()
[1] 2
[1] 4
In object oriented terminology each nigs[[i]] is an object and the functions Q, K, etc. are methods which act on the object's properties w, p, etc. Using the proto package we set each nigs[[i]] to a proto object and then update the object as indicated. Note that all methods take the object as the first argument so if p is a proto object containing method Q then p$Q(t) means to look in p for Q and then run it with the arguments p and t so p$Q(t) is the same as with(p, Q(p, t)). Thus we have added the extra first argument to each of the methods below. See proto home page for more.
library(proto)
# initialize
x <- 1
nigs <- lapply(1:2, function(i) proto(w = i/3, p = i/3, s = i/3, u = i/3))
for(p in nigs) with(p, {
beta <- w / s * p * (1-p^2)^(-1/2)
phi <- w^2 / s^2
z <- (x-u)/s
alpha_bar <- w * (1-p^2)^(-1/2)
y_bar <- sqrt(1+z^2)
Q <- function(., t) { sqrt(1 - (2*beta*t+t^2)/phi) }
K <- function(., t) { u*t - w*.$Q(t) + w }
K1 <- function(., t) { (u + w * (beta+t) / (.$Q(t)*phi)) }
K2 <- function(., t) {
qt = .$Q(t)
(w/(qt * phi) + w * (beta+t)^2 / (qt^3 * phi^2))
}
})
EDIT: A second possible design would be to create a parent object, meths to hold the methods instead of defining them over again in each separate proto object. In that case, within each method we must be sure that we use the properties of the object passed in the first argument since the methods and properties are now located in different objects:
meths <- proto(
Q = function(., t) sqrt(1 - (2*.$beta*t+t^2)/.$phi),
K = function(., t) .$u*t - .$w*.$Q(t) + .$w,
K1 = function(., t) (.$u + .$w * (.$beta+t) / (.$Q(t)*.$phi)),
K2 = function(., t) {
qt = .$Q(t)
(.$w/(qt * .$phi) + .$w * (.$beta+t)^2 / (qt^3 * .$phi^2))
}
)
# initialize - meths$proto means define proto object with parent meths
x <- 1
nigs <- lapply(1:2, function(i) meths$proto(w = i/3, p = i/3, s = i/3, u = i/3))
for(p in nigs) with(p, {
beta <- w / s * p * (1-p^2)^(-1/2)
phi <- w^2 / s^2
z <- (x-u)/s
alpha_bar <- w * (1-p^2)^(-1/2)
y_bar <- sqrt(1+z^2)
})
Now the following works by looking up Q in nigs[[1]] but not finding it there looking into its parent, meths, and running the Q found there. In nigs[[1]]$Q(.1) the call implicitly passes nigs[[1]] to Q as its first argument and we have defined all properties within the body of Q relative to the first argument so everything works:
> nigs[[1]]$Q(.1)
[1] 0.9587958

Resources