(1) The simple version of the problem:
How to calculate log(P1+P2+...+Pn), given log(P1), log(P2), ..., log(Pn), without taking the exp of any terms to get the original Pi. I don't want to get the original Pi because they are super small and may cause numeric computer underflow.
(2) The long version of the problem:
I am using Bayes' Theorem to calculate a conditional probability P(Y|E).
P(Y|E) = P(E|Y)*P(Y) / P(E)
I have a thousand probabilities multiplying together.
P(E|Y) = P(E1|Y) * P(E2|Y) * ... * P(E1000|Y)
To avoid computer numeric underflow, I used log(p) and calculate the summation of 1000 log(p) instead of calculating the product of 1000 p.
log(P(E|Y)) = log(P(E1|Y)) + log(P(E2|Y)) + ... + log(P(E1000|Y))
However, I also need to calculate P(E), which is
P(E) = sum of P(E|Y)*P(Y)
log(P(E)) does not equal to the sum of log(P(E|Y)*P(Y)). How should I get log(P(E)) without solving for P(E|Y)*P(Y) (they are extremely small numbers) and adding them.
You can use
log(P1+P2+...+Pn) = log(P1[1 + P2/P1 + ... + Pn/P1])
= log(P1) + log(1 + P2/P1 + ... + Pn/P1])
which works for any Pi. So factoring out maxP = max_i Pi results in
log(P1+P2+...+Pn) = log(maxP) + log(1+P2/maxP + ... + Pn/maxP)
where all the ratios are less than 1.
Related
I working on computer vision task and have this equation:
R0*c + t0 = R1*c + t1 = Ri*c + ti = ... = Rn*c + tn ,
n is about 20 (but can be more if needs)
where each pair of R,t (rotation matrix and translation vector in 3D) is a result of i-measurement and they are known, and vector c is what I whant to know.
I've got result with ceres solver. It's good that it can handle outliers but I think it's overkill for this task.
So what methods I should use for two situations:
With outliers
Without outliers
To handle outliers you can use RANSAC:
* In each iteration randomly pick i,j (a "sample") and solve c:
Ri*c + ti = Rj*c + tj
- Set Y = Ri*c + ti
* Apply to a larger population:
- Select S={k} for which ||Rk*c + tk - Y||<e
e ~ 3*RMS of errors without outliers
- Find optimal c for all k equations (with least mean square)
- Give it a "grade": size of S
* After few iterations use optimal c found for Max "grade".
* Number of iterations: log(1-p)/log(1-w^2)
[https://en.wikipedia.org/wiki/Random_sample_consensus]
p = 0.001 (for example. It is the required certainty of the result)
w is an assumption of nonoutliers/n.
Is there any way to combine co-variance from two data sets instead of calculating the new co-variance by merging the data. Suppose I have already calculated co-variance from 1 million data and then if I get another 2 million data that has already calculated co-variance, can i combine the already calculated co-variance to produce the new co-variance. I am mostly interested in reducing the computation that is required when i calculate the co-variance from the combined 3 million data.
This can be easily done for mean.
new mean = (data_size_1* mean_1 + data_size_2*mean_2)/((data_size_1 + data_size_2)
Is there any similar way to calculate co-variance so that i can take advantage of the pre-computed data. I can also store some information while calculating co-variance of data_size_1 and data data_size_2 if that can help me to find the new merged co-variance easily.
The complete derivation is given in this pdf http://prod.sandia.gov/techlib/access-control.cgi/2008/086212.pdf
I found formula for combining variances of two sets here:
https://www.emathzone.com/tutorials/basic-statistics/combined-variance.html
Replacing (X1–Xc)2 with
(X1–Xc)(Y1–Yc),
and (X2–Xc)2 with
(X2–Xc)(Y2–Yc)
gives the correct results for covariances.
Unlike the formula from the first answer, which is only approximately correct.
Here is a code fragment that combines covariances a and b
into resulting covariance r.
r.n = a.n + b.n;
r.mean_x = (a.n * a.mean_x + b.n * b.mean_x) / r.n;
r.mean_y = (a.n * a.mean_y + b.n * b.mean_y) / r.n;
r.sum = a.sum + a.n * (a.mean_x - r.mean_x) * (a.mean_y - r.mean_y)
+ b.sum + b.n * (b.mean_x - r.mean_x) * (b.mean_y - r.mean_y);
a, b and r are structs that contain:
n – number of elements,
mean_x – mean of X,
mean_y – mean of Y,
sum – the covariance multiplied by n.
How to calculate the logarithm of the sum of the probabilities, i.e. ln(p1 + p2), where p1 = a very small number and p2 = a very small number. Using the values of lp1 = ln(p1) and lp2 = ln(p2)
If you p1 and p2 are very small numbers underflow will happen. How to avoid this?
In general the following tips are useful for taking logs in r:
If you are taking log(1+x) for a very small x there is a function log1p that is more accurate (see also expm1).
log(x^a) = a*log(x)
log(a*x) = log(a) + log(x)
Calculating log(x) for small x is fine. log(1e-308) does not suffer from underflow. Calculating exp(-1e308) is different, but that is far smaller than any representable answer anyway.
One way to solve your question might be (assuming p1 and p2 are less than $10^-308$) is to calculate log(p2) and p1/p2, and then
log(p1 + p2) = log(1 + p1/p2) + log(p2)
calculate the first term using log1p and you already have the second.
I am trying to replicate a graph on an example on Danish Data set used in the text Non-Life Insurance Mathematics.
I want to create the following new variable from my data set so I can plot the graph. My biggest challenge is how to sum(sigma) over w over j given that I have to start from max of two values to min of two values. I don't have the faintest idea how to do it in R. Guess I have a lot to still learn in how to do operations in R.
I would appreciate if some how can give me useful tip on how to go about it.
Below is the equation in question I couldn't replace the sigma sign so I used the literally interpretation (sum)
1/λ(i)) = 1/(2m + 1) * sum Wj from {j=max(1,i−m) to min(n,i+m)} for m = 50.
Try this
m = 50
total = 0
for (j in seq(max(1, i-m), min(n, i+m)) {
total = total + W[j]
}
total = total / (2 * m + 1)
lambda = 1 / total
or this
m = 50
lambda = 1 / (sum(W[max(i,i-m) : min(n,i+m)]) / (2 * m + 1))
I have the logarithms of very large values, for example:
log_a = 1347
log_b = 1351
And I am trying to solve this expression:
exp(log_a) - (0.1 * exp(log_b))
Or equivalently this (same expression just in a different form):
exp( log_a ) - exp( log(0.1) + log_b )
But of course every time I try to compute exp(log_a) or exp(log_b)
values I get Inf. Are there any tricks I can use to get a real result
for exp(log_a) - (0.1 * exp(log_b)), either in logarithm or
exponential form?
Thank you very much for the help!
library(Brobdingnag)
a <- as.brob(exp(1))^1347
a*(1-0.1*exp(4))
#[1] -exp(1348.5)
or calculated manually:
-(exp(1347+log(0.1*exp(4)-1))=-exp(1347+1.4951...)=-exp(1348.4951...)
X = exp(log_a) - (0.1 * exp(log_b))
= exp(log_a) * (1 - 0.1 * exp(log_b) / exp(log_b))
= exp(log_a) * (1 - exp(-log(10) + log_b - log_a))
= -exp(log_a) * expm1(-log(10) + log_b - log_a)
expm1 is a built-in function that accurately computes exp(x)-1 for x close to zero. You can get the logarithm of this only if the argument to expm1 is negative so that the entire expression is positive. Then you can just take the logarithm of the absolute value.
log X = log_a + log(-expm1(-log(10) + log_b - log_a))
You can use the gmp library for R, which supports large numbers (arbitrarily big, as far as I know)
for example
> bigz('11111111111111111111111111111111111111111111111111111111111111111111111111111')
Big Integer ('bigz') :
[1] 11111111111111111111111111111111111111111111111111111111111111111111111111111
I presume the exponentiation operator is included somewhere in the package. The manual is here: http://cran.r-project.org/web/packages/gmp/gmp.pdf