Related
I have this problem where I need to compute a continuous exponential moving average of a value in a discrete data stream. It's impossible to predict when I will receive the next sample, but EMA formulas expect the amount of time between each sample of data to be equal.
I found this article with a demonstration of how to work around this:
double exponentialMovingAverageIrregular( double alpha,
double sample,
double prevSample,
double deltaTime,
double emaPrev
)
{
double a = deltaTime / ( 1 - alpha );
double u = exp( a * -1 ); // e^(-a)
double v = ( 1 - u ) / a;
double emaNext = ( emaPrev * u )
+ ( prevSample * ( v - u ) )
+ ( sample * ( 1 - v ) );
return emaNext;
}
I compute alpha by using the following formula: 2 / (period + 1) where period is the number of milliseconds I want my EMA to pay attention to.
When I use this, the EMA moves way too quickly. I could have a 30 minute window that takes only two or three samples for the EMA to equal the input.
Here are some things I could be doing wrong:
I use milliseconds for computing alpha because that's the resolution of the timestamps on my input
I use milliseconds for deltaTime because that's what everything else is using
Per the suggestion of commenters on the article, I use a = deltaTime / (a - alpha) instead of a = deltaTime / alpha. Neither fixes the problem, but the latter causes more problems.
Here is a contrived example in which all the samples are exactly one minute apart. When computing alpha, I used 11 * 60 * 1000, or 11 minutes, leaving me with alpha = 0.0000030302984389417593. Notice how each ema has followed the sample almost exactly. This is not supposed to happen with an 11 minute window.
sample 10766.26, ema 10766.260001166664, time 1518991800000
sample 10750.75, ema 10750.750258499216, time 1518991860000
sample 10750.76, ema 10750.759999833333, time 1518991920000
sample 10750.75, ema 10750.750000166665, time 1518991980000
sample 10750.76, ema 10750.759999833333, time 1518992040000
sample 10750.76, ema 10750.759999999998, time 1518992100000
sample 10750.76, ema 10750.759999999998, time 1518992160000
sample 10750, ema 10750.000012666627, time 1518992220000
sample 10719.99, ema 10719.990500165151, time 1518992280000
sample 10720, ema 10719.999999833333, time 1518992340000
sample 10719.99, ema 10719.990000166667, time 1518992400000
sample 10719.99, ema 10719.99, time 1518992460000
sample 10709.27, ema 10709.270178666126, time 1518992520000
sample 10690.26, ema 10690.260316832373, time 1518992580000
sample 10690.27, ema 10690.269999833334, time 1518992640000
sample 10690.27, ema 10690.27, time 1518992700000
sample 10695, ema 10694.999921166906, time 1518992760000
sample 10699.98, ema 10699.979917000252, time 1518992820000
sample 10702.05, ema 10702.049965500104, time 1518992880000
sample 10744.99, ema 10744.989284335501, time 1518992940000
sample 10744.12, ema 10744.120014499955, time 1518993000000
The way the function was derived was not explained, and I didn't pay attention in math class. Any pointers would be greatly appreciated.
You Get Exactly What You've Defined:
given the way you defined alpha, the rest is a causal-chain:
|>>> a = 60000 / 0.999997
|>>> u = exp( -a )
|>>> v = ( 1 - u ) / a
|>>> u, ( v - u ), ( 1 - v )
( 0.0, 1.6666616666666667e-05, 0.99998333338333334 )
thus a
return ( ( emaPrev * u ) // -> 0. * emaPrev
+ ( prevSample * ( v - u ) ) // -> 0.000016 * prevSample
+ ( sample * ( 1 - v ) ) // -> 0.999983 * sample
); // ~= sample
returns nothing much different from the sample ( all the powers of the smoothing effect has been efficiently short-cut off the wannabe-smoothing-filter )
There are different motivations in different fields of use of the signal-filtering / smoothing. Strategies that may work fine in domains of mass-bound models for noisy sensor readouts, need not meet your expectations in other domains, like quant-modelling in trading and other domains that enjoy mass-less or otherwise absent products of inertia for processes and similar principal discontinuities of the subject of the study phenomena.
Out of question, it is worth spending some time both on math and on quant subjects of the study, both of these help you a lot in future work.
I was reading the cat and mouse Markov model on wikipedia, and decided to write some Julia code to empirically confirm the analytical results:
P = [
0 0 0.5 0 0.5 ;
0 0 1 0 0 ;
0.25 0.25 0 0.25 0.25;
0 0 0.5 0 0.5 ;
0 0 0 0 1
]
prob_states = transpose([0.0, 1, 0, 0, 0])
prob_end = [0.0]
for i in 1:2000
prob_states = prob_states * P
prob_end_new = (1 - sum(prob_end)) * prob_states[end]
push!(prob_end, prob_end_new)
println("Ending probability: ", prob_end_new)
println("Cumulative: ", sum(prob_end))
end
println("Expected lifetime: ", sum(prob_end .* Array(1:2001)))
Here P is the transition matrix, prob_states is the probability distribution of the states at each iteration, prob_end is an array of probilities of termination at each step (e.g. prob_end[3] is probability of termination at step 3).
According to the result of this script, the expected lifetime of the mouse is about 4.3, while the analytical result is 4.5. The script makes sense to me so I really don't know where it could have gone wrong. Could anyone help?
P.S. Increasing the number of iteration by an order of magnitude almost changes nothing.
The probability of the mouse surviving approaches zero very quickly. This is not only unfortunate for the mouse, but also unfortunate for us as we cannot use 64-bit floating point numbers (which Julia is using here by default) to accurately approximate these tiny values of survival time.
In fact, most of the values prob_end are identically zero after a relatively low number of iterations, but evaluated analytically these values should be not-quite zero. The Float64 type simply cannot represent such small positive numbers.
This is why multiplying and summing the arrays never quite gets to 4.5; steps which should nudge the sum closer to this value fail cannot make the contribution as they are equal to zero. We see convergence to the lower value instead.
Using a different type which can represent arbitrarily tiny positive values, is a possibility, maybe. There are some suggestions here but you may find them very slow and memory-heavy when performing anything more than a few hundred iterations of this Markov chain model.
Another solution could be to convert the code to work with log probabilities instead (which are often used to overcome exactly this limitation of floating point numbers).
If you just want to empirically confirm the result, you can simulate the model directly:
const first_index = 1
const last_index = 5
const cat_start = 2
const mouse_start = 4
function move(i)
if i == first_index
return first_index + 1
elseif i == last_index
return last_index - 1
else
return i + rand([-1,1])
end
end
function step(cat, mouse)
return move(cat), move(mouse)
end
function game(cat, mouse)
i = 1
while cat != mouse
cat, mouse = step(cat, mouse)
i += 1
end
return i
end
function run()
res = [game(cat_start, mouse_start) for i=1:10_000_000]
return mean(res), std(res)/sqrt(length(res))
end
μ,σ = run()
println("Mean lifetime: $μ ± $σ")
Example output:
Mean lifetime: 4.5004993 ± 0.0009083568998918751
On recent interview I was asked the following question. There is a function random2(), wich returns 0 or 1 with equal probability (0.5). Write implementation of random4() and random3() using random2().
It was easy to implement random4() like this
if(random2())
return random2();
return random2() + 2;
But I had difficulties with random3(). The only realization I could represent:
uint32_t sum = 0;
for (uint32_t i = 0; i != N; ++i)
sum += random2();
return sum % 3;
This implementation of random4() is based only my intuition only. I'm not sure if it is correct actually, because I can't mathematically prove its correctness. Can somebody help me with this question, please.
random3:
Not sure if this is the most efficient way, but here's my take:
x = random2 + 2*random2
What can happen:
0 + 0 = 0
0 + 2 = 2
1 + 0 = 1
1 + 2 = 3
The above are all the possibilities of what can happen, thus each has equal probability, so...
(p(x=c) is the probability that x = c)
p(x=0) = 0.25
p(x=1) = 0.25
p(x=2) = 0.25
p(x=3) = 0.25
Now while x = 3, we just keep generating another number, thus giving equal probability to 0,1,2. More technically, you would distribute the probability from x=3 across all of them repeatedly such that p(x=3) tends to 0, thus the probability of the others will tend to 0.33 each.
Code:
do
val = random2() + 2*random2();
while (val != 3);
return val;
random4:
Let's run through your code:
if(random2())
return random2();
return random2() + 2;
First call has 50% chance of 1 (true) => returns either 0 or 1 with 50% * 50% probability, thus 25% each
First call has 50% chance of 0 (false) => returns either 2 or 3 with 50% * 50% probability, thus 25% each
Thus your code generates 0,1,2,3 with equal probability.
Update inspired by e4e5f4's answer:
For a more deterministic answer than the one I provided above...
Generate some large number by calling random2 a bunch of times and mod the result by the desired number.
This won't be exactly the right probability for each, but it will be close.
So, for a 32-bit integer by calling random2 32 times, target = 3:
Total numbers: 4294967296
Number of x's such that x%3 = 1 or 2: 1431655765
Number of x's such that x%3 = 0: 1431655766
Probability of 1 or 2 (each): 0.33333333325572311878204345703125
Probability of 0: 0.3333333334885537624359130859375
So within 0.00000002% of the correct probability, seems pretty close.
Code:
sum = 0;
for (int i = 0; i < 32; i++)
sum = 2*sum + random2();
return sum % N;
Note:
As pjr pointed out, this is, in general, far less efficient than the rejection method above. The probability of getting to the same number of calls of random2 (i.e. 32) (assuming this is the slowest operation) with the rejection method is 0.25^(32/2) = 0.0000000002 = 0.00000002%. This together with the fact that this method isn't exact, gives way more preference to the rejection method. Lower this number decreases the running time, but increases the error, and it would probably need to be lowered quite a bit (thus reaching a high error) to approach the average running time of the rejection method.
It is useful to note the above algorithm has a maximum running time. The rejection method does not. If your random number generator is totally broken for some reason, it could keep generating the rejected number and run for quite a while or forever with the rejection method, but the for-loop above will run 32 times, regardless of what happens.
Using modulo(%) is not recommended because it introduces bias. Mapping will be nice only if n is power of 2. Otherwise some kind of rejection is involved as suggested by other answer.
Another generic approach would be to emulate built-in PRNGs by -
Generate 32 random2() and map it to a 32-bit integer
Get random number in range (0,1) by dividing it by max integer value
Simply multiply this number by n (=3,4...73 so on) and floor to get desired output
I am trying to translate a function in a book into code, using MATLAB and C#.
I am first trying to get the function to work properly in MATLAB.
Here are the instructions:
The variables are:
xt and m can be ignored.
zMax = Maximum Sensor Range (100)
zkt = Sensor Measurement (49)
zkt* = What sensor measurement should have been (50)
oHit = Std Deviation of my measurement (5)
I have written the first formula, N(zkt;zkt*,oHit) in MATLAB as this:
hitProbabilty = (1/sqrt( 2*pi * (oHit^2) ))...
* exp(-0.5 * (((zkt- zktStar) ^ 2) / (oHit^2)) );
This gives me the Gaussian curve I expect.
I have an issue with the definite integral below, I do not understand how to turn this into a real number, because I get horrible values out my code, which is this:
func = #(x) hitProbabilty * zkt * x;
normaliser = quad(func, 0, max) ^ -1;
hitProbabilty = normaliser * hitProbabilty;
Can someone help me with this integral? It is supposed to normalize my curve, but it just goes crazy.... (I am doing this for zkt 0:1:100, with everything else the same, and graphing the probability it should output.)
You should use the error function ERF (available in basic MATLAB)
EDIT1:
As #Jim Brissom mentioned, the cumulative distribution function (CDF) is related to the error function by:
normcdf(X) = (1 + erf(X/sqrt(2)) / 2 , where X~N(0,1)
Note that NORMCDF requires the Statistics Toolbox
EDIT2:
I think there's been a small confusion seeing the comments.. The above only compute the normalizing factor, so if you want to compute the final probability over a certain range of values, you should do this:
zMax = 100; %# Maximum Sensor Range
zktStar = 50; %# What sensor measurement should have been
oHit = 5; %# Std Deviation of my measurement
%# p(0<z<zMax) = p(z<zMax) - p(z<0)
ncdf = diff( normcdf([0 zMax], zktStar, oHit) );
normaliser = 1 ./ ncdf;
zkt = linspace(0,zMax,500); %# Sensor Measurement, 500 values in [0,zMax]
hitProbabilty = normpdf(zkt, zktStar, oHit) * normaliser;
plot(zkt, hitProbabilty)
xlabel('z^k_t'), ylabel('P_{hit}(z^k_t)'), title('Measurement Probability')
The N in your code is just the well-known gaussian or normal distribution. I am mentioning this because since you re-implemented it in Matlab, it seems you missed that, seeing as how it is obviously already implemented in Matlab.
Integrating the normal distribution will yield a cumulative distribution function, available in Matlab for the normal distribution via normcdf. The ncdf can be written in terms of erf, which is probably what Amro was talking about.
Using normcdf avoids integrating manually.
In case you still need the result for the integral.
From Mathematica. The Calc is
hitProbabilty[zkt_] := (1/Sqrt[2*Pi*oHit^2])*Exp[-0.5*(((zkt - zktStar)^2)/(oHit^2))];
Integrate[hitProbabilty[zkt], {zkt, 0, zMax}];
The result is (just for copy/paste)
((1.2533141373155001*oHit*zktStar*Erf[(0.7071067811865476*Sqrt[zktStar^2])/oHit])/
Sqrt[zktStar^2] +
(1.2533141373155001*oHit*(zMax-zktStar)*Erf[(0.7071067811865476*Sqrt[(zMax-zktStar)^2])/oHit])/
Sqrt[(zMax-zktStar)^2])/(2*oHit*Sqrt[2*Pi])
Where Erf[] is the error function
HTH!
I have done this before, but now I'm struggling with it again, and I think I am not understanding the math underlying the issue.
I want to set a random number on within a small range on either side of 1. Examples would be .98, 1.02, .94, 1.1, etc. All of the examples I find describe getting a random number between 0 and 100, but how can I use that to get within the range I want?
The programming language doesn't really matter here, though I am using Pure Data. Could someone please explain the math involved?
Uniform
If you want a (psuedo-)uniform distribution (evenly spaced) between 0.9 and 1.1 then the following will work:
range = 0.2
return 1-range/2+rand(100)*range/100
Adjust the range accordingly.
Pseudo-normal
If you wanted a normal distribution (bell curve) you would need special code, which would be language/library specific. You can get a close approximation with this code:
sd = 0.1
mean = 1
count = 10
sum = 0
for(int i=1; i<count; i++)
sum=sum+(rand(100)-50)
}
normal = sum / count
normal = normal*sd + mean
Generally speaking, to get a random number within a range, you don't get a number between 0 and 100, you get a number between 0 and 1. This is inconsequential, however, as you could simply get the 0-1 number by dividing your # by 100 - so I won't belabor the point.
When thinking about the pseudocode of this, you need to think of the number between 0 and 1 which you obtain as a percentage. In other words, if I have an arbitrary range between a and b, what percentage of the way between the two endpoints is the point I have randomly selected. (Thus a random result of 0.52 means 52% of the distance between a and b)
With this in mind, consider the problem this way:
Set the start and end-points of your range.
var min = 0.9;
var max = 1.1;
Get a random number between 0 and 1
var random = Math.random();
Take the difference between your start and end range points (b - a)
var range = max - min;
Multiply your random number by the difference
var adjustment = range * random;
Add back in your minimum value.
var result = min + adjustment;
And, so you can understand the values of each step in sequence:
var min = 0.9;
var max = 1.1;
var random = Math.random(); // random == 0.52796 (for example)
var range = max - min; // range == 0.2
var adjustment = range * random; // adjustment == 0.105592
var result = min + adjustment; // result == 1.005592
Note that the result is guaranteed to be within your range. The minimum random value is 0, and the maximum random value is 1. In these two cases, the following occur:
var min = 0.9;
var max = 1.1;
var random = Math.random(); // random == 0.0 (minimum)
var range = max - min; // range == 0.2
var adjustment = range * random; // adjustment == 0.0
var result = min + adjustment; // result == 0.9 (the range minimum)
var min = 0.9;
var max = 1.1;
var random = Math.random(); // random == 1.0 (maximum)
var range = max - min; // range == 0.2
var adjustment = range * random; // adjustment == 0.2
var result = min + adjustment; // result == 1.1 (the range maximum)
return 0.9 + rand(100) / 500.0
or am I missing something?
If rand() returns you a random number between 0 and 100, all you need to do is:
(rand() / 100) * 2
to get a random number between 0 and 2.
If on the other hand you want the range from 0.9 to 1.1, use the following:
0.9 + ((rand() / 100) * 0.2)
You can construct any distribution you like form uniform in range [0,1) by changing variable. Particularly, if you want random of some distribution with cumulative distribution function F, you just substitute uniform random from [0,1) to inverse function for desired CDF.
One special (and maybe most popular) case is normal distribution N(0,1). Here you can use Box-Muller transform. Scaling it with stdev and adding a mean you get normal distribution with desired parameters.
You can sum uniform randoms and get some approximation of normal distribution, this case is considered by Nick Fortescue above.
If your source randoms are integers you should firstly construct a random in real domain with some known distribution. For example, uniform distribution in [0,1) you can construct such way. You get first integer in range from 0 to 99, multiply it by 0.01, get second integer, multiply it by 0.0001 and add to first and so on. This way you get a number 0.XXYYZZ... Double precision is about 16 decimal digits, so you need 8 integer randoms to construct double uniform one.
Box-Müller to the rescue.
var z2_cached;
function normal_random(mean, variance) {
if ( z2_cached ) {
var z2 = z2_cached;
z2_cached = 0
return z2 * Math.sqrt(variance) + mean;
}
var x1 = Math.random();
var x2 = Math.random();
var z1 = Math.sqrt(-2 * Math.log(x1) ) * Math.cos( 2*Math.PI * x2);
var z2 = Math.sqrt(-2 * Math.log(x1) ) * Math.sin( 2*Math.PI * x2);
z2_cached = z2;
return z1 * Math.sqrt(variance) + mean;
}
Use with values of mean 1 and variance e.g. 0.01
for ( var i=0; i < 20; i++ ) console.log( normal_random(1, 0.01) );
0.937240893365304
1.072511121460833
0.9950053748909895
1.0034139439164074
1.2319710866884104
0.9834737343090275
1.0363970887198277
0.8706648577217094
1.0882382154101415
1.0425139197341595
0.9438723605883214
0.935894021237943
1.0846400276817076
1.0428213927823682
1.020602499547105
0.9547701472093025
1.2598174560413493
1.0086997644531541
0.8711594789918106
0.9669499056660755
Function gives approx. normal distribution around mean with given variance.
low + (random() / 100) * range
So for example:
0.90 + (random() / 100) * 0.2
How near? You could use a Gaussian (a.k.a. Normal) distribution with a mean of 1 and a small standard deviation.
A Gaussian is suitable if you want numbers close to 1 to be more frequent than numbers a bit further away from 1.
Some languages (such as Java) will have support for Gaussians in the standard library.
Divide by 100 and add 1. (I assume you are looking for a range from 0 to 2?)
You want a range from -1 to 1 as output from your rand() expression.
( rand(2) - 1 )
Then scale that -1 to 1 range as needed. Say, for a .1 variation on either side:
(( rand(2) - 1 ) / 10 )
Then just add one.
(( rand(2) - 1 ) / 10 ) + 1
Rand() already gives you a random number between 0 and 100. The maximum different random number you can get with this are 100 thus Assuming that you want up to three decimal numbers 0.950-1.050 is the range you would be looking at.
The distribution can then be achieved by
0.95 + ((rand() / 100)
Are you looking for the random no. from range 1 to 2, like 1.1,1.5,1.632, etc. if yes then here is a simple python code:
import random
print (random.random%2)+1
var randomNumber = Math.random();
while(randomNumber<0.9 && randomNumber>0.1){
randomNumber = Math.random();
}
if(randomNumber>=0.9){
alert(randomNumber);
}
else if(randomNumber<=0.1){
alert(1+randomNumber);
}
For numbers from 0.9 to 1.1
seed = 1
range = 0,1
if your random is from 0..100
f_rand = random/100
the generated number
gen_number = (seed+f_rand*range*2)-range
You will get
1,04; 1,08; 1,01; 0,96; ...
with seed 3, range 2 => 1,95; 4,08; 2,70; 3,06; ...
I didn't understand this (sorry):
I am trying to set a random number on either side of 1: .98, 1.02, .94, 1.1, etc.
So, I'll provide a general solution for the problem instead.
Converting a random number generator
If you have a random number generator in a give range [0, 1)* with uniform distribution you can convert it to any distribution using the following method:
1 - Describe the distribution as a function defined in the output range and with total area of 1. So this function is f(x) = the probability of getting the value x.
2 - Integrate** the function.
3 - Equate it to the "randomic"*.
4 - Solve the equation for x. So ti gives you the value of x in function of the randomic.
*: Generalization for any input distribution is below.
**: The constant term of the integrated function is 0 (that is, you just discard it).
**: That is a variable the represents the result of generating a random number with uniform distribution in the range [0, 1). [I'm not sure if that's the correct name in English]
Example:
Let's say you want a value with the distribution f(x)=x^2 from 0 to 100. Well that function is not normalized because the total area below the function in the range is 1000000/3 not 1. So you normalize it scaling the curve in the vertical axis (keeping the relative proportions), that is dividing by the total area: f(x)=3*x^2 / 1000000 from 0 to 100.
Now, we have a function with the a total area of 1. The next step is to integrate it (you may have already have done that to get the area) and equte it to the randomic.
The integrated function is: F(x)=x^3/1000000+c. And equate it to the randomic: r=x^3/1000000 (remember that we discard the constant term).
Now, we need to solve the equation for x, the resulting expression: x=100*r^(1/3). Now you can use this formula to generate numbers with the desired distribution.
Generalization
If you have a random number generator with a custom distribution and want another different arbitrary distribution, you first need the source distribution function and then use it to express the target arbirary random number generator. To get the distribution function do the steps up to 3. For the target do all the steps, and then replace the randomic with the expression you got from the source distribution.
This is better understood with an example...
Example:
You have a random number generator with uniform distribution in the range [0, 100) and you want.. the same distribution f(x)=3*x^2 / 1000000 from 0 to 100 for simplicity [Since for that one we already did all the steps giving us x=100*r^(1/3)].
Since the source distribution is uniform the function is constant: f(z)=1. But we need to normalize for the range, leaving us with: f(z)=1/100.
Now, we integrate it: F(z)=z/100. And equate it to the randomic: r=z/100, but this time we don't solve it for x, instead we use it to replace r in the target:
x=100*r^(1/3) where r = z/100
=>
x=100*(z/100)^(1/3)
=>
x=z^(1/3)
And now you can use x=z^(1/3) to calculate random numbers with the distribution f(x)=3*x^2 / 1000000 from 0 to 100 starting with a random number in the distribution f(z)=1/100 from 0 to 100 [uniform].
Note: If you have normal distribution, use the bell function instead. The same method works for any other distribution. Take care of possible asymptote some distributions make create, you may need to try different ways to solve the equations.
On discrete distributions
Some times you need to express a discrete distribution, for example, you want to get 0 with 95% chance and 1 with 5% chance. So how do you do that?
Well, you divide it in rectangular distributions in such way that the ranges join to [0, 1) and use the randomic to evaluate:
0 if r is in [0, 0.95)
f(r) = {
1 if r is in [0.95, 1)
Or you can take the complex path, which is to write a distribution function like this (making each option exactly a range of length 1):
0.95 if x is in [0, 1)
f(x) = {
0.5 if x is in [1, 2)
Since each range has a length of 1 and the assigned values sum up to 1 we know that the total area is 1. Now the next step would be to integrate it:
0.95*x if x is in [0, 1)
F(x) = {
(0.5*(x-1))+0.95 = 0.5*x + 0.45 if x is in [1, 2)
Equate it to the randomic:
0.95*x if x is in [0, 1)
r = {
0.5*x + 0.45 if x is in [1, 2)
And solve the equation...
Ok, to solve that kind of equation, start by calculating the output ranges by applying the function:
[0, 1) becomes [0, 0.95)
[1, 2) becomes [0.95, {(0.5*(x-1))+0.95 where x = 2} = 1)
Now, those are the ranges for the solution:
? if r is in [0, 0.95)
x = {
? if r is in [0.95, 1)
Now, solve the inner functions:
r/0.95 if r is in [0, 0.95)
x = {
2*(r-0.45) = 2*r-0.9 if r is in [0.95, 1)
But, since the output is discrete, we end up with the same result after doing integer part:
0 if r is in [0, 0.95)
x = {
1 if r is in [0.95, 1)
Note: using random to mean pseudo random.
Edit: Found it on wikipedia (I knew I didn't invent it).