Related
Ok so I figured out how to plot the credible intervals for a univariate linear model in Turing.jl using the following code (I'm replicating Statistical rethinking by McElreath) This particular exercise is in chapter 4. If anyone has already plotted these types of models with Turing and can give me a guide, it would be great!!!
Univariate model code:
using Turing
using StatsPlots
using Plots
height = df2.height
weight = df2.weight
#model heightmodel(y, x) = begin
#priors
α ~ Normal(178, 100)
σ ~ Uniform(0, 50)
β ~ LogNormal(0, 10)
x_bar = mean(x)
#model
μ = α .+ (x.-x_bar).*β
y ~ MvNormal(μ, σ)
end
chns = sample(heightmodel(height, weight), NUTS(), 100000)
## code 4.43
describe(chns) |> display
# covariance and correlation
alph = get(chns, :α)[1].data
bet = get(chns, :β)[1].data
sigm = get(chns, :σ)[1].data
vecs = (alph[1:352], bet[1:352])
arr = vcat(transpose.(vecs)...)'
ss = [vec(alph + bet.*(x)) for x in 25:1:70]
arrr = vcat(transpose.(ss)...)'
plot([mean(arrr[:,x]) for x in 1:46],25:1:70, ribbon = ([-1*(quantile(arrr[:,x],[0.1,0.9])[1] - mean(arrr[:,x])) for x in 1:46], [quantile(arrr[:,x],[0.1,0.9])[2] - mean(arrr[:,x]) for x in 1:46]))
Credible interval Univariate:
However, when I try to replicate it with a multivatiate function, very strange things are drawn:
Multivariate model code:
weight_s = (df.weight .-mean(df.weight))./std(df.weight)
weight_s² = weight_s.^2
#model heightmodel(height, weight, weight²) = begin
#priors
α ~ Normal(178, 20)
σ ~ Uniform(0, 50)
β1 ~ LogNormal(0, 1)
β2 ~ Normal(0, 1)
#model
μ = α .+ weight.*β1 + weight².*β2
height ~ MvNormal(μ, σ)
end
chns = sample(heightmodel(height, weight_s, weight_s²), NUTS(), 100000)
describe(chns) |> display
### painting the fit
alph = get(chns, :α)[1].data
bet1 = get(chns, :β1)[1].data
bet2 = get(chns, :β2)[1].data
vecs = (alph[1:99000], bet1[1:99000], bet2[1:99000])
arr = vcat(transpose.(vecs)...)'
polinomial = [vec(alph + bet1.*(x) + bet2.*(x.^2)) for x in -2:0.01:2]
arrr = vcat(transpose.(polinomial)...)'
plot([mean(arrr[:,x]) for x in 1:401],-2:0.01:2, ribbon = ([-1*(quantile(arrr[:,x],[0.1,0.9])[1] - mean(arrr[:,x])) for x in 1:46], [quantile(arrr[:,x],[0.1,0.9])[2] - mean(arrr[:,x]) for x in 1:46]))
Credible interval Univariate:
In the Julia Slack channel (https://slackinvite.julialang.org/) Jens was kind enough to give me the answer. The credit goes to him --He doesn't have a SO account.
The main problem was that I was lacking simplicity and was trying to plot the mean the wrong way --in a very inefficient and strange way might I add. Each parameter has a vector which corresponds to 99000 draws from the posterior distribution, I was trying to draw the mean through the matrix but it's much easier to define the median first and then plot it, then you don't make mistakes as I did calculating the mean.
old code
vecs = (alph[1:99000], bet1[1:99000], bet2[1:99000])
arr = vcat(transpose.(vecs)...)'
[mean(arrr[:,x]) for x in 1:401]
can be written as:
testweights = -2:0.01:2
arr = [fheight.(w, res.α, res.β1, res.β2) for w in testweights]
m = [mean(v) for v in arr]
Moreover, the way Jens defined the Credible intervals is much more elegant and Julianic:
Jens' code:
quantiles = [quantile(v, [0.1, 0.9]) for v in arr]
lower = [q[1] - m for (q, m) in zip(quantiles, m)]
upper = [q[2] - m for (q, m) in zip(quantiles, m)]
My code:
ribbon = ([-1*(quantile(arrr[:,x],[0.1,0.9])[1] - mean(arrr[:,x])) for x in 1:46], [quantile(arrr[:,x],[0.1,0.9])[2] - mean(arrr[:,x]) for x in 1:46]))
Complete Solution:
weight_s = (d.weight .-mean(d.weight))./std(d.weight)
height = d.height
#model heightmodel(height, weight) = begin
#priors
α ~ Normal(178, 20)´
σ ~ Uniform(0, 50)
β1 ~ LogNormal(0, 1)
β2 ~ Normal(0, 1)
#model
μ = α .+ weight .* β1 + weight.^2 .* β2
# or μ = fheight.(weight, α, β1, β2) if we are defining fheigth anyways
height ~ MvNormal(μ, σ)
end
chns = sample(heightmodel(height, weight_s), NUTS(), 10000)
describe(chns) |> display
res = DataFrame(chns)
fheight(weight, α, β1, β2) = α + weight * β1 + weight^2 * β2
testweights = -2:0.01:2
arr = [fheight.(w, res.α, res.β1, res.β2) for w in testweights]
m = [mean(v) for v in arr]
quantiles = [quantile(v, [0.1, 0.9]) for v in arr]
lower = [q[1] - m for (q, m) in zip(quantiles, m)]
upper = [q[2] - m for (q, m) in zip(quantiles, m)]
plot(testweights, m, ribbon = [lower, upper])
I've tried to reproduce the model from a PYMC3 and Stan comparison. But it seems to run slowly and when I look at #code_warntype there are some things -- K and N I think -- which the compiler seemingly calls Any.
I've tried adding types -- though I can't add types to turing_model's arguments and things are complicated within turing_model because it's using autodiff variables and not the usuals. I put all the code into the function do_it to avoid globals, because they say that globals can slow things down. (It actually seems slower, though.)
Any suggestions as to what's causing the problem? The turing_model code is what's iterating, so that should make the most difference.
using Turing, StatsPlots, Random
sigmoid(x) = 1.0 / (1.0 + exp(-x))
function scale(w0::Float64, w1::Array{Float64,1})
scale = √(w0^2 + sum(w1 .^ 2))
return w0 / scale, w1 ./ scale
end
function do_it(iterations::Int64)::Chains
K = 10 # predictor dimension
N = 1000 # number of data samples
X = rand(N, K) # predictors (1000, 10)
w1 = rand(K) # weights (10,)
w0 = -median(X * w1) # 50% of elements for each class (number)
w0, w1 = scale(w0, w1) # unit length (euclidean)
w_true = [w0, w1...]
y = (w0 .+ (X * w1)) .> 0.0 # labels
y = [Float64(x) for x in y]
σ = 5.0
σm = [x == y ? σ : 0.0 for x in 1:K, y in 1:K]
#model turing_model(X, y, σ, σm) = begin
w0_pred ~ Normal(0.0, σ)
w1_pred ~ MvNormal(σm)
p = sigmoid.(w0_pred .+ (X * w1_pred))
#inbounds for n in 1:length(y)
y[n] ~ Bernoulli(p[n])
end
end
#time chain = sample(turing_model(X, y, σ, σm), NUTS(iterations, 200, 0.65));
# ϵ = 0.5
# τ = 10
# #time chain = sample(turing_model(X, y, σ), HMC(iterations, ϵ, τ));
return (w_true=w_true, chains=chain::Chains)
end
chain = do_it(1000)
https://en.wikipedia.org/wiki/Superellipse
I have read the SO questions on how to point-pick from a circle and an ellipse.
How would one uniformly select random points from the interior of a super-ellipse?
More generally, how would one uniformly select random points from the interior of the curve described by an arbitrary super-formula?
https://en.wikipedia.org/wiki/Superformula
The discarding method is not considered a solution, as it is mathematically unenlightening.
In order to sample the superellipse, let's assume without loss of generality that a = b = 1. The general case can be then obtained by rescaling the corresponding axis.
The points in the first quadrant (positive x-coordinate and positive y-coordinate) can be then parametrized as:
x = r * ( cos(t) )^(2/n)
y = r * ( sin(t) )^(2/n)
with 0 <= r <= 1 and 0 <= t <= pi/2:
Now, we need to sample in r, t so that the sampling transformed into x, y is uniform. To this end, let's calculate the Jacobian of this transform:
dx*dy = (2/n) * r * (sin(2*t)/2)^(2/n - 1) dr*dt
= (1/n) * d(r^2) * d(f(t))
Here, we see that as for the variable r, it is sufficient to sample uniformly the value of r^2 and then transform back with a square root. The dependency on t is a bit more complicated. However, with some effort, one gets
f(t) = -(n/2) * 2F1(1/n, (n-1)/n, 1 + 1/n, cos(t)^2) * cos(t)^(2/n)
where 2F1 is the hypergeometric function.
In order to obtain uniform sampling in x,y, we need now to sample uniformly the range of f(t) for t in [0, pi/2] and then find the t which corresponds to this sampled value, i.e., to solve for t the equation u = f(t) where u is a uniform random variable sampled from [f(0), f(pi/2)]. This is essentially the same method as for r, nevertheless in that case one can calculate the inverse directly.
One small issue with this approach is that the function f is not that well-behaved near zero - the infinite slope makes it quite challenging to find a root of u = f(t). To circumvent this, we can sample only the "upper part" of the first quadrant (i.e., area between lines x=y and x=0) and then obtain all the other points by symmetry (not only in the first quadrant but also for all the other ones).
An implementation of this method in Python could look like:
import numpy as np
from numpy.random import uniform, randint, seed
from scipy.optimize import brenth, ridder, bisect, newton
from scipy.special import gamma, hyp2f1
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import matplotlib.gridspec as gridspec
seed(100)
def superellipse_area(n):
#https://en.wikipedia.org/wiki/Superellipse#Mathematical_properties
inv_n = 1. / n
return 4 * ( gamma(1 + inv_n)**2 ) / gamma(1 + 2*inv_n)
def sample_superellipse(n, num_of_points = 2000):
def f(n, x):
inv_n = 1. / n
return -(n/2)*hyp2f1(inv_n, 1 - inv_n, 1 + inv_n, x)*(x**inv_n)
lb = f(n, 0.5)
ub = f(n, 0.0)
points = [None for idx in range(num_of_points)]
for idx in range(num_of_points):
r = np.sqrt(uniform())
v = uniform(lb, ub)
w = bisect(lambda w: f(n, w**n) - v, 0.0, 0.5**(1/n))
z = w**n
x = r * z**(1/n)
y = r * (1 - z)**(1/n)
if uniform(-1, 1) < 0:
y, x = x, y
x = (2*randint(0, 2) - 1)*x
y = (2*randint(0, 2) - 1)*y
points[idx] = [x, y]
return points
def plot_superellipse(ax, n, points):
coords_x = [p[0] for p in points]
coords_y = [p[1] for p in points]
ax.set_xlim(-1.25, 1.25)
ax.set_ylim(-1.25, 1.25)
ax.text(-1.1, 1, '{n:.1f}'.format(n = n), fontsize = 12)
ax.scatter(coords_x, coords_y, s = 0.6)
params = np.array([[0.5, 1], [2, 4]])
fig = plt.figure(figsize = (6, 6))
gs = gridspec.GridSpec(*params.shape, wspace = 1/32., hspace = 1/32.)
n_rows, n_cols = params.shape
for i in range(n_rows):
for j in range(n_cols):
n = params[i, j]
ax = plt.subplot(gs[i, j])
if i == n_rows-1:
ax.set_xticks([-1, 0, 1])
else:
ax.set_xticks([])
if j == 0:
ax.set_yticks([-1, 0, 1])
else:
ax.set_yticks([])
#ensure that the ellipses have similar point density
num_of_points = int(superellipse_area(n) / superellipse_area(2) * 4000)
points = sample_superellipse(n, num_of_points)
plot_superellipse(ax, n, points)
fig.savefig('fig.png')
This produces:
I want to plot a surface whose elevation is the result of 3D interpolation on scattered data:
I have coordinates x, y, z and the scalar value v in three column vectors. I would like to interpolate v on, say, a constant x plane with coordinates xi, yi, zi obtaining vi and eventually plot the surface (yi,zi,vi).
How can I do that?
Here comes the solution I have found, thanks to andy1978 on #octave irc channel and Nicola:
x1 = linspace (min(x),max(x),50) ;
y1 = linspace (min(y),max(y),50) ;
z1 = linspace (min(z),max(z),50) ;
[xx, yy, zz] = meshgrid (x1, y1, z1) ;
vv = griddata3 (x, y, z, v, xx, yy, zz ) ;
x0 = 0.05 ; % constant x of chosen plane
[~,i] = min (abs (x - x0)) ;
vv_sect = vv (:, i, :) ; % here I slice the matrix of interpolated results at x=x0
vv_out = squeeze (vv_sect) ; % and this is what I needed to reduce the dimensionality of
% the sliced matrix
[yy2,zz2] = meshgrid (y1, z1) ;
surf (yy2, zz2, vv_out ) ;
How can I draw a perpendicular on a line segment from a given point? My line segment is defined as (x1, y1), (x2, y2), If I draw a perpendicular from a point (x3,y3) and it meets to line on point (x4,y4). I want to find out this (x4,y4).
I solved the equations for you:
k = ((y2-y1) * (x3-x1) - (x2-x1) * (y3-y1)) / ((y2-y1)^2 + (x2-x1)^2)
x4 = x3 - k * (y2-y1)
y4 = y3 + k * (x2-x1)
Where ^2 means squared
From wiki:
In algebra, for any linear equation
y=mx + b, the perpendiculars will all
have a slope of (-1/m), the opposite
reciprocal of the original slope. It
is helpful to memorize the slogan "to
find the slope of the perpendicular
line, flip the fraction and change the
sign." Recall that any whole number a
is itself over one, and can be written
as (a/1)
To find the perpendicular of a given
line which also passes through a
particular point (x, y), solve the
equation y = (-1/m)x + b, substituting
in the known values of m, x, and y to
solve for b.
The slope of the line, m, through (x1, y1) and (x2, y2) is m = (y1 - y2) / (x1 - x2)
I agree with peter.murray.rust, vectors make the solution clearer:
// first convert line to normalized unit vector
double dx = x2 - x1;
double dy = y2 - y1;
double mag = sqrt(dx*dx + dy*dy);
dx /= mag;
dy /= mag;
// translate the point and get the dot product
double lambda = (dx * (x3 - x1)) + (dy * (y3 - y1));
x4 = (dx * lambda) + x1;
y4 = (dy * lambda) + y1;
You know both the point and the slope, so the equation for the new line is:
y-y3=m*(x-x3)
Since the line is perpendicular, the slope is the negative reciprocal. You now have two equations and can solve for their intersection.
y-y3=-(1/m)*(x-x3)
y-y1=m*(x-x1)
You will often find that using vectors makes the solution clearer...
Here is a routine from my own library:
public class Line2 {
Real2 from;
Real2 to;
Vector2 vector;
Vector2 unitVector = null;
public Real2 getNearestPointOnLine(Real2 point) {
unitVector = to.subtract(from).getUnitVector();
Vector2 lp = new Vector2(point.subtract(this.from));
double lambda = unitVector.dotProduct(lp);
Real2 vv = unitVector.multiplyBy(lambda);
return from.plus(vv);
}
}
You will have to implement Real2 (a point) and Vector2 and dotProduct() but these should be simple:
The code then looks something like:
Point2 p1 = new Point2(x1, y1);
Point2 p2 = new Point2(x2, y2);
Point2 p3 = new Point2(x3, y3);
Line2 line = new Line2(p1, p2);
Point2 p4 = getNearestPointOnLine(p3);
The library (org.xmlcml.euclid) is at:
http://sourceforge.net/projects/cml/
and there are unit tests which will exercise this method and show you how to use it.
#Test
public final void testGetNearestPointOnLine() {
Real2 p = l1112.getNearestPointOnLine(new Real2(0., 0.));
Real2Test.assertEquals("point", new Real2(0.4, -0.2), p, 0.0000001);
}
Compute the slope of the line joining points (x1,y1) and (x2,y2) as m=(y2-y1)/(x2-x1)
Equation of the line joining (x1,y1) and (x2,y2) using point-slope form of line equation, would be y-y2 = m(x-x2)
Slope of the line joining (x3,y3) and (x4,y4) would be -(1/m)
Again, equation of the line joining (x3,y3) and (x4,y4) using point-slope form of line equation, would be y-y3 = -(1/m)(x-x3)
Solve these two line equations as you solve a linear equation in two variables and the values of x and y you get would be your (x4,y4)
I hope this helps.
cheers
Find out the slopes for both the
lines, say slopes are m1 and m2 then
m1*m2=-1 is the condition for
perpendicularity.
Matlab function code for the following problem
function Pr=getSpPoint(Line,Point)
% getSpPoint(): find Perpendicular on a line segment from a given point
x1=Line(1,1);
y1=Line(1,2);
x2=Line(2,1);
y2=Line(2,1);
x3=Point(1,1);
y3=Point(1,2);
px = x2-x1;
py = y2-y1;
dAB = px*px + py*py;
u = ((x3 - x1) * px + (y3 - y1) * py) / dAB;
x = x1 + u * px;
y = y1 + u * py;
Pr=[x,y];
end
Mathematica introduced the function RegionNearest[] in version 10, 2014. This function could be used to return an answer to this question:
{x4,y4} = RegionNearest[Line[{{x1,y1},{x2,y2}}],{x3,y3}]
This is mostly a duplicate of Arnkrishn's answer. I just wanted to complete his section with a complete Mathematica code snippet:
m = (y2 - y1)/(x2 - x1)
eqn1 = y - y3 == -(1/m)*(x - x3)
eqn2 = y - y1 == m*(x - x1)
Solve[eqn1 && eqn2, {x, y}]
This is a C# implementation of the accepted answer. It's also using ArcGis to return a MapPoint as that's what we're using for this project.
private MapPoint GenerateLinePoint(double startPointX, double startPointY, double endPointX, double endPointY, double pointX, double pointY)
{
double k = ((endPointY - startPointY) * (pointX - startPointX) - (endPointX - startPointX) * (pointY - startPointY)) / (Math.Pow(endPointY - startPointY, 2)
+ Math.Pow(endPointX - startPointX, 2));
double resultX = pointX - k * (endPointY - startPointY);
double resultY = pointY + k * (endPointX - startPointX);
return new MapPoint(resultX, resultY, 0, SpatialReferences.Wgs84);
}
Thanks to Ray as this worked perfectly for me.
c#arcgis
Just for the sake of completeness, here is a solution using homogeneous coordinates.
The homogeneous points are:
p1 = (x1,y1,1), p2 = (x2,y2,1), p3 = (x3,y3,1)
a line through two points is their cross-product
l_12 := p1 x p2 = (y1-y2, x2-x1, x1*y2 - x2*y1)
The (signed) distance of a point to a line is their dot product.
d := l_12 * p3 = x3*(y1-y2) + y3*(x2-x1) + x1*y2 - x2*y1
The vector from p4 to p3 is d times the normal vector of l_12 divided by the squared length of the normal vector.
n2 := (y1-y2)^2 + (x2-x1)^2
p4 := p3 + d/n2*(y1-y2, x2-x1, 0)
Note: if you divide l_12 by the length of the normal vector
l_12 := l_12 / sqrt((y1-y2)^2 + (x2-x1)^2)
the distance d will be the euclidean distance.
First, calculate the linear function determined by the points
(x1,y2),(x2,y2).
We get:
y1 = mx+b1 where m and b1 are constants.
This step is easy to calculate by the formula of linear function between two points.
Then, calculate the linear function y that goes through (x3,y3).
The function slope is -m, where m is the slope of y1.
Then calculate the const b2 by the coordinates of the point (x3,y3).
We get y2 = -mx+b2 where m and b2 are constants.
The last thing to do is to find the intersection of y1, y2.
You can find x by solving the equation: -mx+b2 = mx+b1, then place x in one of the equations to find y.
This is a vectorized Matlab function for finding pairwise projections of m points onto n line segments. Here xp and yp are m by 1 vectors holding coordinates of m different points, and x1, y1, x2 and y2 are n by 1 vectors holding coordinates of start and end points of n different line segments.
It returns m by n matrices, x and y, where x(i, j) and y(i, j) are coordinates of projection of i-th point onto j-th line.
The actual work is done in first few lines and the rest of the function runs a self-test demo, just in case where it is called with no parameters. It's relatively fast, I managed to find projections of 2k points onto 2k line segments in less than 0.05s.
function [x, y] = projectPointLine(xp, yp, x1, y1, x2, y2)
if nargin > 0
xd = (x2-x1)';
yd = (y2-y1)';
dAB = xd.*xd + yd.*yd;
u = bsxfun(#rdivide, bsxfun(#times, bsxfun(#minus, xp, x1'), xd) + ...
bsxfun(#times, bsxfun(#minus, yp, y1'), yd), dAB);
x = bsxfun(#plus, x1', bsxfun(#times, u, xd));
y = bsxfun(#plus, y1', bsxfun(#times, u, yd));
else
nLine = 3;
nPoint = 2;
xp = rand(nPoint, 1) * 2 -1;
yp = rand(nPoint, 1) * 2 -1;
x1 = rand(nLine, 1) * 2 -1;
y1 = rand(nLine, 1) * 2 -1;
x2 = rand(nLine, 1) * 2 -1;
y2 = rand(nLine, 1) * 2 -1;
tic;
[x, y] = projectPointLine(xp, yp, x1, y1, x2, y2);
toc
close all;
plot([x1'; x2'], [y1'; y2'], '.-', 'linewidth', 2, 'markersize', 20);
axis equal;
hold on
C = lines(nPoint + nLine);
for i=1:nPoint
scatter(x(i, :), y(i, :), 100, C(i+nLine, :), 'x', 'linewidth', 2);
scatter(xp(i), yp(i), 100, C(i+nLine, :), 'x', 'linewidth', 2);
end
for i=1:nLine
scatter(x(:, i)', y(:, i)', 100, C(i, :), 'o', 'linewidth', 2);
end
end
end