I'm new in Julia and I'm trying to learn to manipulate calculus on it. How do I do if I calculate the gradient of a function with "ForwardDiff" like in the code below and see the function next?
I know if I input some values it gives me the gradient value in that point but I just want to see the function (the gradient of f1).
julia> gradf1(x1, x2) = ForwardDiff.gradient(z -> f1(z[1], z[2]), [x1, x2])
gradf1 (generic function with 1 method)
To elaborate on Felipe Lema's comment, here are some examples using SymPy.jl for various tasks:
#vars x y z
f(x,y,z) = x^2 * y * z
VF(x,y,z) = [x*y, y*z, z*x]
diff(f(x,y,z), x) # ∂f/∂x
diff.(f(x,y,z), [x,y,z]) # ∇f, gradiant
diff.(VF(x,y,z), [x,y,z]) |> sum # ∇⋅VF, divergence
J = VF(x,y,z).jacobian([x,y,z])
sum(diag(J)) # ∇⋅VF, divergence
Mx,Nx, Px, My,Ny,Py, Mz, Nz, Pz = J
[Py-Nz, Mz-Px, Nx-My] # ∇×VF
The divergence and gradient are also part of SymPy, but not exposed. Their use is more general, but cumbersome for this task. For example, this finds the curl:
import PyCall
PyCall.pyimport_conda("sympy.physics.vector", "sympy")
RF = sympy.physics.vector.ReferenceFrame("R")
v1 = get(RF,0)*get(RF,1)*RF.x + get(RF,1)*get(RF,2)*RF.y + get(RF,2)*get(RF,0)*RF.z
sympy.physics.vector.curl(v1, RF)
Related
I am trying to use ForwardDIff to compute the gradient of a function. My code is very simple:
buffer1 = 1.
buffer2 = 1.0-buffer1
buffers = [buffer1; buffer2];
K0 = [[100., 200.] [0.01, 0.001]];
q_sat = [100, 150];
K = exp.(log.(K0) * buffers);
f(x::Vector) = q_sat[1]*(K[1]*x[1]/(1+alpha[1]*K[1]*x[1]+alpha[2]*K[2]*x[2]))
x = [0.5; 0.5]
g = ForwardDiff.gradient(f(x), x)
The first lines are about defining some constants and then I define the function that takes in a vector and returns a real number. When trying to comput the gradient of f I get the error:
MethodError: objects of type Float64 are not callable
What could the problem be?
You want to differentiate f (the function), not f(x) (some scalar), so it must be
ForwardDiff.gradient(f, x)
I am trying to implement ridge-regression from scratch in Julia but something is going wrong.
# Imports
using DataFrames
using LinearAlgebra: norm, I
using Optim: optimize, LBFGS, minimizer
# Read Data
out = CSV.read(download("https://raw.githubusercontent.com/jbrownlee/Datasets/master/housing.csv"), DataFrame, header=0)
# Separate features and response
y = Vector(out[:, end])
X = Matrix(out[:, 1:(end-1)])
λ = 0.1
# Functions
loss(beta) = norm(y - X * beta)^2 + λ*norm(beta)^2
function grad!(G, beta)
G = -2*transpose(X) * (y - X * beta) + 2*λ*beta
end
function hessian!(H, beta)
H = X'X + λ*I
end
# Optimization
start = randn(13)
out = optimize(loss, grad!, hessian!, start, LBFGS())
However, the result of this is terrible and we essentially get back start since it is not moving. Of course, I know I could simply use (X'X + λ*I) \ X'y or IterativeSolvers.lmsr(X, y) but I would like to implement this myself.
The problem is with the implementation of the grad! and hessian! functions: you should use dot assignment to change the content of the G and H matrices:
G .= -2*transpose(X) * (y - X * beta) + 2*λ*beta
H .= X'X + λ*I
Without the dot you replace the matrix the function parameter refers to, but the matrix passed to the function (which will then be used by the optimizer) remains unchanged (presumably a zero matrix, that's why you got back the start vector).
I need to operate on a sequence of functions
h_k(x) = (I + f_k( ) )^k g(x)
for each k=1,...,N.
A basic example (N=2, f_k=f) is the following:
f(x) = x^2
g(x) = x
h1(x) = g(x) + f(g(x))
h2(x) = g(x) + f(g(x)) + f(g(x) + f(g(x)))
println(h1(1)) # returns 2
println(h2(1)) # returns 6
I need to write this in a loop and it would be best to redefine g(x) at each iteration. Unfortunately, I do not know how to do this in Julia without conflicting with the syntax for a recursive definition of g(x). Indeed,
f(x) = x^2
g(x) = x
for i=1:2
global g(x) = g(x) + f(g(x))
println(g(1))
end
results in a StackOverflowError.
In Julia, what is the proper way to redefine g(x), using its previous definition?
P.S. For those who would suggest that this problem could be solved with recursion: I want to use a for loop because of how the functions f_k(x) (in the above, each f_k = f) are computed in the real problem that this derives from.
I am not sure if it is best, but a natural approach is to use anonymous functions here like this:
let
f(x) = x^2
g = x -> x
for i=1:2
l = g
g = x -> l(x) + f(l(x))
println(g(1))
end
end
or like this
f(x) = x^2
g = x -> x
for i=1:4
l = g
global g = x -> l(x) + f(l(x))
println(g(1))
end
(I prefer the former option using let as it avoids using global variables)
The issue is that l is a loop local variable that gets a fresh binding at each iteration, while g is external to the loop.
You might also check out this section of the Julia manual.
Say I have prices of a stock and I want to find the slope of the regression line in rolling manner with a given window size. How can I get it done in Julia? I want it to be really fast hence don't want to use a for loop.
You should not, in general, be worried about for loops in Julia, as they do not have the overhead of R or Python for loops. Thus, you only need to worry about asymptotic complexity and not the potentially large constant factor introduced by interpreter overhead.
Nevertheless, this operation can be done much more (asymptotically) efficiently with convolutions than with the naïve O(n²) slice-and-regress approach. The DSP.jl package provides convolution functionality. The following is an example with no intercept (it computes the rolling betas); support for an intercept should be possible by modifying the formulas.
using DSP
# Create some example x (signal) and y (stock prices)
# such that strength of signal goes up over time
const x = randn(100)
const y = (1:100) .* x .+ 100 .* randn(100)
# Create the rolling window
const window = Window.rect(20)
# Compute linear least squares estimate (X^T X)^-1 X^T Y
const xᵗx = conv(x .* x, window)[length(window):end-length(window)+1]
const xᵗy = conv(x .* y, window)[length(window):end-length(window)+1]
const lls = xᵗy ./ xᵗx # desired beta
# Check result against naïve for loop
const βref = [dot(x[i:i+19], y[i:i+19]) / dot(x[i:i+19], x[i:i+19]) for i = 1:81]
#assert isapprox(βref, lls)
Edit to add: To support an intercept, i.e. X = [x 1], so X^T X = [dot(x, x) sum(x); sum(x) w] where w is the window size, the formula for inverse of a 2D matrix can be used to get (X^T X)^-1 = [w -sum(x); -sum(x) dot(x, x)]/(w * dot(x, x) - sum(x)^2). Thus, [β, α] = [w dot(x, y) - sum(x) * sum(y), dot(x, x) * sum(y) - sum(x) * dot(x, y)] / (w * dot(x, x) - sum(x)^2). This can be translated to the following convolution code:
# Compute linear least squares estimate with intercept
const w = length(window)
const xᵗx = conv(x .* x, window)[w:end-w+1]
const xᵗy = conv(x .* y, window)[w:end-w+1]
const 𝟙ᵗx = conv(x, window)[w:end-w+1]
const 𝟙ᵗy = conv(y, window)[w:end-w+1]
const denom = w .* xᵗx - 𝟙ᵗx .^ 2
const α = (xᵗx .* 𝟙ᵗy .- 𝟙ᵗx .* xᵗy) ./ denom
const β = (w .* xᵗy .- 𝟙ᵗx .* 𝟙ᵗy) ./ denom
# Check vs. naive solution
const ref = vcat([([x[i:i+19] ones(20)] \ y[i:i+19])' for i = 1:81]...)
#assert isapprox([β α], ref)
Note that, for weighted least squares with a different window shape, some minor modifications will be needed to disentangle length(window) and sum(window) which are used interchangeably in the code above.
Since I dont need a x variable, I created a numeric series. Using RollingFunctions Package I was able to get rolling regressions through below function.
using RollingFunctions
function rolling_regression(price,windowsize)
sum_x = sum(collect(1:windowsize))
sum_x_squared = sum(collect(1:windowsize).^2)
sum_xy = rolling(sum,price,windowsize,collect(1:windowsize))
sum_y = rolling(sum,price,windowsize)
b = ((windowsize*sum_xy) - (sum_x*sum_y))/(windowsize*sum_x_squared - sum_x^2)
c = [repeat([missing],windowsize-1);b]
end
I have the following code:
f=tan(x)*x**2
q=Wild('q')
s=f.match(tan(q))
s={q_ : x}
How to work with the result of the "wild"? How to not address the array, for example, s[0], s{0}?
Wild can be used when you have an expression which is the result of some complicated calculation, but you know it has to be of the form sin(something) times something else. Then s[q] will be the sympy expression for the "something". And s[p] for the "something else". This could be used to investigate both p and q. Or to further work with a simplified version of f, substituting p and q with new variables, especially if p and q would be complex expressions involving multiple variables.
Many more use cases are possible.
Here is an example:
from sympy import *
from sympy.abc import x, y, z
p = Wild('p')
q = Wild('q')
f = tan(x) * x**2
s = f.match(p*tan(q))
print(f'f is the tangent of "{s[q]}" multiplied by "{s[p]}"')
g = f.xreplace({s[q]: y, s[p]:z})
print(f'f rewritten in simplified form as a function of y and z: "{g}"')
h = s[p] * s[q]
print(f'a new function h, combining parts of f: "{h}"')
Output:
f is the tangent of "x" multiplied by "x**2"
f rewritten in simplified form as a function of y and z: "z*tan(y)"
a new function h, combining parts of f: "x**3"
If you're interested in all arguments from tan that appear in f written as a product, you might try:
from sympy import *
from sympy.abc import x
f = tan(x+2)*tan(x*x+1)*7*(x+1)*tan(1/x)
if f.func == Mul:
all_tan_args = [a.args[0] for a in f.args if a.func == tan]
# note: the [0] is needed because args give a tupple of arguments and
# in the case of tan you'ld want the first (there is only one)
elif f.func == tan:
all_tan_args = [f.args[0]]
else:
all_tan_args = []
prod = 1
for a in all_tan_args:
prod *= a
print(f'All the tangent arguments are: {all_tan_args}')
print(f'Their product is: {prod}')
Output:
All the tangent arguments are: [1/x, x**2 + 1, x + 2]
Their product is: (x + 2)*(x**2 + 1)/x
Note that neither method would work for f = tan(x)**2. For that, you'ld need to write another match and decide whether you'ld want to take the same power of the arguments.