The following is ok,
function f!(F,x)
F[1]=x[1]^3+8
F[2]=x[2]^2 -8
end
nlsolve(f!, [1.,2])
however, the following is bad:
function f!(F,x)
F[1]=x[1]^3+8
end
nlsolve(f!, [1.])
This is not an issue with 1D. The problem is with Jacobian of your function.
Here is an example of the same in 2D:
julia> function f!(F,x)
F[1]=x[1]^3+8
F[2]=x[2]^3-2
end
f! (generic function with 1 method)
julia> nlsolve(f!, [0.,0.])
Results of Nonlinear Solver Algorithm
* Algorithm: Trust-region with dogleg and autoscaling
* Starting Point: [0.0, 0.0]
* Zero: [NaN, NaN]
* Inf-norm of residuals: 8.000000
* Iterations: 1000
* Convergence: false
* |x - x'| < 0.0e+00: false
* |f(x)| < 1.0e-08: false
* Function Calls (f): 1001
* Jacobian Calls (df/dx): 2
Now let us go back to your function. If you start from [1.0] you are unlucky and nlsolve in the root finding process hits [0.0] exactly with its default parameters and you have a problem that then Jacobian is [0.0].
You can see it by running:
julia> function f!(F,x)
F[1]=x[1]^3+8
end
f! (generic function with 1 method)
julia> nlsolve(f!, [1.], show_trace=true, extended_trace=true, iterations=3);
Iter f(x) inf-norm Step 2-norm
------ -------------- --------------
0 9.000000e+00 NaN
* f(x): [9.0]
* g(x): [3.0]
* x: [1.0]
* delta: NaN
* rho: NaN
1 8.000000e+00 1.000000e+00
* f(x): [8.0]
* g(x): [0.0]
* x: [0.0]
* delta: 2.9999999999239875
* rho: 0.3777777777854353
2 NaN NaN
* f(x): [NaN]
* g(x): [0.0]
* x: [NaN]
* delta: 2.9999999999239875
* rho: NaN
3 NaN NaN
* f(x): [NaN]
* g(x): [0.0]
* x: [NaN]
* delta: 2.9999999999239875
* rho: NaN
You can fix it by changing a starting point or by changing factor:
julia> nlsolve(f!, [10.])
Results of Nonlinear Solver Algorithm
* Algorithm: Trust-region with dogleg and autoscaling
* Starting Point: [10.0]
* Zero: [-2.0]
* Inf-norm of residuals: 0.000000
* Iterations: 18
* Convergence: true
* |x - x'| < 0.0e+00: false
* |f(x)| < 1.0e-08: true
* Function Calls (f): 17
* Jacobian Calls (df/dx): 11
julia> nlsolve(f!, [1.], factor=0.5)
Results of Nonlinear Solver Algorithm
* Algorithm: Trust-region with dogleg and autoscaling
* Starting Point: [1.0]
* Zero: [-2.0]
* Inf-norm of residuals: 0.000000
* Iterations: 7
* Convergence: true
* |x - x'| < 0.0e+00: false
* |f(x)| < 1.0e-08: true
* Function Calls (f): 8
* Jacobian Calls (df/dx): 8
Also - as Chris suggests - Roots.jl has methods that are more robust because you have derivative free methods in it.
It's fine, but Roots.jl is probably better suited for this kind of equation. Its methods are directly for 1-dimensional rootfinding problems and can be more robust.
Related
How do I minimize a simple scalar function in Julia using Newton's method? (or any other suitable optimization scheme)
using Optim
# Function to optimize
function g(x)
return x^2
end
x0 = 2.0 # Initial value
optimize(g, x0, Newton())
The above doesn't seem to work and returns
ERROR: MethodError: no method matching optimize(::typeof(g), ::Float64, ::Newton{LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64,Base.RefValue{Bool}}})
Optim is designed for vector problems and not scalar ones like in your example. You can adjust the example to be a vector-problem with one variable though:
julia> using Optim
julia> function g(x) # <- g accepts x as a vector
return x[1]^2
end
julia> x0 = [2.0] # <- Make this a vector
1-element Vector{Float64}:
2.0
julia> optimize(g, x0, Newton())
* Status: success
* Candidate solution
Final objective value: 0.000000e+00
The optimize function expects an interval, not a starting point:
optimize(g, -10, 10)
returns
Results of Optimization Algorithm
* Algorithm: Brent's Method
* Search Interval: [-10.000000, 10.000000]
* Minimizer: 0.000000e+00
* Minimum: 0.000000e+00
* Iterations: 5
* Convergence: max(|x - x_upper|, |x - x_lower|) <= 2*(1.5e-08*|x|+2.2e-16): true
* Objective Function Calls: 6
Concerning available methods I have not read the doc, but you can directly have a look at the source code using the #edit macro:
#edit optimize(g, -10, 10)
Browsing the source you will see:
function optimize(f,
lower::Union{Integer, Real},
upper::Union{Integer, Real},
method::Union{Brent, GoldenSection};
kwargs...)
T = promote_type(typeof(lower/1), typeof(upper/1))
optimize(f,
T(lower),
T(upper),
method;
kwargs...)
end
Hence I think that you have only two methods for unidimensional minimization: Brent and GoldenSection.
By example you can try:
julia> optimize(g, -10, 10, GoldenSection())
Results of Optimization Algorithm
* Algorithm: Golden Section Search
* Search Interval: [-10.000000, 10.000000]
* Minimizer: 1.110871e-16
* Minimum: 1.234035e-32
* Iterations: 79
* Convergence: max(|x - x_upper|, |x - x_lower|) <= 2*(1.5e-08*|x|+2.2e-16): true
* Objective Function Calls: 80
I would like to minimize a distance function ||dz - z|| under the constraint that g(z) = 0.
I wanted to use Lagrange Multipliers to solve this problem. Then I used NLsolve.jl to solve the non-linear equation that I end up with.
using NLsolve
using ForwardDiff
function ProjLagrange(dz, g::Function)
λ_init = ones(size(g(dz...),1))
initial_x = vcat(dz, λ_init)
function gradL!(F, x)
len_dz = length(dz)
z = x[1:len_dz]
λ = x[len_dz+1:end]
F = Array{Float64}(undef, length(x))
my_distance(z) = norm(dz - z)
∇f = z -> ForwardDiff.gradient(my_distance, z)
F[1:len_dz] = ∇f(z) .- dot(λ, g(z...))
if length(λ) == 1
F[end] = g(z...)
else
F[len_dz+1:end] = g(z)
end
end
nlsolve(gradL!, initial_x)
end
g_test(x1, x2, x3) = x1^2 + x2 - x2 + 5
z = [1000,1,1]
ProjLagrange(z, g_test)
But I always end up with Zero: [NaN, NaN, NaN, NaN] and Convergence: false.
Just so you know I have already solved the equation by using Optim.jl and minimizing the following function: Proj(z) = b * sum(abs.(g(z))) + a * norm(dz - z).
But I would really like to know if this is possible with NLsolve. Any help is greatly appreciated!
Starting almost from scratch and wikipedia's Lagrange multiplier page because it was good for me, the code below seemed to work. I added an λ₀s argument to the ProjLagrange function so that it can accept a vector of initial multiplier λ values (I saw you initialized them at 1.0 but I thought this was more generic). (Note this has not been optimized for performance!)
using NLsolve, ForwardDiff, LinearAlgebra
function ProjLagrange(x₀, λ₀s, gs, n_it)
# distance function from x₀ and its gradients
f(x) = norm(x - x₀)
∇f(x) = ForwardDiff.gradient(f, x)
# gradients of the constraints
∇gs = [x -> ForwardDiff.gradient(g, x) for g in gs]
# Form the auxiliary function and its gradients
ℒ(x,λs) = f(x) - sum(λ * g(x) for (λ,g) in zip(λs,gs))
∂ℒ∂x(x,λs) = ∇f(x) - sum(λ * ∇g(x) for (λ,∇g) in zip(λs,∇gs))
∂ℒ∂λ(x,λs) = [g(x) for g in gs]
# as a function of a single argument
nx = length(x₀)
ℒ(v) = ℒ(v[1:nx], v[nx+1:end])
∇ℒ(v) = vcat(∂ℒ∂x(v[1:nx], v[nx+1:end]), ∂ℒ∂λ(v[1:nx], v[nx+1:end]))
# and solve
v₀ = vcat(x₀, λ₀s)
nlsolve(∇ℒ, v₀, iterations=n_it)
end
# test
gs_test = [x -> x[1]^2 + x[2] - x[3] + 5]
λ₀s_test = [1.0]
x₀_test = [1000.0, 1.0, 1.0]
n_it = 100
res = ProjLagrange(x₀_test, λ₀s_test, gs_test, n_it)
gives me
julia> res = ProjLagrange(x₀_test, λ₀s_test, gs_test, n_it)
Results of Nonlinear Solver Algorithm
* Algorithm: Trust-region with dogleg and autoscaling
* Starting Point: [1000.0, 1.0, 1.0, 1.0]
* Zero: [9.800027199717013, -49.52026655749088, 51.520266557490885, -0.050887973682118504]
* Inf-norm of residuals: 0.000000
* Iterations: 10
* Convergence: true
* |x - x'| < 0.0e+00: false
* |f(x)| < 1.0e-08: true
* Function Calls (f): 11
* Jacobian Calls (df/dx): 11
I altered your code as below (see my comments in there) and got the following output. It doesn't throw NaNs anymore, reduces the objective and converges. Does this differ from your Optim.jl results?
Results of Nonlinear Solver Algorithm
* Algorithm: Trust-region with dogleg and autoscaling
* Starting Point: [1000.0, 1.0, 1.0, 1.0]
* Zero: [9.80003, -49.5203, 51.5203, -0.050888]
* Inf-norm of residuals: 0.000000
* Iterations: 10
* Convergence: true
* |x - x'| < 0.0e+00: false
* |f(x)| < 1.0e-08: true
* Function Calls (f): 11
* Jacobian Calls (df/dx): 11
using NLsolve
using ForwardDiff
using LinearAlgebra: norm, dot
using Plots
function ProjLagrange(dz, g::Function, n_it)
λ_init = ones(size(g(dz),1))
initial_x = vcat(dz, λ_init)
# These definitions can go outside as well
len_dz = length(dz)
my_distance = z -> norm(dz - z)
∇f = z -> ForwardDiff.gradient(my_distance, z)
# In fact, this is probably the most vital difference w.r.t. your proposal.
# We need the gradient of the constraints.
∇g = z -> ForwardDiff.gradient(g, z)
function gradL!(F, x)
z = x[1:len_dz]
λ = x[len_dz+1:end]
# `F` is memory allocated by NLsolve to store the residual of the
# respective call of `gradL!` and hence doesn't need to be allocated
# anew every time (or at all).
F[1:len_dz] = ∇f(z) .- λ .* ∇g(z)
F[len_dz+1:end] .= g(z)
end
return nlsolve(gradL!, initial_x, iterations=n_it, store_trace=true)
end
# Presumable here is something wrong: x2 - x2 is not very likely, also made it
# callable directly with an array argument
g_test = x -> x[1]^2 + x[2] - x[3] + 5
z = [1000,1,1]
n_it = 10000
res = ProjLagrange(z, g_test, n_it)
# Ugly reformatting here
trace = hcat([[state.iteration; state.fnorm; state.stepnorm] for state in res.trace.states]...)
plot(trace[1,:], trace[2,:], label="f(x) inf-norm", xlabel="steps")
Evolution of inf-norm of f(x) over iteration steps
[Edit: Adapted solution to incorporate correct gradient computation for g()]
I am learning how to solve non-linear equations and Julia (1.3.1) in general and want to ask how we should use NLsolve.
As a first step, I try the following:
using NLsolve
uni = 1
z = 3
x = [3,2,4,5,6]
y = z .+ x
print(y)
function g!(F, x)
F[1:uni+4] = y .- z - x
end
nlsolve(g!, [0.5,0.9,1,2,3])
And, I confirm it works as below:
Results of Nonlinear Solver Algorithm
* Algorithm: Trust-region with dogleg and autoscaling
* Starting Point: [0.5, 0.9, 1.0, 2.0, 3.0]
* Zero: [2.999999999996542, 2.000000000003876, 4.000000000008193, 4.999999999990685, 5.999999999990221]
* Inf-norm of residuals: 0.000000
* Iterations: 2
* Convergence: true
* |x - x'| < 0.0e+00: false
* |f(x)| < 1.0e-08: true
* Function Calls (f): 3
* Jacobian Calls (df/dx): 3
Then, I try a more complicated model as following
using SpecialFunctions, NLsolve, Random
Random.seed!(1234)
S = 2
# Setting parameters
ε = 1.3
β = 0.4
γ = gamma((ε-1)/ε)
T = rand(S)
E = rand(S)
B = rand(S)
w = rand(S)
Q = rand(S)
d = [1 2 ;2 1 ]
# Construct a model
rvector = T.*Q.^(1-β).*B.^ε
svector = E.* w.^ε
Φ_all = (sum(sum(rvector * svector' .* d )))
π = rvector * svector' .* d ./ Φ_all
# These two are outcome the model
πR = (sum(π,dims=1))'
πM = sum(π,dims=2)
# E is now set as unknown and we want to estimate it given the outcome of the model
function f!(Res, Unknown)
rvector = T.*Q.^(1-β).*B.^ε
svector = Unknown[1:S].* w.^ε
Φ_all = (sum(sum(rvector * svector' .* d )))
π = rvector * svector' .* d ./ Φ_all
Res = ones(S+S)
Res[1:S] = πR - (sum(π,dims=1))'
Res[S+1:S+S] = πM - sum(π,dims=2)
end
nlsolve(f!, [0.5,0.6])
and this code produces a weird result as follows.
Results of Nonlinear Solver Algorithm
* Algorithm: Trust-region with dogleg and autoscaling
* Starting Point: [0.5, 0.6]
* Zero: [0.5, 0.6]
* Inf-norm of residuals: 0.000000
* Iterations: 0
* Convergence: true
* |x - x'| < 0.0e+00: false
* |f(x)| < 1.0e-08: true
* Function Calls (f): 1
* Jacobian Calls (df/dx): 1
So, essentially, the function always yields 0 as the return, and thus the initial input always becomes the solution. I cannot understand why this does not work and also why this behaves differently from the first example. Could I have your suggestion to fix it?
You need to update Res in place. Now you do
Res = ones(S+S)
which shadows the input Res. Just update Res directly.
I need to find the fixed point of a multivariable function in Julia.
Consider the following minimal example:
function example(p::Array{Float64,1})
q = -p
return q
end
Ideally I'd use a package like Roots.jl and call find_zeros(p -> p - example(p)), but I can't find the analogous package for multivariable functions. I found one called IntervalRootFinding, but it oddly requires unicode characters and is sparsely documented, so I can't figure out how to use it.
There are many options. The choice of the best one depends on the nature of example function (you have to understand the nature of your example function and check against a documentation of a specific package if it would support it).
Eg. you can use fixedpoint from NLsolve.jl:
julia> using NLsolve
julia> function example!(q, p::Array{Float64,1})
q .= -p
end
example! (generic function with 1 method)
julia> fixedpoint(example!, ones(1))
Results of Nonlinear Solver Algorithm
* Algorithm: Anderson m=1 beta=1 aa_start=1 droptol=0
* Starting Point: [1.0]
* Zero: [0.0]
* Inf-norm of residuals: 0.000000
* Iterations: 3
* Convergence: true
* |x - x'| < 0.0e+00: true
* |f(x)| < 1.0e-08: true
* Function Calls (f): 3
* Jacobian Calls (df/dx): 0
julia> fixedpoint(example!, ones(3))
Results of Nonlinear Solver Algorithm
* Algorithm: Anderson m=3 beta=1 aa_start=1 droptol=0
* Starting Point: [1.0, 1.0, 1.0]
* Zero: [-2.220446049250313e-16, -2.220446049250313e-16, -2.220446049250313e-16]
* Inf-norm of residuals: 0.000000
* Iterations: 3
* Convergence: true
* |x - x'| < 0.0e+00: false
* |f(x)| < 1.0e-08: true
* Function Calls (f): 3
* Jacobian Calls (df/dx): 0
julia> fixedpoint(example!, ones(5))
Results of Nonlinear Solver Algorithm
* Algorithm: Anderson m=5 beta=1 aa_start=1 droptol=0
* Starting Point: [1.0, 1.0, 1.0, 1.0, 1.0]
* Zero: [0.0, 0.0, 0.0, 0.0, 0.0]
* Inf-norm of residuals: 0.000000
* Iterations: 3
* Convergence: true
* |x - x'| < 0.0e+00: true
* |f(x)| < 1.0e-08: true
* Function Calls (f): 3
* Jacobian Calls (df/dx): 0
If your function would require a global optimization tools to find a fixed point then you can e.g. use BlackBoxOptim.jl with norm(f(x) .-x) as an objective:
julia> using LinearAlgebra
julia> using BlackBoxOptim
julia> function example(p::Array{Float64,1})
q = -p
return q
end
example (generic function with 1 method)
julia> f(x) = norm(example(x) .- x)
f (generic function with 1 method)
julia> bboptimize(f; SearchRange = (-5.0, 5.0), NumDimensions = 1)
Starting optimization with optimizer DiffEvoOpt{FitPopulation{Float64},RadiusLimitedSelector,BlackBoxOptim.AdaptiveDiffEvoRandBin{3},RandomBound{ContinuousRectSearchSpace}}
0.00 secs, 0 evals, 0 steps
Optimization stopped after 10001 steps and 0.15 seconds
Termination reason: Max number of steps (10000) reached
Steps per second = 68972.31
Function evals per second = 69717.14
Improvements/step = 0.35090
Total function evaluations = 10109
Best candidate found: [-8.76093e-40]
Fitness: 0.000000000
julia> bboptimize(f; SearchRange = (-5.0, 5.0), NumDimensions = 3);
Starting optimization with optimizer DiffEvoOpt{FitPopulation{Float64},RadiusLimitedSelector,BlackBoxOptim.AdaptiveDiffEvoRandBin{3},RandomBound{ContinuousRectSearchSpace}}
0.00 secs, 0 evals, 0 steps
Optimization stopped after 10001 steps and 0.02 seconds
Termination reason: Max number of steps (10000) reached
Steps per second = 625061.23
Function evals per second = 631498.72
Improvements/step = 0.32330
Total function evaluations = 10104
Best candidate found: [-3.00106e-12, -5.33545e-12, 5.39072e-13]
Fitness: 0.000000000
julia> bboptimize(f; SearchRange = (-5.0, 5.0), NumDimensions = 5);
Starting optimization with optimizer DiffEvoOpt{FitPopulation{Float64},RadiusLimitedSelector,BlackBoxOptim.AdaptiveDiffEvoRandBin{3},RandomBound{ContinuousRectSearchSpace}}
0.00 secs, 0 evals, 0 steps
Optimization stopped after 10001 steps and 0.02 seconds
Termination reason: Max number of steps (10000) reached
Steps per second = 526366.94
Function evals per second = 530945.88
Improvements/step = 0.29900
Total function evaluations = 10088
Best candidate found: [-9.23635e-8, -2.6889e-8, -2.93044e-8, -1.62639e-7, 3.99672e-8]
Fitness: 0.000000391
I'm an author of IntervalRootFinding.jl. I'm happy to say that the documentation has been improved a lot recently, and no unicode characters are necessary. I suggest using the master branch.
Here's how to solve your example with the package. Note that this package should be able to find all of the roots within a box, and guarantee that it has found all of them. Yours only has 1:
julia> using IntervalArithmetic, IntervalRootFinding
julia> function example(p)
q = -p
return q
end
example (generic function with 2 methods)
julia> X = IntervalBox(-2..2, 2)
[-2, 2] × [-2, 2]
julia> roots(x -> example(x) - x, X, Krawczyk)
1-element Array{Root{IntervalBox{2,Float64}},1}:
Root([0, 0] × [0, 0], :unique)
For more information, I suggest looking at https://discourse.julialang.org/t/ann-intervalrootfinding-jl-for-finding-all-roots-of-a-multivariate-function/9515.
Question is about the modulo operator on very large numbers.
For example consider a question where the total number of permutations are to be calculated.
Consider a number of 90 digits with each of the 9 numbers (1 to 9) repeating 10 times
so 90!/(10!)^9) is to be calculated
After reading many answers on StackOverflow I used logarithms to do it.
Now consider the log value to be 1923.32877864.
Now my question is how can I display the answer (i.e. 10 ^ log10(value) ) modulo of "m"?
And is this the best method for calculating the possible number of permutations?
Edit
Got the solution :)
Thanks to duedl0r.
Did it the way you specified using Modular Multiplicative Inverse.Thanks :)
I'm not sure whether this is actually possible and correct, but let me summarize my comments and extend the answer from Miky Dinescu.
As Miky already wrote:
a × b ≣m am × bm
You can use this in your equality:
90! / 10!^9 ≣m x
Calculate each term:
90!m / 10!^9m ≣m x
Then find out your multiplicative inverse from 10!^9m. Then multiplicate the inverse with 90!m.
update
This seems to be correct (at least for this case :)). I checked with wolfram:
(90!/10!^9) mod (10^9+7) = 998551163
This leads to the same result:
90! mod (10^9+7) = 749079870
10!^9 mod (10^9+7) = 220052161
do the inverse:
(220052161 * x) mod(10^9+7) = 1 = 23963055
then:
(749079870*23963055) mod (10^9+7) = 998551163
No proof, but some evidence that it might work :)
I would argue that the way to compute the total number of permutations modulo m, where m is an arbitrary integer (usually chosen to be a large prime number) is to use the following property:
(a * b) % m = ((a % m) * (b % m)) % m
Considering that the total number of permutations of N is N! = 1 * 2 * 3 * .. * N, if you need to compute N! % m, you can essentially apply the property above for multiplication modulo m, and you have:
((((1 * (2 % m)) % m) * (3 % m)) % m) * ..
EDIT
In order to compute the 90! / (10! ^ 9) value you could simplify the factors and then use multiplication modulo m to compute the final result modulo m.
Here's what I'm thinking:
90! = 10! * (11 * 12 * .. * 20) * (21 * 22 * .. * 30) * .. * (81 * 82 * .. * 90)
You can then rewrite the original expression as:
(10! * (11 * 12 * .. * 20) * (21 * 22 * .. * 30) * .. * (81 * 82 * .. * 90)) / (10! * 10! * ... * 10!)
At the numerator, you have a product of 9 factors - considering each expression in parenthesis a factor. The same is true for the denominator (you have 9 factors, each equal to 10!).
The first factor at the denominator is trivial to simplify. After that you still have 8 pairs that need simplification.
So, you can factor each term of the products and simplify the denominator away. For example:
11 * 12 * 13 * 14 * 15 * 16 * 17 * 18 * 19 * 20 <=> 11 * 2 * 2 * 3 * 13 * 2 * 7 * 3 * 5 * 2 * 2 * 2 * 2 * 17 * 2 * 9 * 2 * 2 * 5
The denominator will always be: 2 * 3 * 2 * 2 * 5 * 2 * 3 * 7 * 2 * 2 * 2 * 2 * 3 * 3 * 2 * 5
After the simplification the second pair reduces to : 2 * 2 * 11 * 13 * 17 * 19
The same can be applied to each subsequent pair and you will end up with a simple product that can be computed modulo m using the formula above.
Of course, efficiently implementing the algorithm to perform the simplification will be tricky so ultimately there has to be a better way that eludes me now.