I am new to the package CVXR. I am using it to do the convex optimization within each iteration of EM algorithms. Everything is fine at first but after 38 iterations, I have an error:
Error in valuesById(object, results_dict, sym_data, solver) :
Solver failed. Try another.
I am not sure why the solver works fine at first but then fails to work later. I looked up the manual about how to change the solver but could not find the answer. I am also curious about whether we can specify learning step size in CVXR. Really appreciate any help
The list of installed solvers in CVXR you can get with
installed_solvers()
In my case that is:
# "ECOS" "ECOS_BB" "SCS"
You can change the one that is used just using argument solver, e.g. to change from the default ECOS to SCS:
result <- solve(prob, solver="SCS")
I think the developers are planning to support other solvers in the future, e.g. gurobi...
Related
I am new in Julia language. I have a a large system of ODEs (around 500). When I use the AutoTsit5(Rosenbrock23()) solver, I receive this error:
This solver is not able to use mass matrices.
Does it mean that I have to use solvers for DAE problems? What other options exist?
Thanks!
I tried different solvers. Some work, some not.
Does it mean that I have to use solvers for DAE problems? What other options exist?
If you are using a solver to solve an ODEProblem in mass matrix form, you must use one of the methods that's documented as capable of doing this. This is shown in the DAE solver page. AutoTsit5 is not a compatible method with this as no explicit method is compatible with mass matrices (for clear mathematical reasons). FBDF would likely be recommended in a scenario like this.
In general, we would highly recommend not choosing an ODE solver and instead relying on the default given by DifferentialEquations.jl's default algorithm unless you have a clear idea of why you're choosing a specific solver.
I had to add some circular dependencies to my model and thus adding NonlinearBlockGS and LinearBlockGS to the Group with the circular dependency. I get messages like this
LN: LNBGSSolver 'LN: LNBGS' on system 'XXX' failed to converge in 10
iterations.
in the phase where it's finding the Coloring of the problem. There is a Dymos trajectory as part of the problem, but the circular dependency is not in the Trajectory group, it's upstream. It however converges very easily when actually solving the problem. The number of FWD solves is the same as it was before-- everything seem to work fine. Should I be worried about anything?
the way our total derivative coloring works is that we replace partial derivatives with random numbers and then solve the linear system. So the linear solver should be converging. Now, whether or not it should converge with LNBGS in 10 iterations... probably not.
Its hard to speak diffinitively when putting random numbers into a matrix to invert it... but generally speaking it should remain invertible (though we can't promise). That does not mean that it will remain easily invertible. How close does the linear residual get during the coloring? it is decreasing, but slowly. Would more iteration let it get there?
If your problem is working well, I don't think you need to freak out about this. If you would like it to converge better, it won't hurt anything and might give you better coloring. You can increase the iprint of that solver to get more information on the convergence history.
Another option, if your system is small enough, is to try using the DirectSolver instead of LNBGS. For most models with less than 10,000 variables in them a DirectSolver will be overall faster than the LNBGS. There is a nice symetry to using LNBGS with NLGBS ... but while the nonlinear solver tends to be a good choice (i.e. fast and stable) for cyclic dependencies the same can't be said for its linear counter part.
So my go-to combination if NLBGS and DirectSolver. You can't always use the DirectSolver. If you have distributed components in your model, or components that use the matrix-free derivative APIs (apply_linear, compute_jacvec_product), then LNBGS is a good option. But if everything is explicit components with compute_partials or implicit components that provide partials in the linearize method then I suggest using the DirectSolver as your first option.
I think you may have discovered a coloring performance issue in OpenMDAO. When we compute coloring, internally we replace the component partials with random arrays matching the declared sparsity. Since we're not trying to find an actual solution when we compute coloring, we probably don't need to iterate more than once in any given system. And we shouldn't be generating convergence warnings when computing the coloring. I don't think you need to be worried in this case. I'll put a story in our bug tracker to look into this.
I have two openmdao groups with cyclic dependency between the groups. I calculate the derivatives using Complex step. I have a non-linear solver for the dependency and use SLSQP to optimize my objective function. The issue is with the choice of the non-linear solver. When I use NonlinearBlockGS the optimization is successful in 12 iterations. But when I use NewtonSolver with Directsolver or ScipyKrylov the optimization fails (Iteration limit exceeded), even with maxiter=2000. The cyclic connections converge, but it is just that the design variables does not reach the optimal values. The difference between the design variables in consecutive iterations is in the order 1e-5. And this increases the iterations needed. Also when I change the initial guess to a value closer to the optimal value it works.
To check further, I converted the model into IDF (by creating copies of coupling variables and consistency constraints) thereby removing the need for a solver. Now the optimization is successful in 5 iterations and the results are similar to the results when NonlinearBlockGS is used.
Why does this happen? Am I missing something? When should I use NewtonSolver over others? I know that it is difficult to answer without seeing the code. But it is just that my code is long with multiple components and I couldn't recreate the issue with a toy model. So any general insight is much appreciated.
Without seeing the code, you're right that its hard to give specifics.
Very broadly speaking, Newton can sometimes have a lot more trouble converging than NLBGS (Note: this is not absolutely true, but is a good rule of thumb). So what I would guess is happening is that on your first or second iteration, the newton solver isn't actually converging. You can check this by setting newton.options['iprint']=2 and looking at the iteration history as the optimizer iterates.
When you have a solver in your optimization, its critical that you also make sure that you set it to throw an error on non-convergence. Some optimizers can handle this error, and will backtrack on the line search. Others will just die. Either way, its important. Otherwise, you end up giving the optimizer an unconverged case that it doesn't know is unconverged.
This is bad for two reasons. First, the objective and constraints values it gets are going to be wrong! Second, and perhaps more importantly, the derivatives it computes are going to be wrong! You can read the details [in the theory manual,] but in summary the analytic derivative methods that OpenMDAO uses assume that the residuals have gone to 0. If thats not the case, the math breaks down. Even if you were doing full model finite-difference, non-convergenced models are a problem. You'll just get noisy garbage when you try to FD it.2
So, assuming you have set up your model correctly, and that you have the linear solvers set up problems (it sounds like you do since it works with NLBGS), then its most likely that the newton solver isn't converging. Use iprint, possibly combined with driver debug printing, to check this for yourself. If thats the case, you need to figure out how to get newton to behave better.
There are some tips here that are pretty general. You could also try using the armijo line search, which can often stablize a newton solve at the cost of some speed.
Finally... Newton isn't the best answer in all situations. If NLBGS is more stable, and computational cheaper you should use it. I applaud your desire to get it to work with Newton. You should definitely track down why its not, but if it turns out that Newton just can't solve your coupled problem reliably thats ok too!
the set it to throw an error on non-convergence is broken on your answer. I have added the link which I think is the right one. Please correct if the linked one is not the one you were thinking to link.
I am trying to solve differential equations with indeterminate forms and I am playing with Julia and DifferentialEquations.jl package to do so.
I started with a simple differential equation with indeterminate form that I know of (as a test case) to see if this might work ( My eventual goal is to solve a complex system of differential equations and couple it with Machine Learning. This requires substantial amount of effort on my part to analyze and code, which I would only like to begin once I know that some basic test cases work).
Here is the test case code I started with:
bondi(u,p,t) = -(2*k/t^2)*(1 - (2*a^2*t/k))/(1 - a^2/u)
u0 = 0.01
p = (1e4, 10)
k,a = p
tc = k/(2*a^2)
tspan = (tc/10, tc*5)
prob = ODEProblem(bondi,u0,tspan,p)
The analytical solution to this problem (known as the Bondi flow problem in Astrophysics) is well known (even for the indeterminate cases). I am interested in the indeterminate solutions obtained from the solver.
When I solve using sol = solve(prob) I am getting an oscillating solution quite different from the smooth analytical solution I am aware exists (see the figure in the link below).
Plot u vs t
I was expecting to encounter some 'issues' as t approaches 50 (while simultaneously the y axis variable (representing speed) denoted by u would approach 100) as only then the numerator (and denominator) would vanish together. Any ideas why the solutions starts to oscillate?
I also tried with sol = solve(prob, alg_hints = [:stiff]) and got the following warning:
Warning: Interrupted. Larger maxiters is needed.
# DiffEqBase C:\Users\User.julia\packages\DiffEqBase\1yTcS\src\integrator_interface.jl:329
The solution still oscillates (similar to the solutions obtained without imposing stiffness).
Am I doing something incorrect here?
Is there another way to solve such indeterminate equations with the DifferentialEquations.jl package?
That ODE is highly numerically unstable at the singularity. Time stepping based methods are prone to explode when encountering that kind of problem. You need to use a collocation based method, like something in ApproxFun.jl, to avoid directly evaluating at the singularity in order to stabilize the solve.
I have done it in Excel but need to run a proper simulation in R.
I need to minimize function F(x) (x is a vector) while having constraints that sum(x)=1, all values in x are [0,1] and another function G(x) > G_0.
I have tried it with optim and constrOptim. None of them give you this option.
The problem you are referring to is (presumably) a non-linear optimization with non-linear constraints. This is one of the most general optimization problems.
The package I have used for these purposes is called nloptr: see here. From my experience, it is both versatile and fast. You can specify both equality and inequality constaints by setting eval_g_eq and eval_g_ineq, correspondingly. If the jacobians are known explicitly (can be derived analytically), specify them for faster convergence; otherwise, a numerical approximation is used.
Use this list as a general reference to optimization problems.
Write the set of equations using the Lagrange multiplier, then solve using the R command nlm.
You can do this in the OpenMx Package (currently host at the site listed below. Aiming for 2.0 relase on cran this year)
It is a general purpose package mostly used for Structural Equation Modelling, but handling nonlinear constraints.
FOr your case, make an mxModel() with your algebras expressed in mxAlgebras() and the constraints in mxConstraints()
When you mxRun() the model, the algebras will be solved within the constraints, if possible.
http://openmx.psyc.virginia.edu/