Checking if the optimizer support gradient - openmdao

I want to distinguish optimizers (gradients based and free). If I use the sample optimization in the main webpage of OpenMDAO which uses SLSQP and check if the optimizer supports gradients I get "False" as in;
prob.driver.supports['gradients']
is this OpenMDAO or Scipy related issue?
Is there another way to see if the optimizer will use the gradient calculations or not before the problem is run.
Based on the answer below I added this in the beginning of my script; Thanks!
prob = om.api.Problem()
prob.driver = user_driver_object
prob.setup()
prob.final_setup()
grads.append(prob.driver.supports['gradients'])

In the ScipyOptimizeDriver not all optimizers support gradient optimization, hence you cannot determine the correct value until you setup your driver This is done in final_setup() of your problem (which calls _setup_driver() in your driver). This method is called in run_model() and run_driver(), but you can also call it in itself to get the correct properties of your optimizer.
In the example below I am asking the driver 3 times if it supports gradients. The first time, after the problem setup it gives a False answer (the default), because the driver was not touched yet. If I call final_setup(), this will setup the driver, and all the properties of the driver will be correct. If run_model() or run_driver() is called, of course this will also setup the driver.
So my advice is to just use final_setup() before querying anything from your driver, that can change during the setup (which are mostly optimizer specific properties).
import openmdao.api as om
# build the model
prob = om.Problem()
indeps = prob.model.add_subsystem('indeps', om.IndepVarComp())
indeps.add_output('x', 3.0)
indeps.add_output('y', -4.0)
prob.model.add_subsystem('paraboloid', om.ExecComp('f = (x-3)**2 + x*y + (y+4)**2 - 3'))
prob.model.connect('indeps.x', 'paraboloid.x')
prob.model.connect('indeps.y', 'paraboloid.y')
# setup the optimization
driver = prob.driver = om.ScipyOptimizeDriver()
prob.driver.options['optimizer'] = 'SLSQP'
prob.model.add_design_var('indeps.x', lower=-50, upper=50)
prob.model.add_design_var('indeps.y', lower=-50, upper=50)
prob.model.add_objective('paraboloid.f')
prob.setup()
print("\nSupports gradients (after setup)?")
print(prob.driver.supports['gradients'])
prob.final_setup()
print("\nSupports gradients (after final setup)?")
print(prob.driver.supports['gradients'])
prob.run_driver()
print("\nSupports gradients (after run)?")
print(prob.driver.supports['gradients'])
This results in the following output:
Supports gradients (after setup)?
False
Supports gradients (after final setup)?
True
Optimization terminated successfully. (Exit mode 0)
Current function value: -27.33333333333333
Iterations: 5
Function evaluations: 6
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Supports gradients (after run)?
True

Related

Terminal process terminated with exit code 3221226356 (Julia and VS Code)

I am trying to run the code below in VS Code for Julia (or directly on Julia). It is a simple example that computes the Maximum Likelihood estimator of the mean and the variance of a normal distribution (source):
Random.seed!(1234)
n = 1_000
data = randn(n)
mle = Model(optimizer_with_attributes(Ipopt.Optimizer, "print_level" => 0))
#NLparameter(mle, problem_data[i = 1:n] == data[i])
μ0 = randn()
σ0 = rand() + 1
#info "Starting guess, mean: $μ0, std: $σ0"
#variable(mle, μ, start = μ0)
#variable(mle, σ >= 0.0, start = σ0)
#NLexpression(mle, loglikelihood,
-(n / 2) * (log(2π) + 2 * log(σ)) - inv(2 * σ^2) * sum((xi - μ)^2 for xi in problem_data)
)
#NLobjective(mle, Max, loglikelihood)
optimize!(mle)
This is a nonlinear optimization problem using JuMP, and when running optimize!(mle) I am getting 'terminal process terminated with exit code 3221226356' in VS Code. Similarly, when I run it directly in Julia, it just shuts down entirely. (I have the latest versions) (I have tried in a different computer and everything works fine). Any help would be greatly appreciated!
P.S. I have seen it might have to do with a 'heap corruption problem', but I have no idea what that means/how to solve it.
This has been cross-posted on the Julia discourse, we'll continue to debug it there: https://discourse.julialang.org/t/cant-run-simple-jump-example/67938
It's either:
An issue in VS-Code (although "when I run it directly in Julia" may rule this out)
An issue with Ipopt, which is either due to it installing an old version, or a weird incompatibility with this user's system
Either way, this is likely hard to debug.

Is there a way to expand groups with the XDSM diagram creation in OpenMDAO?

Most of my test files involve the creation of an IndepVarComp that gets connected to a group. When I go to create an XDSM from the test file, it only shows the IndepVarComp Box and the Group Box. Is there a way to get it to expand the group and show what's inside?
This would also be useful when dealing with a top level model that contains many levels of groups where I want to expand one or two levels and leave the rest closed.
There is a recurse option, which controls if groups are expanded or not. Here is a small example with the Sellar problem to explore this option. The disciplines d1 and d2 are part of a Group called cycle.
import numpy as np
import openmdao.api as om
from openmdao.test_suite.components.sellar import SellarNoDerivatives
from omxdsm import write_xdsm
prob = om.Problem()
prob.model = model = SellarNoDerivatives()
model.add_design_var('z', lower=np.array([-10.0, 0.0]),
upper=np.array([10.0, 10.0]), indices=np.arange(2, dtype=int))
model.add_design_var('x', lower=0.0, upper=10.0)
model.add_objective('obj')
model.add_constraint('con1', equals=np.zeros(1))
model.add_constraint('con2', upper=0.0)
prob.setup()
prob.final_setup()
# Write output. PDF will only be created, if pdflatex is installed
write_xdsm(prob, filename='sellar_pyxdsm', out_format='pdf', show_browser=True,
quiet=False, output_side='left', recurse=True)
The same code with recurse=False (d1 and d2 are not shown, instead their Group cycle):
To enable the recursion from the command line, use the --recurse flag:
openmdao xdsm sellar_pyxdsm.py -f pdf --recurse
With the function it is turned on by default, in the command line you have to include the flag. If this does not work as expected for you, please provide an example.
You can find a lot of examples with different options in the tests of the XDSM plugin. Some of the options, like recurse, include_indepvarcomps, include_solver and model_path control what is included in the XDSM.

Can a parameter be used to set the unit attribute for a component?

So far, using Wolfram System Modeler 4.3 and 5.1 the following minimal example would compile without errors:
model UnitErrorModel
MyComponent c( hasUnit = "myUnit" );
block MyComponent
parameter String hasUnit = "1";
output Real y( unit = hasUnit );
equation
y = 10;
end MyComponent;
end UnitErrorModel;
But with the new release of WSM 12.0 (the jump in version is due to an alignment with the current release of Wolfram's flagship Mathematica) I am getting an error message:
Internal error: Codegen.getValueString: Non-constant expression:c.hasUnit
(Note: The error is given by WSMLink'WSMSimulate in Mathematica 12.0 which is running System Modeler 12.0 internally; here asking for the "InternalValues" property of the above model since I have not installed WSM 12.0 right now).
Trying to simulate the above model in OpenModelica [OMEdit v. 1.13.2 (64-bit)] reveals:
SimCodeUtil.mo: 8492:9-8492:218]: Internal error Unexpected expression (should have been handled earlier, probably in the front-end. Unit/displayUnit expression is not a string literal: c.hasUnit
So it seems that to set the unit attribute I cannot make use of a variable that has parameter variability? Why is this - after all shouldn't it suffice that the compiler can hard-wire the unit when compiling for runtime (after all the given model will run without any error in WSM 4.3 and 5.1)?
EDIT: From the answer to an older question of mine I had believed that at least final parameters might be used to set the unit-attribute. Making the modification final (e.g. c( final hasUnit = "myUnit" ) does not resolve the issue.
I have been given feedback on Wolfram Community by someone from Wolfram MathCore regarding this issue:
You are correct in that it's not in violation with the specification,
although making it a constant makes more sense since you would
invalidate all your static unit checking if you are allowed to change
the unit after building the simulation. We filed an issue on the
specification regarding this (Modelica Specification Issue # 2362).
So, MatheCore is a bit ahead of the game in proposing a Modelica specification change that they have already implemented. ;-)
Note: That in Wolfram System Modeler (12.0) using the annotation Evaluate = true will not cure the problem (cf. the comment above by #matth).
As a workaround variables used to set the unit attribute should have constant variability, but can nevertheless by included in user dialogs to be interactively changed using annotation(Dialog(group = "GroupName")).

Why are promoted names invalid when running in parallel?

I am trying to run a simple mathematical problem in parallel in OpenMDAO 2.5.0. The problem is an adapted version of the example in the OpenMDAO docs found here: http://openmdao.org/twodocs/versions/latest/features/core_features/grouping_components/parallel_group.html. It has some extra components and connections and uses promotions instead of connections.
from openmdao.api import Problem, IndepVarComp, ParallelGroup, ExecComp, Group, NonlinearBlockGS
prob = Problem()
model = prob.model
model.add_subsystem('p1', IndepVarComp('x1', 1.0), promotes=['x1'])
model.add_subsystem('p2', IndepVarComp('x2', 1.0), promotes=['x2'])
cycle = model.add_subsystem('cycle', Group(), promotes=['*'])
parallel = cycle.add_subsystem('parallel', ParallelGroup(), promotes=['*'])
parallel.add_subsystem('c1', ExecComp(['y1=(-2.0*x1+z)/3']), promotes=['x1', 'y1', 'z'])
parallel.add_subsystem('c2', ExecComp(['y2=(5.0*x2-z)/6']), promotes=['x2', 'y2', 'z'])
cycle.add_subsystem('c3', ExecComp(['z=(3.0*y1+7.0*y2)/10']), promotes=['y1', 'y2', 'z'])
model.add_subsystem('c4', ExecComp(['z2 = y1+y2']), promotes=['z2', 'y1', 'y2'])
cycle.nonlinear_solver = NonlinearBlockGS()
prob.setup(mode='fwd')
prob.set_solver_print(level=2)
prob.run_model()
print(prob['z2'])
print(prob['z'])
print(prob['y1'])
print(prob['y2'])
When I run this code in series, it works as expected with no errors.
However, when I run this code in parallel with:
mpirun -n 2 python Test.py
I get this error for the first process:
RuntimeError: The promoted name y1 is invalid because it refers to multiple inputs: [cycle.c3.y1 ,c4.y1]. Access the value from the connected output variable cycle.parallel.c1.y1 instead.
and this error for the second process:
RuntimeError: The promoted name y2 is invalid because it refers to multiple inputs: [cycle.c3.y2 ,c4.y2]. Access the value from the connected output variable cycle.parallel.c2.y2 instead.
So my question is: why does this example give an error in the promotes names when running in parallel, while it is running without problems in series? Is it only allowed to use connections when running in parallel or are promoted variables okay as well?
As of OpenMDAO V2.5, you have to be a little careful about how you access problem variables in parallel.
If you look at the full stack track of your model, you can see that the error is being thrown right at the end when you call
print(prob['y1'])
print(prob['y2'])
What is happening here is that you have set the model up so that y1 only exists on proc 0 and y2 only exists on proc 1. Then you try to get values that don't exist on that proc and you get (an admittedly not very clear) error.
You can fix this with the following minor modification to your script:
from openmdao.api import Problem, IndepVarComp, ParallelGroup, ExecComp, Group, NonlinearBlockJac
prob = Problem()
model = prob.model
model.add_subsystem('p1', IndepVarComp('x1', 1.0), promotes=['x1'])
model.add_subsystem('p2', IndepVarComp('x2', 1.0), promotes=['x2'])
cycle = model.add_subsystem('cycle', Group(), promotes=['*'])
parallel = cycle.add_subsystem('parallel', ParallelGroup(), promotes=['*'])
parallel.add_subsystem('c1', ExecComp(['y1=(-2.0*x1+z)/3']), promotes=['x1', 'y1', 'z'])
parallel.add_subsystem('c2', ExecComp(['y2=(5.0*x2-z)/6']), promotes=['x2', 'y2', 'z'])
cycle.add_subsystem('c3', ExecComp(['z=(3.0*y1+7.0*y2)/10']), promotes=['y1', 'y2', 'z'])
model.add_subsystem('c4', ExecComp(['z2 = y1+y2']), promotes=['z2', 'y1', 'y2'])
cycle.nonlinear_solver = NonlinearBlockJac()
prob.setup(mode='fwd')
prob.set_solver_print(level=2)
prob.run_model()
print(prob['z2'])
print(prob['z'])
if prob.model.comm.rank == 0:
print(prob['y1'])
if prob.model.comm.rank == 0:
print(prob['y2'])
There are a few minor issues with this. 1) it means your script is now different for serial and parallel. 2) Its annoying. So we're working on a fix that will let things work more cleanly by automatically doing an MPI broadcast when you try to get a value thats no on your proc. That will be released in V2.6.
One other small note. I changed you NL solver over to NonLinearBlockJac. That is for Block Jacobi, which is designed to work in parallel. You could also use the Newton solver in parallel The Gauss-Seidel solver won't actually allow you to get parallel speedups.

how to use chainer.using_config to stop F.dropout in evaluate/predict process in chainer?

F.dropout is only used in train, I confused how to use chainer.using_config in it?
How does it works and chainer how to know it is in training or predict?
From Chainer v2, function behavior is controlled by the config variable, according to the official doc,
chainer.config.train
Training mode flag. If it is True, Chainer runs in the training mode. Otherwise, it runs in the testing (evaluation) mode. The default value is True.
You may control this config by following two ways.
1. Simply assign the value.
chainer.config.train = False
here, code runs in the test mode, and dropout won't drop any unit.
model(x)
chainer.config.train = True
2. with using_config(key, value) notation
If we use above case, you might need to set True and False often, chainer provides with using_config(key, value) notation to ease this setting.
with chainer.using_config('train', False):
# train config is set to False, thus code runs in the test mode.
model(x)
# Here, train config is recovered to original value.
...
Note1: If you are using trainer moudule, Evaluator will handle these configuration automatically during validation/evaluation (see document, or source code). It means train config is set to False and dropout runs as evaluate mode when calculating validation loss.
Note2: train config is used to switch "train" mode and "evaluate/validation".
If you need "predict" code, you need to implement separately.

Resources