One can determine the lower and upper bounds of the design variable e.g. within the paraboloid sample :
top.model.add_design_var('p1.x', lower=-50, upper=50)
but is it possible to force the optimizer to sweep the design variables with a user input step?
something like
top.model.add_design_var('p1.x', lower=-50, upper=50, increment=2)
or maybe introduce it as an array
top.model.add_design_var('p1.x', [-50,-25,25,50])
Using a gradient based optimizer that is not possible. You would need to use a gradient free method. As over OpenMDAO 2.2 there is not any built in way to enforce that kind of discretization. You would need to use an external loop around the problem class to get that to work.
Here is a simple example:
import numpy as np
from openmdao.api import Problem, ScipyOptimizeDriver, ExecComp, IndepVarComp
# build the model
prob = Problem()
indeps = prob.model.add_subsystem('indeps', IndepVarComp())
indeps.add_output('x', 3.0)
indeps.add_output('y', -4.0)
prob.model.add_subsystem('paraboloid', ExecComp('f = (x-3)**2 + x*y + (y+4)**2 - 3'))
prob.model.connect('indeps.x', 'paraboloid.x')
prob.model.connect('indeps.y', 'paraboloid.y')
# setup the optimization
prob.driver = ScipyOptimizeDriver()
prob.driver.options['optimizer'] = 'SLSQP'
prob.model.add_design_var('indeps.y', lower=-50, upper=50)
prob.model.add_objective('paraboloid.f')
prob.setup()
for x in np.arange(-10,12,2):
prob['indeps.x'] = x
# could call just run_model if no optimization was desired
#prob.run_model()
# for each value of x, optimize for y
prob.run_driver()
# minimum value
print(prob['paraboloid.f'])
# location of the minimum
print(prob['indeps.x'])
print(prob['indeps.y'])
Related
I'm trying to setup a dynamic optimization with dymos where I have an analysis upstream of my dymos trajectory. This upstream analysis computes some 2D-matrix K. I want to pass this matrix into my dymos problem. According to the documentation (and how I've done this in the past) is to add K as a paramter to the trajectory:
traj.add_parameter('K',targets={'phase0':['K'],opt=False,static_target=True).
However, this returns an error because static_target expects K to be a scalar. If I have static_target=False, this also returns an error because it expects K to have some dimension related to the number of nodes in the trajectory.
Is there something I'm missing here?
Is it sufficient to manually connect K to the trajectory via
p.model.connect('K','traj.phase0.rhs_disc.K') and
p.model.connect('K','traj.phase0.rhs_col.K')? Or will that create issues in how dymos works the problem.
It doesn't seem appropriate to vectorize K either.
Any suggestions are greatly appreciated.
In my opinion, the easiest way to connect parameters from trajectory to phase is to add the parameter to both the Trajectory and the phases in which it is to be used.
Consider a simple oscillator where the mass, spring constant, and dampening coefficient are given as a single size-3 input.
In this case, I used OpenMDAO's tags feature and a special dymos tag dymos.static_target so that dymos realizes the target isn't shaped with a different value at each node. I think its a bit easier to do it this way as opposed to having to add it later at the add_parameter call.
class OscillatorODEVectorParam(om.ExplicitComponent):
"""
A Dymos ODE for a damped harmonic oscillator.
"""
def initialize(self):
self.options.declare('num_nodes', types=int)
def setup(self):
nn = self.options['num_nodes']
# Inputs
self.add_input('x', shape=(nn,), desc='displacement', units='m')
self.add_input('v', shape=(nn,), desc='velocity', units='m/s')
self.add_input('constants', shape=(3,), units=None,
desc='a vector of mass, spring constant, and damping coefficient [m, k, c]',
tags=['dymos.static_target'])
self.add_output('v_dot', val=np.zeros(nn), desc='rate of change of velocity', units='m/s**2')
self.declare_coloring(wrt='*', method='fd')
def compute(self, inputs, outputs):
x = inputs['x']
v = inputs['v']
m, k, c = inputs['constants']
f_spring = -k * x
f_damper = -c * v
outputs['v_dot'] = (f_spring + f_damper) / m
To use the ODE, we have a problem with a single trajectory and in this case, as single phase.
Again, in my opinion, the clearest way to link parameters from the trajectory to phases is to add them in both places with the same name.
Dymos will perform some introspection and automatically link them up.
def test_ivp_driver_shaped_param(self):
import openmdao.api as om
import dymos as dm
import matplotlib.pyplot as plt
# plt.switch_backend('Agg') # disable plotting to the screen
from dymos.examples.oscillator.oscillator_ode import OscillatorODEVectorParam
# Instantiate an OpenMDAO Problem instance.
prob = om.Problem()
# We need an optimization driver. To solve this simple problem ScipyOptimizerDriver will work.
prob.driver = om.ScipyOptimizeDriver()
# Instantiate a Phase
phase = dm.Phase(ode_class=OscillatorODEVectorParam, transcription=dm.Radau(num_segments=10))
# Tell Dymos that the duration of the phase is bounded.
phase.set_time_options(fix_initial=True, fix_duration=True)
# Tell Dymos the states to be propagated using the given ODE.
phase.add_state('x', fix_initial=True, rate_source='v', targets=['x'], units='m')
phase.add_state('v', fix_initial=True, rate_source='v_dot', targets=['v'], units='m/s')
# The spring constant, damping coefficient, and mass are inputs to the system that are
# constant throughout the phase.
# Declare this parameter on phase and then we'll feed its value from the parent trajectory.
phase.add_parameter('constants', units=None)
# Since we're using an optimization driver, an objective is required. We'll minimize
# the final time in this case.
phase.add_objective('time', loc='final')
# Instantiate a Dymos Trajectory and add it to the Problem model.
traj = prob.model.add_subsystem('traj', dm.Trajectory())
traj.add_phase('phase0', phase)
# This parameter value will connect to any phase with a parameter named constants by default.
# This is the easiest way, in my opinion, to pass parameters from trajectory to phase.
traj.add_parameter('constants', units=None, opt=False)
# Setup the OpenMDAO problem
prob.setup()
# Assign values to the times and states
prob.set_val('traj.phase0.t_initial', 0.0)
prob.set_val('traj.phase0.t_duration', 15.0)
prob.set_val('traj.phase0.states:x', 10.0)
prob.set_val('traj.phase0.states:v', 0.0)
# m k c
prob.set_val('traj.parameters:constants', [1.0, 1.0, 0.5])
# Now we're using the optimization driver to iteratively run the model and vary the
# phase duration until the final y value is 0.
prob.run_driver()
# Perform an explicit simulation of our ODE from the initial conditions.
sim_out = traj.simulate(times_per_seg=50)
# Plot the state values obtained from the phase timeseries objects in the simulation output.
t_sol = prob.get_val('traj.phase0.timeseries.time')
t_sim = sim_out.get_val('traj.phase0.timeseries.time')
states = ['x', 'v']
fig, axes = plt.subplots(len(states), 1)
for i, state in enumerate(states):
sol = axes[i].plot(t_sol, prob.get_val(f'traj.phase0.timeseries.states:{state}'), 'o')
sim = axes[i].plot(t_sim, sim_out.get_val(f'traj.phase0.timeseries.states:{state}'), '-')
axes[i].set_ylabel(state)
axes[-1].set_xlabel('time (s)')
fig.legend((sol[0], sim[0]), ('solution', 'simulation'), 'lower right', ncol=2)
plt.tight_layout()
plt.show()
I am trying to include some ref0= scaling in my optimization problem, but I keep getting a bounds error: ValueError: SLSQP Error: lb > ub in bounds True, False.. Without ref0= scaling the optimization runs fine. Taking the paraboloid optimization example from the docs (with slight modifications to get larger magnitudes in the values):
import openmdao.api as om
# build the model
prob = om.Problem()
prob.model.add_subsystem('paraboloid', om.ExecComp('f = (x-10)**2 + x*y + (y+40)**2 - 3'))
# setup the optimization
prob.driver = om.ScipyOptimizeDriver()
prob.driver.options['optimizer'] = 'SLSQP'
prob.model.add_design_var('paraboloid.x', lower=20, upper=60, ref=10, ref0=40)
prob.model.add_design_var('paraboloid.y', lower=-100, upper=50, ref=10)
prob.model.add_objective('paraboloid.f')
prob.setup()
# Set initial values.
prob.set_val('paraboloid.x', 40)
prob.set_val('paraboloid.y', -4.0)
# run the optimization
prob.run_driver()
# minimum value
print(prob.get_val('paraboloid.f'))
# location of the minimum
print(prob.get_val('paraboloid.x'))
print(prob.get_val('paraboloid.y'))
Now of course it may be unnecessary for this simple problem to add such scaling, but it serves the purpose of recreating the problem. Adding ref0=40 to the 'paraboloid.x' design var will cause the bounds error. How can/should I correctly setup the bounds and scale factors in order for the optimizer to work properly?
ref0 and ref specify those values which the optimizer sees as 0 and 1, respectively. When you set ref0 > ref, you're essentially flipping the axis of that variable, as far as the optimizer is aware.
The scaling is then applied to your lower and upper bounds. Because the axis is flipped, upper is now less than lower and the optimizer is unhappy.
In this case the correct behavior can be obtained by swapping the values of lower and upper, although I'd argue that providing ref > ref0 is more logical.
Recently, I have started working in the field of multidisciplinary design optimization. I am using OpenMDAO framework for weight optimization of Golinski's speed reducer in the MDO test suit. I want to apply MDF architecture for this problem. I am referring the paper "Benchmarking Multidisciplinary Design Optimization Algorithms" by Tedford and Martins for problem formulation and decomposition. Where they have decomposed this problem into three disciplines and their individual constraints.
While coding, I referred Seller problem from OpenMDAO documentation. I have made three disciplines and a group (speed_mda()) to implement multidisciplinary analysis. I have added objective function and constraints in the speed_mda() group as a subsystem. I have made their connections with discipline outputs (coupled variables). But I have not applied these constraints on individual discipline (Actually, I don't know how to do it). So I have applied all of them on top level group. I am getting output 2713.678 by violating some of the constraints.
Here is my code:
# Discipline 1
class speed_1(om.ExplicitComponent):
def setup(self):
self.add_input('z1', val = 0)
self.add_input('z2', val = 0)
self.add_output('y1', val = 1)
def setup_partials(self):
# Finite difference all partials.
self.declare_partials('*', '*', method='fd')
def compute(self,inputs,outputs):
outputs['y1'] = max(27/(inputs['z1']**2*(inputs['z2'])),397.5/(inputs['z1']**2*inputs['z2']**2), 5*inputs['z1'], 2.6)
# Discipline 2
class speed_2(om.ExplicitComponent):
def setup(self):
self.add_input('z1', val = 0)
self.add_input('z2', val = 0)
self.add_input('x21', val = 0)
self.add_output('y2', val = 0)
def setup_partials(self):
self.declare_partials('*', '*', method='fd')
def compute(self,inputs,outputs):
outputs['y2'] = max((1.93*inputs['x21']**3/(inputs['z1']*inputs['z2']))**0.25, 1/(0.5*(((1.69*10**7)**2)*inputs['x21']**2/(inputs['z1']**2*inputs['z2']**2) + 745)**0.5)**(0.3333), 2.9)
# Discipline 3
class speed_3(om.ExplicitComponent):
def setup(self):
self.add_input('z1', val = 0)
self.add_input('z2', val = 0)
self.add_input('x31', val = 0)
self.add_output('y3', val = 0)
def setup_partials(self):
self.declare_partials('*', '*', method='fd')
def compute(self,inputs,outputs):
outputs['y3'] = max((1.93*inputs['x31']**3/(inputs['z1']*inputs['z2']))**0.25, 1/(85*(((1.69*10**7)**2)*inputs['x31']**2/(inputs['z1']**2*inputs['z2']**2) + 1.575*(10**8))**0.5)**(0.3333), 5)
class speed_mda(om.Group):
def setup(self):
# Adding all discipline to MDA
cycle = self.add_subsystem('cycle',om.Group(),promotes_inputs=['z1', 'z2', 'x21', 'x31'])
cycle.add_subsystem('d1', speed_1(), promotes_inputs = ['z1', 'z2'])
cycle.add_subsystem('d2', speed_2(), promotes_inputs=['z1','z2', 'x21'])
cycle.add_subsystem('d3', speed_3(), promotes_inputs=['z1','z2', 'x31'])
# No need of connections for the discipline
cycle.set_input_defaults('x21', 7.8)
cycle.set_input_defaults('x31', 8.3)
cycle.set_input_defaults('z1', 0.75)
cycle.set_input_defaults('z2', 22.0)
# Add solver to MDA: Nonlinear Block Gauss Seidel is a gradient free solver
cycle.nonlinear_solver = om.NonlinearBlockGS()
# Adding obj. function and constraints as a subsystem
self.add_subsystem('obj_fun',om.ExecComp('obj = (0.7854*y1*z1**2)*(3.3333*z2**2+14.933*z2-43.0934) - 1.5079*y1*(y2**2+y3**2)+7.477*(y2**3+y3**3)+0.7854*(x21*y2**2+x31*y3**2)', z1=0.0,z2=0.0,x21=0.0,x31=0.0), promotes=['x21','x31','z2','z1','obj'])
self.add_subsystem('con1',om.ExecComp('c1 = z1*z2 - 40.0'), promotes=['c1']) # Global
self.add_subsystem('con10',om.ExecComp('c10 = y1 - 12.0*z1'), promotes=['c10']) # 1
self.add_subsystem('con11',om.ExecComp('c11 = y1 - 3.6'), promotes=['c11'])
self.add_subsystem('con12',om.ExecComp('c12 = y2 - 3.9'), promotes=['c12']) # 2
self.add_subsystem('con13',om.ExecComp('c13 = 2.85*y2 - x21'), promotes=['c13'])
self.add_subsystem('con14',om.ExecComp('c14 = y3 - 5.5'), promotes=['c14']) # 3
self.add_subsystem('con15',om.ExecComp('c15 = 2.09*y3 - x31'), promotes=['c15'])
# Connect outputs from MDA (coupled variables) to obj. function and constraints
self.connect('cycle.d1.y1',['obj_fun.y1','con10.y1','con11.y1'])
self.connect('cycle.d2.y2',['obj_fun.y2','con12.y2','con13.y2'])
self.connect('cycle.d3.y3',['obj_fun.y3','con14.y3','con15.y3'])
# Form topmost group (Problem) and add above MDF model to it
prob = om.Problem()
model = prob.model = speed_mda()
prob.driver = om.ScipyOptimizeDriver()
prob.driver.options['optimizer'] = 'SLSQP'
prob.driver.options['tol'] = 1e-9
prob.driver.options['disp'] = True
model.add_design_var('x21', lower=7.3, upper=8.3)
model.add_design_var('x31', lower=7.3, upper=8.3)
model.add_design_var('z1', lower=0.7, upper=0.8)
model.add_design_var('z2', lower=17.0, upper=28.0)
model.add_objective('obj')
model.add_constraint('c1', upper = 0)
model.add_constraint('c10',upper = 0)
model.add_constraint('c11',upper = 0)
model.add_constraint('c12',upper = 0)
model.add_constraint('c13',upper = 0)
model.add_constraint('c14',upper = 0)
model.add_constraint('c15',upper = 0)
prob.model.approx_totals()
prob.setup()
prob.set_solver_print(level=0)
prob.set_val('x21', 7.8)
prob.set_val('x31', 8.3)
prob.set_val('z1', 0.75)
prob.set_val('z2', 22.0)
prob.run_model()
prob.run_driver()
print('minimum found at')
print((prob.get_val('z1')[0],prob.get_val('z2')[0],prob.get_val('x21')[0],prob.get_val('x31')[0],prob.get_val('cycle.d1.y1')[0], prob.get_val('cycle.d2.y2')[0],prob.get_val('cycle.d3.y3')[0]))
print('minumum objective')
print(prob.get_val('obj')[0])
I am getting following output:
Positive directional derivative for linesearch (Exit mode 8)
Current function value: [2713.67806668]
Iterations: 8
Function evaluations: 4
Gradient evaluations: 4
Optimization FAILED.
Positive directional derivative for linesearch
-----------------------------------
minimum found at
(0.7, 17.00007920093152, 7.300007915120889, 7.300015825246984, 3.5, 2.9, 5.0)
minumum objective
2713.6780666813797
Here, objective value is far less than the actual and it says optimization has failed. I looked for the above error but my initial guess is inside the bound. My all problem constraints are not satisfied. I also tried to apply constraints on individual disciplines but I couldn't do it. I don't know what is the actual problem, probably I am making some basic conceptual mistake. Can please anyone help me with this.
There is something strange about the way the speed reducer problem is posed in that paper. The use of max function is not technically differentiable at all, which makes this an less-than-ideal way to implement a problem for use in gradient based optimization
Also, that formulation looks really different from other descriptions of the speed-reducer problem I've seen in other papers. This formulation traces back to original work on an MDO test problem suite, where problems that were not really multidisciplinary were broken up into separate blocks and additional constraints were added to ensure compatibility. In the case of this paper, I think the changes to the problem formulation resulted in some less than ideal problem structure. I recommend you look toward a more well posed formulation such as this one
Regardless, when I set the given inputs from the states optimum of their paper I don't get the optimum value that they reported. I get a lower number, so there must be something subtly different about your code. Somehow your code is returning lower values, so check your equations carefully.
I ran your code on OpenMDAO V3.8 and got the following:
/Users/jsgray/work/packages/OpenMDAO/openmdao/core/total_jac.py:1713: UserWarning:Constraints or objectives ['con1.c1', 'con12.c12', 'con13.c13', 'con14.c14', 'con15.c15'] cannot be impacted by the design variables of the problem.
Positive directional derivative for linesearch (Exit mode 8)
Current function value: [2713.66402046]
Iterations: 9
Function evaluations: 5
Gradient evaluations: 5
Optimization FAILED.
Positive directional derivative for linesearch
-----------------------------------
minimum found at
(0.7, 17.000000001268262, 7.3, 7.3, 3.5, 2.9, 5.0)
minumum objective
2713.6640204584155
So I see the same value you do, but I also got a helpful additional warning (newly added in V3.8) about constraints that are not affected by any of the design variables. When I commented out those constraints, the result changed to
Optimization terminated successfully (Exit mode 0)
Current function value: [2713.66402024]
Iterations: 10
Function evaluations: 6
Gradient evaluations: 6
Optimization Complete
-----------------------------------
minimum found at
(0.7, 17.0, 7.3, 7.3, 3.5, 2.9, 5.0)
minumum objective
2713.6640202393
Which is the same answer as before, but sans the scary warnings from the optimizer. So the error you were seeing was due to having a large number of constraints that, while inherently satisfied, were not actually controllable by the optimizer. This causes rows of all zeros to show up in the Jacobian and hence makes the optimization problem singular. While SLSQP was able to work around the singularity, it caused enough numerical headaches to cause it to throw the warning.
I'd like to check the total derivatives of an output with respect to a large array of inputs, but I don't want to check the derivative with respect to every member of the array, since the array is too large, and the complex steps (or finite differences) across each member of the array would take too long. Is there a way to check_totals wrt just a single member of an array?
Alternatively, is there a way to perform a directional derivative across the entire array for check_totals? This feature seems to exist for check_partials only?
As of Version 3.1.1 of OpenMDAO we don't have directional checking for totals, but it is a good idea and we are probably going to implement it when we figure out the best way.
As a workaround for now, I think the easiest way to take a directional derivative of your model is to temporarily modify your model by creating a component that takes a "step" in some random direction, and then inserting it in front of your component with wide inputs. I've put together a simple example here:
import numpy as np
import openmdao.api as om
n = 50
class DirectionalComp(om.ExplicitComponent):
def setup(self):
self.add_input('x', 1.0)
self.add_output('y', np.ones(n))
self.A = -1.0 + 2.0 * np.random.random(n)
self.declare_partials('y', 'x', rows=np.arange(n), cols=np.repeat(0, n), val=self.A)
def compute(self, inputs, outputs, discrete_inputs=None, discrete_outputs=None):
x = inputs['x']
outputs['y'] = x * self.A
prob = om.Problem()
model = prob.model
# Add something like this
model.add_subsystem('p', om.IndepVarComp('x', 1.0))
model.add_subsystem('direction', DirectionalComp())
model.connect('p.x', 'direction.x')
model.connect('direction.y', 'comp.x')
model.add_design_var('p.x')
# Old Model
model.add_subsystem('comp', om.ExecComp('y = 2.0*x', x=np.ones((n, )), y=np.ones((n, ))))
model.add_constraint('comp.y', lower=0.0)
prob.setup()
prob.run_model()
totals = prob.check_totals()
I am using Conv2D model of Keras 2.0. However, I cannot fully understand what the function is doing mathematically. I try to understand the math using randomly generated data and a very simple network:
import numpy as np
import keras
from keras.layers import Input, Conv2D
from keras.models import Model
from keras import backend as K
# create the model
inputs = Input(shape=(10,10,1)) # 1 channel, 10x10 image
outputs = Conv2D(32, (3, 3), activation='relu', name='block1_conv1')(inputs)
model = Model(outputs=outputs, inputs=inputs)
# input
x = np.random.random(100).reshape((10,10))
# predicted output for x
y_pred = model.predict(x.reshape((1,10,10,1))) # y_pred.shape = (1,8,8,32)
I tried to calculate, for example, the value of the first row, the first column in the first feature map, following the demo in here.
w = model.layers[1].get_weights()[0] # w.shape = (3,3,1,32)
w0 = w[:,:,0,0]
b = model.layers[1].get_weights()[1] # b.shape = (32,)
b0 = b[0] # b0 = 0
y_pred_000 = np.sum(x[0:3,0:3] * w0) + b0
But relu(y_pred_000) is not equal to y_pred[0][0][0][0].
Could anyone point out what's wrong with my understanding? Thank you.
It's easy and it comes from Theano dim ordering. The result of applying filter in stored in a so called channel dimension. In case of TensorFlow this is the last dimension and that's why results are good. In case of Theano it's second dimension (convolution result has shape (cases, channels, width, height) so in order to solve your problem you need to change prediction line to:
y_pred = model.predict(x.reshape((1,1,10,10)))
Also you need to change the way you get the weights as weights in Theano has shape (output_channels, input_channels, width, height) you need to change the weight getter to:
w = model.layers[1].get_weights()[0] # w.shape = (32,1,3,3)
w0 = w[0,0,:,:]