Approximating the whole problem with Mixed Analytical strategy - openmdao

Question

I have problem where I have implemented analytical derivatives for some components and I'm using complex step for the rest. There is a cyclic dependency between them so I also use a solver to converge them. It converges when I use NonlinearBlockGS. But when I use NewtonSolver in combination with a linear solver the optimization fails (Iteration limit exceeded), even with high iteration count. But I found that it converges easily and works perfectly when I use prob.model.approx_totals(). I read that approx_totals uses fd or cs to find the model gradients. So I have two questions.
In general, Will I lose the benefits from the mixed-analytical approach when I use approx_totals()? Is there a way to find the derivatives of whole model (or group) with mixed analytical strategy ? (Anyway In my case the explicitcomponents which are coupled use 'complex step`. But I'm just curious about this.)
In general (not in this scenario), will Openmdao automatically detect the mixed strategy or should I specify it some how ?
I will also be grateful, if you could point me to some examples where mixed derivatives are used. I didnt have any luck finding them myself.
Edit:Adding Example. I am not able to reproduce the issue in a sample code. Also I dont want to waste your time with my code(there more than 30 ExplicitComponents and 7 Groups). So I made a simple structure below to explain it better. In this there are 7 components A to G and only F and G doesn't have analytical derivatives and uses FD.
import openmdao.api as om
import numpy as np
class ComponentA_withDerivatives(om.ExplicitComponent):
def setup(self):
#setup inputs and outputs
def setup_partials(self):
#partial declaration
def compute(self, inputs, outputs):
def compute_partials(self, inputs, J):
#Partial definition
class ComponentB_withDerivatives(om.ExplicitComponent):
.....
class ComponentC_withDerivatives(om.ExplicitComponent):
......
class ComponentD_withDerivatives(om.ExplicitComponent):
......
class ComponentE_withDerivatives(om.ExplicitComponent):
......
class ComponentF(om.ExplicitComponent):
def setup(self):
#setup inputs and outputs
self.declare_partials(of='*', wrt='*', method='fd')
def compute(self,inputs,outputs):
# Computation
class ComponentG(om.ExplicitComponent):
def setup(self):
#setup inputs and outputs
self.declare_partials(of='*', wrt='*', method='fd')
def compute(self,inputs,outputs):
# Computation
class GroupAB(om.Group):
def setup(self):
self.add_subsystem('A', ComponentA_withDerivatives(), promotes_inputs=['x','y'], promotes_outputs=['z'])
self.add_subsystem('B', ComponentB_withDerivatives(), promotes_inputs=['x','y','w','u'], promotes_outputs=['k'])
class GroupCD(om.Group):
def setup(self):
self.add_subsystem('C', ComponentC_withDerivatives(), .....)
self.add_subsystem('D', ComponentD_withDerivatives(), ...)
class Final(om.Group):
def setup(self):
cycle1 = self.add_subsystem('cycle1', om.Group(), promotes=['*'])
cycle1.add_subsystem('GroupAB', GroupAB())
cycle1.add_subsystem('ComponentF', ComponentF())
cycle1.linear_solver = om.DirectSolver()
cycle1.nonlinear_solver = om.NewtonSolver(solve_subsystems=True)
cycle2 = self.add_subsystem('cycle2', om.Group(), promotes=['*'])
cycle2.add_subsystem('GroupCD', GroupCD())
cycle2.add_subsystem('ComponentE_withDerivatives', ComponentE_withDerivatives())
cycle2.linear_solver = om.DirectSolver()
cycle2.nonlinear_solver = om.NewtonSolver(solve_subsystems=True)
self.add_subsystem('ComponentG', ComponentG(), promotes_inputs=['a1','a2','a3'], promotes_outputs=['b1'])
prob = om.Problem()
prob.model = Final()
prob.driver = om.pyOptSparseDriver()
prob.driver.options['optimizer'] = 'SNOPT'
prob.driver.options['print_results']= True
## Design Variables
## Costraints
## Objectives
# Setup
prob.setup()
##prob.model.approx_totals(method='fd')
prob.run_model()
prob.run_driver()
Here this doesn't work. The cycle1 doesn't converge. The code works when I completely remove cycle1 or use NonlinearBlockGS instead of Newton or if I uncomment prob.model.approx_total(method='FD'). (no problem with cycle2. Work with Newton)
So if I don't use approx_totals(), I am assuming Openmdao uses a mixed strategy. Or should I manually mention it somehow ? And when I do use approx_totals() , will I lose the benefits from the analytical derivatives that I do have?

Answer 1

The code example you provided isn't runnable, so I'll have to make a few guesses. You call both run_model() and run_driver(). You bothered to include an optimizer in your sample code though, and you've show approx_totals to be called at the top of the model hierarchy.
So when you say it does not work, I will assume you mean that the optimizer doesn't converge.
You have understood the behavior of approx_totals correctly. When you set that at the top of your model, then OpenMDAO will FD the relevant variables from the group level. In this case, that means you will also be FD-ing across the solver itself. You say that this seems to work, but the mixed analytic approach does not.
In general, Will I lose the benefits from the mixed-analytical approach when I use approx_totals()?
Yes. You are no long using a mixed approach. You are just FD-ing across the model monolithically.
Is there a way to find the derivatives of whole model (or group) with mixed analytical strategy ?
OpenMDAO is computing total derivatives with a mixed strategy when you don't use approx_totals. The issue is that for your model, it seems not to be working.
In general (not in this scenario), will Openmdao automatically detect the mixed strategy?
It will "detect" it (it doesn't actually detect anything, but the underlying algorithms will use a mixed strategy UNLESS you tell it not to with approx_totals. Again, the issue is not that a mixed strategy is not being used, but that it is not working.
So why isn't the mixed strategy working?
I can only guess, since I can't run the code... so YMMV.
You mention that you are using complex-step for partials of your explicit components. Complex-step is a much more accurate approximation scheme than FD, but it is not without its own flaws. Not every computation is complex-safe. Some can be re-written to be complex-safe, others can not.
By "complex-safe" I mean that the computation correctly handles the complex-part to give a derivatives.
Two commonly used-complex-safe methods are np.linalg.norm and np.abs. Both will happily accept complex-numbers and give you an answer, but it is not the correct answer for when you need derivatives.
Because of this, OpenMDAO ships with a set of custom functions that are cs-safe --- custom norm and abs are provided.
What typically happens with non cs-safe methods is that the complex-part somehow gets dropped off and you get 0 partial derivatives. Wrong partials, wrong totals.
To check this, make sure you call check_partials on your components that are being complex-stepped, using a finite-difference check. You'll probably find some discrepancies.
The fixes available to you are:
Switch those components to use FD partials. Less accurate, but will probably work
Correct whatever problems in your compute are making your code non-cs-safe. Use OpenMDAO's custom functions if thats the problem, or possibly you need to be more careful about how you allocate and use numpy arrays in your compute (if you're allocating your own arrays, then you need to be careful to make sure they are complex too!).