Meaning of factor is exactly singular / singular entry found in group - openmdao

Question

I am struggling with an error regarding a singular entry in the group caused by an implicit component, and I don't manage to figure out how to solve it.
We created a louvered fin heat exchanger model, based on the effectiveness-NTU method. It uses an ImplicitComponent to solve the system in such a way that the "guessed" outlet temperatures (used to calculate fluid properties) are equal to the actual outlet temperatures calculated based on the actual heat transfer. This components seems to run fine and the N2 diagram of this base heat exchanger can be found here.
Among others, two inputs are the mass flow rates of the cold and hot side (see indeps in original N2). However, the validation data uses cold side flow velocity and hot side volumetric flow rate instead of mass flow rates. Although not direct group inputs, these properties are calculated within the heat exchanger group. Manually changing the mass flow rates until the flow velocity and volumetric flow rate meet that of the validation data works fine. But I figured I could add an additional implicit component around the heat exchanger group to do that work for me. The resulting N2 diagram can be found here. However, this implicit component results in an error:
Traceback (most recent call last):
File "...\openmdao\solvers\linear\direct.py", line 275, in _linearize
self._lu = scipy.sparse.linalg.splu(matrix)
File "...\scipy\sparse\linalg\dsolve\linsolve.py", line 326, in splu
ilu=False, options=_options)
RuntimeError: Factor is exactly singular
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "louveredfin3.py", line 91, in <module>
p.run_model()
File "...\openmdao\core\problem.py", line 527, in run_model
self.model.run_solve_nonlinear()
File "...\openmdao\core\system.py", line 3734, in run_solve_nonlinear
self._solve_nonlinear()
File "...\openmdao\core\group.py", line 1886, in _solve_nonlinear
self._nonlinear_solver.solve()
File "...\openmdao\solvers\solver.py", line 597, in solve
raise err
File "...\openmdao\solvers\solver.py", line 593, in solve
self._solve()
File "...\openmdao\solvers\solver.py", line 384, in _solve
self._single_iteration()
File "...\openmdao\solvers\nonlinear\newton.py", line 230, in _single_iteration
self._linearize()
File "...\openmdao\solvers\nonlinear\newton.py", line 161, in _linearize
self.linear_solver._linearize()
File "...\openmdao\solvers\linear\direct.py", line 278, in _linearize
raise RuntimeError(format_singular_error(system, matrix))
RuntimeError: Singular entry found in Group (<model>) for column associated with state/residual 'ConvertInputs.m_dot_hot' index 0.
What does this error mean in practical terms? Is the solver not able to reduce the residual by changing the output (m_dot_hot and m_dot_cold in this case)? However, if this is the case I am failing to understand why, as manually changing the mass flow rates does result in a change in 'V_cold' and 'flowrate_hot'.
As an alternative I tried to use only one solver for all the components on the same level (N2 here), but this results in the same error. Also removing the original temperature implicit component (i.e. only having one implicit component, which changes the mass flow rate) did not solve the problem.
In case it helps, the implicit component looks like this (using single values instead of array with length n for the time being):
class ConvertInputs(om.ImplicitComponent):
def initialize(self):
self.options.declare('n', default=1, desc='length of the array')
def setup(self):
self.add_input('flowrate_hot_required', val=1.33, desc='required flowrate', units='L/s')
self.add_input('flowrate_hot', val=1.33, desc='actual flowrate', units='L/s')
self.add_input('V_cold_required', val=8., desc='required air velocity', units='m/s')
self.add_input('V_cold', val=8., desc='actual air velocity', units='m/s')
self.add_output('m_dot_hot', val=1.296, desc='hot side mass flow rate', units='kg/s')
self.add_output('m_dot_cold', val=1.655, desc='cold side mass flow rate', units='kg/s')
self.declare_partials('*', '*', method='fd')
def apply_nonlinear(self, inputs, outputs, residuals):
residuals['m_dot_hot'] = inputs['flowrate_hot'] - inputs['flowrate_hot_required']
residuals['m_dot_cold'] = inputs['V_cold'] - inputs['V_cold_required']
and the related lines in FluidProperties like this:
outputs['flowrate_hot'] = inputs['m_dot_hot'] / (outputs['rho_hot_in']*1e-3)
outputs['V_cold'] = inputs['m_dot_cold'] / (outputs['rho_cold_in'] * inputs['A_flow_cold'])
The solvers used are the NewtonSolver (solve_subsystems=True) and DirectSolver. Additionally I double checked that the derivatives are declared everywhere (e.g. self.declare_partials(' * ', ' * ', method='fd') in all components for now), but no success so far.
EDIT – based on the answer from Justin:
Thank you for the answer and the tips! I implemented the BalanceComp replacing the top-level implicit component, but unfortunately it did not make a difference. Setting maxiter=0 for the top solver still throws the error, but the lower solver seems to solve without problem:
+
+ ========
+ original
+ ========
+ NL: Newton 0 ; 14.3235199 1
+ NL: Newton 1 ; 0.0668341831 0.00466604463
+ NL: Newton 2 ; 0.000273898972 1.91223229e-05
+ NL: Newton 3 ; 1.17390481e-06 8.1956448e-08
+ NL: Newton 4 ; 5.0212449e-09 3.50559425e-10
+ NL: Newton 5 ; 2.14662005e-11 1.49866797e-12
+ NL: Newton Converged
NL: Newton 0 ; 0.252556049 1
Traceback (most recent call last):
[…]
RuntimeError: Factor is exactly singular
To get a feeling for the magnitudes: using only the HX subsystem and manually changing the indeps “m_dot_hot” and “m_dot_cold” with stepsize 1e-6:
Diff. flowrate_hot = -1.0271477852707989e-06
Diff. V_cold = -4.6808071525461514e-06
I did not realise yet that you can actually add bounds to the implicit component, that is very useful and I added them for both implicit components. Unfortunately, no improvement for the error from that neither. Setting the partials to complex step for the top-level implicit component and the FluidProperties did also not improve the situation. Should this be done for all components or only the involved ones here?.
However, I noticed the following when printing the “m_dot_* ” outputs in the implicit component, and the “m_dot_* ” inputs in the FluidProperties component in apply_nonlinear() and compute() respectively. It shows that after the heat exchanger subsystem has solved, the top-level implicit component “m_dot_* ” outputs change with the stepsize of 1e-6 as I would expect it to do for gradient calculation. However, the inputs in FluidProperties are not printed anymore at all after this and the singular error occurs soon after. Hence, to me it seems that the lower level FluidProperties compute method is not called anymore. Since no analytical derivatives are given or values set (i.e. only FD is used of all output w.r.t. all inputs), it appears to me that the FluidProperties component is never executed with the “m_dot_* ” stepsize and the gradient is not (successfully or correctly) calculated. Nevertheless, the N2 chart shows that the inputs/outputs of top-level to subsystem are correctly connected. Can this be a pointer to something specifically going wrong?

Answer 1

Your error (RuntimeError: Singular entry found in Group (<model>) for column associated with state/residual 'ConvertInputs.m_dot_hot' index 0.) indicates that, all of the partial derivatives in that column are 0.
Practically, that means that as far as OpenMDAO is concerned changes to 'ConvertInputs.m_dot_hot' don't affect any of the residuals in your model.
One thing you can try is to use the BalanceComp from OpenMDAO's standard library. This component is deigned specifically for what you were trying to accomplish, but has derivatives already defined. This is a small chance that this will fix your problem, but likely not.
What I recommend is the following:
Set the max_iter option on your top level newton solver to 0 (the sub-solves will still run). Then you can run your model and manually change your guess for m_dot_hot to see if you really can manually converge it in the model with the coupling built in. Perhaps there is a bug in the way you did the connections in this model that is causing the problem. You said that you could manually converge the original model, this step will make sure that you coupled model also has a solution
I noticed that you did not define any bounds on your design variables. Perhaps the solver is driving m_dot_hot to 0 or a negative number in its iterations. I suggest setting both lower and upper bounds to something reasonable as follows
self.add_output('m_dot_hot', val=1.296, desc='hot side mass flow rate', units='kg/s', lower=1e-4, upper=10)
self.add_output('m_dot_cold', val=1.655, desc='cold side mass flow rate', units='kg/s', lower=1e-4, upper=10)
For OpenMDAO V3.0 and up, the default for the newton solver is to use the bounds enforcing line search which will respect these limits when trying to converge
Consider switching to the cs method for your partial derivatives. Its possible that the FD values are just not very good (at least with the default step sizes and you're getting 0s that are numerical instead of physical.