I wanna create a Neural Network in PyTorch, that will have 2 inputs and 3 outputs with 1 hidden layer. The two inputs will be float numbers that represents features of an image and 3 outputs will be real numbers between 0 and 1. For example output (1, 0, 0) would mean that it is square and (0,1,0) would mean it is rectangle. Any idea how to do it in pytorch?
The network can be defined like this:
import torch
import torch.nn as nn
import torch.optim as optim
import torch.autograd as autograd
import torch.nn.functional as F
from torch.autograd import Variable
class Net(nn.Module):
def __init__(self, num_inputs=2, num_outputs=3,hidden_dim=5):
# define your network here
super(Net, self).__init__()
self.layer1 = nn.Linear(num_inputs,hidden_dim)
self.layer2 = nn.Linear(hidden_dim,num_outputs)
def forward(self, x):
# implement the forward pass
x = F.relu(self.layer1(x))
x = F.sigmoid(self.layer2(x))
return x
Although I have defined the network here, you should maybe look at some examples on the official pytorch website for example on how to train your model.
Related
I'm trying to setup a dynamic optimization with dymos where I have an analysis upstream of my dymos trajectory. This upstream analysis computes some 2D-matrix K. I want to pass this matrix into my dymos problem. According to the documentation (and how I've done this in the past) is to add K as a paramter to the trajectory:
traj.add_parameter('K',targets={'phase0':['K'],opt=False,static_target=True).
However, this returns an error because static_target expects K to be a scalar. If I have static_target=False, this also returns an error because it expects K to have some dimension related to the number of nodes in the trajectory.
Is there something I'm missing here?
Is it sufficient to manually connect K to the trajectory via
p.model.connect('K','traj.phase0.rhs_disc.K') and
p.model.connect('K','traj.phase0.rhs_col.K')? Or will that create issues in how dymos works the problem.
It doesn't seem appropriate to vectorize K either.
Any suggestions are greatly appreciated.
In my opinion, the easiest way to connect parameters from trajectory to phase is to add the parameter to both the Trajectory and the phases in which it is to be used.
Consider a simple oscillator where the mass, spring constant, and dampening coefficient are given as a single size-3 input.
In this case, I used OpenMDAO's tags feature and a special dymos tag dymos.static_target so that dymos realizes the target isn't shaped with a different value at each node. I think its a bit easier to do it this way as opposed to having to add it later at the add_parameter call.
class OscillatorODEVectorParam(om.ExplicitComponent):
"""
A Dymos ODE for a damped harmonic oscillator.
"""
def initialize(self):
self.options.declare('num_nodes', types=int)
def setup(self):
nn = self.options['num_nodes']
# Inputs
self.add_input('x', shape=(nn,), desc='displacement', units='m')
self.add_input('v', shape=(nn,), desc='velocity', units='m/s')
self.add_input('constants', shape=(3,), units=None,
desc='a vector of mass, spring constant, and damping coefficient [m, k, c]',
tags=['dymos.static_target'])
self.add_output('v_dot', val=np.zeros(nn), desc='rate of change of velocity', units='m/s**2')
self.declare_coloring(wrt='*', method='fd')
def compute(self, inputs, outputs):
x = inputs['x']
v = inputs['v']
m, k, c = inputs['constants']
f_spring = -k * x
f_damper = -c * v
outputs['v_dot'] = (f_spring + f_damper) / m
To use the ODE, we have a problem with a single trajectory and in this case, as single phase.
Again, in my opinion, the clearest way to link parameters from the trajectory to phases is to add them in both places with the same name.
Dymos will perform some introspection and automatically link them up.
def test_ivp_driver_shaped_param(self):
import openmdao.api as om
import dymos as dm
import matplotlib.pyplot as plt
# plt.switch_backend('Agg') # disable plotting to the screen
from dymos.examples.oscillator.oscillator_ode import OscillatorODEVectorParam
# Instantiate an OpenMDAO Problem instance.
prob = om.Problem()
# We need an optimization driver. To solve this simple problem ScipyOptimizerDriver will work.
prob.driver = om.ScipyOptimizeDriver()
# Instantiate a Phase
phase = dm.Phase(ode_class=OscillatorODEVectorParam, transcription=dm.Radau(num_segments=10))
# Tell Dymos that the duration of the phase is bounded.
phase.set_time_options(fix_initial=True, fix_duration=True)
# Tell Dymos the states to be propagated using the given ODE.
phase.add_state('x', fix_initial=True, rate_source='v', targets=['x'], units='m')
phase.add_state('v', fix_initial=True, rate_source='v_dot', targets=['v'], units='m/s')
# The spring constant, damping coefficient, and mass are inputs to the system that are
# constant throughout the phase.
# Declare this parameter on phase and then we'll feed its value from the parent trajectory.
phase.add_parameter('constants', units=None)
# Since we're using an optimization driver, an objective is required. We'll minimize
# the final time in this case.
phase.add_objective('time', loc='final')
# Instantiate a Dymos Trajectory and add it to the Problem model.
traj = prob.model.add_subsystem('traj', dm.Trajectory())
traj.add_phase('phase0', phase)
# This parameter value will connect to any phase with a parameter named constants by default.
# This is the easiest way, in my opinion, to pass parameters from trajectory to phase.
traj.add_parameter('constants', units=None, opt=False)
# Setup the OpenMDAO problem
prob.setup()
# Assign values to the times and states
prob.set_val('traj.phase0.t_initial', 0.0)
prob.set_val('traj.phase0.t_duration', 15.0)
prob.set_val('traj.phase0.states:x', 10.0)
prob.set_val('traj.phase0.states:v', 0.0)
# m k c
prob.set_val('traj.parameters:constants', [1.0, 1.0, 0.5])
# Now we're using the optimization driver to iteratively run the model and vary the
# phase duration until the final y value is 0.
prob.run_driver()
# Perform an explicit simulation of our ODE from the initial conditions.
sim_out = traj.simulate(times_per_seg=50)
# Plot the state values obtained from the phase timeseries objects in the simulation output.
t_sol = prob.get_val('traj.phase0.timeseries.time')
t_sim = sim_out.get_val('traj.phase0.timeseries.time')
states = ['x', 'v']
fig, axes = plt.subplots(len(states), 1)
for i, state in enumerate(states):
sol = axes[i].plot(t_sol, prob.get_val(f'traj.phase0.timeseries.states:{state}'), 'o')
sim = axes[i].plot(t_sim, sim_out.get_val(f'traj.phase0.timeseries.states:{state}'), '-')
axes[i].set_ylabel(state)
axes[-1].set_xlabel('time (s)')
fig.legend((sol[0], sim[0]), ('solution', 'simulation'), 'lower right', ncol=2)
plt.tight_layout()
plt.show()
I'd like to check the total derivatives of an output with respect to a large array of inputs, but I don't want to check the derivative with respect to every member of the array, since the array is too large, and the complex steps (or finite differences) across each member of the array would take too long. Is there a way to check_totals wrt just a single member of an array?
Alternatively, is there a way to perform a directional derivative across the entire array for check_totals? This feature seems to exist for check_partials only?
As of Version 3.1.1 of OpenMDAO we don't have directional checking for totals, but it is a good idea and we are probably going to implement it when we figure out the best way.
As a workaround for now, I think the easiest way to take a directional derivative of your model is to temporarily modify your model by creating a component that takes a "step" in some random direction, and then inserting it in front of your component with wide inputs. I've put together a simple example here:
import numpy as np
import openmdao.api as om
n = 50
class DirectionalComp(om.ExplicitComponent):
def setup(self):
self.add_input('x', 1.0)
self.add_output('y', np.ones(n))
self.A = -1.0 + 2.0 * np.random.random(n)
self.declare_partials('y', 'x', rows=np.arange(n), cols=np.repeat(0, n), val=self.A)
def compute(self, inputs, outputs, discrete_inputs=None, discrete_outputs=None):
x = inputs['x']
outputs['y'] = x * self.A
prob = om.Problem()
model = prob.model
# Add something like this
model.add_subsystem('p', om.IndepVarComp('x', 1.0))
model.add_subsystem('direction', DirectionalComp())
model.connect('p.x', 'direction.x')
model.connect('direction.y', 'comp.x')
model.add_design_var('p.x')
# Old Model
model.add_subsystem('comp', om.ExecComp('y = 2.0*x', x=np.ones((n, )), y=np.ones((n, ))))
model.add_constraint('comp.y', lower=0.0)
prob.setup()
prob.run_model()
totals = prob.check_totals()
I am trying to solve an optimization problem where I need to specify the problem and the constraints using a 2D matrix. I have been using SCIPY, where the 1D arrays are the requirements. I want to check if GEKKO allows one to specify the objective function, bounds and constraints using a 2D matrix.
I have provided details and a reproducible version of the problem in the post here:
SCIPY - building constraints without listing each variable separately
Thanks
C
You can use the m.Array function in gekko. I don't recommend that you use the np.triu() with the Gekko array because the eliminated variables will still solve but potentially be hidden from the results. Here is a solution:
import numpy as np
import scipy.optimize as opt
from gekko import GEKKO
p= np.array([4, 5, 6.65, 12]) #p = prices
pmx = np.triu(p - p[:, np.newaxis]) #pmx = price matrix, upper triangular
m = GEKKO(remote=False)
q = m.Array(m.Var,(4,4),lb=0,ub=10)
# only upper triangular can change
for i in range(4):
for j in range(4):
if j<=i:
q[i,j].upper=0 # set upper bound = 0
def profit(q):
profit = np.sum(q.flatten() * pmx.flatten())
return profit
for i in range(4):
m.Equation(np.sum(q[i,:])<=10)
m.Equation(np.sum(q[:,i])<=8)
m.Maximize(profit(q))
m.solve()
print(q)
This gives the solution:
[[[0.0] [2.5432017412] [3.7228765674] [3.7339217013]]
[[0.0] [0.0] [4.2771234426] [4.2660783187]]
[[0.0] [0.0] [0.0] [0.0]]
[[0.0] [0.0] [0.0] [0.0]]]
I'm new to OpenMDAO and I want to build an OpenMDAO model consisting of a windfarm (Group) which contains several wind turbines (Component). When all turbines are calculated some further calculations based on the results of all turbines (wind farm) shall be performed. For example: Each wind turbine component calulates its specific power output and in the end the total power shall be calculated as sum of the turbine specific power.
As it is mentioned in the conversion guide following functionality is not supported in new OpenMDAO:
asm.connect('windfarm.windturbine5.power', 'windfarm.eval.power[5]')
Is there any work around to achieve the same/similar result?
Thank you,
Jerome
EDIT:
Following example now works:
from openmdao.api import IndepVarComp, Component, Problem, Group
class Summer(Component):
def __init__(self):
super(Summer, self).__init__()
self.add_param('summand:x0', val=0.0)
self.add_param('summand:x1', val=0.0)
self.add_output('sum', shape=1)
def solve_nonlinear(self, params, unknowns, resids):
x0 = params['summand:x0']
x1 = params['summand:x1']
unknowns['sum'] = x0 + x1
if __name__ == "__main__":
top = Problem()
root = top.root = Group()
root.add('wt0', IndepVarComp('power', 1000.0))
root.add('wt1', IndepVarComp('power', 2000.0))
root.add('eval', Summer())
root.connect('wt0.power', 'eval.summand:x0')
root.connect('wt1.power', 'eval.summand:x1')
top.setup()
top.run()
print(top['eval.sum'])
PS: Is there an easy way to loop over a variable tree (e.g. params['summand'])?
You have to make a separate input for each wind-far output. I suggest the following:
asm.connect('windfarm.windturbine0.power', 'windfarm.eval.power:0')
asm.connect('windfarm.windturbine1.power', 'windfarm.eval.power:1')
I am using Conv2D model of Keras 2.0. However, I cannot fully understand what the function is doing mathematically. I try to understand the math using randomly generated data and a very simple network:
import numpy as np
import keras
from keras.layers import Input, Conv2D
from keras.models import Model
from keras import backend as K
# create the model
inputs = Input(shape=(10,10,1)) # 1 channel, 10x10 image
outputs = Conv2D(32, (3, 3), activation='relu', name='block1_conv1')(inputs)
model = Model(outputs=outputs, inputs=inputs)
# input
x = np.random.random(100).reshape((10,10))
# predicted output for x
y_pred = model.predict(x.reshape((1,10,10,1))) # y_pred.shape = (1,8,8,32)
I tried to calculate, for example, the value of the first row, the first column in the first feature map, following the demo in here.
w = model.layers[1].get_weights()[0] # w.shape = (3,3,1,32)
w0 = w[:,:,0,0]
b = model.layers[1].get_weights()[1] # b.shape = (32,)
b0 = b[0] # b0 = 0
y_pred_000 = np.sum(x[0:3,0:3] * w0) + b0
But relu(y_pred_000) is not equal to y_pred[0][0][0][0].
Could anyone point out what's wrong with my understanding? Thank you.
It's easy and it comes from Theano dim ordering. The result of applying filter in stored in a so called channel dimension. In case of TensorFlow this is the last dimension and that's why results are good. In case of Theano it's second dimension (convolution result has shape (cases, channels, width, height) so in order to solve your problem you need to change prediction line to:
y_pred = model.predict(x.reshape((1,1,10,10)))
Also you need to change the way you get the weights as weights in Theano has shape (output_channels, input_channels, width, height) you need to change the weight getter to:
w = model.layers[1].get_weights()[0] # w.shape = (32,1,3,3)
w0 = w[0,0,:,:]