After ver. 3.13 src_indices is changed, I could not find it in docs. Could you share a sample code for connecting a smaller portion of a large matrix?
i.e. source is 5x5 matrix and target is 2x2 starting from [2,3]
I find it easiest to think about connections as slices. We added om.slicer to handle indices via slicing notation. The following example connects a 2x2 portion of a 5x5 matrix, starting from indices [2, 3]:
import openmdao.api as om
import numpy as np
print('Connecting a 2x2 submatrix of M to B')
p = om.Problem()
ivc = p.model.add_subsystem('ivc', om.IndepVarComp())
ivc.add_output('M', val=np.arange(25).reshape((5, 5)))
exec = p.model.add_subsystem('exec', om.ExecComp())
exec.add_expr('A = B', A={'shape': (2, 2)}, B={'shape': (2, 2)})
p.model.connect('ivc.M', 'exec.B', src_indices=om.slicer[2:4, 3:5])
p.setup()
p.run_model()
print('M')
print(p.get_val('ivc.M'))
print('A')
print(p.get_val('exec.A'))
Connecting a 2x2 submatrix of M to B
M
[[ 0. 1. 2. 3. 4.]
[ 5. 6. 7. 8. 9.]
[10. 11. 12. 13. 14.]
[15. 16. 17. 18. 19.]
[20. 21. 22. 23. 24.]]
A
[[13. 14.]
[18. 19.]]
Or, for instance, connecting only rows [0, 2, 3] of the 5x5 matrix to the target:
rint('Connecting rows 0, 2, and 3 of M to B')
p = om.Problem()
ivc = p.model.add_subsystem('ivc', om.IndepVarComp())
ivc.add_output('M', val=np.arange(25).reshape((5, 5)))
exec = p.model.add_subsystem('exec', om.ExecComp())
exec.add_expr('A = B', A={'shape': (3, 5)}, B={'shape': (3, 5)})
p.model.connect('ivc.M', 'exec.B', src_indices=om.slicer[[0, 2 ,3], :])
p.setup()
p.run_model()
print('M')
print(p.get_val('ivc.M'))
print('A')
print(p.get_val('exec.A'))
Connecting rows 0, 2, and 3 of M to B
M
[[ 0. 1. 2. 3. 4.]
[ 5. 6. 7. 8. 9.]
[10. 11. 12. 13. 14.]
[15. 16. 17. 18. 19.]
[20. 21. 22. 23. 24.]]
A
[[ 0. 1. 2. 3. 4.]
[10. 11. 12. 13. 14.]
[15. 16. 17. 18. 19.]]
While we discuss this feature in some of the parallelization docs, you're correct that we need to discuss it in our connection documentation.
Related
I am trying to set up a problem in OpenMDAO and would like to make use of parallel finite difference computations. However, when I call compute_totals() each MPI process actually computes all the perturbed points.
I have made a minimal example that demonstrates the problem. Consider the simple case of a model which can be represented by a matrix multiplication. The Jacobian of this model is simply the matrix of the model. See the code below:
import numpy as np
import time
from openmdao.api import ExplicitComponent, Problem, IndepVarComp, Group
from openmdao.utils.mpi import MPI
rank = 0 if not MPI else MPI.COMM_WORLD.rank
class MatMultComp(ExplicitComponent):
def __init__(self, matrix, **kwargs):
super().__init__(**kwargs)
self.matrix = matrix
def setup(self):
self.add_input('x', val=np.ones(self.matrix.shape[1])))
self.add_output('y', val=np.ones(self.matrix.shape[0])))
def compute(self, inputs, outputs, **kwargs):
outputs['y'] = self.matrix.dot(inputs['x'])
print('{} :: x = {}'.format(rank, np.array_str(inputs['x'])))
class Model(Group):
def setup(self):
matrix = np.arange(25, dtype=float).reshape(5, 5)
self.add_subsystem('ivc', IndepVarComp('x', np.ones(matrix.shape[1])), promotes=['*'])
self.add_subsystem('mat', MatMultComp(matrix), promotes=['*'])
self.approx_totals(step=0.1)
self.num_par_fd = matrix.shape[1]
if __name__ == '__main__':
p = Problem()
p.model = Model()
p.setup()
p.run_model()
t0 = time.time()
jac = p.compute_totals(of=['y'], wrt=['x'], return_format='array')
dt = time.time() - t0
if rank == 0:
print('Took {:2.3f} seconds.'.format(dt))
print('J = ')
print(np.array_str(jac, precision=0))
When I run this code without MPI, I get the following output:
0 :: x = [1. 1. 1. 1. 1.]
0 :: x = [1.1 1. 1. 1. 1. ]
0 :: x = [1. 1.1 1. 1. 1. ]
0 :: x = [1. 1. 1.1 1. 1. ]
0 :: x = [1. 1. 1. 1.1 1. ]
0 :: x = [1. 1. 1. 1. 1.1]
Took 5.008 seconds.
J =
[[ 0. 1. 2. 3. 4.]
[ 5. 6. 7. 8. 9.]
[10. 11. 12. 13. 14.]
[15. 16. 17. 18. 19.]
[20. 21. 22. 23. 24.]]
This is the correct result, and takes about 5 seconds, as expected. Now, when I run this with MPI, using 5 processes, with the command mpirun -np 5 python matmult.py, I get the following output:
0 :: x = [1. 1. 1. 1. 1.]
1 :: x = [1. 1. 1. 1. 1.]
2 :: x = [1. 1. 1. 1. 1.]
3 :: x = [1. 1. 1. 1. 1.]
4 :: x = [1. 1. 1. 1. 1.]
0 :: x = [1.001 1. 1. 1. 1. ]
1 :: x = [1.001 1. 1. 1. 1. ]
2 :: x = [1.001 1. 1. 1. 1. ]
3 :: x = [1.001 1. 1. 1. 1. ]
4 :: x = [1.001 1. 1. 1. 1. ]
3 :: x = [1. 1.001 1. 1. 1. ]
0 :: x = [1. 1.001 1. 1. 1. ]
1 :: x = [1. 1.001 1. 1. 1. ]
2 :: x = [1. 1.001 1. 1. 1. ]
4 :: x = [1. 1.001 1. 1. 1. ]
2 :: x = [1. 1. 1.001 1. 1. ]
3 :: x = [1. 1. 1.001 1. 1. ]
0 :: x = [1. 1. 1.001 1. 1. ]
1 :: x = [1. 1. 1.001 1. 1. ]
4 :: x = [1. 1. 1.001 1. 1. ]
1 :: x = [1. 1. 1. 1.001 1. ]
2 :: x = [1. 1. 1. 1.001 1. ]
3 :: x = [1. 1. 1. 1.001 1. ]
0 :: x = [1. 1. 1. 1.001 1. ]
4 :: x = [1. 1. 1. 1.001 1. ]
0 :: x = [1. 1. 1. 1. 1.001]
1 :: x = [1. 1. 1. 1. 1.001]
2 :: x = [1. 1. 1. 1. 1.001]
3 :: x = [1. 1. 1. 1. 1.001]
4 :: x = [1. 1. 1. 1. 1.001]
Took 5.072 seconds.
J =
[[ 0. 1. 2. 3. 4.]
[ 5. 6. 7. 8. 9.]
[10. 11. 12. 13. 14.]
[15. 16. 17. 18. 19.]
[20. 21. 22. 23. 24.]]
The final result is correct, of course. However, this defies the purpose of using MPI, because each of the 5 processes computed all the perturbed points, and the total execution takes about 5 seconds like before. I expected the following output:
0 :: x = [1. 1. 1. 1. 1.]
1 :: x = [1. 1. 1. 1. 1.]
2 :: x = [1. 1. 1. 1. 1.]
3 :: x = [1. 1. 1. 1. 1.]
4 :: x = [1. 1. 1. 1. 1.]
0 :: x = [1.1 1. 1. 1. 1. ]
1 :: x = [1. 1.1 1. 1. 1. ]
2 :: x = [1. 1. 1.1 1. 1. ]
3 :: x = [1. 1. 1. 1.1 1. ]
4 :: x = [1. 1. 1. 1. 1.1]
Took 1.000 seconds.
J =
[[ 0. 1. 2. 3. 4.]
[ 5. 6. 7. 8. 9.]
[10. 11. 12. 13. 14.]
[15. 16. 17. 18. 19.]
[20. 21. 22. 23. 24.]]
Note that in reality the order in which the processes finish is arbitrary, and the time it took will be a little more than 1 second.
How can I get this to work as expected? Note that I am using OpenMDAO 2.5.0.
There are a few of issues here. The first is that num_par_fd should typically be passed as an __init__ arg to your Group or your Component. Setting it in the Component or Group's setup() function is too late, because OpenMDAO does all of its MPI communicator splitting in the _setup_procs function, which happens before the Component/Group setup call. The same timing issue applies to calling the approx_totals function. It must be called prior to the Problem setup call. Finally, the name of the attribute we use internally to specify the number of parallel FD computations is actually self._num_par_fd and not self.num_par_fd. Setting of the internal _num_par_fd attribute isn't recommended, but if you must, you'll have to set it before Problem setup is called.
Note: this is a heavily edited version of my original answer.
Using Prolog:
Write a predicate dispnth to display the nth element of a list. You may assume that the input list always has n or more elements.
For Example:
?- dispnth([1, [2, 3], 4, 5], 2, X). should return X = [2, 3]
I have this so far:
dispnth([X|_], 0, X).
dispnth([_|Xs], N, X) :-
dispnth(N1, X, Xs),
N is N1 + 1.
First let's give the predicate a more descriptive name, say list_nth_element/3. Next you might like to consider an auxiliary predicate list_nth_element_/4 with an additional argument, that holds the current position. From your given example I assume that you start counting at 1, so that's going to be the start value for the fourth argument. Then the predicates might look something like this:
list_nth_element(L,N,E) :-
list_nth_element_(L,N,E,1).
list_nth_element_([X|Xs],N,X,N). % if the 2nd and 4th elements are equal X is the nth element
list_nth_element_([_X|Xs],N,E,P0) :- % if the 2nd and 4th arguments
dif(P0,N), % differ
P1 is P0+1, % increment current position
list_nth_element_(Xs,N,E,P1). % and recurse
So essentially the fourth argument is used as a position indicator that is being incremented until you reached the desired position. However, there is no need to have this additional argument in the actual predicates interface, so it is "hidden" in the auxiliary predicate's interface.
Querying this predicate yields your desired result:
?- list_nth_element([1, [2, 3], 4, 5], 2, X).
X = [2,3] ? ;
no
You can also ask things like Which element is at what position?
?- list_nth_element([1, [2, 3], 4, 5], N, X).
N = X = 1 ? ;
N = 2,
X = [2,3] ? ;
N = 3,
X = 4 ? ;
N = 4,
X = 5 ? ;
no
In pandas, there is the clip function (see https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.clip.html), which constrains values within the lower and upper bound provided by the user. What is the Julia equivalent? I.e., I would like to have:
> clip.([2 3 5 10],3,5)
> [3 3 5 5]
Obviously, I can write it myself, or use a combination of min and max, but I was surprised to find out there is none. StatsBase provides the trim and winsor functions, but these do not allow fixed values as input, but rather counts or percentiles (https://juliastats.github.io/StatsBase.jl/stable/robust.html).
You are probably looking for clamp:
help?> clamp
clamp(x, lo, hi)
Return x if lo <= x <= hi. If x > hi, return hi. If x < lo, return lo. Arguments are promoted to a common type.
This is a function for scalar x, but we can broadcast it over the vector using dot-notation:
julia> clamp.([2, 3, 5, 10], 3, 5)
4-element Array{Int64,1}:
3
3
5
5
If you don't care about the original array you can also use the in-place version clamp!, which modifies the input:
julia> A = [2, 3, 5, 10];
julia> clamp!(A, 3, 5);
julia> A
4-element Array{Int64,1}:
3
3
5
5
How can I include an outer product (of the previous feature vector and itself) as a layer in chainer, especially in a way that's compatible with batching?
F.matmul is also very handy.
Depending on the input shapes, you can combine it with F.expand_dims (of course F.reshape works, too) or use transa/transb arguments.
For details, refer to the official documentation of functions.
Code
import chainer.functions as F
import numpy as np
print("---")
x = np.array([[[1], [2], [3]], [[4], [5], [6]]], 'f')
y = np.array([[[1, 2, 3]], [[4, 5, 6]]], 'f')
print(x.shape)
print(y.shape)
z = F.matmul(x, y)
print(z)
print("---")
x = np.array([[[1], [2], [3]], [[4], [5], [6]]], 'f')
y = np.array([[[1], [2], [3]], [[4], [5], [6]]], 'f')
print(x.shape)
print(y.shape)
z = F.matmul(x, y, transb=True)
print(z)
print("---")
x = np.array([[1, 2, 3], [4, 5, 6]], 'f')
y = np.array([[1, 2, 3], [4, 5, 6]], 'f')
print(x.shape)
print(y.shape)
z = F.matmul(
F.expand_dims(x, -1),
F.expand_dims(y, -1),
transb=True)
print(z)
Output
---
(2, 3, 1)
(2, 1, 3)
variable([[[ 1. 2. 3.]
[ 2. 4. 6.]
[ 3. 6. 9.]]
[[ 16. 20. 24.]
[ 20. 25. 30.]
[ 24. 30. 36.]]])
---
(2, 3, 1)
(2, 3, 1)
variable([[[ 1. 2. 3.]
[ 2. 4. 6.]
[ 3. 6. 9.]]
[[ 16. 20. 24.]
[ 20. 25. 30.]
[ 24. 30. 36.]]])
---
(2, 3)
(2, 3)
variable([[[ 1. 2. 3.]
[ 2. 4. 6.]
[ 3. 6. 9.]]
[[ 16. 20. 24.]
[ 20. 25. 30.]
[ 24. 30. 36.]]])
You can use F.reshape and F.broadcast_to to explicitly handle array.
Assume you have 2-dim array h with shape (minibatch, feature).
If you want to calculate outer product of h and h, try below code.
Is this what you want to do?
import numpy as np
from chainer import functions as F
def outer_product(h):
s0, s1 = h.shape
h1 = F.reshape(h, (s0, s1, 1))
h1 = F.broadcast_to(h1, (s0, s1, s1))
h2 = F.reshape(h, (s0, 1, s1))
h2 = F.broadcast_to(h2, (s0, s1, s1))
h_outer = h1 * h2
return h_outer
# test code
h = np.arange(12).reshape(3, 4).astype(np.float32)
h_outer = outer_product(h)
print(h.shape)
print(h_outer.shape, h_outer.data)
Right now I have a code that can find the number of combinations of a sum of a value using numbers greater than zero and less than the value.
I need to alter the value in order to expand the combinations so that they include more than just the value.
For example:
The number 10 yields the results:
[1, 2, 3, 4], [1, 2, 7],
[1, 3, 6], [1, 4, 5],
[1, 9], [2, 3, 5], [2, 8],
[3, 7], [4, 6]
But I need to expand this to including any number that collapses to 1 as well. Because in essence, I need 100 = n in that the sum of the individual numbers within the digits = n. So in this case 100 = 1 because 100 --> 1+0+0 = 1
Therefore the number 1999 will also be a valid combination to list for value = 100 because 1999 = 1+9+9+9 = 28, and 28 = 2+8 = 10, and 10 = 1+0 = 1
Now I realize that this will yield an infinite series of combinations, so I will need to set limits to the range I want to acquire data for. This is the current code I am using to find my combinations.
def a(lst, target, with_replacement=False):
def _a(idx, l, r, t, w):
if t == sum(l): r.append(l)
elif t < sum(l): return
for u in range(idx, len(lst)):
_a(u if w else (u + 1), l + [lst[u]], r, t, w)
return r
return _a(0, [], [], target, with_replacement)
for val in range(100,101):
s = range(1, val)
solutions = a(s, val)
print(solutions)
print('Value:', val, "Combinations", len(solutions))
You seem to have multiple issues.
To repeatedly add the decimal digits of an integer until you end with a single digit, you could use this code.
d = val
while d > 9:
d = sum(int(c) for c in str(d))
This acts in just the way you describe. However, there is an easier way. Repeatedly adding the decimal digits of a number is called casting out nines and results in the digital root of the number. This almost equals the remainder of the number when divided by nine, except that you want to get a result of 9 rather than 1. So easier and faster code is
d = val % 9
if d == 0:
d == 9
or perhaps the shorter but trickier
d = (val - 1) % 9 + 1
or the even-more-tricky
d = val % 9 or 9
To find all numbers that end up at 7 (for example, or any digit from 1 to 9) you just want all numbers with the remainder 7 when divided by 9. So start at 7 and keep adding 9 and you get all such values.
The approach you are using to find all partitions of 7 then arranging them into numbers is much more complicated and slower than necessary.
To find all numbers that end up at 16 (for example, or any integer greater than 9) your current approach may be best. It is difficult otherwise to avoid the numbers that directly add to 7 or to 25 without going through 16. If this is really what you mean, say so in your question and we can look at this situation further.