I have a config group called scheduler which contains many optimizers scheduler, each one has it's own parameters. Is it possible to create a sweep such that I can explore many schedulers and in the same time at each scheduler explore different parameters of that scheduler.
What I have currently is the following:
sweeper:
ax_config:
params:
general.batch_size:
type: choice
values:
- 4
- 8
- 16
- 32
- 64
general.lr:
type: range
bounds: [0.00025, 0.025]
scheduler:
type: choice
values:
- step_lr
- cyclic_lr
- one_cycle_lr
What I want something like this:
values:
- step_lr
gamma:
type: range
bounds: [0.9, 0.99]
- cyclic_lr
- one_cycle_lr
Basically I think you want to use grid search sweeper to sweep over the AX sweeper . This is not supported. You will need to create different sweeps for each parameter in this case.
Related
I am having trouble navigating the source code to see how the design variables in the initial population for the SimpleGA and DifferentialEvolution Drivers are set. Is there some sort of Latin Hypercube sampling of the design variable ranges? Do the initial values I set in my problem instance get used like they would for the other drivers (Scipy and pyOptSparse)?
Many thanks,
Garrett
For these two drivers, the initial value in the model is not used. Its not even clear to me what it would mean to use that value directly, since you need a stochastically generated population --- but I'm admittedly not an expert on the latest GA population initialization methods. However, I can answer the question of how they do get initialized as of OpenMDAO V3.17:
Simple GA Driver:
This driver does seem to use an LHS sampling like this:
new_gen = np.round(lhs(self.lchrom, self.npop, criterion='center',
random_state=random_state))
new_gen[0] = self.encode(x0, vlb, vub, bits)
Differential Evolution Driver:
This driver uses a uniform random distribution like this:
population = rng.random([self.npop, self.lchrom]) * (vub - vlb) + vlb # scale to bounds
Admittedly, it doesn't make a whole lot of sense why the intialization methods are different, and perhaps there should be some option to pick from a set of methods or provide your own initial population somehow. A POEM and/or pull-request to improve this would be most welcome.
I have a unidirectional graph.
The structure is as follows:
There are about 20,000 nodes in the graph.
I make the simplest request: MATCH (b1)-[:NEXT_BAR*10]->(b2) RETURN b1.id, b2.id LIMIT 5
The request is processed quickly.
But if I increase the number of relationships, the query takes much longer to process. In other words, the speed depends on the number of relationships.
This request takes longer than 5 minutes to complete: MATCH (b1)-[:NEXT_BAR*10000]->(b2) RETURN b1.id, b2.id LIMIT 5
This is still a simplified version. The request can have more than two nodes and the number of relationships can still be a range.
How can I optimize a query with a large number of relationships?
Perhaps there are other graph DBMS where there is no such problem?
Variable-length relationship queries have exponential time and memory complexity.
If R is the average number of suitable relationships per node, and D is the depth of the search, then the complexity is O(R ** D). This complexity will exist in any DBMS.
The theory is simple here, but there are a couple of intricacies in the query execution.
-[:NEXT_BAR*10000]- matches a path that is precisely 10000 edges in size, so query engine spends some time to find these paths. Another thing to mention is that in (b1)-[...]- >(b2), b1 and b2 are not specific, which means that the query engine has to scall all nodes. If there is a limit, yea, scall all should return a limited number of items. The whole execution also depends on the efficiency of variable-length path implementation.
Some of the following might help:
Is it feasible to start from a specific node?
If there are branches, the only hope is aggressive filtering because of exponential complexity (as cybersam well explained).
Use a smaller number in the variable expand, or a range, e.g., [NEXT_BAR*..10000]. In this case, the query engine will match any path up to 10000 in size (different semantics, but maybe applicable).
* means the DFS type of execution. On the other hand, BFS might be the right approach. Memgraph (DISCLAIMER: I'm the co-founder and CTO) also supports BFS type of execution with filtering lambda.
Here is a Python script I've used to generate and import data into Memgraph. By using small nodes_no you can quickly notice the execution patterns.
import mgclient
# Make a connection to the database.
connection = mgclient.connect(
host='127.0.0.1',
port=7687,
sslmode=mgclient.MG_SSLMODE_REQUIRE)
connection.autocommit = True
cursor = connection.cursor()
# Clean and setup database instance.
cursor.execute("""MATCH (n) DETACH DELETE n;""")
cursor.execute("""CREATE INDEX ON :Node(id);""")
# Import dataset.
nodes_no = 10
# Create nodes.
for identifier in range(0, nodes_no):
cursor.execute("""CREATE (:Node {id: "%s"});""" % identifier)
# Create edges.
for identifier in range(1, nodes_no):
cursor.execute("""
MATCH (start_node:Node {id: "%s"})
MATCH (end_node:Node {id: "%s"})
CREATE (start_node)-[:NEXT_BAR]->(end_node);
""" % (identifier - 1, identifier))
Sometimes a device has commands to be sent to another device. In simple cases, I often use strings of digits and letters like
"123X456YG"
with the meaning
set the value of parameter X to 123, 456 to Y, and then Go.
The interpretation algorithm is quite simple : process char by char, "push" the digit to build a number (n = 10*n + ch-'0'), execute action (and reset number) when it is a letter.
It very convenient when there are not too many different actions (and you can remember the letters), of course you may use SCPI for more complicated things.
Is there a name for this (rather obvious) way to to?
This is an example of postfix notation. It's a notation that fairly naturally falls out of anything that uses a stack.
Examples include:
Reverse Polish Notation -- common on programmable calculators from the late 1960s onwards.
... in which 4 3 - 5 + results in 6 being left at the head of the stack, because the stack builds up like this:
Stack Unconsumed input
1 [ ] 4 3 - 5 +
2 [ 4 ] 3 - 5 +
3 [ 4 3 ] - 5 +
4 [ 1 ] 5 +
5 [ 1 5 ] +
6 [ 6 ]
At steps 4 and 6, the operator is executed by popping elements off the stack, then pushing the result onto the stack.
Many stack-based programming languages -- including Forth and Postscript. Postfix syntax is also fairly common in the world of esoteric programming languages.
If you're curious, I thoroughly recommend having a go at hand-writing Postscript. It's a fully-fledged programming language; for example, you can write a recursive Postscript program in a few bytes, to draw a fractal curve. Test it in Ghostscript. PDF is closely related to Postscript.
The language you have created differs from these in your approach to tokenisation, but at its core -- putting the operand(s) before the operator -- it's a postfix language.
You are unlikely to get a patent granted. :)
by use of FFT in R I want to determine all frequency components (amplitude, frequency and phase) of any signal. My FFT routine has worked proper on my self-provided testfiles except on my last testfile. Here was an offset on one phase – the other phases were calculated correctly.
By trial and error I detected that there is a constant offset about 1.255.
In R I implemented the FFT as described in the following:
At steady state I take 5 seconds of the signal
Of this segment of signal – below only signal – the mean must be subtracted (if necessary)
Now the signal will be windowed (Hanning) and some zeros were padded (zero padding)
Next you apply the FFT to the signal
A matrix is created with two columns – column one is for the bins and column two for the magnitude of the FFT
The rear half of the matrix is cut off
Now maxima and minima of the magnitude were identified and saved with related bin in a new matrix (column one bins, column two magnitude)
And again the new matrix will be shortend. Any amplitude under 0.45 will be cut off
Finally the calculation of amplitude, frequency and phase will take place
amplitude
magnitude/(length(signal)/4)
length(signal) depends on the sampling rate
frequency
bin*sampling rate/length(signalw)
length(signalw) depends on the sampling rate and the zeros padded
phase - case analysis because the magnitude is a complex number fft(signalw)=x+iy
x>0,y>0 - - - - atan(y/x) - - - - - - - Case 1
x>0,y<0 - - - - atan(y/x)+2*pi - - - Case 2
x>0 - - - - - - - atan(y/x)+pi - - - - - Case 3
x==0,y>0 - - - 0.5*pi - - - - - - - - - - Case 4
x==0,y<0 - - - 1.5*pi - - - - - - - - - - Case 5
bin-number, amplitude, frequency and phase will be saved in a matrix
this is my result matrix
To examine this phenomenon of the constant phaseshift of 1.255 I created a testfile (with a sampling rate of 1000 Hz, whereby… whatever…).
Frequency: - - seq(from=0.3,to=64.3,by=2)
Amplitude: - - -rep(x=c(0.5,1.1,1.134,0.75),length.out=33)
Phase: - - - - - seq(from=0,to=8,by=0.25)
With signal=ampl[i]*cos(2*pi*freq[i]*t+phi[i]) you get a signal with 33 cosine waves.
The next picture is showing the table of setpoints.
table of setpoints
When I do the FFT routine as described above I get following results:
my result matrix
Now let’s have a quick comparison with the table of setpoints:
The red marked values have a constant offset of 1.255
The yellow marked values are reduced by 2*pi (periodicity)
The green marked value isn’t calculated correctly – see question…
The non-marked values were calculated right
Which case is used for calculating the phase
Case 2 (frequency 38.3 and 40.3) - red marked
Case 1 (frequency 42.3 ... 50.3) - red marked
Case 1 (frequency 52.3) - yellow marked
Case 3 (frequency 54.3 ... 64.3) - yellow marked
Just for some further examination I putted all these frequencies in an underdamped step.
Again a comparison with the table of setpoints:
all phases have the constant offset of 1.255
Notes:
The constant offset is occurring in every case of the phase case analysis
In this forum I found the hint, to use atan2(y,x) instead of atan(x)
But this hint doesn’t solve my problem – hopefully I implemented it right…
Watch the frequency of 42.3 Hz, for example
In the first result you need to add the offset of 1.255
3.9926288+1.255=5.2476288
In the second result it is probably necessary to consider the periodicity
0.2223801+2*pi-1.255=5.250565
Questions regarding the constant offset of 1.255:
How emerge the constant offset?
As you can see above some values are calculated correctly. The formulas I am using are always the same!
The constant offset of 1.255 I got through trial and error.
Is there an efficient way to get the exact value?
How should I complete my phase calculation to get always the right phase?
Additional question:
How to deal with the green marked value? The phase of the frequency is hard to calculate…
Will it be sufficient to raise the zero padded zeros and the sampling rate?
When you need more information please let me know – I will edit my post…
Furthermore I'm very sorry that I'm not able to include more pictures... I have not enough reputation...
Many thanks in advance!
I have a nodeset that contains various number of nodes with unique values. I
want the rule to fire if the nodeset contains nodes with some of the possible
values.
Example:
Instance containing a nodeset with one node with Y=1 AND another node with
Y=2 should fire rule.
Instance that should fire:
- X
- - Y - 1
- - Y - 2
- - Y - 3
- - Y - 4
Instance that should NOT fire:
- X
- - Y - 1
- - Y - 3
- - Y - 4
I tried with AND, OR and others, but the "problem" is that since the engine
does pattern-matching it tests every node for the condition and it will
either always fire, or never.
It's not that complex problem. How hard can it be :) When I found "Set of values" I thought I was home safe, but that was more of "enums".Grateful for any suggestion.
Regards
Martin Bring
Try something along the lines you'll find here
It creates an exclusive-OR (When this or that but not this and that) functionality. Not saying it'll fix it for you, but it may help put you on the right track.