I would like to check whether 2 vectors are the same in APL. Right now I am using the following solution (comparing element by element, summing the elements and comparing with size of vector a):
a←1 2 3
b←1 2 3
(+/a=b)=⍴a ⍝ it needs to return 0 or 1
Is there any quicker or more idiomatic solution?
You can use the match function which compares its entire arguments rather than equals which is a scalar function that compares the elements of each argument:
a←1 2 3
b←1 2 3 4 5
c←1 2 3
a≡b
0
a≡c
1
The match primitive, as mentioned above, returns 1 if the arguments are exactly identical. This means that they have the exact same rank, shape, data type, and content. In a few cases match will return a false negative because of data-type issues (division resulting in a floating point representation, even though it is within the comparison tolerance of an integer), or because a scaler will not match a 1-element vector.
^/a=b
will return a 1 if all elements of a test equal to corresponding elements of b, but it will fail with a LENGTH error if a and b are of different lengths, and it will use scaler extension, so that if a is 1 1 1 and b is a scaler 1, the result will be 1.
Match is usually better for this, and it is also more efficient on large arrays.
Related
Per DICOM specification, a UID is defined by: 9.1 UID Encoding Rules. In other words the following are valid DICOM UIDs:
"1.2.3.4.5"
"1.3.6.1.4.35045.103501438824148998807202626810206788999"
"1.2.826.0.1.3680043.2.1143.5028470438645158236649541857909059554"
while the following are illegal DICOM UIDs:
".1.2.3.4.5"
"1..2.3.4.5"
"1.2.3.4.5."
"1.2.3.4.05"
"12345"
"1.2.826.0.1.3680043.2.1143.50284704386451582366495418579090595540"
Therefore I know that the string is at most 64 bytes, and should match the following regex [0-9\.]+. However this regex is really a superset, since there are a lot less than (10+1)^64 (=4457915684525902395869512133369841539490161434991526715513934826241L) possibilities.
How would one computes precisely the number of possibilities to respect the DICOM UID rules ?
Reading the org root / suffix rule clearly indicates that I need at least one dot ('.'). In which case the combination is at least 3 bytes (char) in the form: [0-9].[0-9]. In which case there are 10x10=100 possibilities for UID of length 3.
Looking at the first answer, there seems to be something unclear about:
The first digit of each component shall not be zero unless the
component is a single digit.
What this means is that:
"0.0" is valid
"00.0" or "1.01" are not valid
Thus I would say a proper expression would be:
(([1-9][0-9]*)|0)(\.([1-9][0-9]*|0))+
Using a simple C code, I could find:
f(0) = 0
f(1) = 0
f(2) = 0
f(3) = 100
f(4) = 1800
f(5) = 27100
f(6) = 369000
f(7) = 4753000
f(8) = 59049000
The validation of the Root UID part is outside the scope of this question. A second validation step could take care of rejecting some OID that cannot possibly be registered (some people mention restriction on first and second arc for example). For simplicity we'll accept all possible (valid) Root UID.
While my other answer takes good care of this specific application, here is a more generic approach. It takes care of situations where you have a different regular expression describing the language in question. It also allows for considerably longer string lengths, since it only requires O(log n) arithmetic operations to compute the number of combinations for strings of length up to n. In this case the number of strings grows so quickly that the cost of these arithmetic operations will grow dramatically, but that may not be the case for other, otherwise similar situations.
Build a finite state automaton
Start with a regular expression description of your language in question. Translate that regular expression into a finite state automaton. In your case the regular expression can be given as
(([1-9][0-9]*)|0)(\.([1-9][0-9]*|0))+
The automaton could look like this:
Eliminate ε-transitions
This automaton usually contains ε-transitions (i.e. state transitions which do not correspond to any input character). Remove those, so that one transition corresponds to one character of input. Then add an ε-transition to the accepting state(s). If the accepting states have other outgoing transitions, don't add ε-loops to them, but instead add an ε-transition to an accepting state with no outgoing edges and then add the loop to that. This can be seen as padding the input with ε at its end, without allowing ε in the middle. Taken together, this transformation ensures that performing exactly n state transitions corresponds to processing an input of n characters or less. The modified automaton might look like this:
Note that both the construction of the first automaton from the regular expression and the elimination of ε-transitions can be performed automatically (and perhaps even in a single step. The resulting automata might be more complicated than what I constructed here manually, but the principle is the same.
Ensuring unique paths
You don't have to make the automaton deterministic in the sense that for every combination of source state and input character there is only one target state. That's not the case in my manually constructed one either. But you have to make sure that every complete input has only one possible path to the accepting state, since you'll essentially be counting paths. Making the automaton deterministic would ensure this weaker property, too, so go for that unless you can ensure unique paths without this. In my example the length of each component clearly dictates which path to use, so I didn't make it deterministic. But I've included an example with a deterministic approach at the end of this post.
Build transition matrix
Next, write down the transition matrix. Associate the rows and columns with your states (in order a, b, c, d, e, f in my example). For each arrow in your automaton, write the number of characters included in the label of that arrow in the column associated with the source state and the row associated with the target state of that arrow.
⎛ 0 0 0 0 0 0⎞
⎜ 9 10 0 0 0 0⎟
⎜10 10 0 10 10 0⎟
⎜ 0 0 1 0 0 0⎟
⎜ 0 0 0 9 10 0⎟
⎝ 0 0 0 10 10 1⎠
Read result off that matrix
Now applying this matrix with a column vector once has the following meaning: if the number of possible ways to arrive in a given state is encoded in the input vector, the output vector gives you the number of ways one transition later. Take the 64th power of that matrix, concentrate on the first column (since ste start situation is encoded as (1,0,0,0,0,0), meaning only one way to end up in the start state) and sum up all the entries that correspond to accepting states (only the last one in this case). The bottom left element of the 64th power of this matrix is
1474472506836676237371358967075549167865631190000000000000000000000
which confirms my other answer.
Compute matrix powers efficiently
In order to actually compute the 64th power of that matrix, the easiest approach would be repeated squaring: after squaring the matrix 6 times you have an exponent of 26 = 64. If in some other scenario your exponent (i.e. maximal string length) is not a power of two, you can still perform exponentiation by squaring by multiplying the relevant squares according to the bit pattern of the exponent. This is what makes this approach take O(log n) arithmetic operations to compute the result for string length n, assuming a fixed number of states and therefore fixed cost for each matrix squaring.
Example with deterministic automaton
If you were to make my automaton deterministic using the usual powerset construction, you'd end up with
and sorting the states as a, bc, c, d, cf, cef, f one would get the transition matrix
⎛ 0 0 0 0 0 0 0⎞
⎜ 9 10 0 0 0 0 0⎟
⎜ 1 0 0 0 0 0 0⎟
⎜ 0 1 1 0 1 1 0⎟
⎜ 0 0 0 1 0 0 0⎟
⎜ 0 0 0 9 0 10 0⎟
⎝ 0 0 0 0 1 1 1⎠
and could sum the last three elements of the first column of its 64th power to obtain the same result as above.
Single component
Start by looking for ways to form a single component. The corresponding regular expression for a single component is
0|[1-9][0-9]*
so it is either zero or a non-zero digit followed by arbitrary many zero digits. (I had missed the possible sole zero case at first, but the comment by malat made me aware of this.) If the total length of such a component is to be n, and you write h(n) to denote the number of ways to form such a component of length exactly n, then you can compute that h(n) as
h(n) = if n = 1 then 10 else 9 * 10^(n - 1)
where the n = 1 case allows for all possible digits, and the other cases ensure a non-zero first digit.
One or more components
Subsection 9.1 only writes that a UID is a bunch of dot-separated number components, as outlined above. So in regular expressions that would be
(0|[1-9][0-9]*)(\.(0|[1-9][0-9]*))*
Suppose f(n) is the number of ways to write a UID of length n. Then you have
f(n) = h(n) + sum h(i) * f(n-i-1) for i from 1 to n-2
The first term describes the case of a single component, while the sum takes care of the case where it consists of more than one component. In that case you have a first component of length i, then a dot which accounts for the -1 in the formula, and then the remaining digits form one or more components which is expressed via the recursive use of f.
Two or more components
As the comment by cneller indicates, the part of section 9 before subsection 9.1 indicates that there has to be at least two components. So the proper regular expression would be more like
(0|[1-9][0-9]*)(\.(0|[1-9][0-9]*))+
with a + at the end indicating that we want at least one repetition of the parenthesized expression. Deriving an expression for this simply means leaving out the one-component-only case in the definition of f:
g(n) = sum h(i) * f(n-i-1) for i from 1 to n-2
If you sum all the g(n) for n from 3 (the minimal possible UID length) through 64 you get the number of possible UIDs as
1474472506836676237371358967075549167865631190000000000000000000000
or approximately 1.5e66. Which is considerably less than the 4.5e66 you get from your computation, in terms of absolute difference, although it's definitely on the same order of magnitude. By the way, your estimate doesn't explicitely mention UIDs shorter than 64, but you can always consider padding them with dots in your setup. I did the computation using a few lines of Python code:
f = [0]
g = [0]
h = [0, 10] + [9 * (10**(n-1)) for n in range(2, 65)]
s = 0
for n in range(1, 65):
x = 0
if n >= 3:
for i in range(1, n - 1):
x += h[i] * f[n-i-1]
g.append(x)
f.append(x + h[n])
s += x
print(h)
print(f)
print(g)
print(s)
what is the Python notation a[i-j] translated to R? As far as I understand it, it should be the array element at position i-j. But in R it seems to be the array until the ith element subtracted by the element at position j.
R and Python have somewhat similar indexing properties, with the main difference being that indexing in Python starts at 0 while in R it starts at 1. Beyond the index start, there is also the fact that Python supports negative indexing, while in R negative indexing means that you are removing the element at that exact index from your list. To be specific to your case, the indexing list[i-j] could be somewhat the same thing if i - j returns a positive integer. Otherwise, you are talking about two completely different things. The illustration below should be helpful to you:
Python:
#Create a list
lst = [1,3,5,6,7,7]
#index element at 4-2 (which is 2)
lst[4-2] # returns 5
#index element at 2-4 (which is -2) or lst[len(lst)-2]
lst[2-4] # returns 7
R:
lst <- c(1,3,5,6,7,7)
#indexing element at 4-2 (which is 2)
lst[4-2] # returns 3 (because R indexing starts at 1, not 0)
[1] 3
#BUT indexing element at 2-4 (which is -2) does not work,
#because it means that you are removing the element at index 2, i.e. 3
lst[2-4] #returns the original list without element at index 2
[1] 1 5 6 7 7
These are the main differences in indexing a list that I could offer to help with your question. The differences in indexing become more prominent as you tackle more complicated data structures in both languages.
I hope this is helpful.
I'd like to understand better how outer works and how to vectorize functions. Below is a minimal example for what I am trying to do:I have a set of numbers 2,3,4. for each combination (a,b) of create a diagonal matrix with a a b b b on the diagonal, and then do something with it, e.g. calculating its determinant (this is just for demonstration purposes). The results of the calculation should be written in a 3 by 3 matrix, one field for each combination.
The code below isn't working - apparently, outer (or my.func) doesn't understand that I don't want the whole lambdas vector to be applied - you can see that this is the case when you uncomment the print command included.
lambdas <- c(1:2)
my.func <- function(lambda1,lambda2){
# print(diag(c(rep(lambda1,2),rep(lambda2,2))))
det(diag(c(rep(lambda1,2),rep(lambda2,2))))
}
det.summary <- outer(lambdas,lambdas, FUN="my.func")
How do I need to modify my function or the call of outer so things behave like I'd like to?
I guess I need to vectorize my function somehow, but I don't know how, and in which way the outer call would be processed differently.
Edit:
I've changed size of the matrices to make it a bit less messy. I'd like to generate 4 diagonal 4 by 4 matrices, with the following diagonals; in are brackets the corresponding parameters lambda1, lambda2:
1 1 1 1 (1,1), 1 1 2 2 (1,2), 2 2 1 1 (2,1), 2 2 2 2 (2,2).
Then, I want to calculate their determinants (which is an arbitrary choice here) and put the results into a matrix, whose first column corresponds to lambda1=1, the second to lambda1=2, and the rows correspond to the choice of lambda2. det.summary should be a 2 by to matrix with the following values:
1 4
4 16
as these are the determinants of the diagonal matrices listed above.
What do you know, there is a Vectorize function (capital "V")!
outer(lambdas,lambdas, Vectorize(my.func))
# [,1] [,2]
# [1,] 1 4
# [2,] 4 16
As you figured out (and as it took me a while to figure out) outer requires the function to be vectorized. In some ways, it is the opposite of the *pply functions which effectively vectorize an operation by feeding the operator/function each value in turn. But this is easily dealt with, as shown above.
When I compute the difference between the largest and the smallest number in an empty vector(v←⍳0) using ⌈⌿(⌈/c)- ⌊⌿(⌊/c) , it gives me a domain error. This statement works fine with normal vectors and matrices.
How do I handle the exception such that it does not give me an error when the vector is empty? It should not return anything or just return a zero.
A guard is the best way to do this:
{0=⍴⍵:0 ⋄ (⌈/⍵)-⌊/⍵}
Note that the use of two reductions, one with axis specfication, is not really needed or correct actually. That is, if you want it to work on all of the elements of a simple array of any dimension, simply ravel the argument first:
{0=⍴⍵:0 ⋄ (⌈/⍵)-⌊/⍵},10 10 ⍴⍳100
99
Or for an array of any structure or depth, you can use "super ravel":
{0=⍴⍵:0 ⋄ (⌈/⍵)-⌊/⍵}∊(1 2 3)(7 8 9 10)
9
Note that quadML (Migration Level) must be set to 3 to ensure that epsilon is "super ravel."
Note also the equivalence of the following when operating on a matrix:
⌈⌿⌈/10 10 ⍴⍳100
99
⌈/⌈/10 10 ⍴⍳100
99
⌈/⌈⌿10 10 ⍴⍳100
99
⌈⌿⌈⌿10 10 ⍴⍳100
99
Using reduction with axis is not needed in this case, and obscures the intent and is also potentially more expensive. Better to just ravel the whole thing.
As I mentioned in the comments, Dyalog APL has guards, which can be used for conditional execution, and thus you can simply check for the empty vector and give a different answer.
This can be implemented in a more traditional/pure APL method however.
This version only works in 1-dimension
In the APL font:
Z←DIFFERENCE V
⍝ Calculate difference between vectors, with empty set protection
⍝ Difference is calculated by a reduced ceiling subtracted from the reduced floor
⍝ eg. (⌈⌿(⌈V)) - (⌊⌿(⌊V))
⍝ Protection is implemented by comparison against the empty set ⍬≡V
⍝ Which yields 0 or 1, and using that result to select an answer from a tuple
⍝ If empty, then it drops the first element, yielding just a zero, otherwise both are retained
⍝ eg. <condition>↓(a b) => 0 = (a b), 1 = (b)
⍝ The final operation is first ↑, to remove the first element from the tuple.
Z←↑(⍬≡V)↓(((⌈⌿(⌈V)) - (⌊⌿(⌊V))) 0)
Or in brace notation, for people without the font.
Z{leftarrow}DIFFERENCE V
{lamp} Calculate difference between vectors, with empty set protection
{lamp} Difference is calculated by a reduced ceiling subtracted from the reduced floor
{lamp} eg. ({upstile}{slashbar}({upstile}V)) - ({downstile}{slashbar}({downstile}V))
{lamp} Protection is implemented by comparison against the empty set {zilde}{equalunderbar}V
{lamp} Which yields 0 or 1, and using that result to select an answer from a tuple
{lamp} If empty, then it drops the first element, yielding just a zero, otherwise both are retained
{lamp} eg. <condition>{downarrow}(a b) => 0 = (a b), 1 = (b)
{lamp} The final operation is first {uparrow}, to remove the first element from the tuple.
Z{leftarrow}{uparrow}({zilde}{equalunderbar}V){downarrow}((({upstile}{slashbar}({upstile}V)) - ({downstile}{slashbar}({downstile}V))) 0)
and an image for the sake of preservation...
Updated. multi-dimensional
Z←DIFFERENCE V
⍝ Calculate difference between vectors, with empty set protection
⍝ Initially enlist the vector to get reduce to single dimension
⍝ eg. ∊V
⍝ Difference is calculated by a reduced ceiling subtracted from the reduced floor
⍝ eg. (⌈/V) - (⌊/V)
⍝ Protection is implemented by comparison against the empty set ⍬≡V
⍝ Which yields 0 or 1, and using that result to select an answer from a tuple
⍝ If empty, then it drops the first element, yielding just a zero, otherwise both are retained
⍝ eg. <condition>↓(a b) => 0 = (a b), 1 = (b)
⍝ The final operation is first ↑, to remove the first element from the tuple.
V←∊V
Z←↑(⍬≡V)↓(((⌈/V) - (⌊/V)) 0)
I have a factor RFyhat which I'm looking to convert to a numeric vector. I've already discovered that
as.numeric(levels(RFyhat))[RFyhat]
works as desired, and I've played around a bit with this construction:
c(1,2,20,4,5,6,7)[RFyhat]
also works as expected (RFyhat has 7 levels).
So I understand the behavior of this construction, but I'm wondering if anyone can explain how this syntax is intended to work, or whether it is just 'syntactic sugar'. More specifically, does [RFyhat] act as an index vector? If it does, how do factors generally behave when used as an index?
Yes, I believe that factors gets converted to integers when used for indexing, rather than characters or anything else.
Look at this example
> fac <- factor(letters[c(1,1,2,1,3,3,2,1)])
> vec <- c(b=1, a=2, c=3)
> vec[fac]
b b a b c c a b
1 1 2 1 3 3 2 1
So element 1 of fac has returned element 1 of vec, regardless of the different order of names.
Personally I'd prefer as.integer(as.character(RFyhat)) to as.numeric(levels(RFyhat))[...].