Can you help me understand how R interprets square brackets with forms such as y[i:j - k]?
dummy data:
y <- c(1, 2, 3, 5, 7, 8)
Here's what I do understand:
y[i] is the ith element of vector y.
y[i:j] is the ith to jth element (inclusive) of vector y.
y[-i] is vector y without the first i elements. etc. etc.
However, what I don't understand is what happens when you start mixing these options, and I haven't found a good resource for explaining it.
For example:
y[1-1:4]
[1] 5 7 8
So y[1-1:4] returns the vector without the first three elements. But why?
and
y[1-4]
[1] 1 2 5 7 8
So y[1-4] returns the vector without the third element. Is that because 1-4 = -3 and it's interpretting it the same as y[-3]? If so, that doesn't seem consistent with my previous example where y[1-1:4] would presumably be interpretted as y[0:4], but that isn't the case.
and
y[1:1+2-1]
[1] 2
Why does this return the second element? I encountered this while I was trying to code something along the lines of: y[i:i + j - k] and it took me a while to figure out that I should write y[i:(i + j - k)] so the parenthesis captured the whole of the right-hand-side of the colon. But I still can't figure out what logic R was doing when I didn't have those brackets.
Thanks!
It's best to look closer at precedence and the integer sequences you use for subsetting. These are evaluated before subsetting with []. Note that - is a function with two arguments (1, 1:4) which are evaluated beforehand and so
> 1-1:4
[1] 0 -1 -2 -3
Negative indices in [] mean exclusion of the corresponding elements. There is no "0" element (and so subsetting at 0 returns an empty vector of the present type -- numeric(0)). We thus expect y[1-1:4] to drop the first three elements in y and return the remainder.
As you write correctly y[1-4] is y[-3], i.e. omission of the third element.
Similar as above, in 1:1+2-1, 1:1 evaluates to a one-element vector 1, the rest is simple arithmetic.
For more on operator precedence, see Hadley's excellent book.
I have the following basic code. The first line sums p along dimension 1 to create a 1 x column array. The next line plot A. Unfortunately, it seems that Julia assumes it must plot many lines (in this case just points) along dimension 2.
A = sum(p,dims = 1)
plot(A)
So, my question is, how can I plot a simple line when the data is in a 1 x column array?
I assume you use Plots.jl. The following is from Plots.jl's documentation.
If the argument [to plot] is a "matrix-type", then each column will map to a series, cycling through columns if there are fewer columns than series. In this sense, a vector is treated just like an "nx1 matrix".
The number of series plot(a) tries to plot is the number of columns in a.
To get a single series, you can do one of the followings
plot(vec(a)) # `vec` will give you a vector view of `a` without an allocation
plot(a') # or `plot(transpose(a))`. `transpose` does not allocate a new array
plot(a[:]) # this allocates a new array so you should probably avoid it
I'm trying to make a LibreOffice spreadsheet formula that populates a column based on another input column, comparing each input with a series of range pairs defined in another sheet and finally outputting a symbol based on matched criteria. I have a series of ranges that specify a - output, and another series that corresponds to +, but not all inputs will fall into a category. I am using this trinary output later for another expression, which I already have in place.
My question becomes: how can I test input against each range pair without spelling out the cell coordinates for each individual cell (ie OR(AND(">= $A$1", "< $B$1"), AND(">=$A$2", "<$B$2"), ...))? Ideally I could just specify an array to compare against like $A$1:$B$4. Writing it in a python macro would work, too, since I don't plan on sharing this file.
I wrote a really quick list comp in python to illustrate what I'm after. This snippet would be one half, such as testing - qualification, and these values may be fed into a condition that outputs the symbol:
>>> def cmp(f, r):
... return r[0] <= f < r[1]
>>> f = (1, 2, 3)
>>> ranges = ((2, 5), (4, 6), (3, 8))
>>> [any([cmp(i, r) for r in ranges]) for i in f]
[False, True, True]
Here is a small test example with real input and real ranges.
Change the range pairs so that they are in two columns starting from A13. Be sure that they are in sorted order (Data -> Sort).
A B C
~~~~~~~~ ~~~~~~~~ ~
145.1000 145.5000 -
146.0000 146.4000 +
146.6000 147.0000 -
147.0000 147.4000 +
147.6000 148.0000 -
440.0000 445.0000 +
In each row, specify whether it is negative or positive. To do this, I entered the following formula in C13 and filled down. If the range pairs are not consistent enough then enter values for C13 and below manually.
=IF(ISODD(ROW());"-";"+")
Now, enter the following formula in cell C3 and fill down.
=IFNA(IF(
VLOOKUP(A3;A$13:C$18;2;1) >= A3;
VLOOKUP(A3;A$13:C$18;3;1);
"None");"None")
The formula finds the closest pair and then checks if the number is inside that range or not. For better testing, I would also suggest using 145.7000 as input, which should result in no shift if I understood the question correctly.
The results in column C:
-
+
None
None
Documentation: VLOOKUP, IFNA, ROW.
EDIT:
The following formula produces correct results for the example data you gave, and it works for anything between 144.0 and 148.0.
=IFNA(VLOOKUP(A3;A$13:C$18;3;1); "None")
However, 150.0 produces - and 550.0 produces +. If that is not what you want, then use the formula above that has two VLOOKUP expressions.
I'm mucking about with Julia and can't seem to get multidimensional array comprehensions to work. I'm using a nightly build of 0.20-pre for OSX; this could conceivably be a bug in the build. I suspect, however, it's a bug in the user.
Lets say I want to wind up with something like:
5x2 Array
1 6
2 7
3 8
4 9
5 10
And I don't want to just call reshape. From what I can tell, a multidimensional array should be generated something like: [(x, y) for x in 1:5, y in 6:10]. But this generates a 5x5 Array of tuples:
julia> [(x, y) for x in 1:5, y in 6:10]
5x5 Array{(Int64,Int64),2}:
(1,6) (1,7) (1,8) (1,9) (1,10)
(2,6) (2,7) (2,8) (2,9) (2,10)
(3,6) (3,7) (3,8) (3,9) (3,10)
(4,6) (4,7) (4,8) (4,9) (4,10)
(5,6) (5,7) (5,8) (5,9) (5,10)
Or, maybe I want to generate a set of values and a boolean code for each:
5x2 Array
1 false
2 false
3 false
4 false
5 false
Again, I can only seem to create an array of tuples with {(x, y) for x in 1:5, y=false}. If I remove the parens around x, y I get ERROR: syntax: missing separator in array expression. If I wrap x, y in something, I always get output of that kind -- Array, Array{Any}, or Tuple.
My guess: there's something I just don't get here. Anybody willing to help me understand what?
I don't think a comprehension is appropriate for what you're trying to do. The reason can be found in the Array Comprehension section of the Julia Manual:
A = [ F(x,y,...) for x=rx, y=ry, ... ]
The meaning of this form is that F(x,y,...) is evaluated with the variables x, y, etc. taking on each value in their given list of values. Values can be specified as any iterable object, but will commonly be ranges like 1:n or 2:(n-1), or explicit arrays of values like [1.2, 3.4, 5.7]. The result is an N-d dense array with dimensions that are the concatenation of the dimensions of the variable ranges rx, ry, etc. and each F(x,y,...) evaluation returns a scalar.
A caveat here is that if you set one of the variables to a >1 dimensional Array, it seems to get flattened first; so the statement that the "the result is... an array with dimensions that are the concatenation of the dimensions of the variable ranges rx, ry, etc" is not really accurate, since if rx is 2x2 and ry is 3, then you will not get a 2x2x3 result but rather a 4x3. But the result you're getting should make sense in light of the above: you are returning a tuple, so that's what goes in the Array cell. There is no automatic expansion of the returned tuple into the row of an Array.
If you want to get a 5x2 Array from a comprhension, you'll need to make sure x has a length of 5 and y has a length of 2. Then each cell would contain the result of the function evaluated with each possible pairing of elements from x and y as arguments. The thing is that the values in the cells of your example Arrays don't really require evaluating a function of two arguments. Rather what you're trying to do is just to stick two predetermined columns together into a 2D array. For that, use hcat or a literal:
hcat(1:5, 6:10)
[ 1:5 5:10 ]
hcat(1:5, falses(5))
[ 1:5 falses(5) ]
If you wanted to create a 2D Array where column 2 contained the result of a function evaluated on column 1, you could do this with a comprehension like so:
f(x) = x + 5
[ y ? f(x) : x for x=1:5, y=(false,true) ]
But this is a little confusing and it seems more intuitive to me to just do
x = 1:5
hcat( x, map(f,x) )
I think you are just reading the list comprehension wrong
julia> [x+5y for x in 1:5, y in 0:1]
5x2 Array{Int64,2}:
1 6
2 7
3 8
4 9
5 10
When you use them in multiple dimensions you get two variables and need a function for the cell values based on the coordinates
For your second question I think that you should reconsider your requirements. Julia uses typed arrays for performance and storing different types in different columns is possible. To get an untyped array you can use {} instead of [], but I think the better solution is to have an array of tuples (Int, Bool) or even better just use two arrays (one for the ints and one for the bool).
julia> [(i,false) for i in 1:5]
5-element Array{(Int64,Bool),1}:
(1,false)
(2,false)
(3,false)
(4,false)
(5,false)
I kind of like the answer #fawr gave for the efficiency of the datatypes while retaining mutability, but this quickly gets you what you asked for (working off of Shawn's answer):
hcat(1:5,6:10)
hcat({i for i=1:5},falses(5))
The cell-array comprehension in the second part forces the datatype to be Any instead of IntXX
This also works:
hcat(1:5,{i for i in falses(5)})
I haven't found another way to explicitly convert an array to type Any besides the comprehension.
Your intuition was to write [(x, y) for x in 1:5, y in 6:10], but what you need is to wrap the ranges in zip, like this:
[i for i in zip(1:5, 6:10)]
Which gives you something very close to what you need, namely:
5-element Array{(Int64,Int64),1}:
(1,6)
(2,7)
(3,8)
(4,9)
(5,10)
To get exactly what you're looking for, you'll need:
hcat([[i...] for i in zip(1:5, 6:10)]...)'
This gives you:
5x2 Array{Int64,2}:
1 6
2 7
3 8
4 9
5 10
This is another (albeit convoluted) way:
x1 = 1
x2 = 5
y1 = 6
y2 = 10
x = [x for x in x1:x2, y in y1:y2]
y = [y for x in x1:x2, y in y1:y2]
xy = cat(2,x[:],y[:])
As #ivarne noted
[{x,false} for x in 1:5]
would work and give you something mutable
I found a way to produce numerical multidimensional arrays via vcat and the splat operator:
R = [ [x y] for x in 1:3, y in 4:6 ] # make the list of rows
A = vcat(R...) # make n-dim. array from the row list
Then R will be a 3x3 Array{Array{Int64,2},2} while A is a 9x2 Array{Int64,2}, as you want.
For the second case (a set of values and a Boolean code for each), one can do something like
R = [[x y > 5] for x in 1:3, y in 4:6] # condition is y > 5
A = vcat(R...)
where A will be a 9x2 Array{Int64,2}, where true/false is denote by 1/0.
I have tested those in Julia 0.4.7.
I am trying to get a matrix that contains the distances between the points in two lists.
The vector of points contain the latitude and longitude, and the distance can be calculated between any two points using the function distCosine in the geosphere package.
> Points_a
lon lat
1 -77.69271 45.52428
2 -79.60968 43.82496
3 -79.30113 43.72304
> Points_b
lon lat
1 -77.67886 45.48214
2 -77.67886 45.48214
3 -77.67886 45.48214
4 -79.60874 43.82486
I would like to get a matrix out that would look like:
d_11 d_12 d_13
d_21 d_22 d_23
d_31 d_32 d_33
d_41 d_42 d_43
I am struggling to think of a way to generate the matrix without just looping over Points_a and Points_b and calculating each combination, can anyone suggest a more elegant solution?
You can use this:
outer(seq(nrow(Points_a)),
seq(nrow(Points_b)),
Vectorize(function(i, j) distCosine(Points_a[i,], Points_b[j,]))
)
(based on tip by #CarlWitthoft)
According to the desired output you post, maybe you'll want the transpose t() of this, or simply replace _a with _b above.
EDIT: some explanation:
seq(nrow(Points_x)): creates a sequence from 1 to the number of rows of Points_x;
distCosine(Points_a[i,], Points_b[j,]): expression to compute the distance between points given by row i of Points_a and row j of Points_b;
function(i, j): makes the above an unnamed function in two parameters;
Vectorize(...): ensure that, given inputs i and j of length greater than one, the unnamed function above is called only once for each element of the vectors (see this for more info);
outer(x, y, f): creates "expanded" vectors x and y such that all combinations of its elements are present, and calls f using this input (see link above). The result is then reassembled into a nice matrix.