I have two 3D arrays, the dimensions are specifically [Longitudes][Latitudes][Time] and the two array are temperature and precipitation observations in space and time.
I would like to obtain a 2D matrix [Longitudes][Latitudes] of the correlation (along the [Time] dimension) between the temperature and precipitation array for each specific Longitude-Latitude point.
The function apply() only work with one array at the time and the only solution I came out with is a basic loop, being Ts and Ps respectively the Temperature and Precipitation 3D arrays this is what I wrote:
corr.matix <- array(dim = dim(Ts)[c(1,2)])
for (i in seq(dim(Ts)[1])){
for (j in seq(dim(Ts)[2])){
corr.matix[i,j] <- cor(Ts[i,j,],Ps[i,j,])
}
}
It works, however it is slow.
My question is, is there a faster (vectorised?) way to solve this simple problem in R?
It would help to have some sample working code to test, but could map2() from package purrrhelp?
See https://purrr.tidyverse.org/reference/map2.html
Related
I am trying to find a function that matches two time series such that the datetime corresponds to reality.
So I need a function that minimizes the distance between the two curves shown above and outputs a new dataframe that has TAIR time-shifted towards the values of tre200h0.
From my bare eyes, it looks like this shift is about 22h.
ggplot
Best,
Fabio
I don't know a function that does this job for me.
Solved by Ric Villalba in the comments to OG Question.
Two R base functions to analyze time series lags are acf and pacf. i.e. given you have x and y you can use acf(y-x) and seek the zeroes in the plot (if your series have adequate seasonal behaviour), or, if you prefer, acf(y-x, plot=F) and get the data. Try which.min( acf(x-y)$acf^2 ).
Of course, it is a simplification of otherwise complex matter
I want to know the fastest algorithms for obtaining the cartesian distances between each point in a SpatialPointsDataFrame (X) and either (a) the closest point in a second SpatialPointsDataFrame (Y), or (b) the closest line segment in a SpatialLinesDataFrame (Y). So this is basically 2 questions, with perhaps the same answer.
For the lines, I know I can use dist2Line(X,Y, distfun=distGeo) but this is insanely slow. I also tried using nncross, after converting both X and Y to ppp objects, as below. This is did NOT work; heat mapping the new distance measure showed that it does not radiate from Y.
X_ppp <- as(X, "ppp")
Y_psp <- as(Y, "psp")
distR <- nncross(X_ppp,Y_ppp,what="dist",k=1)
X$dist2road <- distR
For lines, I also tried using gDistance(X,Y) but was met with the error, for i=1,2: Spatial object i is not projected; GEOS expects planar coordinates. I think this is because I'm using lat-lon, and it needs a true projection. But all the files i'm working with are lat-lon, and I'm not sure how to choose and specify a projection (for tanzania) w/out coping it from another file.
For points, again using the nncross approach resulted in definitely wrong distances. (In each the point and line case, is this because the output vector is not ordered in the same way that the points within X are? If so, I see now way of outputting an ID for the point within X.)
Also for points, this knn code below did work. But it's clearly not in cartesian distance, and so I'd like to convert it or find some other algorithm that provides cartesian distance.
knn.results = knn(data=coordinates(market.shp),
query=coordinates(tzprice.shp), k=1)
knn.results <- data.frame(knn.results)
tzprice.shp$dist2market <- knn.results[,2]
Basically, my hope is to find the fastest algorithm for each purpose (distance to nearest point, distance to nearest line), with output either in cartesian distance or convertible to cartesian distance. Thanks!
Somebody pointed me towards one possible answer for finding the cartesian distance between each point in a SpatialPointsDataFrame (X) and the closest point in a second SpatialPointsDataFrame (let's call it Y). So that's the first half of my question... perhaps there's a faster method out there, but this way is quite fast, and it DOES return answers in Km, at least if proj=longlat.
tree <- createTree(coordinates(Y))
inds <- knnLookup(tree, newdat=coordinates(X), k=1)
distkm <- sapply(seq_len(nrow(inds)), function(i) spDists(X[i, ], Y[inds[i, ],]))
Still looking for an algorithm that (quickly) finds meters/km from each point in X to the nearest line in a SpatialLinesDataFrame.
I'm trying to find out the peak frequencies hidden in my data using the fft() method in R. While preparing the data, a more experienced user recommends to create a "mask" (more after explaining the details), that does give me the exact diagram I'm looking for. The problem is, I don't understand what it does or why it's needed.
To give some context, I'm working with .txt files with around 12000 entries each. It's voltage vs. time information, and the expected result is just a sinusoidal wave with a clear peak frequency that should be close to 1-2 Hz. This is an example of what one of those files look like:
I've been trying to use the Fast Fourier Transform method fft() implemented in R to find the peak frequencies and get a diagram that reflected them clearly. At first, I calculate some things that I understand are going to be useful, like the Nyquist frequency and the range of frequencies I'll show in the final graph:
n = length(variable)
dt = time[5]-time[4]
df = 1/(max(time)) #Find out the "unit" frequency
fnyquist = 1/(2*dt) #The Nyquist frequency
f = seq(-fnyquist, fnyquist-df, by=df) #These are the frequencies I'll plot
But when I plot the absolute value of what fft(data) calculates vs. the range of frequencies, I get this:
The peak frequency seems to be close to 50 Hz, but I know that's not the case. It should be close to 1 Hz. I'm a complete newbie in R and in Fourier analysis, so after researching a little, I found in a Swiss page that this can be solved by creating a "mask", which is actually just a vector with a repeatting patern (1, -1, 1, -1...) with the same length as my data vector itself:
mask=rep(c(1, -1),length.out=n)
Then if I multiply my data vector by this mask and plot the results:
results = mask*data
plot(f,abs(fft(results)),type="h")
I get what I was looking for. (This is the graph after limiting the x-axis to a reasonable scale).
So, what's the mask actually doing? I undestand it's changing my data point signs in an alternate manner, but I don't get why it would take the infered peak frequencies from ~50 Hz to the correct result of ~1 Hz.
Thanks in advance!
Your "mask" is one of two methods of performing an fftshift, which is commonly done to center the 0 Hz output of an FFT in the middle of a graph or plot (instead of at the left edge, with the negative frequencies wrapping around to the right edge).
To perform an fftshift, you can hetrodyne or modulate your data (by Fs/2) before the FFT, or simply do a circular shift by 50% after the FFT. Both produce the same result. They are the same due to the shift property of the DFT.
I've been working with a spatial model which contains 21,000 grid cells of unequal size (i by j, where i is [1:175] and j is[1:120]). I have the latitude and longitude values in two seperate arrays (lat_array,lon_array) of i and j dimensions.
Plotting the coordinates:
> plot(lon_array, lat_array, main='Grid Coordinates')
Result:
My question: Is it possible to plot these spatial coordinates as a grid rather than as points? Does anyone know of a package or function that might be able to do this? I haven't been able to find anything online to this nature.
Thanks.
First of all it is always a bit dangerous to plot inherently spherical coordinates (lat,long) directly in the plane. Usually you should project them in some way, but I will leave it for you to explore the sp package and the function spTransform or something like that.
I guess in principle you could simply use the deldir package to calculate the Dirichlet tessellation of you points which would give you a nice grid. However, you need a bounding region for this to avoid large cells radiating out from the border of your region. I personally use spatstat to call deldir so I can't give you the direct commands in deldir, but in spatstat I would do something like:
library(spatstat)
plot(lon_array, lat_array, main='Grid Coordinates')
W <- clickpoly(add = TRUE) # Now click the region that contains your grid
i_na <- is.na(lon_array) | is.na(lat_array) # Index of NAs
X <- ppp(lon_array[!i_na], lat_array[!i_na], window = W)
grid <- dirichlet(X)
plot(grid)
I have not tested this yet and I will update this answer once I get the chance to test it with some artificial data. A major problem is the size of your dataset which may take a long time to calculate the Dirichlet tessellation of. I have only tried to call dirichlet on dataset of size up to 3000 points...
I would like to know the angular difference between the orientation of two 3D matrices (4x4). Two matrices that are both oriented in the same direction would be zero, and two matrices that are oriented in opposite directions would be 180ยบ. By 'orientation' I am referring to the direction that an object transformed by the matrix would be facing. So I'm only concerned with rotation, not translation or scale.
Specifically, I am using instances of WebKitCSSMatrix which refer to the 16 3D matrix values as .m11 through .m44.
in that case compare only one axis from the matrices
extract direction vector from your matrix
which one it is depends on your mesh models
it is the one the object is going forward
in mine models it is usually Z-axis
but I also see that other people often use X-axis
look here: matrix vectors extraction
I am not familiar with your matrix library
but there is a chance that your matrices are transposed !!!
so if it not works like it should extract the transposed vectors ... (rows instead columns)
compute the difference
just compute this: angle = acos ( (V1.V2)/(|V1|.|V2|) )
where V1,V2 are the direction vectors