I have the following vector
37 15 30 37 4 11 35 37
I want to extract intervals of number. The interval starts and ends with the same number. This number appears more than once in the vector.
For example in this case 37: 15, 30 and 4, 11, 35 and 15, 30, 37, 4, 11, 35.
Can this example reproduced to a matrix?
After finding out the start value , using split and cumsum
names(table(v)[table(v)>2])
[1] "37"
split(v[v!=37],cumsum(v==37)[v!=37])
$`1`
[1] 15 30
$`2`
[1] 4 11 35
Related
I have a data matrix of order 2000 x 20, and I want to choose a specific order of entire rows from the matrix, for example, 1st,7th, 8th, 14th, 15th, 21th, 22th, is such a sequence of rows until the last 2000th rows.
[1, 7, 8, 14, 15, 21, 22, ...]
Manually it's very difficult to select such a sequence, is there is an alternative to do the same task in R? are the for looping is helpful in solving such a problem.
Using the updated question data, something like:
cumsum(rep(c(1,6), 2000/7))
# [1] 1 7 8 14 15
# ...
#[566] 1981 1982 1988 1989 1995
Since your pattern is +1/+6 up until 2000, you can repeat the two values c(1,6) as many times as sum(c(1,6)) goes in to 2000, and then take a cumulative sum.
Try this
mat[sort(c(k <- seq(6, 2000, by = 7), k + 1)),]
You can define your sequence first, using e.g. sequence, and then subset using [].
n = 2000 / 7
s = sequence(nvec = c(1, rep(2,n)), from = c(1, 7*1:n))
# s
# [1] 1 7 8 14 15 21 22 28 29 35 36 42 43 49 50 56 57 63 64 70 71 ...
yourMatrix[s, ]
sequence creates a sequence of sequences of length nvec and of starting point from.
I am working with R version i386 3.1.1 and RStudio 0.99.442.
I have large datasets of tree species that I've collected from 7 plots, each of which are divided into 5 subplots (i.e. 35 distinct subplots). I am trying to get R to run through my dataset and print the species which are present within each plot.
I thought I could use "aggregate" to apply the "levels" function to the Species data column and have it return the Species present for each Plot and Subplot, however it returns the levels of the entire data frame (for 12 species, total) rather than the 3 or 4 species that are actually present in the Subplot.
To provide a reproducible example of what I'm trying to do, we can use the "warpbreaks" dataset that comes with R.
I convert the 'breaks' variable in warpbreaks to a factor variable to recreate the problem; It thus exemplifies my 'species' variable, whereas 'warpbreaks$wool' would represent 'plot', and 'warpbreaks$tension' would represent 'subplot'.
require(stats)
warpbreaks$breaks = as.factor(warpbreaks$breaks)
aggregate(breaks ~ wool + tension, data = warpbreaks, FUN="levels")
If we look at the warpbreaks data, then for "Plot" A (wool) and "Subplot" L (tension) - the desired script would print the species "26, 30, 54, 25, etc."
breaks wool tension
1 26 A L
2 30 A L
3 54 A L
4 25 A L
5 70 A L
6 52 A L
7 51 A L
8 26 A L
9 67 A L
10 18 A M
11 21 A M
12 29 A M
...
Instead, R returns something of this sort, where it is printing ALL of the levels of the factor variable for ALL of the plots:
wool tension breaks.1 breaks.2 breaks.3 breaks.4 breaks.5 breaks...
1 A L 10 12 13 14 15 ...
2 B L 10 12 13 14 15 ...
3 A M 10 12 13 14 15 ...
4 B M 10 12 13 14 15 ...
5 A H 10 12 13 14 15 ...
6 B H 10 12 13 14 15 ...
How do I get it to print only the factors that are present within that Plot/Subplot combination? Am I totally off in my use of "aggregate"? I'd imagine this is a relatively easy task for an experience R user...
First time stackoverflow post - would appreciate any help or nudges towards the right code!
Many kind thanks.
Try FUN=unique rather than FUN=levels. levels will return every level of the factor, as you have surmised already. unique(...) will only return the unique levels.
y <- aggregate(breaks ~ wool + tension, data = warpbreaks, FUN=unique)
wool tension breaks
1 A L 14, 18, 29, 13, 31, 28, 27, 30
2 B L 15, 4, 17, 9, 19, 23, 10, 26
3 A M 8, 11, 17, 7, 2, 20, 18, 21
4 B M 24, 14, 9, 6, 22, 16, 11, 17
5 A H 21, 11, 12, 8, 1, 25, 16, 5, 14
6 B H 10, 11, 12, 7, 3, 5, 6, 16
NOTE the breaks column is a little weird, as in each row of that column instead of having one value (which makes sense for a dataframe), you have a vector of values. i.e. each cell of that breaks column is NOT a string; it's a vector!
> class(y$wool)
[1] "factor"
> class(y$breaks) # list !!
[1] "list"
> y$breaks[[1]] # first row in breaks
[1] 26 30 54 25 70 52 51 67
Levels: 10 12 13 14 15 16 17 18 19 20 21 24 25 26 27 28 29 30 31 35 36 39 41 42 43 44 51 52 54 67 70
Note that to access the first element of the breaks column, instead of doing y$breaks[1] (like you would with the wool or tension column) you need to do y$breaks[[1]] because of this.
Data frames are not really meant to work like this; a single cell in a dataframe is supposed to have a single value, and most functions will expect a dataframe to conform to this, so just keep this in mind when doing future processing.
If you wanted to convert to a string, use (e.g.) FUN=function (x) paste(unique(x), collapse=', '); then y$breaks will be a column of strings and behaves as normal.
I have two vectors of uneven lengths.
> starts
[1] 1 4 7 11 13 15 18 20 37 41 53 61
> ends
[1] 3 6 10 17 19 35 52 60 63
Each corresponding part in starts and ends are supposed to form a boundary, e.g. (1, 3) for the first, (4, 6) for second, etc. However you will notice that starts has 10 elements, and ends has just 9. What happened is for some anomaly, there may be consecutive starts, e.g. 4th to 6th elements of starts (11, 13, 15) are all smaller than the 4th element of ends (17).
Edit: please note also corresponding ends are not always 1 higher than starts, sample above edited to reflect so i.e. after ends 35, the next starts is 37.
My question is, how to find all these extranuous unpaired starts? My aim is to lengthen ends to be same length as starts, and pair all extranuous starts with a corresponding NA in ends. The actual vector lengths are in thousands, with mismatches in hundreds. I can imagine a nested for loop to address this, but am wondering if there is a more efficient solution.
Edit: the expected result would be (starts unchanged, displayed for comparison):
> starts
[1] 1 4 7 11 13 15 18 20 37 41 53 61
> ends
[1] 3 6 10 NA NA 17 19 35 NA 52 60 63
or equivalent, not particular about format.
> starts = c(1, 4, 7, 11, 15, 19, 23, 27)
> ends = c(3, 5, 14, 22, 25)
> e = ends[findInterval(starts, ends)+1]
> e
[1] 3 5 14 14 22 22 25 NA
> e[duplicated(e, fromLast=T)]=NA
> e
[1] 3 5 NA 14 NA 22 25 NA
findInterval seems to work
Assuming both starts and ends are sorted and that it's only in ends where the values are missing, you might be able to do something as straightforward as:
ends[c(match(starts, ends + 1)[-1], length(ends))]
# [1] 3 6 10 NA 17 19 36 52 60 63
I have imported a simple CSV file into a data frame. In a particular column, I would like to replace all values under 15.0 with 15.0, all values above 25.0 with 25.0, and all values between 15.0 and 25.0 with 20.0.
The below snippet works well for the first task, and switching the directional sign and replacing '15' with '25' works fine for the second task.
ff$temp[ ff$temp<15 ] <- 15
How can I accomplish replacing the values between 15 and 25 with 20?
You could use the same premise, just with a second logical operator to catch the "between" values.
For example,
> ff <- data.frame(temp = c(14, 17, 19, 24, 30))
> within(ff, temp[temp > 15 & temp < 25] <- 20)
# temp
# 1 14
# 2 20
# 3 20
# 4 20
# 5 30
set.seed(25)
v1 <- sample(5:35, 25, replace=TRUE)
c(15, 20, 25)[cut(v1, breaks=c(-Inf, 15, 25, Inf), labels=FALSE)]
#[1] 20 25 15 25 15 25 20 15 15 15 15 20 25 20 25 15 20 25 20 25 15 20 15 15 15
I have a data frame with a string of values, with certain anomalous readings I want to identify. I would like to make a third column in my data frame marking certain readings as "anomaly", and the rest as "normal". Looking over a plot of my data, by eye it seems pretty obvious when I get these odd dips, but I am having trouble figuring out how to get R to recognize the odd readings since the baseline average changes over time. The best I can come up with is three rules to use to classify something as "anomaly".
1: Starting with the second value, if the second value is within a close range of the first value, then mark as "N" for normal in the third column. And so on through the rest of the data set.
2: If the second value represents a large increase or decrease from the first value, mark as "A" for anomaly in the third column.
3: If a value is marked as "A", the following value will be marked as "A" as well if it is within a small range the previous anomalous value. If the following value represents a large increase or decrease from the previous anomalous value, it is to be marked as "N".
This was my best logic I could come up with, but looking at the data below if you can come up with a better idea I'm all for it.
So given a dummy data set:
SampleNum<-1:50
Value <- c(1, 2, 2, 2, 23, 22, 2, 3, 2, -23, -23, 4, 4, 5, 5, 25, 24,
6, 7, 6, 35, 38, 20, 21, 22, -22, 2, 2, 6, 7, 7, 6, 30, 31,
6, 6, 6, 5, 22, 22, 4, 5, 4, 5, 30, 39, 18, 18, 19, 18)
DF<-data.frame(SampleNum,Value)
This is how I might see the final data, with a third column identifying which values are anomalous.
SampleNum Value Name
1 1 N
2 2 N
3 2 N
4 2 N
5 23 A
6 22 A
7 2 N
8 3 N
9 2 N
10 -23 A
11 -23 A
12 4 N
13 4 N
14 5 N
15 5 N
16 25 A
17 24 A
18 6 N
19 7 N
20 6 N
21 35 A
22 38 A
23 20 N
24 21 N
25 22 N
26 -22 A
27 2 N
28 2 N
29 6 N
30 7 N
31 7 N
32 6 N
33 30 A
34 31 A
35 6 N
36 6 N
37 6 N
38 5 N
39 22 A
40 22 A
41 4 N
42 5 N
43 4 N
44 5 N
45 30 A
46 39 A
47 18 N
48 18 N
49 19 N
50 18 N
You need to distinguish anomalies from mixtures of different distributions. This is usually NOT a statistical question but rahter soemthing that comes from domain-specific knowledge. If you plot the desnity estimates from you data you get:
png(); plot( density(DF$Value)) ; dev.off()
So how are we supposed to know that the two values below zero are not real? They are 4% of your sample so applying a rule: "anomalies == items being outside the 99% confidence interval" would not define them as "anomalies. Are these activity measurements of some sort where the instrument should have given a positive value? The much larger bump peaking at 20 is surely not an anomaly by any reasonable definition.
You should do some searching on the topic of statistical prcess control. There are R packages with SPC oriented functions in them.