R ifelse condition: frequency of continuously NA - r

Question

I'm new to R, and I was looking for similar questions, but was not able to find one to fix mine, any help would be appreciated.
I have a data frame M:
date value
1 182-2002-01-01 23.95
2 182-2002-01-02 17.47
3 182-2002-01-03 NA
4 183-2002-01-01 NA
5 183-2002-01-02 5.50
6 183-2002-01-03 17.02
What I need to do is: if there are less than 5 NA (continuously), I will just repeat the previous number(17.47), and if there are more than 5 NA in a row, I will need to delete the whole month.
I tried function rle many times, but didn't work, many thanks for your help.

Answer 1

I'm going to adjust your question a little bit for the purposes of demonstration.
I'm going to use a similar dataset to you, but for 2 NAs in a row. This generalises to 5 very easily, don't worry. I'm also going to use a data set that better demonstrates the solution
So first, how to get your data to look like what I'm going to use:
library(reshape)
M2<-data.frame(colsplit(M$date, "-", c("ID", "year", "month", "day")),
value=M$value)
Now that's out of the road, this is the data I'm going to work with:
library(reshape)
M2<-data.frame(colsplit(M$date, "-", c("ID", "year", "month", "day")),
value=M$value)
set.seed(1234)
M2<-expand.grid(ID=182, year=2002:2004, month=1:2, day=1:3, KEEP.OUT.ATTRS=FALSE)
M2 <- M2[with(M2, order(year, month, day, ID)),] #sort the data
M2$value <- sample(c(NA, rnorm(100)), nrow(M2),
prob=c(0.5, rep(0.5/100, 100)), replace=TRUE)
M2
ID year month day value
1 182 2002 1 1 -0.5012581
7 182 2002 1 2 1.1022975
13 182 2002 1 3 NA
4 182 2002 2 1 -0.1623095
10 182 2002 2 2 1.1022975
16 182 2002 2 3 -1.2519859
2 182 2003 1 1 NA
8 182 2003 1 2 NA
14 182 2003 1 3 NA
5 182 2003 2 1 0.9729168
11 182 2003 2 2 0.9594941
17 182 2003 2 3 NA
3 182 2004 1 1 NA
9 182 2004 1 2 -1.1088896
15 182 2004 1 3 0.9594941
6 182 2004 2 1 -0.4027320
12 182 2004 2 2 -0.0151383
18 182 2004 2 3 -1.0686427
First, we're going to remove all cases where, within a month, there are 2 or more NAs in a row:
NA_run <- function(x, maxlen){
runs <- rle(is.na(x$value))
if(any(runs$lengths[runs$values] >= maxlen)) NULL else x
}
library(plyr)
rem <- ddply(M2, .(ID, year, month), NA_run, 2)
rem
ID year month day value
1 182 2002 1 1 -0.5012581
2 182 2002 1 2 1.1022975
3 182 2002 1 3 NA
4 182 2002 2 1 -0.1623095
5 182 2002 2 2 1.1022975
6 182 2002 2 3 -1.2519859
7 182 2003 2 1 0.9729168
8 182 2003 2 2 0.9594941
9 182 2003 2 3 NA
10 182 2004 1 1 NA
11 182 2004 1 2 -1.1088896
12 182 2004 1 3 0.9594941
13 182 2004 2 1 -0.4027320
14 182 2004 2 2 -0.0151383
15 182 2004 2 3 -1.0686427
You can see that the two in a row NAs have been removed. The one remaining is there because it belongs to two different months. Now we're going to fill in the remaining NAs. The na.rm=FALSE argument is there to keep the NAs if they're right at the beginning (which is what you want, I think).
library(zoo)
rem$value <- na.locf(rem$value, na.rm=FALSE)
rem
ID year month day value
1 182 2002 1 1 -0.5012581
2 182 2002 1 2 1.1022975
3 182 2002 1 3 1.1022975
4 182 2002 2 1 -0.1623095
5 182 2002 2 2 1.1022975
6 182 2002 2 3 -1.2519859
7 182 2003 2 1 0.9729168
8 182 2003 2 2 0.9594941
9 182 2003 2 3 0.9594941
10 182 2004 1 1 0.9594941
11 182 2004 1 2 -1.1088896
12 182 2004 1 3 0.9594941
13 182 2004 2 1 -0.4027320
14 182 2004 2 2 -0.0151383
15 182 2004 2 3 -1.0686427
Now all you need to do to make this 5 or more with your data is to change the value of the maxlen argument in NA_run to 5.
EDIT: Alternatively, if you don't want values to copy over from previous months:
library(zoo)
rem$value <- ddply(rem, .(ID, year, month), summarise,
value=na.locf(value, na.rm=FALSE))$value
rem
ID year month day value
1 182 2002 1 1 -0.5012581
2 182 2002 1 2 1.1022975
3 182 2002 1 3 1.1022975
4 182 2002 2 1 -0.1623095
5 182 2002 2 2 1.1022975
6 182 2002 2 3 -1.2519859
7 182 2003 2 1 0.9729168
8 182 2003 2 2 0.9594941
9 182 2003 2 3 0.9594941
10 182 2004 1 1 NA
11 182 2004 1 2 -1.1088896
12 182 2004 1 3 0.9594941
13 182 2004 2 1 -0.4027320
14 182 2004 2 2 -0.0151383
15 182 2004 2 3 -1.0686427

Answer 2

I'd do this in two steps:
An rle, rollapply, or shift-based strategy to fill in the small gaps (fewer than 5 NAs in a row).
A by, aggregate, or ddply-based strategy to take any month with NAs remaining after step 1 and make the whole month NA.