Replace duplicate values using multiple conditions in r - r

Question

I am new to R and I have the following data (an example) as a csv file, and I want to replace any duplicate values if they occurred on the consecutive days during similar year and month by zero or a letter. I only need to keep one average.
Year Month Day Average
2013 8 28 2.3
2013 8 29 2.3
2013 8 30 1.7
2013 8 31 1.7
2014 8 7 3
2014 8 6 3
2014 8 8 3
2014 8 9 3
2014 9 11 5.8
2014 9 12 5.8
2014 9 13 5.8
The result I expect is something like this
Year Month Day Average
2013 8 28 2.3
2013 8 29 0
2013 8 30 1.7
2013 8 31 0
2014 8 7 3
2014 8 6 0
2014 8 8 0
2014 8 9 0
2014 9 11 5.8
2014 9 12 0
2014 9 13 0
Also I would like to be able delete the rows that have the duplicate values that were replaced like this:
Year Month Day Average
2013 8 28 2.3
2013 8 30 1.7
2014 8 7 3
2014 9 11 5.8
I have to have two files one with the duplicate values replaced by Zero or a letter and another one has only the averages without the duplicate values.
Thank you in advance!!

Answer 1

Using dplyr for the data.frame manipulation, lubridate for date
manipulation and diff to find consecutive repeated values.
Note that I've also sorted the dates to keep the earliest one which makes it not exactly match with the example solution.
library(dplyr)
##
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
##
## filter, lag
## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, union
library(lubridate)
##
## Attaching package: 'lubridate'
## The following object is masked from 'package:base':
##
## date
df1 <- read.table(
text = "
Year Month Day Average
2013 8 28 2.3
2013 8 29 2.3
2013 8 30 1.7
2013 8 31 1.7
2014 8 7 3
2014 8 6 3
2014 8 8 3
2014 8 9 3
2014 9 11 5.8
2014 9 12 5.8
2014 9 13 5.8",
header = T)
df2 <- read.table(
text = "
Year Month Day Average
2013 8 28 2.3
2013 8 29 0
2013 8 30 1.7
2013 8 31 0
2014 8 7 3
2014 8 6 0
2014 8 8 0
2014 8 9 0
2014 9 11 5.8
2014 9 12 0
2014 9 13 0",
header = T)
df3 <- read.table(
text = "
Year Month Day Average
2013 8 28 2.3
2013 8 30 1.7
2014 8 7 3
2014 9 11 5.8",
header = T)
df2 <- df1 %>%
mutate(date = ymd(paste(Year, Month, Day, sep = "-"))) %>%
arrange(date) %>%
mutate(is_consecutive_average = c(FALSE, diff(Average) == 0)) %>%
mutate(is_consecutive_day = c(FALSE, diff(date) == 1)) %>%
mutate(Average = Average * !(is_consecutive_average & is_consecutive_day)) %>%
select(-is_consecutive_average, -is_consecutive_day, -date)
df2
## Year Month Day Average
## 1 2013 8 28 2.3
## 2 2013 8 29 0.0
## 3 2013 8 30 1.7
## 4 2013 8 31 0.0
## 5 2014 8 6 3.0
## 6 2014 8 7 0.0
## 7 2014 8 8 0.0
## 8 2014 8 9 0.0
## 9 2014 9 11 5.8
## 10 2014 9 12 0.0
## 11 2014 9 13 0.0
df3 <- df2 %>%
filter(Average != 0)
df3
## Year Month Day Average
## 1 2013 8 28 2.3
## 2 2013 8 30 1.7
## 3 2014 8 6 3.0
## 4 2014 9 11 5.8

Answer 2

Here's a data.table solution:
Read in the data
data <- readr::read_csv(
text,
col_names = TRUE,
trim_ws = TRUE
)
library( data.table )
setDT( data )
Convert the date values to a nicer format, and sort
data[ , date := as.Date( paste0( Year, "-", Month, "-", Day ) ) ]
setorder( data, date )
Create new columns for previous date and average values
data[ , prev.date := shift( date, 1L, type = "lag" ) ]
data[ , prev.average := shift( Average, 1L, type = "lag" ) ]
Mark the points where a new "group" should be created, based on your criteria. Also mark the very first record as the start of a new group, since we can assume that it is.
data[ , group := 0L
][ as.integer( date - prev.date ) > 1L |
Average != prev.average, group := 1L
][ 1L, group := 1L ]
Get your first desired output by replacing particular values with zeros
data[ group != 1L, Average := 0 ]
first.output <- data[ , .( date, Average ) ]
head( first.output, 3 )
date Average
1: 2013-08-28 2.3
2: 2013-08-29 0.0
3: 2013-08-30 1.7
Now mark the groups as unique numbers
data[ , group := cumsum( group ) ]
And get your second output by aggregating to maximum "Average" value (which will be the only one not equal to zero), and the minimum "date" value (the first in that group):
second.output <- data[ , .( date = min( date ),
Average = max( Average ) ),
by = group ][ , .( date, Average ) ]
head( second.output, 3 )
date Average
1: 2013-08-28 2.3
2: 2013-08-30 1.7
3: 2014-08-06 3.0
NOTE: you could likely get second.output by simply removing rows with a zero "Average" value from the first.output, but it would remove any groups where the "Average" really is zero, so I think this method is safer.