Remove values based on specific relation to previous value in same column - r

Question

I have daily stock return data for several companies and need to remove those values, which have a specific relation to the previous (= the day before) return value.
In a mathematical formula it looks something like that:(1+r)*(1+e)-1<= 50%, where r is the return on the current day and e the return on the previous day, and at least either r or e being greater than 100%.
The data frame DF looks like that.
Date A B C D
01.01.2015 0.15 0.17 0.70 0.65
02.01.2015 1.01 0.75 0.01 -0.18
01.02.2015 -0.50 0.64 1.20 0.1
06.02.2015 0.12 0.54 0.13 1.50
01.03.2016 0.45 0.54 1.89 0.56
Afte apllying this filter DF should look like this.
Date A B C D
01.01.2015 0.15 0.17 0.70 0.65
02.01.2015 1.01 0.75 0.01 -0.18
01.02.2015 NA 0.64 1.20 0.1
06.02.2015 0.12 0.54 0.13 1.50
01.03.2016 0.45 0.54 1.89 0.56
Thanks for your help!

Answer 1

I would try this:
library(tidyverse)
check_fn <- function(Z){
ifelse(((lag(Z, n=1) > 1 | Z >1) & ((1+lag(Z, n=1))*(1+Z) <= 1.5)), NA, Z)
}
Y <- X %>%
mutate_at(vars(2:5), check_fn)
Y[1, 2:5] <- X[1, 2:5]
Y
... which generates:
Date A B C D
1 01.01.2015 0.15 0.17 0.70 0.65
2 02.01.2015 1.01 0.75 0.01 -0.18
3 01.02.2015 NA 0.64 1.20 0.10
4 06.02.2015 0.12 0.54 0.13 1.50
5 01.03.2016 0.45 0.54 1.89 0.56
I hope it helps you.

Answer 2

There might for shure be a more elegang solution.
> M=matrix(c(0.15,0.17,0.70,0.65,1.01,0.75,0.01,-0.18,-0.50,0.64,1.20,0.1,0.12,0.54,0.13,1.50,0.45,0.54,1.89,0.56),nrow = 5, byrow = TRUE)
> M
[,1] [,2] [,3] [,4]
[1,] 0.15 0.17 0.70 0.65
[2,] 1.01 0.75 0.01 -0.18
[3,] -0.50 0.64 1.20 0.10
[4,] 0.12 0.54 0.13 1.50
[5,] 0.45 0.54 1.89 0.56
> ifelse(rbind(c(T,T,T,T), !(((M[2:5,]>1)|(M[1:4,]>1))&(((1+M[2:5,])*(1+M[1:4,])-1)<.5))), M, NA)
[,1] [,2] [,3] [,4]
[1,] 0.15 0.17 0.70 0.65
[2,] 1.01 0.75 0.01 -0.18
[3,] NA 0.64 1.20 0.10
[4,] 0.12 0.54 0.13 1.50
[5,] 0.45 0.54 1.89 0.56

Answer 3

Sorry, I misread some of your post. I have corrected it so it matches your expected output.
library(data.table)
setDT(dat)
dat = dat[ , lapply(.SD, relationship), .SDcols = c("A", "B", "C", "D")]
relationship = function(x){
return(ifelse(((1 + x)*(1 + shift(x)) - 1) < .5 & !is.na(shift(x)) & (x > 1 | shift(x) > 1), NA, x))
}
> dat[ , lapply(.SD, relationship), .SDcols = c("A", "B", "C", "D")]
A B C D
1: 0.15 0.17 0.70 0.65
2: 1.01 0.75 0.01 -0.18
3: NA 0.64 1.20 0.10
4: 0.12 0.54 0.13 1.50
5: 0.45 0.54 1.89 0.56
You can cbind the dates back onto the data.table
I should add incase there are many more columns that this needs to be done on and you don't want to write them all out you could do something like this.
Dates = dat$Date
dat[ , "Date" := NULL]
dat = dat[ , lapply(.SD, relationship)]
That will apply the function to every column in the data.table.