95% winsorization by groups over multiple variables - r

Question

In my real data, I have multiple outliers for multiple variables. My data looks something like the example below but the numbers here are completely random. I would like to pull in all data points that are greater than or less than 2 SD using a 95% winsorization.
df <- read.csv(header=TRUE, text="
id, group, test1, test2
1, 0, 57, 82
2, 0, 77, 80
3, 0, 67, 90
4, 0, 15, 70
5, 0, 58, 72
6, 1, 18, 44
7, 1, 44, 44
8, 1, 18, 46
9, 1, 20, 44
10, 1, 14, 38")
I am aware of the 'winsorize' function in the 'robustHD' package but am not sure: how to ensure the winsorization accounts for the 2 different groups, and including multiple variables in that winsorization.
I have tried this code to fix the problem but the code is not complete:
library(robustHD)
library(dplyr)
new.df.wins = df %>%
group_by(group) %>%
mutate(measure_winsorized = winsorize(c(test1,test2)))
An error is returned indicating
Error: Column `measure_winsorized` must be length 45 (the group size) or one, not 90
I am open to other ideas too. Thanks!

Answer 1

Consider creating two new fields for each numeric field to be winsorized:
new.df.wins <- df %>%
group_by(group) %>%
mutate(measure_winsorized_test1 = winsorize(test1),
measure_winsorized_test2 = winsorize(test2))
Alternatively with base R's ave:
new.df.wins <- within(df, {
measure_winsorized_test2 <- ave(test2, group, FUN=winsorize)
measure_winsorized_test1 <- ave(test1, group, FUN=winsorize)
})
Should you want to winsorize both simultaneously, assign to two new columns at once:
# TIDYVERSE (dplyr)
new.df.wins <- df %>%
group_by(group) %>%
mutate_at(.funs = list(wins = winsorize), .vars = vars(test1:test2))
# TINYVERSE (I.E. BASE R)
df[c("test1_wins", "test2_wins")] <- with(df, ave(cbind(test1, test2),
group, FUN=winsorize))

Answer 2

You can make a version of winsorize() that works on data frames, and use that with by()
# Example data
set.seed(1)
df2 <- round(matrix(rt(100, 4), 20), 3)
df2 <- data.frame(id=seq_len(nrow(df2)),
group=sort(rep(1:2, length=nrow(df2))),
test=df2)
df2[c(1:3, 11:13),]
# id group test.1 test.2 test.3 test.4 test.5
# 1 1 1 -0.673 -1.227 0.015 -0.831 0.024
# 2 2 1 -0.584 1.059 1.492 0.833 -0.377
# 3 3 1 0.572 0.613 -1.924 -0.672 1.184
# 11 11 2 0.054 0.020 2.241 -0.103 -0.047
# 12 12 2 1.746 -0.788 -0.268 -1.921 4.577
# 13 13 2 -0.472 -1.294 -0.258 0.795 -1.110
# data frame version of winsorize
winsorizedf <- function(x, ...) {
do.call(cbind, lapply(x, winsorize, ...))
}
# winsorize every column, except the two first ones, grouped by df2$group
w <- do.call(rbind,
by(df2[, -(1:2)], df2$group, winsorizedf))
# combine the winsorized columns with the original id and group columns
dfw <- data.frame(df2[, 1:2], round(w, 2))
dfw[c(1:3, 11:13),]
# id group test.1 test.2 test.3 test.4 test.5
# 1 1 1 -0.63 -1.23 0.02 -0.83 0.02
# 2 2 1 -0.58 1.06 1.49 0.26 -0.38
# 3 3 1 0.57 0.61 -1.60 -0.67 1.18
# 11 11 2 0.05 0.02 1.23 -0.10 -0.05
# 12 12 2 1.70 -0.79 -0.27 -1.92 4.58
# 13 13 2 -0.47 -1.07 -0.26 0.80 -1.11