In data frame, choose column that is most variable from others - r

Question

I have a data frame with, for example, replicates from an experiment in different columns. If each row in my data frame is a sample, with columns a, b, c as replicates, I want to:
Determine the variation between the replicates (What is the difference between the highest and lowest values in each row)? Put this in a new column called "variation.")
If the variation is greater than 10, omit the one replicate that is furthest away.
How can I accomplish this in this data frame? I want new columns:
"max" - highest value of a, b, c for each row
"min" - lowest value of a, b, c for each row
"variation" - max/min for each row
Then, I want to omit the data points in a, b, or c that are furthest away from the others so the remaining points have <10 variation.
df <- data.frame(a = rnorm(10, 100, 20),
b = rnorm(10, 2000, 500),
c = rnorm(10, 50, 20))
df$max = apply(df, 1, max, na.rm = T)
df$min = apply(df, 1, min, na.rm = T)
df$variation = df$max/df$min
(Also, how can I calculate the max and min using dplyr and %>% notation?)

Answer 1

Example using dplyr pipes, with mutate and group_by. I reshaped the data in long format using tidyr gather and reshaped it in wide format at the end using spread.
library(dplyr)
library(tidyr)
set.seed(100)
dtf_wide <- data.frame(a = rnorm(10, 100, 20),
b = rnorm(10, 2000, 500),
c = rnorm(10, 50, 20))
Reshape data in long format. Group by id (row number in the wide format) Then compute the variation and the distance from the median value.
dtf <- dtf_wide %>%
# Explicitely add an identification column (for the grouping)
mutate(id = row_number()) %>%
# put data in tidy format, one observation per row
gather(key, value, a:c) %>%
arrange(id) %>%
group_by(id) %>%
mutate(variation = max(value, na.rm = TRUE) / min(value, na.rm = TRUE),
median = median(value),
distancefrommedian = abs(value-median),
maxdistancefrommedian = max(distancefrommedian))
head(dtf)
# # A tibble: 6 x 7
# # Groups: id [2]
# id key value variation median distancefrommedian maxdistancefrommedian
# <int> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
# 1 1 a 89.95615 49.58856 89.95615 0.00000 1954.987
# 2 1 b 2044.94307 49.58856 89.95615 1954.98692 1954.987
# 3 1 c 41.23820 49.58856 89.95615 48.71795 1954.987
# 4 2 a 102.63062 31.37407 102.63062 0.00000 1945.507
# 5 2 b 2048.13723 31.37407 102.63062 1945.50661 1945.507
# 6 2 c 65.28121 31.37407 102.63062 37.34941 1945.507
If varation is greater than 10, remove the line where the value is further away from the median (you could change that rule here to take away more lines if needed).
dtf <- dtf %>%
# For each id,
# Take all lines where variation is smaller than 10
filter(variation <= 10 |
# If varation is greater than 10,
# Filter out lines were the value is further away from the median
(variation > 10 & distancefrommedian < maxdistancefrommedian)) %>%
# Keep only interesting variables
select(id, key, value) %>%
# Compute the variations again (just to check)
mutate(variation = max(value, na.rm = TRUE) / min(value, na.rm = TRUE))
head(dtf)
# id key value variation
# <int> <chr> <dbl> <dbl>
# 1 1 a 89.95615 2.181379
# 2 1 c 41.23820 2.181379
# 3 2 a 102.63062 1.572131
# 4 2 c 65.28121 1.572131
# 5 3 a 98.42166 1.781735
# 6 3 c 55.23923 1.781735
Reshape data to obtain a table in wide format similar to the original data frame.
dtf_wide2 <- dtf %>%
spread(key, value)
head(dtf_wide2)
# id variation a c
# <int> <dbl> <dbl> <dbl>
# 1 1 4.385692 89.95615 41.23820
# 2 2 4.385692 102.63062 65.28121
# 3 3 4.385692 98.42166 55.23923
# 4 4 4.385692 117.73570 65.46809
# 5 5 4.385692 102.33943 33.71242
# 6 6 4.385692 106.37260 41.23099