Compare effect of environment between two dataframes - r

Question

I have two dataframes as follows:
a <- structure(list(Bacteria_A = c(12, 23, 45, 32, 34, 0), Bacteria_B = c(23,
12, 33, 44, 55, 3), Bacteria_C = c(25, 10, 50, 38, 3, 34), Group = structure(c(1L,
1L, 1L, 1L, 1L, 1L), class = "factor", .Label = "soil")), class = "data.frame", row.names = c("Sample_1",
"Sample_2", "Sample_3", "Sample_4", "Sample_5", "Sample_6"))
b <- structure(list(Bacteria_A = c(14, 10, 40, 40, 37, 3), Bacteria_B = c(25,
14, 32, 23, 45, 35), Bacteria_C = c(12, 34, 45, 22, 7, 23), Group = structure(c(1L,
1L, 1L, 1L, 1L, 1L), class = "factor", .Label = "water")), class = "data.frame", row.names = c("Sample_1",
"Sample_2", "Sample_3", "Sample_4", "Sample_5", "Sample_6"))
> a
Bacteria_A Bacteria_B Bacteria_C Group
Sample_1 12 23 25 soil
Sample_2 23 12 10 soil
Sample_3 45 33 50 soil
Sample_4 32 44 38 soil
Sample_5 34 55 3 soil
Sample_6 0 3 34 soil
> b
Bacteria_A Bacteria_B Bacteria_C Group
Sample_1 14 25 12 water
Sample_2 10 14 34 water
Sample_3 40 32 45 water
Sample_4 40 23 22 water
Sample_5 37 45 7 water
Sample_6 3 35 23 water
I want to compare the difference between each group across samples between soil and water.
For exemple For Bacteria_A i want to know if there is a difference between soil and water. Same for Bacteria_B and Bacteria_c (i have 900 bacteria). I though of a t-test but not sure how to do it with two dataframes.
Forgot to mention that not all bacteria are present in both dataframes so it could happen that one bacteria is not present in one of the environements. If bacteria are found in both environements they have exactly the same name.
Teh original dataframe is 160 samples per 500 Bacteria and data is not normally distributed.
Thanks for your help.

Answer 1

First of all, I want to mention that there are statistical methods to do the comparison which are more adequate than a t-test. They take into account the distribution the numbers are coming from (Negative-Binomial usually). You can check our DESeq2 package for instance.
As to your technical issue I would do:
for (bac in setdiff(intersect(colnames(a), colnames(b)), "Group")){
print(t.test(a[,bac], b[,bac]))
}

Answer 2

Your values do not seem to be in a normal or near-normal distribution, so you should stay away from the t-test. If you are unsure which distribution you are dealing with, you could use a wilcox.test.
You can stick your two data frames together quite easily then convert them to long format before running the appropriate tests:
library(tidyr)
library(dplyr)
bind_rows(a,b) %>%
pivot_longer(c(Bacteria_A, Bacteria_B, Bacteria_C)) %>%
group_by(name) %>%
summarise(mean_soil = mean(value[Group == "soil"]),
mean_water = mean(value[Group == "water"]),
pvalue = wilcox.test(value ~ Group)$p.value)
Which gives you
#> # A tibble: 3 x 4
#> name mean_soil mean_water pvalue
#> <chr> <dbl> <dbl> <dbl>
#> 1 Bacteria_A 24.3 24 0.936
#> 2 Bacteria_B 28.3 29 0.873
#> 3 Bacteria_C 26.7 23.8 0.748

Answer 3

This finds the bacteria names that exist in both data frames and then does a t.test between the same names giving a list L whose names are the bacteria names. The last line uses tidy to convert L to a data frame. You can replace t.test with wilcox.test if you prefer a non-parametric test. (Of course this does not take into account the problems of performing multiple hypothesis tests but rather just does the calculations.)
Name <- intersect(names(Filter(is.numeric, a)), names(Filter(is.numeric, b)))
L <- Map(t.test, a[Name], b[Name])
library(broom)
cbind(Name, do.call("rbind", lapply(L, tidy)))
The last line gives the following data frame:
Name estimate estimate1 estimate2 statistic p.value
Bacteria_A Bacteria_A 0.3333333 24.33333 24.00000 0.03485781 0.9728799
Bacteria_B Bacteria_B -0.6666667 28.33333 29.00000 -0.07312724 0.9435532
Bacteria_C Bacteria_C 2.8333333 26.66667 23.83333 0.30754940 0.7650662
parameter conf.low conf.high method alternative
Bacteria_A 9.988603 -20.97689 21.64356 Welch Two Sample t-test two.sided
Bacteria_B 7.765869 -21.80026 20.46692 Welch Two Sample t-test two.sided
Bacteria_C 9.492873 -17.84326 23.50993 Welch Two Sample t-test two.sided
Note
LinesA <- "Bacteria_A Bacteria_B Bacteria_C Group
Sample_1 12 23 25 soil
Sample_2 23 12 10 soil
Sample_3 45 33 50 soil
Sample_4 32 44 38 soil
Sample_5 34 55 3 soil
Sample_6 0 3 34 soil"
LinesB <- "Bacteria_A Bacteria_B Bacteria_C Group
Sample_1 14 25 12 water
Sample_2 10 14 34 water
Sample_3 40 32 45 water
Sample_4 40 23 22 water
Sample_5 37 45 7 water
Sample_6 3 35 23 water"
a <- read.table(text = LinesA, as.is = TRUE)
b <- read.table(text = LinesB, as.is = TRUE)