From a single dataset I created two dataset filtering on the target variable. Now I'd like to compare all the features in the dataset using chi square. The problem is that one of the two dataset is much smaller than the other one so in some features I have some values that are not present in the second one and when I try to apply the chi square test I get this error: "all arguments must have the same length".
How can I add to the dataset with less value the missing value in order to be able to use chi square test?
Example:
I want to use chi square on a the same feature in the two dataset:
chisq.test(table(df1$var1, df2$var1))
but I get the error "all arguments must have the same length" because table(df1$var1) is:
a b c d
2 5 7 18
while table(df2$var1) is:
a b c
8 1 12
so what I would like to do is adding the value d in df2 and set it equal to 0 in order to be able to use the chi square test.
The table output of df2 can be modified if we convert to factor with levels specified
table(factor(df2$var1, levels = letters[1:4]))
a b c d
8 1 12 0
But, table with two inputs, should have the same length. For this, we may need to bind the datasets and then use table
library(dplyr)
table(bind_rows(df1, df2, .id = 'grp'))
var1
grp a b c d
1 2 5 7 18
2 8 1 12 0
Or in base R
table(data.frame(col1 = rep(1:2, c(nrow(df1), nrow(df2))),
col2 = c(df1$var1, df2$var1)))
col2
col1 a b c d
1 2 5 7 18
2 8 1 12 0
data
df1 <- structure(list(var1 = c("a", "a", "b", "b", "b", "b", "b", "c",
"c", "c", "c", "c", "c", "c", "d", "d", "d", "d", "d", "d", "d",
"d", "d", "d", "d", "d", "d", "d", "d", "d", "d", "d")), class = "data.frame",
row.names = c(NA,
-32L))
df2 <- structure(list(var1 = c("a", "a", "a", "a", "a", "a", "a",
"a",
"b", "c", "c", "c", "c", "c", "c", "c", "c", "c", "c", "c", "c"
)), class = "data.frame", row.names = c(NA, -21L))
dat <- data.frame(Comp1Letter = c("A", "B", "D", "F", "U", "A*", "B", "C"),
Comp2Letter = c("B", "C", "E", "U", "A", "C", "A*", "E"),
Comp3Letter = c("D", "A", "C", "D", "F", "D", "C", "A"))
GradeLevels <- c("A*", "A", "B", "C", "D", "E", "F", "G", "U")
I have a dataframe that looks something like the above (but with many other columns I don't want to change).
The columns I am interested in changing contains lists of letter grades, but are currently character vectors and not in the right order.
I need to convert each of these columns into factors with the correct order. I've been able to get this to work using the code below:
factordat <-
dat %>%
mutate(Comp1Letter = factor(Comp1Letter, levels = GradeLevels)) %>%
mutate(Comp2Letter = factor(Comp2Letter, levels = GradeLevels)) %>%
mutate(Comp3Letter = factor(Comp3Letter, levels = GradeLevels))
However this is super verbose and chews up a lot of space.
Looking at some other questions, I've tried to use a combination of mutate() and across(), as seen below:
factordat <-
dat %>%
mutate(across(c(Comp1Letter, Comp2Letter, Comp3Letter) , factor(levels = GradeLetters)))
However when I do this the vectors remain character vectors.
Could someone please tell me what I'm doing wrong or offer another option?
You can do across as an anonymous function like this:
dat <- data.frame(Comp1Letter = c("A", "B", "D", "F", "U", "A*", "B", "C"),
Comp2Letter = c("B", "C", "E", "U", "A", "C", "A*", "E"),
Comp3Letter = c("D", "A", "C", "D", "F", "D", "C", "A"))
GradeLevels <- c("A*", "A", "B", "C", "D", "E", "F", "G", "U")
dat %>%
tibble::as_tibble() %>%
dplyr::mutate(dplyr::across(c(Comp1Letter, Comp2Letter, Comp3Letter) , ~forcats::parse_factor(., levels = GradeLevels)))
# # A tibble: 8 × 3
# Comp1Letter Comp2Letter Comp3Letter
# <fct> <fct> <fct>
# 1 A B D
# 2 B C A
# 3 D E C
# 4 F U D
# 5 U A F
# 6 A* C D
# 7 B A* C
# 8 C E A
You were close, all that was left to be done was make the factor function anonymous. That can be done either with ~ and . in tidyverse or function(x) and x in base R.
I have two data matrices of different dimensions stored as objects in R (I am using Rstudio with R v4.0.2 in Windows 10):
m1 = 1 column x 44 rows (this is a list of names with no spaces).
m2 = 500,000 columns x 164 rows (this contains a string of characters, the first row being a list of names).
I want to check how many (and which) of the rows of m1 are found in m2 (meaning it will be anywhere between 0 and 44). The end goal is that I have 4000 different matrices that will substitute the place of m2, and I need to see the extent of missing entries (found in m1) in all of the m2s (i.e., I am looking at the extent of missing data of those 44 names).
I am still a beginner to R, so apologies if my description is a bit off.
I tried storing each matrix, saved as CSV files, as such:
m1 <- read.csv("names-file.csv")
m2 <- read.csv("data-file.csv")
and tried to use the row.match function in the prodlim package, and ran row.match(m1, m2) but only got numeric values. I am looking to see just a number of how many of the names from m1 (first column) are found in m2 (first column), which values those are, and what the percentage would be (x out of 44).
As an example:
m1 =
Tom
Harry
Cindy
Megan
Jack
`
m2 =
Tom XXXXXXXXXXXX----XXXXXXXX
Stephanie XXXXXXXXXXXXXXXX----XXXX
Megan XXXXXXXXXXXXXXXXXXXXXXXX
Ryan XXXXXXXXXXXXXXXXXXXXXX-X
David XXXXXX---XXXXXXXXXXXXXXX
Josh XXXXXXXXXXXXXXXXXXXXXXXX
In the m2 matrix, each name is column 1, and the each subsequent X (which represents either an A, T, C, or G) are the subsequent columns (so some columns have an A, T, C, or G, or a "-"). I am looking to write a code that would see how many of the names from m1 and found in m2 (and conversely, how much data is missing from m2 as a percentage). In this case, the desired outputs would be:
2
Tom
Megan
60%
Here are my specific datafile using dput() (please let me know if I am using dput() correctly):
m1:
structure(list(V1 = c("Taxon1", "Taxon2", "Taxon3", "Taxon4",
"Taxon5", "Taxon6", "Taxon7", "Taxon8")), class = "data.frame", row.names = c(NA,
-8L))
m2:
structure(list(V1 = c("Taxon1", "Taxon3", "Taxon4", "Taxon6",
"Taxon7", "Taxon9", "Taxon10", "Taxon11", "Taxon12", "Taxon13",
"Taxon14", "Taxon15", "Taxon16", "Taxon17", "Taxon18", "Taxon19",
"Taxon20", "Taxon21", "Taxon22", "Taxon23", "Taxon24", "Taxon25",
"Taxon26", "Taxon27", "Taxon28", "Taxon29", "Taxon30"), V2 = c("A",
"A", "A", "A", "A", "A", "A", "A", "A", "A", "A", "A", "A", "A",
"A", "A", "A", "A", "A", "A", "C", "C", "C", "C", "C", "C", "C"
), V3 = c("G", "G", "G", "G", "G", "C", "C", "G", "G", "G", "G",
"G", "G", "G", "G", "G", "G", "G", "G", "G", "G", "G", "G", "G",
"G", "G", "G"), V4 = c("C", "C", "C", "C", "C", "T", "G", "C",
"C", "C", "C", "C", "C", "C", "C", "C", "C", "C", "C", "C", "C",
"C", "C", "C", "C", "C", "C"), V5 = c("T", "T", "G", "T", "G",
"G", "G", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T",
"T", "T", "T", "T", "T", "T", "T", "T", "T"), V6 = c("G", "G",
"C", "G", "C", "C", "C", "G", "G", "G", "G", "G", "G", "G", "G",
"G", "G", "G", "G", "G", "G", "G", "G", "G", "G", "G", "G"),
V7 = c("C", "C", "A", "C", "A", "A", "A", "C", "C", "C",
"C", "C", "C", "C", "C", "C", "G", "G", "G", "G", "G", "G",
"G", "G", "G", "G", "G"), V8 = c("T", "T", "A", "T", "A",
"A", "A", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T",
"T", "T", "T", "T", "T", "T", "T", "T", "T", "T"), V9 = c("A",
"A", "A", "A", "A", "T", "T", "A", "A", "A", "A", "A", "A",
"A", "A", "A", "A", "T", "T", "T", "T", "T", "T", "T", "T",
"T", "T")), class = "data.frame", row.names = c(NA, -27L))
Thank you!
You might want to have a look at the %in% operator in R. According to your question, you might want something like this:
m1[,1] %in% m2[,1]
#[1] TRUE FALSE TRUE TRUE FALSE TRUE TRUE FALSE
You can then pair it with functions such as mean or sum which will help you to find the percentage as required:
sum(m1[,1] %in% m2[,1])
#[1] 5
mean(m1[,1] %in% m2[,1])
#[1] 0.625
EDIT: As required by the OP in the comments of this post, there are various methods for that, my personal favourite being the which function:
m1[which(m1[,1] %in% m2[,1]),]
#[1] "Taxon1" "Taxon3" "Taxon4" "Taxon6" "Taxon7"
m1[which(!(m1[,1] %in% m2[,1])),]
#[1] "Taxon2" "Taxon5" "Taxon8"
Again, this is only one method, out of many (I can count 3 right now...), so I suggest you to explore the other options...
To get common names in both the dataframes you may use intersect, to calculate missing percentage you can use %in% with mean
common_names <- intersect(m1$V1, m2$V1)
missing_percentage_in_m1 <- mean(!m1$V1 %in% m2$V1) * 100
missing_percentage_in_m2 <- mean(!m2$V1 %in% m1$V1) * 100
common_names
#[1] "Taxon1" "Taxon3" "Taxon4" "Taxon6" "Taxon7"
missing_percentage_in_m1
#[1] 37.5
missing_percentage_in_m2
#[1] 81.48148
This code will get result like this
2
Tom
Megan
60%
1.how many of the names from m1 and found in m2
m1 <- t(m1)
res1 <-m2 %>%
rowwise %>%
mutate(n = m1 %in% c_across(V1:V9) %>% sum)
res1
# A tibble: 27 x 10
# Rowwise:
V1 V2 V3 V4 V5 V6 V7 V8 V9 n
<chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <int>
1 Taxon1 A G C T G C T A 1
2 Taxon3 A G C T G C T A 1
3 Taxon4 A G C G C A A A 1
4 Taxon6 A G C T G C T A 1
5 Taxon7 A G C G C A A A 1
6 Taxon9 A C T G C A A T 0
7 Taxon10 A C G G C A A T 0
8 Taxon11 A G C T G C T A 0
9 Taxon12 A G C T G C T A 0
10 Taxon13 A G C T G C T A 0
# ... with 17 more rows
res1 %>% select(n) %>% sum
[1] 5
res2 <-res1 %>%
filter(n >0) %>%
pull(V1) %>%
unique
res2
[1] "Taxon1" "Taxon3" "Taxon4" "Taxon6" "Taxon7"
2.how much data is missing from m2 as a percentage
res3 <- res2 %>% length
1 - res3 / length(unique(m2$V1))
[1] 0.8148148
I have two dataframes, each with the same columns. Some columns have the same values in the same order in both dataframes (X1, X2 below). Other columns have the same values, but in a different order (Y1). This is only a problem for some levels of first variables (here, the order of rows in Y1 differs for X1 == "a", but not X1 == "b"). Example:
df1 <- data.frame("X1" = c("a", "a", "a", "b", "b", "b"),
"X2" = c("1", "2", "3", "1", "2", "3"),
"Y1" = c("d", "d", "f", "g", "h", "i"))
df2 <- data.frame("X1" = c("a", "a", "a", "b", "b", "b"),
"X2" = c("1", "2", "3", "1", "2", "3"),
"Y1" = c("f", "d", "d", "g", "h", "i"))
I would like to change the values of df2$X1 and df2$X2 such that the two dataframes are matched on values of Y1.
I would like to change X1 and X2 rather than Y1 because there are many Y variables. I would like to do this only for df$X1 == "a".
The output should looks like this:
df2 <- data.frame("X1" = c("a", "a", "a", "b", "b", "b"),
"X2" = c("3", "1", "2", "1", "2", "3"),
"Y1" = c("f", "d", "d", "g", "h", "i"))
What is a little tricky in your situation is that you have duplicates in the Y1 columns which correspond to different values in the X2 columns. So you will have to make these unique.
First, make sure that your Y1 columns are character vectors and not factors:
df1 <- data.frame("X1" = c("a", "a", "a", "b", "b", "b"),
"X2" = c("1", "2", "3", "1", "2", "3"),
"Y1" = c("d", "d", "f", "g", "h", "i"),
stringsAsFactors = F)
df2 <- data.frame("X1" = c("a", "a", "a", "b", "b", "b"),
"X2" = c("1", "2", "3", "1", "2", "3"),
"Y1" = c("f", "d", "d", "g", "h", "i"),
stringsAsFactors = F)
Give unique names to your Y1 duplicates:
df1$Y1uniq <- make.unique(df1$Y1)
df2$Y1uniq <- make.unique(df2$Y1)
Then you can use match() using those uniques values (and remove that column once you don't need it anymore):
df1[match(df2$Y1uniq, df1$Y1uniq), ][ , 1:3]
Output:
X1 X2 Y1
3 a 3 f
1 a 1 d
2 a 2 d
4 b 1 g
5 b 2 h
6 b 3 i