combining 2 columns using ifelse dropping a variable R - r

I am trying to combine the male and female columns I have created into one column. I tried using some answers I found on stack, but the second sex I queried was excluded.
Build Data Frame:
ID <- 1:10
SPAYDT <- c("", "2011-12-01", "", "2006-05-01", "", "", "", "", "", "")
SPAYDTU <- c(1, NA, NA, NA, NA, NA, NA, NA, NA, NA)
NEUTDT <- c("", "", "", "", "", "", "2013-03-01", "", "", "")
NEUTDTU <- c(NA, NA, NA, NA, NA, NA, NA, 1, NA, NA)
df <- as.data.frame(cbind(ID, SPAYDT, SPAYDTU, NEUTDT, NEUTDTU))
df
The goal is to have a column for sex, formated as a factor with 2 levels - Male and Female
It should say female if the SPAYDT or SPAYDTU have a value in them, and male if the NEUTDT or NEUTDTU have a value in them.
What I have tried:
using a nested if-else statement to build one sex column
making two columns then combining using
df$male <- ifelse(NEUTDT!="", "Male",
ifelse(NEUTDTU=1, "Male", NA))
df$female <- ifelse(SPAYDT!="", "Female",
ifelse(SPAYDTU==1, "Female", NA))
df$sex <- ifelse(!is.na(df$female), df$female, df$male)
and
df$sex <- ifelse(SPAYDT!="", "Female",
ifelse(SPAYDTU==1, "Female",
ifelse(NEUTDT!="", "Male",
ifelse(NEUTDTU=1, "Male", NA))))
However, no matter what I do, the sex column at the end only has one sex. I made sure my df was attached for use of column names as variables. I tried restarting R and running the setup code again. I just don't know why the ifelse statement is ignoring the second sex input.
Any help is greatly appreciated!
Clarifications:
In the larger dataframe I am working with I have done data clean up so that each ID only corresponds to 1 sex. Sorry about the mistake in the code.
Desired output:
ID <- 1:10
SPAYDT <- c("", "2011-12-01", "", "2006-05-01", "", "", "", "", "", "")
SPAYDTU <- c(1, NA, NA, NA, NA, NA, NA, NA, NA, NA)
NEUTDT <- c("", "", "", "", "", "", "2013-03-01", "", "", "")
NEUTDTU <- c(NA, NA, NA, NA, NA, NA, NA, 1, NA, NA)
SEX <- c("Female", "Female", NA, "Female", NA, NA, "Male", "Male", NA, NA)
df <- as.data.frame(cbind(ID, SPAYDT, SPAYDTU, NEUTDT, NEUTDTU, SEX))
df

Is this what you are after?
ID <- 1:10
SPAYDT <- c("", "2011-12-01", "", "2006-05-01", "", "", "", "", "", "")
SPAYDTU <- c(1,NA,NA,NA,NA,NA,NA,NA,NA,NA)
NEUTDT <- c("", "", "", "", "", "", "2013-03-01", "", "", "")
NEUTDTU <- c(NA,NA,NA,1,NA,NA,NA,NA,NA,NA)
df <- data.frame(ID, SPAYDT, SPAYDTU, NEUTDT, NEUTDTU)
df %>%
mutate(
sex = case_when(
NEUTDT!="" | NEUTDTU==1 ~ "Male",
SPAYDT!="" | SPAYDTU==1 ~ "Female",
TRUE ~ NA_character_))

Related

Conditional str_remove based on data frame column

I have a dataframe (pasted below), in which I am trying to set to blank the value of one column based on the value of another column. The idea is that if X6 equals Nbre CV or if X6equals Nbre BVD then I want X6for that row to be blank.
Unfortunately using the following code the entire X6 column turns to NA or missing.
extractstack <- extractstack %>%
mutate(across(everything(), as.character) %>%
mutate(X6 = if_else(X6 == `Nbre CV`, str_remove(X6, `Nbre CV`), X6)) %>%
mutate(X6 = if_else(X6 == `Nbre CV`, str_remove(X6, `Nbre BVD`), X6)))
structure(list(X1 = c("", "", "40", "", "", "41", "", "", "42",
"", "", "43", "", "", "44", ""), X2 = c("", "", "EP. KAPALA",
"", "", "INST. MOTULE", "", "", "CABANE BABOA", "", "", "CABANE BANANGI",
"", "", "E.P.BINZI", ""), X3 = c("", "", "MOBATI-BOYELE", "",
"", "MOBATI-BOYELE", "", "", "MOBATI-BOYELE", "", "", "AVURU-GATANGA",
"", "", "AVURU-GATANGA", ""), X4 = c("", "", "BOGBASA", "", "",
"BOSOBEA", "", "", "BOSOBEA", "", "", "BANANGI", "", "", "GURUZA",
""), X5 = c("", "", "", "", "", "MOBENGE", "", "", "BABOA", "",
"", "DIFONGO", "", "", "DULIA", ""), X6 = c("", "", "BOGBASA",
"", "", "", "1", "", "", "1", "", "", "1", "", "", "1"), X7 = c("1",
"", "", "1", "", "", "4", "", "", "1", "", "", "1", "", "", "5"
), X8 = c("2", "", "", "2", "", "", "510 110", "", "", "510 111",
"", "", "510 112", "", "", "510 113"), X9 = c("510 108", "",
"", "510 109", "", "", "A - D", "", "", "A", "", "", "A", "",
"", "A - E"), page = c("4", "4", "4", "4", "5", "5", "5", "5",
"5", "5", "5", "5", "5", "5", "5", "5"), Plage = c("A - B", NA,
NA, "A - B", NA, NA, "A - D", NA, NA, "A", NA, NA, "A", NA, NA,
"A - E"), `Code SV` = c("510 108", NA, NA, "510 109", NA, NA,
"510 110", NA, NA, "510 111", NA, NA, "510 112", NA, NA, "510 113"
), `Nbre BVD` = c("2", NA, NA, "2", NA, NA, "4", NA, NA, "1",
NA, NA, "1", NA, NA, "5"), `Nbre CV` = c("1", NA, NA, "1", NA,
NA, "1", NA, NA, "1", NA, NA, "1", NA, NA, "1")), class = "data.frame", row.names = c(NA,
-16L))
That's basically Chris Ruehlemann's answer (I don't know why he removed it, I would remove this one for the original one):
library(dplyr)
extractstack %>%
mutate(across(everything(), as.character),
X6 = coalesce(ifelse(X6 == `Nbre BVD` | X6 == `Nbre CV`, "", X6), X6))
compares X6 with the columns Nbre BVD and Nbre CV. If there is matching content, X6 will be changed to an empty string "", else X6 stays unchanged. But for your given data, this code doesn't replace anything, since there are simply no matches in X6 with Nbre BVD and Nbre CV besides NA-values.

Remove rows until columns are identical over multiple data frames

I have 4 data frames named w, x, y, z each with 3 columns and identical column names. I now execute an operation that removes rows until the column named Type is identical over all four data frames.
To achieve this I am using a while loop with the following code:
list_df <- list(z, w, x, y)
tmp <- lapply(list_df, `[[`, 'Type')
i <- as.integer(as.logical(all(sapply(tmp, function(x) all(x == tmp[[1]])))))
while (i == 0) {
z <- z[(z$Type %in% x$Type),]
y <- y[(y$Type %in% x$Type),]
w <- w[(w$Type %in% x$Type),]
z <- z[(z$Type %in% w$Type),]
y <- y[(y$Type %in% w$Type),]
x <- x[(x$Type %in% w$Type),]
z <- z[(z$Type %in% y$Type),]
x <- x[(x$Type %in% y$Type),]
w <- w[(w$Type %in% y$Type),]
x <- x[(x$Type %in% z$Type),]
w <- w[(w$Type %in% z$Type),]
y <- y[(y$Type %in% z$Type),]
list_df <- list(z, w, x, y)
tmp <- lapply(list_df, `[[`, 'Type')
i <- as.integer(as.logical(all(sapply(tmp, function(x) all(x == tmp[[1]])))))
}
In this code, a list is created for the column Type of every data frame. Then the value i tests for identicality and produces 0 if false and 1 if true. The while loop then performs the deletion of rows not included in every data frame and only stops until i becomes 1.
This code works, but applying it to bigger data can result in a long time for the code to go through. Does anybody have an idea on how to simplify this execution?
For reproducible example:
w <- structure(list(Type = c("26809D", "28503C", "360254", "69298N",
"32708V", "680681", "329909", "696978", "32993F", "867609", "51206K",
"130747"), X1980 = c(NA, NA, NA, 271835, NA, NA, NA, NA, NA,
NA, NA, NA), X1981 = c(NA, NA, NA, 290314, NA, NA, NA, NA, NA,
NA, NA, NA)), row.names = c("2", "4", "7", "8", "10", "11", "13",
"16", "17", "21", "22", "23"), class = "data.frame")
x <- structure(list(Type = c("26809D", "28503C", "360254", "69298N",
"32708V", "680681", "329909"), X1980 = c(NA, NA, NA, 1026815,
NA, NA, NA), X1981 = c(NA, NA, NA, 826849, NA, NA, NA)), row.names = c("2",
"4", "7", "8", "10", "11", "13"), class = "data.frame")
y <- structure(list(Type = c("26809D", "28503C", "360254", "69298N",
"32708V"), X1980 = c(NA_real_, NA_real_, NA_real_, NA_real_,
NA_real_), X1981 = c(NA_real_, NA_real_, NA_real_, NA_real_,
NA_real_)), row.names = c("2", "4", "7", "8", "10"), class = "data.frame")
z <- structure(list(Type = c("26809D", "28503C", "360254", "69298N",
"32708V", "680681", "329909", "696978", "32993F", "867609", "51206K",
"130747", "50610H"), X1980 = c(NA, NA, NA, 0.264736101439889,
NA, NA, NA, NA, NA, NA, NA, NA, NA), X1981 = c(NA, NA, NA, 0.351108848169376,
NA, NA, NA, NA, NA, NA, NA, NA, NA)), row.names = c("2", "4",
"7", "8", "10", "11", "13", "16", "17", "21", "22", "23", "24"
), class = "data.frame")
We assume that the question is how to get the values of Type that are common to 4 data frames each of which has a Type column containing unique values.
Form a list L of the data frames, extract the Type column using lapply and [ and iterate merge over that using Reduce :
L <- list(w, x, y, z)
L.Type <- lapply(L, "[", TRUE, "Type", drop = FALSE) # list of DFs w only Type col
Reduce(merge, L.Type)$Type
## [1] "26809D" "28503C" "32708V" "360254" "69298N"
or replace last line with this giving the same result except for order:
Reduce(intersect, L.Type)$Type
## [1] "26809D" "28503C" "360254" "69298N" "32708V"
Another approach which is a bit tedious but does reduce the calulation to one line is to manually iterate intersect:
intersect(w$Type, intersect(x$Type, intersect(y$Type, z$Type)))
## [1] "26809D" "28503C" "360254" "69298N" "32708V"
Another example
The example data is not very good to illustrate this because every data frame has the same values of Type so let us create another example. BOD is a built-in data frame has 6 rows. We assign it to X and rename the columns so that the first one has the name Type. Then for i equals 1, 2, 3, 4 we remove the i-th row giving 4 data frames with 5 rows each and 2 values of Type common to all 4. The result correctly shows that 5 and 7 are the only common Type values.
# set up input L, a list of 4 data frames
X <- BOD
names(X) <- c("Type", "X")
L <- lapply(1:4, function(i) X[-i, ])
L.Type <- lapply(L, "[", TRUE, "Type", drop = FALSE)
Reduce(merge, L.Type)$Type
## [1] 5 7

right_join and mutate does not preserve the index in R

I am Mapping column_data to master and if column value is present in master than it saves it Key
ex:Parent for P and Child for C
Problem is i am getting the output but output is indexed differently
DATA
column_data <- c("", "", "", "", "", "", "", "", "", "", "", "", "", "", "",
"", "P", "C", "C")
master <- list("Parent" = c("P"),
"Child" = c("C")
)
CODE
library(dplyr)
df <- data.frame("column" = column_data)
df <-stack(master) %>%
type.convert(as.is = TRUE) %>%
right_join(df, by = c('values' = 'column')) %>%
mutate(output = coalesce(ind, values))
This Should be the output:
structure(list(values = c("", "", "", "", "", "", "", "", "",
"", "", "", "", "", "", "", "P", "C", "C"), ind = c(NA, NA, NA,
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, "Parent",
"Child", "Child"), output = c("", "", "", "", "", "", "", "",
"", "", "", "", "", "", "", "", "Parent", "Child", "Child")), class = "data.frame", row.names = c(NA,
-19L))
but instead i get this as output:
structure(list(values = c("P", "C", "C", "", "", "", "", "",
"", "", "", "", "", "", "", "", "", "", ""), ind = c("Parent",
"Child", "Child", NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA,
NA, NA, NA, NA, NA), output = c("Parent", "Child", "Child", "",
"", "", "", "", "", "", "", "", "", "", "", "", "", "", "")), row.names = c(NA,
-19L), class = "data.frame")
With dplyr, if you do a right_join(x, y) then the result will include a subset of the matched rows for x, then unmatched rows for y.
From R documentation on mutating joins, the value returned will be:
An object of the same type as x. The order of the rows and columns of
x is preserved as much as possible. The output has the following
properties:
For inner_join(), a subset of x rows. For left_join(), all x rows. For
right_join(), a subset of x rows, followed by unmatched y rows. For
full_join(), all x rows, followed by unmatched y rows.
That is why you have the 3 matched rows at the beginning of your resulting data.frame.
To get the desired result preserving the row order of df, try a left_join as follows:
df2 <- stack(master) %>%
type.convert(as.is = TRUE)
df %>%
left_join(df2, by = c('column' = 'values')) %>%
mutate(output = coalesce(ind, column))
Output
column ind output
1 <NA>
2 <NA>
3 <NA>
4 <NA>
5 <NA>
6 <NA>
7 <NA>
8 <NA>
9 <NA>
10 <NA>
11 <NA>
12 <NA>
13 <NA>
14 <NA>
15 <NA>
16 <NA>
17 P Parent Parent
18 C Child Child
19 C Child Child

summarize and spread by almost identical strings

I started with several raw df's with similar items ,cleaned and merged to a long format which i later combine to wide format using dplyr... However, i'm left with duplicates because i'm dealing with almost identical strings, can anyone please suggest an easier way to remove the duplicates while spreading my data.
here is a sample of my code
library(tidyverse)
library(readxl)
library(reprex)
all_data_final_wider<-all_data_final %>%
mutate(cases = case_when(cases=='X' ~ 'x', cases=='x' ~ 'x'))%>%
group_by(Species) %>%
mutate(row = row_number()) %>%
tidyr::pivot_wider(names_from = location, values_from =cases)%>%
select(-row)
and below is a dput of my sample data
structure(list(`Wall type (Kaminski 2014)` = c("", "", "hyaline",
"hyaline", "hyaline", "hyaline", "", "hyaline", "", "hyaline",
"hyaline", "", "", "porcelaneous (imperforate)", "porcelaneous (imperforate)",
"porcelaneous (imperforate)", "porcelaneous (imperforate)", "porcelaneous (imperforate)",
"", "", "", "", "", "", "", "", "", "porcelaneous (imperforate)",
"porcelaneous (imperforate)", "porcelaneous (imperforate)", "porcelaneous (imperforate)",
"porcelaneous (imperforate)", "porcelaneous (imperforate)", "porcelaneous (imperforate)",
"", "", "", "", "", "", "porcelaneous (imperforate)", "", "",
"", "porcelaneous (imperforate)", "", "", "", "", ""), Order = c("",
"", "Rotaliida", "Rotaliida", "Rotaliida", "Rotaliida", "", "Rotaliida",
"", "Rotaliida", "Rotaliida", "", "", "Miliolida", "Miliolida",
"Miliolida", "Miliolida", "Miliolida", "Miliolida", "", "", "",
"", "", "", "", "", "Miliolida", "Miliolida", "Miliolida", "Miliolida",
"Miliolida", "Miliolida", "Miliolida", "", "", "", "", "", "",
"Miliolida", "", "", "", "Miliolida", "", "", "", "", ""), Superfamily = c("",
"", "Planorbulinoidea", "Acervulinoidea", "Acervulinoidea", "Acervulinoidea",
"", "Acervulinoidea", "Acervulinoidea ", "Acervulinoidea", "Acervulinoidea",
"Milioloidea", "Milioloidea", "Milioloidea", "Milioloidea", "Milioloidea",
"Milioloidea", "Milioloidea", "", "", "", "", "", "", "", "",
"", "Milioloidea", "Milioloidea", "Milioloidea", "Milioloidea",
"Milioloidea", "Milioloidea", "Milioloidea", "", "", "", "",
"", "", "Milioloidea", "", "", "", "Milioloidea", "", "", "",
"", ""), Family = c("", "", "Planorbulinidae", "Acervulinoidae",
"Acervulinoidae", "Acervulinoidae", "", "Acervulinoidae", "Acervulinidae",
"Acervulinoidae", "Acervulinoidae", "Cribrolinoididae", "Cribrolinoididae",
"Cribrolinoididae", "Cribrolinoididae", "Hauerinidae", "Hauerinidae",
"Hauerinidae", "Hauerinidae", "", "", "", "", "", "", "", "",
"Cribrolinoididae", "Cribrolinoididae", "Cribrolinoididae", "Cribrolinoididae",
"Cribrolinoididae", "Cribrolinoididae", "Cribrolinoididae", "",
"", "", "", "", "", "Cribrolinoididae", "", "", "", "Cribrolinoididae",
"", "", "", "", ""), Genus = c("", "", "?Planorbulina", "Acervulina",
"Acervulina", "Acervulina", "", "Acervulina", "Acervulina", "Acervulina",
"Acervulina", "Adelosina", "Adelosina", "Adelosina", "Adelosina",
"Adelosina", "Adelosina", "Adelosina", "Quinqueloculina", "",
"", "", "", "", "", "", "", "Adelosina", "Adelosina", "Adelosina",
"Adelosina", "Adelosina", "Adelosina", "Adelosina", "", "", "",
"", "", "", "Adelosina", "", "", "", "Adelosina", "Adelosina",
"Adelosina", "", "", ""), Species = c("", "", "?Planorbulina sp . 1",
"Acervulina cf. A. mahabethi", "Acervulina cf. A. mahabeti",
"Acervulina inhaerens", "Acervulina inhaerens ", "Acervulina mabahethi",
"Acervulina mabahethi ", "Acervulina sp. 01", "Acervulina sp. 01",
"Adelosina bicornis ", "Adelosina bicornis ", "Adelosina carinatastriata",
"Adelosina carinatastriata", "Adelosina carinatastriata", "Adelosina carinatastriata",
"Adelosina carinatastriata", "Adelosina carinatastriata", "Adelosina carinatastriata ",
"Adelosina carinatastriata ", "Adelosina carinatastriata ", "Adelosina carinatastriata ",
"Adelosina carinatastriata ", "Adelosina carinatastriata ", "Adelosina carinatastriata ",
"Adelosina carinatastriata ", "Adelosina cf. A. mediterranensis",
"Adelosina crassicarinata", "Adelosina crassicarinata", "Adelosina crassicarinata",
"Adelosina crassicarinata", "Adelosina dagornae", "Adelosina dagornae",
"Adelosina dagornae", "Adelosina dagornae", "Adelosina dagornae",
"Adelosina dagornae", "Adelosina dagornae", "Adelosina dagornae",
"Adelosina echinata", "Adelosina echinata ", "Adelosina echinata ",
"Adelosina echinata ", "Adelosina honghensis", "Adelosina honghensis",
"Adelosina honghensis", "Adelosina honghensis ", "Adelosina honghensis ",
"Adelosina honghensis "), authority = c("Haynesina sp.", "Haynesina sp.",
"d'Orbigny, 1826", " Said, 1949 ", "", "Schulze, 1854", "Schulze, 1854",
" Said, 1949 ", "Said, 1949 ", "Schultze, 1854", "", "Walker & Jacob, 1798 ",
"Walker & Jacob, 1798 ", " Wiesner, 1923 ", " Wiesner, 1923 ",
" Wiesner, 1923 ", " Wiesner, 1923 ", " Wiesner, 1923 ", "Wiesner, 1923",
"Wiesner 1923 ", "Wiesner 1923 ", "Wiesner 1923 ", "Wiesner 1923 ",
"Wiesner 1923 ", "Wiesner 1923 ", "Wiesner 1923 ", "Wiesner 1923 ",
" Le Calvez & Le Calvez, 1958 ", "", "", "", "", "", "", "Levi et al. 1990 ",
"Levi et al. 1990 ", "Levi et al. 1990 ", "Levi et al. 1990 ",
"Levi et al. 1990 ", "Levi et al. 1990 ", "", "d'Orbigny, 1826",
"d'Orbigny, 1826", "d'Orbigny, 1826", "", "", "", "Lak, 1982",
"Lak, 1982", "Lak, 1982"), location = c(" Parkar and Gischler 2015 ",
"Present study", "Cherif et al. 1997", "Amao et al. 2016 PG",
"Amao_et_al_2019_Persian_Gulf_paper", "Murray 1965", " Shublak 1977 ",
"Parker and Gischler 2015", " Parkar and Gischler 2015 ", "Amao et al. 2016 PG",
"Amao_et_al_2019_Persian_Gulf_paper", " Shublak 1977 ", "Khader 2020 ",
"Al-Zamel et al 1996", "Al-Zamel et al 2009", "Parker and Gischler 2015",
"Amao et al. 2016 MP", "Amao et al. 2016 Salwa", "Amao_et_al_2019_baseline_paper",
"Al-Zamel et al. 1996 ", "Khader 1997 ", " Cherif et al. 1997 ",
"Al-Ghadban 2000 ", "Al-Zamel et al. 2009 ", "Al-Theyabi 2012b ",
"Al-Enezi et al. 2019 ", "Khader 2020 ", "Amao et al. 2016 MP",
"Al-Zamel et al 1996", "Cherif et al. 1997", "Al-Zamel & Cherif 1998",
"Al-Enezi & Frontalini 2015", "Al-Zamel et al 2009", "Al-Enezi & Frontalini 2015",
"Khader 1997 ", "Al-Ghadban 2000 ", "Al-Zamel et al. 2009 ",
"Al-Ammar 2011 ", "Al-Enezi and Frontalini 2015 ", "Khader 2020 ",
"Cherif et al. 1997", "Al-Shuaibi 1997 ", "Al-Ghadban 2000 ",
"Khader 2020 ", "Cherif et al. 1997", "Clark and Keiji 1975",
"Nabavi 2014", " Cherif et al. 1997 ", "Al-Ghadban 2000 ",
"Khader 2020 "), cases = c("X", "X", "x", "x", "x", "x", "X",
"x", "X", "x", "x", "X", "X", "x", "x", "x", "x", "x", "x", "X",
"X", "X", "X", "X", "X", "X", "X", "x", "x", "x", "x", "x", "x",
"x", "X", "X", "X", "X", "X", "X", "x", "X", "X", "X", "x", "x",
"x", "X", "X", "X")), row.names = c(NA, -50L), class = c("tbl_df",
"tbl", "data.frame"))
At the moment, my result look like Before but my target is After
Thank you in anticipation for your help.
As #hendrikvanb points our, your duplicate output rows are not only due to strings, but also incomplete data and slight differences in some of your input strings. Even if two strings contain the same information for a human reader, R treats them as different unless every single character is the same. Once we resolve this the solution is much easier.
Step 1: ensure entries with similar names have the same name
The following code begins with some simple tidying (removing excess white space, making everything lower case). It then searches your table for text that is similar and for every pair asks if you want to replace one with the other.
E.g. if you dataset contains "levi et al. 1990" and "levi et al 1990" one with a full stop and the other without, you will receive a message:
Do you want to replace "levi et al. 1990" with "levi et al 1990"?
You will also be asked the same question in reverse order. If you click 'yes' then all instances of the first will be replaced by the second in your database.
library(dplyr)
library(tidyr)
# standardise
standardized <- all_data_final %>%
rename(walltype = `Wall type (Kaminski 2014)`) %>% # first column in example data has odd name
mutate_all(as.character) %>% # ensures all columns are string not factor
mutate_all(trimws) %>% # leading and trailing white space
mutate_all(function(x){gsub(" +"," ",x)}) %>% # remove internal duplicate spaces
mutate_all(tolower) %>% # cast everything to lower
mutate(row = row_number())
# prompt user to merge text that is very close together
tollerance = 2
cols <- c("walltype", "Order", "Superfamily", "Family", "Genus", "Species", "authority", "location")
for(col in cols){
unique_vals = standardized[[col]] %>% unique() %>% sort()
for(val in unique_vals){
for(val2 in unique_vals){
# check if text strings are within edit distance of each other
if(adist(val, val2) > 0 & adist(val, val2) <= tollerance){
msg = paste0("Do you want [", val, "] replaced with [", val2, "] ?")
ans = FALSE
ans = askYesNo(msg) # ask user for every pair of close values
if(ans)
standardized <- mutate_all(standardized, function(x){ifelse(x == val, val2, x)})
}
}
}
}
You can control the sensitivity of this check by adjusting the tollerance parameter. You can think of it as the number of characters between the correct text and a spelling mistake.
Step 2: keep category text information where available
The goal here is to ensure that if one record of the species has an order, family, genus, or authority then this appears on the final table. We can do this by asking for the maximum order/family/genus per species.
When working with text, max returns the last record alphabetically. Blank or white space gets sorted to the top first, hence we must use max as min will return empty text fields.
The code for this is merged into step 3.
Step 3: keep case mark where available
By converting the case column to numeric, we can summarise across cases looking for a maximum value of 1. In some cases NA or NULL gets treated as -Inf so we also handle this.
The following code resolves step 2 and 3 in the same summarise_all statement.
# collapse
final_result <- standardized %>%
mutate(cases = ifelse(!is.na(cases), 1, 0)) %>%
pivot_wider(names_from = location, values_from = cases) %>%
group_by(Species) %>%
summarise_all(max, na.rm = TRUE) %>% # hack, ideally we'd handle strings and numbers differently
mutate_all(function(x){ifelse(is.infinite(x), NA, x)}) # gets rid of -Inf caused by summarise_all
Here is the dput output I get from this code:
structure(list(Species = c("", "?planorbulina sp . 1", "acervulina cf. a. mahabethi",
"acervulina inhaerens", "acervulina mabahethi", "acervulina sp. 01",
"adelosina bicornis", "adelosina carinatastriata", "adelosina cf. a. mediterranensis",
"adelosina crassicarinata", "adelosina dagornae", "adelosina echinata",
"adelosina honghensis"), walltype = c("", "hyaline", "hyaline",
"hyaline", "hyaline", "hyaline", "", "porcelaneous (imperforate)",
"porcelaneous (imperforate)", "porcelaneous (imperforate)", "porcelaneous (imperforate)",
"porcelaneous (imperforate)", "porcelaneous (imperforate)"),
Order = c("", "rotaliida", "rotaliida", "rotaliida", "rotaliida",
"rotaliida", "", "miliolida", "miliolida", "miliolida", "miliolida",
"miliolida", "miliolida"), Superfamily = c("", "planorbulinoidea",
"acervulinoidea", "acervulinoidea", "acervulinoidea", "acervulinoidea",
"milioloidea", "milioloidea", "milioloidea", "milioloidea",
"milioloidea", "milioloidea", "milioloidea"), Family = c("",
"planorbulinidae", "acervulinidae", "acervulinidae", "acervulinidae",
"acervulinidae", "cribrolinoididae", "hauerinidae", "cribrolinoididae",
"cribrolinoididae", "cribrolinoididae", "cribrolinoididae",
"cribrolinoididae"), Genus = c("", "?planorbulina", "acervulina",
"acervulina", "acervulina", "acervulina", "adelosina", "quinqueloculina",
"adelosina", "adelosina", "adelosina", "adelosina", "adelosina"
), authority = c("haynesina sp.", "d'orbigny, 1826", "said, 1949",
"schultze, 1854", "said, 1949", "schultze, 1854", "walker & jacob, 1798",
"wiesner 1923", "le calvez & le calvez, 1958", "", "levi et al. 1990",
"d'orbigny, 1826", "lak, 1982"), row = c(2L, 3L, 5L, 7L,
9L, 11L, 13L, 27L, 28L, 32L, 40L, 44L, 50L), `parkar and gischler 2015` = c(1,
NA, NA, NA, 1, NA, NA, 1, NA, NA, NA, NA, NA), `present study` = c(1,
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA), `cherif et al. 1997` = c(NA,
1, NA, NA, NA, NA, NA, 1, NA, 1, NA, 1, 1), `amao et al. 2016 mp` = c(NA,
NA, 1, NA, NA, 1, NA, 1, 1, NA, NA, NA, NA), amao_et_al_2019_persian_gulf_paper = c(NA,
NA, 1, NA, NA, 1, NA, NA, NA, NA, NA, NA, NA), `murray 1965` = c(NA,
NA, NA, 1, NA, NA, NA, NA, NA, NA, NA, NA, NA), `shublak 1977` = c(NA,
NA, NA, 1, NA, NA, 1, NA, NA, NA, NA, NA, NA), `khader 2020` = c(NA,
NA, NA, NA, NA, NA, 1, 1, NA, NA, 1, 1, 1), `al-zamel et al 1996` = c(NA,
NA, NA, NA, NA, NA, NA, 1, NA, 1, NA, NA, NA), `al-zamel et al 2009` = c(NA,
NA, NA, NA, NA, NA, NA, 1, NA, NA, 1, NA, NA), `amao et al. 2016 salwa` = c(NA,
NA, NA, NA, NA, NA, NA, 1, NA, NA, NA, NA, NA), amao_et_al_2019_baseline_paper = c(NA,
NA, NA, NA, NA, NA, NA, 1, NA, NA, NA, NA, NA), `khader 1997` = c(NA,
NA, NA, NA, NA, NA, NA, 1, NA, NA, 1, NA, NA), `al-ghadban 2000` = c(NA,
NA, NA, NA, NA, NA, NA, 1, NA, NA, 1, 1, 1), `al-theyabi 2012b` = c(NA,
NA, NA, NA, NA, NA, NA, 1, NA, NA, NA, NA, NA), `al-enezi et al. 2019` = c(NA,
NA, NA, NA, NA, NA, NA, 1, NA, NA, NA, NA, NA), `al-zamel & cherif 1998` = c(NA,
NA, NA, NA, NA, NA, NA, NA, NA, 1, NA, NA, NA), `al-enezi & frontalini 2015` = c(NA,
NA, NA, NA, NA, NA, NA, NA, NA, 1, 1, NA, NA), `al-ammar 2011` = c(NA,
NA, NA, NA, NA, NA, NA, NA, NA, NA, 1, NA, NA), `al-enezi and frontalini 2015` = c(NA,
NA, NA, NA, NA, NA, NA, NA, NA, NA, 1, NA, NA), `al-shuaibi 1997` = c(NA,
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 1, NA), `clark and keiji 1975` = c(NA,
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 1), `nabavi 2014` = c(NA,
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 1)), class = c("tbl_df",
"tbl", "data.frame"), row.names = c(NA, -13L))
...even if you decide to use just the species column ignoring every other column.e. Species, location and cases to pivot wide, it still doesn't help.
Actually, with minimal wrangling, it does help.
This is more complex than your comment appear to suggest.
I don't believe it is:
# load libraries
library(tidyverse)
# define data using the structure posted in the initial question
# create all_data_final_wider by taking all_data_final %>% remove all
# leading/trailing white space %>% convert cases column to lowercase %>% select
# columns to retain %>% remove exact duplicates %>% pivot from long to wide
all_data_final_wider <- all_data_final %>%
mutate_all(str_squish) %>%
mutate(cases = str_to_lower(cases)) %>%
select(Species, location, cases) %>%
distinct() %>%
pivot_wider(names_from = location, values_from = cases)
# prove that there are as many rows in all_data_final_wider as there are
# distinct spellings of the Species column
nrow(all_data_final_wider) == length(unique(all_data_final_wider$Species))
#> [1] TRUE
So I stand by my comments:
You'll need to fix these and all other inconsistencies in the input data if you expect to get sensible results from pivot_wider()

r: create data frame with all possible options and number of variable combinations

This question might be obvious or asked already, but I can't find a solution:
I want to create a data frame with all possible combinations (and number of variables) such that it looks like the following example:
dataframe <- data.frame(variable = 1:4,
a = c("gender", NA, NA, NA),
b = c("age", NA, NA, NA),
c = c("city", NA, NA, NA),
d = c("education", NA, NA, NA),
e = c("gender", "age", NA, NA),
f = c("gender", "city", NA, NA),
g = c("gender", "education", NA, NA),
h = c("age", "city", NA, NA),
i = c("age", "education", NA, NA),
j = c("city", "education", NA, NA),
k = c("gender", "age", "city", NA),
l = c("gender", "age", "education", NA),
m = c("gender", "city", "education", NA),
n = c("gender", "age", "city", "education"))
I have too many variables, so it's not worth writing it out, and I want to avoid errors. Thank you for helping!
Here is an option with combn. Get the vector of variable names, loop through the sequence of the vector, apply the combn on the vector with m specified as the sequence from the loop, convert to data.frame and cbind all the list elements together. The cbind.fill from rowr is suitable to fill with NA for list elements that have less number of rows than the maximum row data.frame
library(rowr)
res <- do.call(cbind.fill, c(fill = NA, lapply(seq_along(v1), function(i) {
m1 <- combn(v1, i)
if(is.vector(m1)) as.data.frame.list(m1) else as.data.frame(m1)})))
colnames(res) <- letters[seq_along(res)]
Or as #Moody_Mudskipper suggested,
res1 <- do.call(cbind.fill, c(fill = NA, lapply(seq_along(v1), function(i) combn(v1, i))))
colnames(res1) <- letters[seq_len(ncol(res1))]
data
v1 <- c('gender', 'age', 'city', 'education')

Resources