I have two dataframes. I want to replace the ids in dataframe1 with generic ids. In dataframe2 I have mapped the ids from dataframe1 with the generic ids.
Do I have to merge the two dataframes and after it is merged do I delete the column I don't want?
Thanks.
With dplyr
library(dplyr)
left_join(df1, df2, by = 'ids')
We can use merge and then delete the ids.
dataframe1 <- data.frame(ids = 1001:1010, variable = runif(min=100,max = 500,n=10))
dataframe2 <- data.frame(ids = 1001:1010, generics = 1:10)
result <- merge(dataframe1,dataframe2,by="ids")[,-1]
Alternatively we can use match and replace by assignment.
dataframe1$ids <- dataframe2$generics[match(dataframe1$ids,dataframe2$ids)]
Subsetting data frames isn't very difficult in R: hope this helps, you didn't provide much code so I hope this will be of help to you:
#create 4 random columns (vectors) of data, and merge them into data frames:
a <- rnorm(n=100,mean = 0,sd=1)
b <- rnorm(n=100,mean = 0,sd=1)
c <- rnorm(n=100,mean = 0,sd=1)
d<- rnorm(n=100,mean = 0,sd=1)
df_ab <- as.data.frame(cbind(a,b))
df_cd <- as.data.frame(cbind(c,d))
#if you want column d in df_cd to equal column a in df_ab simply use the assignment operator
df_cd$d <- df_ab$a
#you can also use the subsetting with square brackets:
df_cd[,"d"] <- df_ab[,"a"]
I have a list of data.frames. Some of the data.frames are redundant and among the non-redundant ones the rows (indicated by an id column) are not identical but do overlap:
set.seed(2)
ids.1.2 <- paste0("id",sample(30,10,replace = F))
ids.3.4 <- paste0("id",sample(30,20,replace = F))
df.1 <- data.frame(id = ids.1.2,matrix(rnorm(100),10,10,dimnames = list(NULL,paste0("s.1.2:",1:10))))
df.2 <- df.1
df.3 <- data.frame(id = ids.3.4,matrix(rnorm(300),20,15,dimnames = list(NULL,paste0("s.3.4:",1:15))))
df.4 <- df.3
df.list <- list(df.1, df.2, df.3, df.4)
So in this case, df.1 and df.2 are identical, and so are df.3 and df.4, and both sets intersect on ids:
"id6" "id21" "id17" "id5" "id24" "id11" "id12
Is there a purrr::reduce or similar way to combine this list into a single data.frame with unique columns and the intersecting id's?
I'd use:
purrr::reduce(df.list, dplyr::inner_join,by = "id")
If all data.frames had unique columns. But in my case using this adds the .x, .y, ... suffices to the redundant columns.
I'm not sure if that's what you what, but I'd remove identical dataframes at first and then combine the rest. It's not a pretty solution and you may adjust it here and there, but if I got it right, it gives you your desired result. You might want to include a line that removes identical combinations in the combinations dataframe, so that you can be sure that there are no errors when removing the identical dfs from your list.
library(tidyr)
library(dplyr)
# create all possible combinations
names(df.list) <- 1:length(df.list)
combinations <- crossing(names(df.list), names(df.list))
colnames(combinations) <- c("v1", "v2")
# remove self-combinations
combinations <- combinations[!combinations$v1 == combinations$v2,]
# check which cases are identical
combinations$check <- sapply(1:nrow(combinations), function(x){combinations[x,] <- identical(df.list[[combinations$v1[x]]], df.list[[combinations$v2[x]]])})
combinations <- combinations[combinations$check == T,]
# remove identical cases
for(i in 1:length(df.list)){
if(combinations$v1[i] == names(df.list)[i] & combinations$v1[i] %in% names(df.list)){df.list[i] <- NULL}
}
# combine dataframes
bind_rows(df.list)
So I'm trying to update or retain a dataframe df2 based on a certain condition of another data frame df1.
For Example, Assuming df1 get updated for every 30 seconds, so if the number of rows in df1 i.e nrow(df1)!= 0 then df2 <- df1 else if retain the previous values in df2.
NOTE: On the first iteration, df2 can be initialized to a NULL dataframe.
Following is my code
#Initializing df2 as empty dataframe
df2 <- data.frame(weight = integer(),stringAsFactors = FALSE)
#Condition to check if number of rows in df1 != 0
if(nrow(df1) != 0){
df2 <- df1
temp <- df1 #Another copy of df1
}
else{
df2 <- temp
}
Here I created an another data frame called temp to keep a copy of df1 so that it can be used when nrow(df1) == 0. I don't know if the usage of temp is correct or not.
This code will create an empty dataframe named df2. If nrow(df1)>0 then it will effectively assign the contents of df1 to df2. If nrow(df1)==0 then df2 remains empty.
df2 <- data.frame()
if(nrow(df1)>0) df2 <- df1
I have a hard time imagining why this is useful. If, perhaps, you intended to "grow" df2 by appending on whatever is in df1 - which might be more common - then do something like this:
df2 <- data.frame()
if(nrow(df1)>0) df2 <- rbind(df2, df1)
I'd like to use a data frame (Df2) to recode the variables of another data frame (Df1), so that the end result is a data frame that contains text like local/international rather than 1s/2s (Df3). Missingness is present in the Df1 data frame, and I'd like to make sure it's represented as NA.
This is a minimal working example, the actual data set contains more than a hundred variables (all of which are of the character class) with between one and fifteen levels. Any help would be much appreciated.
Starting point (dfs)
Df1 <- data.frame("buyer_Q1"=c(1,2,1,1),"seller_Q2"=c(2,1,3,2),"price_Q1_2"=c(2,5,7,5))
Df2 <- data.frame("NameOfVariable"=c("buyer_Q1","buyer_Q1","seller_Q2","seller_Q2","seller_Q2","price_Q1_2","price_Q1_2","price_Q1_2"),"VariableLevel"=c(1,2,1,2,3,2,5,7),"VariableDef"=c("local","internat","local","internat","NA","50-100K","100-200K","200+K"))
Desired outcome (df)
Df3 <- data.frame("buyer_Q1"=c("local","internat","local","local"),"seller_Q2"=c("internat","local","NA","internat"),"price_Q1_2"=c("50-100K","100-200K","200+K","100-200K"))
Thoughts, not really code, so far: (If there's a match between a row of the df2 NameOfVariable and a df1 variable name, as well as a match between a row of df2 VariableLevel and a df1 observation, then paste the corresponding row of df2 VariableDef into df1. Wondering if you can use if statements for it.)
if (Df2["NameOfVariable"]==names(Df1))
{
if (Df2["VariableLevel"]==Df1[ ])
{
Df1[ ] <- paste0("VariableDef")
}
}
Here is on method in base R using match and Map. Map applies a function to corresponding list elements. Here, there are two list elements: Df1 and a list that is composed of the second and third columns of Df2, split by column 1. The second list is reordered to match the order of the names in Df1.
The applied function matches elements in a column Df1 to the corresponding column in the second argument and uses it as an index to return the corresponding name of the Df2 argument. Map returns a list, which is converted to a data.frame with the function of the same name.
data.frame(Map(function(x, y) y[[2]][match(x, y[[1]])],
Df1,
split(Df2[2:3], Df2[1])[names(Df1)]))
this returns
buyer_Q1 seller_Q2 price_Q1_2
1 local internat 50-100K
2 internat local 100-200K
3 local NA 200+K
4 local internat 100-200K
Solution using loop and factors. Be careful. Results seem equivalent but they are not. The function fun return data frame with factors. If needed you can convert them to characters.
Df1 <- data.frame("buyer_Q1"=c(1,2,1,1),"seller_Q2"=c(2,1,3,2),"price_Q1_2"=c(2,5,7,5))
Df2 <- data.frame("NameOfVariable"=c("buyer_Q1","buyer_Q1","seller_Q2","seller_Q2","seller_Q2","price_Q1_2","price_Q1_2","price_Q1_2"),"VariableLevel"=c(1,2,1,2,3,2,5,7),"VariableDef"=c("local","internat","local","internat","NA","50-100K","100-200K","200+K"))
Df3 <- data.frame("buyer_Q1"=c("local","internat","local","local"),"seller_Q2"=c("internat","local","NA","internat"),"price_Q1_2"=c("50-100K","100-200K","200+K","100-200K"))
fun <- function(df, mdf) {
for (varn in names(df)) {
dat <- mdf[mdf$NameOfVariable == varn & !is.na(mdf$VariableDef),]
df[[varn]] <- factor(df[[varn]], dat$VariableLevel, dat$VariableDef)
}
return(df)
}
fun(Df1, Df2)
Df3
A solution from dplyr and tidyr. The code will work fine even with warning messages because the columns are in factor. If you don't want to see any warning messages, set stringsAsFactors = FALSE when creating the data frame like the example I provided.
library(dplyr)
library(tidyr)
Df3 <- Df1 %>%
mutate(ID = 1:n()) %>%
gather(NameOfVariable, VariableLevel, -ID) %>%
left_join(Df2, by = c("NameOfVariable", "VariableLevel")) %>%
select(-VariableLevel) %>%
spread(NameOfVariable, VariableDef) %>%
select(-ID)
Df3
buyer_Q1 price_Q1_2 seller_Q2
1 local 50-100K internat
2 internat 100-200K local
3 local 200+K NA
4 local 100-200K internat
DATA
Df1 <- data.frame("buyer_Q1"=c(1,2,1,1),
"seller_Q2"=c(2,1,3,2),
"price_Q1_2"=c(2,5,7,5),
stringsAsFactors = FALSE)
Df2 <- data.frame("NameOfVariable"=c("buyer_Q1","buyer_Q1","seller_Q2","seller_Q2","seller_Q2","price_Q1_2","price_Q1_2","price_Q1_2"),
"VariableLevel"=c(1,2,1,2,3,2,5,7),
"VariableDef"=c("local","internat","local","internat","NA","50-100K","100-200K","200+K"),
stringsAsFactors = FALSE)
I have a data frame and I would like to split the first column into two columns but the separate pattern is similar to others and I only want to split the pattern located on number 4.
data frame:
TCGA-TS-A7P1-01A-41D-A39S-05 0.8637304
TCGA-NQ-A57I-01A-11D-A34E-05 0.7812147
TCGA-3H-AB3O-01A-11D-A39S-05 0.8963944
TCGA-LK-A4O2-01A-11D-A34E-05 0.6942843
TCGA-MQ-A4LI-01A-11D-A34E-05 0.8882558
desired output:
TCGA-TS-A7P1-01A 41D-A39S-05 0.8637304
TCGA-NQ-A57I-01A 11D-A34E-05 0.7812147
TCGA-3H-AB3O-01A 11D-A39S-05 0.8963944
TCGA-LK-A4O2-01A 11D-A34E-05 0.6942843
TCGA-MQ-A4LI-01A 11D-A34E-05 0.8882558
I tried:
sapply(strsplit(as.character(df$ID), "-"), '[', 1:4)
However, it is not the desired output above that I want. Thank you very much.
It seems all the elements of your first column are of the same length so one simple way could be:
df <- data.frame(col1 = c("TCGA-TS-A7P1-01A-41D-A39S-05","TCGA-NQ-A57I-01A-11D-A34E-05","TCGA-3H-AB3O-01A-11D-A39S-05"),
col2 = c(0.8637304,0.7812147,0.8963944), stringsAsFactors = FALSE)
df$col1bis <- substr(df$col1,18,28)
df$col1 <- substr(df$col1,1,16)
Then I reaggange the order of the columns:
df <- df[, c(1,3,2)]
resulting in:
> df
col1 col1bis col2
1 TCGA-TS-A7P1-01A 41D-A39S-05 0.8637304
2 TCGA-NQ-A57I-01A 11D-A34E-05 0.7812147
3 TCGA-3H-AB3O-01A 11D-A39S-05 0.8963944
I tried this one and it worked well.
df <- cbind(df[,1],df)
df[,1] <- substr(df[,1],1,16)
df[,2] <- substr(df[,2],18,28)