I'd like to add rows to a dataframe based on a vector within the dataframe. Here are the dataframes (df2 is the one I'd like to add rows to; df1 is the one I'd like to take the rows from):
ID=c(1:5)
x=c(rep("a",3),rep("b",2))
y=c(rep(0,5))
df1=data.frame(ID,x,y)
df2=df1[2:4,1:2]
df2$y=c(5,2,3)
df1
ID x y
1 1 a 0
2 2 a 0
3 3 a 0
4 4 b 0
5 5 b 0
df2
ID x y
2 2 a 5
3 3 a 2
4 4 b 3
I'd like to add to df2 any rows that aren't in df1, based on the ID vector. so my output dataframe would look like this:
ID x y
1 b 0
5 b 0
2 a 5
3 a 2
4 b 3
Can anyone see a way of doing this neatly, please? I need to do it for a lot of dataframes, all with different numbers of rows. I've tried using merge or rbind but I haven't been able to work out how to do it based on the vector.
Thank you!
A solution with dplyr:
bind_rows(df2,anti_join(df1,df2,by="ID"))
# ID x y
#1 2 a 5
#2 3 a 2
#3 4 b 3
#4 1 a 0
#5 5 b 0
You can do the following:
missingIDs <- which(!df1$ID %in% df2$ID) #check which df1 ID's are not in df2, see function is.element()
df.toadd <- df1[missingIDs,] #define the data frame to add to df2
result <- rbind(df.toadd, df2) #use rbind to add it
result
ID x y
1 1 a 0
5 5 b 0
2 2 a 5
3 3 a 2
4 4 b 3
What about this one-liner?
rbind(df2, df1[!df1$ID %in% df2$ID,])
ID x y
2 2 a 5
3 3 a 2
4 4 b 3
1 1 a 0
5 5 b 0
Related
I'm having trouble figuring out how to do the opposite of the answer to this question (and in R not python).
Count the amount of times value A occurs with value B
Basically I have a dataframe with a lot of combinations of pairs of columns like so:
df <- data.frame(id1 = c("1","1","1","1","2","2","2","3","3","4","4"),
id2 = c("2","2","3","4","1","3","4","1","4","2","1"))
I want to count, how often all the values in column A occur in the whole dataframe without the values from column B. So the results for this small example would be the output of:
df_result <- data.frame(id1 = c("1","1","1","2","2","2","3","3","4","4"),
id2 = c("2","3","4","1","3","4","1","4","2","1"),
count = c("4","5","5","3","5","4","2","3","3","3"))
The important criteria for this, is that the final results dataframe is collapsed by the pairs (so in my example rows 1 and 2 are duplicates, and they are collapsed and summed by the total frequency 1 is observed without 2). For tallying the count of occurances, it's important that both columns are examined. I.e. order of columns doesn't matter for calculating the frequency - if column A has 1 and B has 2, this counts the same as if column A has 2 and B has 1.
I can do this very slowly by filtering for each pair, but it's not really feasible for my real data where I have many many different pairs.
Any guidance is greatly appreciated.
First paste the two id columns together to id12 for later matching. Then use sapply to go through all rows to see the records where id1 appears in id12 but id2 doesn't. sum that value and only output the distinct records. Finally, remove the id12 column.
library(dplyr)
df %>% mutate(id12 = paste0(id1, id2),
count = sapply(1:nrow(.),
function(x)
sum(grepl(id1[x], id12) & !grepl(id2[x], id12)))) %>%
distinct() %>%
select(-id12)
Or in base R completely:
id12 <- paste0(df$id1, df$id2)
df$count <- sapply(1:nrow(df), function(x) sum(grepl(df$id1[x], id12) & !grepl(df$id2[x], id12)))
df <- df[!duplicated(df),]
Output
id1 id2 count
1 1 2 4
2 1 3 5
3 1 4 5
4 2 1 3
5 2 3 5
6 2 4 4
7 3 1 2
8 3 4 3
9 4 2 3
10 4 1 3
A full tidyverse version:
library(tidyverse)
df %>%
mutate(id = paste(id1, id2),
count = map(cur_group_rows(), ~ sum(str_detect(id, id1[.x]) & str_detect(id, id2[.x], negate = T))))
A more efficient approach would be to work on a tabulation format:
tab = crossprod(table(rep(seq_len(nrow(df)), ncol(df)), c(df$id1, df$id2)))
#tab
#
# 1 2 3 4
# 1 7 3 2 2
# 2 3 6 1 2
# 3 2 1 4 1
# 4 2 2 1 5
So, now, we have the times each value appears with another (irrespectively of their order in the two columns). Here on, we need a way to subset the above table by each pair and subtract the value of their cooccurence from the value of each id's total appearance.
Make a grid of all combinations:
gr = expand.grid(id1 = colnames(tab), id2 = rownames(tab), stringsAsFactors = FALSE)
Create 2-column matrices to subset the table:
id1.ij = cbind(match(gr$id1, colnames(tab)),
match(gr$id1, rownames(tab)))
id2.ij = cbind(match(gr$id1, colnames(tab)),
match(gr$id2, rownames(tab)))
Subtract the respective values:
cbind(gr, count = tab[id1.ij] - tab[id2.ij])
# id1 id2 count
#1 1 1 0
#2 2 1 3
#3 3 1 2
#4 4 1 3
#5 1 2 4
#6 2 2 0
#7 3 2 3
#8 4 2 3
#9 1 3 5
#10 2 3 5
#11 3 3 0
#12 4 3 4
#13 1 4 5
#14 2 4 4
#15 3 4 3
#16 4 4 0
Of course, if we do not need the full grid of values, we can set:
gr = unique(df)
which results in:
# id1 id2 count
#1 1 2 4
#3 1 3 5
#4 1 4 5
#5 2 1 3
#6 2 3 5
#7 2 4 4
#8 3 1 2
#9 3 4 3
#10 4 2 3
#11 4 1 3
I have data frames with lists of elements such as NAMES. There are different names in dataframes, but most of them match together. I'd like to combine all of them in one list in which I'd see whether some names are missing from any of df.
DATA sample for df1:
X x
1 1 rh_Structure/Focus_S
2 2 rh_Structure/Focus_C
3 3 lh_Structure/Focus_S
4 4 lh_Structure/Focus_C
5 5 RH_Type-Function-S
6 6 RH_REFERENT-S
and for df2
X x
1 1 rh_Structure/Focus_S
2 2 rh_Structure/Focus_C
3 3 lh_Structure/Focus_S
4 4 lh_Structure/Focus_C
5 5 UCZESTNIK
6 6 COACH
and expected result would be:
NAME. df1 df2
1 COACH NA 6
2 lh_Structure/Focus_C 4 4
3 lh_Structure/Focus_S 3 3
4 RH_REFERENT-S 6 NA
5 rh_Structure/Focus_C 2 2
6 rh_Structure/Focus_S 1 1
7 RH_Type-Function-S 5 NA
8 UCZESTNIK NA 5
I can do that with merge.data.frame(df1,df2,by = "x", all=T),
but the I can't do it with more df with similar structure. Any help would be appreciated.
It might be easier to work with this in a long form. Just rbind all the datasets below one another with a flag for which dataset they came from. Then it's relatively straightforward to get a tabulation of all the missing values (and as an added bonus, you can see if you have any duplicates in any of the source datasets):
dfs <- c("df1","df2")
dfall <- do.call(rbind, Map(cbind, mget(dfs), src=dfs))
table(dfall$x, dfall$src)
# df1 df2
# COACH 0 1
# lh_Structure/Focus_C 1 1
# lh_Structure/Focus_S 1 1
# RH_REFERENT-S 1 0
# rh_Structure/Focus_C 1 1
# rh_Structure/Focus_S 1 1
# RH_Type-Function-S 1 0
# UCZESTNIK 0 1
I have two data.frame df1 and df2 .
df1=data.frame(id=c(1,2,2),var1=c(3,5,5),var3=c(2,3,4))
df2=data.frame(id=c(1,1,2,2),var1=c('NONE','NONE','NONE','NONE'),var3=c(2,4,6,5))
now I want to merge to one data.frame. First, I should change the df2$var1. re encoding the df2$var1 with df1$var1 when df2$id match with df1$id. For example, df1$id=1 df1$var1=3 then df2$id=1 and df2$var1=3, so the result should like this:
df1=data.frame(id=c(1,2,2),var1=c(3,5,5),var3=c(2,3,4)).
df2=data.frame(id=c(1,1,2,2),var1=c(3,3,5,5),var3=c(2,4,6,5))
secondly, I want to merge two data.frame and delete the same one.the result should like this:
df=data.frame(id=c(1,1,2,2,2,2),var1=c(3,3,5,5,5,5),var2=c(2,4,3,4,6,5))
Sorry, it's my first to use stackoverflow. And most importantly,English isn't my native language.
library(dplyr)
union_all(df1, df2) %>%
distinct() %>%
arrange(id, var1)
id var1 var3
1 1 3 2
2 1 3 4
3 2 5 3
4 2 5 4
5 2 5 6
6 2 5 5
First,I use dplyr::union,then I found that the order is disrupted.
So,finally I use union_all, then rank it
I think this is what you want.
library(sqldf)
sqldf("select b.id, a.var1, b.var3 from df1 a left join df2 b on a.id = b.id")
id var1 var3
1 1 3 2
2 1 3 4
3 2 5 5
4 2 5 6
5 2 5 5
6 2 5 6
This is the same as the example you gave of your desired result, except for the 3rd column of the 3rd and 4th row. I believe that is due to a typo in your example, however if I am mistaken about this please let me know (and just explain why those values would be different and I will update my answer accordingly).
By the way, there are multiple ways to do this, but I find this one to be quick and easy.
with merge:
df2$var1 <- df1[df2$id,'var1'];
df2
id var1 var3
1 1 3 2
2 1 3 4
3 2 5 6
4 2 5 5
df <- merge(df1, df2, by='id')[-2:-3]
df
id var1.y var3.y
1 1 3 2
2 1 3 4
3 2 5 6
4 2 5 5
5 2 5 6
6 2 5 5
I'm trying to select the column with the highest value for each row in a data.frame. So for instance, the data is set up as such.
> df <- data.frame(one = c(0:6), two = c(6:0))
> df
one two
1 0 6
2 1 5
3 2 4
4 3 3
5 4 2
6 5 1
7 6 0
Then I'd like to set another column based on those rows. The data frame would look like this.
> df
one two rank
1 0 6 2
2 1 5 2
3 2 4 2
4 3 3 3
5 4 2 1
6 5 1 1
7 6 0 1
I imagine there is some sort of way that I can use plyr or sapply here but it's eluding me at the moment.
There might be a more efficient solution, but
ranks <- apply(df, 1, which.max)
ranks[which(df[, 1] == df[, 2])] <- 3
edit: properly spaced!
Consider these two df examples
df1=data.frame(names=c('a','b','c'),value=1:3)
df2=data.frame(names=c('c','a','b'),value=1:3)
so that
> df1
names value
1 a 1
2 b 2
3 c 3
> df2
names value
1 c 1
2 a 2
3 b 3
Now, I would like to sort the df1 to the same order as the names column in df2, to obtain
names value
c 3
a 1
b 2
How can I achieve this?
try
df1[match(df2$names,df1$names),]
> df1[match(df2$names,df1$names),]
names value
3 c 3
1 a 1
2 b 2