how can I split the following data.frame
df <- data.frame(var1 = c("a", 1, 2, 3, "a", 1, 2, 3, 4, 5, 6, "a", 1, 2), var2 = 1:14)
into lists of / groups of
a 1
1 2
2 3
3 4
a 5
1 6
2 7
3 8
4 9
5 10
6 11
a 12
1 13
2 14
So basically, value "a" in column 1 is the tag / identifier I want to split the data frame on. I know about the split function but that means I have to add another column and since, as can be seen from my example, the size of the groups can vary I do not know how to automatically create such a dummy column to fit my needs.
Any ideas on that?
Cheers,
Sven
You could find which values of the indexing vector equal "a", then create a grouping variable based on that and then use split.
df[,1] == "a"
# [1] TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE
#[13] FALSE FALSE
cumsum(df[,1] == "a")
# [1] 1 1 1 1 2 2 2 2 2 2 2 3 3 3
split(df, cumsum(df[,1] == "a"))
#$`1`
# var1 var2
#1 a 1
#2 1 2
#3 2 3
#4 3 4
#
#$`2`
# var1 var2
#5 a 5
#6 1 6
#7 2 7
#8 3 8
#9 4 9
#10 5 10
#11 6 11
#
#$`3`
# var1 var2
#12 a 12
#13 1 13
#14 2 14
You could create a loop that loops through the entire first column of the data frame and saves the positions of non-numeric characters in a vector. Thus, you'd have something like:
data <- df$var1 #this gives you a vector of the values you'll sort through
positions <- c()
for (i in seq(1:length(data))){
if (is.numeric(data[i]) == TRUE) {
#nothing
}
else positions <- append(positions, i) #saves the positions of the non-numeric characters
}
With those positions, you shouldn't have a problem accessing splitting up the data frame from there. It's just a matter of using sequences between the values in the position vector.
Related
How can I insert blank rows every 3 existing rows in a data frame?
After a web scraping process I get a dataframe with the information I need, however the final excel format requires that I add a blank row every 3 rows. I have searched the web for help but have not found a solution yet.
With hypothetical data, the structure of my data frame is as follows:
mi_df <- data.frame(
"ID" = rep(1:3,c(3,3,3)),
"X" = as.character(c("a", "a", "a", "b", "b", "b", "c", "c", "c")),
"Y" = seq(1,18, by=2)
)
mi_df
ID X Y
1 1 a 1
2 1 a 3
3 1 a 5
4 2 b 7
5 2 b 9
6 2 b 11
7 3 c 13
8 3 c 15
9 3 c 17
The result I hope for is something like this
ID X Y
1 1 a 1
2 1 a 3
3 1 a 5
4
5 2 b 7
6 2 b 9
7 2 b 11
8
9 3 c 13
10 3 c 15
11 3 c 17
If the indices of a data frame contain NA, then the output will have NA rows. So my goal is to create a vector like 1 2 3 NA 4 5 6 NA ... and set it as the indices of mi_df.
cut <- rep(1:(nrow(mi_df)/3), each = 3)
mi_df[sapply(split(1:nrow(mi_df), cut), c, NA), ]
# ID X Y
# 1 1 a 1
# 2 1 a 3
# 3 1 a 5
# NA NA <NA> NA
# 4 2 b 7
# 5 2 b 9
# 6 2 b 11
# NA.1 NA <NA> NA
# 7 3 c 13
# 8 3 c 15
# 9 3 c 17
# NA.2 NA <NA> NA
If nrow(mi_df) is not a multiple of 3, then the following is a general solution:
# Version 1
cut <- rep(1:ceiling(nrow(mi_df)/3), each = 3, len = nrow(mi_df))
mi_df[Reduce(c, lapply(split(1:nrow(mi_df), cut), c, NA)), ]
# Version 2
cut <- rep(1:ceiling(nrow(mi_df)/3), each = 3, len = nrow(mi_df))
mi_df[Reduce(function(x, y) c(x, NA, y), split(1:nrow(mi_df), cut)), ]
Don't mind the NA in the output because some functions which write data to an excel file have an optional argument controls if NA values are converted to strings or be empty. E.g.
library(openxlsx)
write.xlsx(df, "test.xlsx", keepNA = FALSE) # defaults to FALSE
tmp <- split(mi_df, rep(1:(nrow(mi_df) / 3), each = 3))
# or split(mi_df, ggplot2::cut_width(seq_len(nrow(mi_df)), 3, center = 2))
do.call(rbind, lapply(tmp, function(x) { x[4, ] <- NA; x }))
ID X Y
1.1 1 a 1
1.2 1 a 3
1.3 1 a 5
1.4 NA <NA> NA
2.4 2 b 7
2.5 2 b 9
2.6 2 b 11
2.4.1 NA <NA> NA
3.7 3 c 13
3.8 3 c 15
3.9 3 c 17
3.4 NA <NA> NA
You can make empty rows like you show by assigning an empty character vector ("") instead of NA, but this will convert your columns to character, and I wouldn't recommend it.
My recommendation is somewhat different from all the other answers: don't make a mess of your dataset inside R . Use the existing packages to write to designated rows in an Excel workbook. For example, with the package xlConnect, the method writeWorksheet (called from writeWorksheetToFile ) includes these arguments:
object The workbook to write to data Data to write
sheet The name or index of the sheet to write to
startRow Index of the first row to write to. The default is startRow = 1.
startCol Index of the first column to write to. The default is startCol = 1.
So if you simply set up a loop that writes 3 rows of your data file at a time, then moves the row index down by 4 and writes the next 3 rows, etc., you're all set.
Here's one method.
Splits into list by ID, adds empty row, then binds list back into data frame.
mi_df2 <- do.call(rbind,Map(rbind,split(mi_df,mi_df$ID),rep("",3)))
rownames(mi_df2) <- NULL
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 2 years ago.
Improve this question
I have a dataframe (df) in R. All columns are character class.
> dim(df)
[1] 1000 6
I'm trying to remove rows where df$entry == c("7795").
entries_to_remove <- subset(df, entry == c("7795"))
> dim(entries_to_remove)
[1] 35 6
So as you can see above, I have 35 entries to remove from the data frame. However, when I go to remove these using subset, it doesn't remove the correct amount:
entries_to_remove <- subset(df, entry != c("7795"))
> dim(entries_to_remove)
[1] 648 6
The above command was supposed to remove 35 entries, but instead it removed 352. Does anyone know why this might be happening?
Here's another solution, which takes up just one line:
df[-which(grepl("7995", apply(df, 1, paste0, collapse = " "))),]
RESULT:
v1 entry1 entry2 entry3
2 2 5 5 2
3 3 2 4 2
4 4 2 3 1
6 6 1 2 1
7 7 2 4 4
8 8 4 5 5
9 9 5 1 5
DATA:
set.seed(121)
df <- data.frame(
v1 = 1:10,
entry1 = c(sample(1:5, 9, replace = T), 7995),
entry2 = c(sample(1:5, 4), 7995, sample(1:5, 5)),
entry3 = c(7995, sample(1:5, 9, replace = T))
)
df[2:4] <- lapply(df[2:4], as.character) # convert to character, as in your data
df
v1 entry1 entry2 entry3
1 1 1 2 7995
2 2 5 5 2
3 3 2 4 2
4 4 2 3 1
5 5 3 7995 2
6 6 1 2 1
7 7 2 4 4
8 8 4 5 5
9 9 5 1 5
10 10 7995 3 5
The above solutions didn't work, I do not think the issue is with NA. However, I solved the problem myself. It is a workaround but it worked:
# list the row numbers for the entries to remove
row_remove <- rownames(entries_to_remove )
# make a list of all the row numbers
all_rows <- 1:dim(df)[1]
# create a vector with only the rows to keep
subset_row <- all_rows[!(all_rows%in%row_remove)]
# subset the dataframe with these rows
df<- df[subset_row,]
The issue has to do with NAs, some of the other solutions will work, but the easiest and I think most inutive is just to use %in% rather than ==
entries_to_remove <- subset(df, !(entry %in% c("7795")))
entries_to_remove <- subset(df, entry %in% c("7795"))
This should explain whats happening. Notice how the ==, returns NA rather than FALSE.
> c( 5, 6, 7) == 5
[1] TRUE FALSE FALSE
> c( 5, 6, 7 , NA) == 5
[1] TRUE FALSE FALSE NA
> c( 5, 6, 7 , NA) %in% 5
[1] TRUE FALSE FALSE FALSE
and you can't subset using an NA
This question already has answers here:
Finding ALL duplicate rows, including "elements with smaller subscripts"
(9 answers)
Closed 4 years ago.
I have one question in R.
I have the following example code for a question.
> exdata <- data.frame(a = rep(1:4, each = 3),
+ b = c(1, 1, 2, 4, 5, 3, 3, 2, 3, 9, 9, 9))
> exdata
a b
1 1 1
2 1 1
3 1 2
4 2 4
5 2 5
6 2 3
7 3 3
8 3 2
9 3 3
10 4 9
11 4 9
12 4 9
> exdata[duplicated(exdata), ]
a b
2 1 1
9 3 3
11 4 9
12 4 9
I tried to use the duplicated() function to find all the duplicate records in the exdata dataframe, but it only finds a part of the duplicated records, so it is difficult to confirm intuitively whether duplicates exist.
I'm looking for a solution that returns the following results
a b
1 1 1
2 1 1
7 3 3
9 3 3
10 4 9
11 4 9
12 4 9
Can use the duplicated() function to find the right solution?
Or is there a way to use another function?
I would appreciate your help.
duplicated returns a logical vector with the length equal to the length of its argument, corresponding to the second time a value exists. It has a method for data frames, duplicated.data.frame, that looks for duplicated rows (and so has a logical vector of length nrow(exdata). Your extraction using that as a logical vector is going to return exactly those rows that have occurred once before. It WON'T however, return the first occurence of those rows.
Look at the index vector your using:
duplicated(exdata)
# [1] FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE TRUE
But you can combine it with fromLast = TRUE to get all of the occurrences of these rows:
exdata[duplicated(exdata) | duplicated(exdata, fromLast = TRUE),]
# a b
# 1 1 1
# 2 1 1
# 7 3 3
# 9 3 3
# 10 4 9
# 11 4 9
# 12 4 9
look at the logical vector for duplicated(exdata, fromLast = TRUE) , and the combination with duplicated(exdata) to convince yourself:
duplicated(exdata, fromLast = TRUE)
# [1] TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE TRUE FALSE
duplicated(exdata) | duplicated(exdata, fromLast = TRUE)
# [1] TRUE TRUE FALSE FALSE FALSE FALSE TRUE FALSE TRUE TRUE TRUE TRUE
how can I split the following data.frame
df <- data.frame(var1 = c("a", 1, 2, 3, "a", 1, 2, 3, 4, 5, 6, "a", 1, 2), var2 = 1:14)
into lists of / groups of
a 1
1 2
2 3
3 4
a 5
1 6
2 7
3 8
4 9
5 10
6 11
a 12
1 13
2 14
So basically, value "a" in column 1 is the tag / identifier I want to split the data frame on. I know about the split function but that means I have to add another column and since, as can be seen from my example, the size of the groups can vary I do not know how to automatically create such a dummy column to fit my needs.
Any ideas on that?
Cheers,
Sven
You could find which values of the indexing vector equal "a", then create a grouping variable based on that and then use split.
df[,1] == "a"
# [1] TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE
#[13] FALSE FALSE
cumsum(df[,1] == "a")
# [1] 1 1 1 1 2 2 2 2 2 2 2 3 3 3
split(df, cumsum(df[,1] == "a"))
#$`1`
# var1 var2
#1 a 1
#2 1 2
#3 2 3
#4 3 4
#
#$`2`
# var1 var2
#5 a 5
#6 1 6
#7 2 7
#8 3 8
#9 4 9
#10 5 10
#11 6 11
#
#$`3`
# var1 var2
#12 a 12
#13 1 13
#14 2 14
You could create a loop that loops through the entire first column of the data frame and saves the positions of non-numeric characters in a vector. Thus, you'd have something like:
data <- df$var1 #this gives you a vector of the values you'll sort through
positions <- c()
for (i in seq(1:length(data))){
if (is.numeric(data[i]) == TRUE) {
#nothing
}
else positions <- append(positions, i) #saves the positions of the non-numeric characters
}
With those positions, you shouldn't have a problem accessing splitting up the data frame from there. It's just a matter of using sequences between the values in the position vector.
how can I split the following data.frame
df <- data.frame(var1 = c("a", 1, 2, 3, "a", 1, 2, 3, 4, 5, 6, "a", 1, 2), var2 = 1:14)
into lists of / groups of
a 1
1 2
2 3
3 4
a 5
1 6
2 7
3 8
4 9
5 10
6 11
a 12
1 13
2 14
So basically, value "a" in column 1 is the tag / identifier I want to split the data frame on. I know about the split function but that means I have to add another column and since, as can be seen from my example, the size of the groups can vary I do not know how to automatically create such a dummy column to fit my needs.
Any ideas on that?
Cheers,
Sven
You could find which values of the indexing vector equal "a", then create a grouping variable based on that and then use split.
df[,1] == "a"
# [1] TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE
#[13] FALSE FALSE
cumsum(df[,1] == "a")
# [1] 1 1 1 1 2 2 2 2 2 2 2 3 3 3
split(df, cumsum(df[,1] == "a"))
#$`1`
# var1 var2
#1 a 1
#2 1 2
#3 2 3
#4 3 4
#
#$`2`
# var1 var2
#5 a 5
#6 1 6
#7 2 7
#8 3 8
#9 4 9
#10 5 10
#11 6 11
#
#$`3`
# var1 var2
#12 a 12
#13 1 13
#14 2 14
You could create a loop that loops through the entire first column of the data frame and saves the positions of non-numeric characters in a vector. Thus, you'd have something like:
data <- df$var1 #this gives you a vector of the values you'll sort through
positions <- c()
for (i in seq(1:length(data))){
if (is.numeric(data[i]) == TRUE) {
#nothing
}
else positions <- append(positions, i) #saves the positions of the non-numeric characters
}
With those positions, you shouldn't have a problem accessing splitting up the data frame from there. It's just a matter of using sequences between the values in the position vector.