R: how to group by nested intervals? - r

Question

I have a dataframe which looks like the following:
d b c a
1 3400 100 3 -1
2 3400 50 3 1
3 3400 100 1 -1
4 3408 50 1 1
5 3412 100 3 1
6 3423 50 1 1
7 3434 100 1 1
8 3436 100 3 1
9 3438 50 3 1
10 3445 50 1 1
11 3454 100 3 1
12 3465 100 1 1
and I want to group by column a and b based on the condition that the group starts with column c value= 3 and the group ends if the column d value is + 30 ahead of the first group entry(So the interval length = 30, but the starting point of every interval can be in another interval). Then I want to count the rows in each group.
So the expected output for this sample should be:
b a rowcount
100 -1 2 ( starting at d = 3400)
50 1 3 ( starting at d = 3400)
100 1 3 (starting at d= 3412)
50 1 2 (starting at d= 3438)
100 1 2 (starting at d= 3454)
I tried:
df<-df%>%
group_by(b,a,first(c) == 3 & lead(d) - d < 30)
summarise(number = n())
but this does not give me the desired output. Any comments are appreciated!
UPDATE: New Example:
d b c a
1 3400 100 3 1
2 3400 100 3 1
3 3400 100 1 1
4 3408 100 1 1
5 3412 100 3 1
6 3434 100 3 1
7 3436 100 1 1
8 3438 100 3 1
9 3445 100 1 1
10 3443 100 3 1
11 3444 100 1 1
12 3463 100 3 1
13 3463 100 1 1
14 3463 100 3 1
Your code gives as output:
a b count desc addition_info
<dbl> <dbl> <int> <chr> <chr>
1 1 100 5 ( starting at d = 3400) There is 3 `c == 3` in this group
2 1 100 6 ( starting at d = 3434) There is 3 `c == 3` in this group
3 1 100 3 ( starting at d = 3463) There is 2 `c == 3` in this group
but the third group is wrong, since the difference in d = 29 and therefore <30. Why is this the case? So the right output in this example should be:
a b count desc addition_info
<dbl> <dbl> <int> <chr> <chr>
1 1 100 5 ( starting at d = 3400) There is 3 `c == 3` in this group
2 1 100 9 ( starting at d = 3434) There is 3 `c == 3` in this group

Answer 1

Tweaked your example a bit, still I think this will suffice largely. (Also see notes below the code)
df <- read.table(text = " d b c a
1 3400 100 3 1
2 3400 100 3 1
3 3400 100 1 1
4 3408 100 1 1
5 3412 100 3 1
6 3434 100 3 1
7 3436 100 1 1
8 3438 100 3 1
9 3445 100 1 1
10 3443 100 3 1
11 3444 100 1 1
12 3463 100 3 1
13 3463 100 1 1
14 3463 100 3 1
15 3465 100 3 1", header = T)
#added one row in df
> df
d b c a
1 3400 100 3 1
2 3400 100 3 1
3 3400 100 1 1
4 3408 100 1 1
5 3412 100 3 1
6 3434 100 3 1
7 3436 100 1 1
8 3438 100 3 1
9 3445 100 1 1
10 3443 100 3 1
11 3444 100 1 1
12 3463 100 3 1
13 3463 100 1 1
14 3463 100 3 1
15 3465 100 3 1
Now follow this strategy
library(tidyverse)
library(data.table) # for rleid()
df %>% mutate(r = row_number()) %>%
group_by(b, a) %>% mutate(grp_no = rleid(accumulate(d, ~ifelse(.y - .x > 30, .y, .x)))) %>%
group_by(b, a, grp_no) %>%
summarise(row_count = n(), r = first(r), d = first(d)) %>%
arrange(r) %>%
mutate(additional = paste("group starts at d =", d)) %>%
select(-r, -d)
# A tibble: 3 x 5
# Groups: b, a [1]
b a grp_no row_count additional
<int> <int> <int> <int> <chr>
1 100 1 1 5 group starts at d = 3400
2 100 1 2 9 group starts at d = 3434
3 100 1 3 1 group starts at d = 3465
With first example, its output is
# A tibble: 5 x 5
# Groups: b, a [3]
b a grp_no row_count additional
<int> <int> <int> <int> <chr>
1 100 -1 1 2 group starts at d = 3400
2 50 1 1 3 group starts at d = 3400
3 100 1 1 3 group starts at d = 3412
4 50 1 2 2 group starts at d = 3438
5 100 1 2 2 group starts at d = 3454
Note: you may also use dplyr::dense_rank instead of rleid in above syntax, like this
df %>% mutate(r = row_number()) %>%
group_by(b, a) %>%
mutate(grp_no = dense_rank(accumulate(d, ~ifelse(.y - .x > 30, .y, .x)) )) %>%
group_by(b, a, grp_no) %>%
summarise(row_count = n(), r = first(r), d = first(d)) %>%
arrange(r) %>%
mutate(additional = paste("group starts at d =", d)) %>%
select(-r, -d)
EndNote: Now I am not how your logic of c==3 fits into this? If you'll clarify I may try again

Answer 2

library(dplyr, warn.conflicts = FALSE)
library(purrr)
options(scipen = 999)
data <- structure(list(d = c(3400L, 3400L, 3400L, 3408L, 3412L, 3423L,
3434L, 3436L, 3438L, 3445L, 3454L, 3645L), b = c(100L, 50L, 100L,
50L, 100L, 50L, 100L, 100L, 50L, 50L, 100L, 100L), c = c(3L,
3L, 1L, 1L, 3L, 1L, 1L, 3L, 3L, 1L, 3L, 1L), a = c(-1L, 1L, -1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L)), row.names = c(NA, -12L), class = "data.frame")
# split data into group of a,b
new_data <- data %>%
group_by(a, b) %>%
group_split()
# function group index - assuming that every c == 1 have a c == 3 before it
# then only need to sort by the d different with first record by less than 30
group_function <- function(df) {
bin <- seq(from = min(df$d), to = max(df$d) + 30, by = 30)
df <- df %>%
mutate(d_group = cut(d, breaks = bin,
include.lowest = TRUE, right = FALSE)) %>%
group_by(d_group)
df$group_index <- group_indices(df)
df %>%
group_by(a, b, group_index) %>%
summarize(count = n(),
desc = sprintf("( starting at d = %s)", first(d)),
# I added the count of c==3 in the group just to show that sample data
# is not follow the logic you mentioned
addition_info = paste0("There is ",
sum(c == 3), " `c == 3` in this group"),
.groups = "drop") %>%
select(-group_index)
}
new_data %>%
map_dfr(group_function)
#> # A tibble: 6 x 5
#> a b count desc addition_info
#> <int> <int> <int> <chr> <chr>
#> 1 -1 100 2 ( starting at d = 3400) There is 1 `c == 3`` in this group
#> 2 1 50 3 ( starting at d = 3400) There is 1 `c == 3`` in this group
#> 3 1 50 2 ( starting at d = 3438) There is 1 `c == 3`` in this group
#> 4 1 100 3 ( starting at d = 3412) There is 2 `c == 3`` in this group
#> 5 1 100 1 ( starting at d = 3454) There is 1 `c == 3`` in this group
#> 6 1 100 1 ( starting at d = 3645) There is 0 `c == 3`` in this group
Created on 2021-04-04 by the reprex package (v1.0.0)
Update: included the logics described.