nest_by on R and running multiple models - r

Question

I am fairly new to this community and R, thank you for all the support.
I encountered the new nest_by option of dplyr and it seems rather good. I have managed to split existing Dataframe and but not to run multiple models with them. I would like to iterate through all the dataframes and get raw and summary data of statistical models (GLM models mainly).
library(tidyverse)
nested <- mtcars %>% nest_by (cyl,carb)
# A tibble: 9 x 3
# Rowwise: cyl, carb
cyl carb data
<dbl> <dbl> <list<tbl_df[,9]>>
1 4 1 [5 x 9]
2 4 2 [6 x 9]
3 6 1 [2 x 9]
4 6 4 [4 x 9]
5 6 6 [1 x 9]
6 8 2 [4 x 9]
7 8 3 [3 x 9]
8 8 4 [6 x 9]
9 8 8 [1 x 9]
#Now i would like to run each line seperately in a lm model. This line should do it, but it doesn't
fit<- nested %>%
mutate(model = map(data, ~lm(mpg~hp, data=.)))
Now, I am trying to make a printable version of all models for my statistics teacher.
nested <- mtcars %>% nest (data = -c(cyl,carb))
regressions <-nested %>%
mutate(
fit = map(data, ~ lm(mpg ~ hp, data = .x))
)
printing<- regressions %>% rowwise() %>%
mutate (printed = paste(carb, cyl, "This model summary is"), summary(fit), sep = '*')
However this doesn't work altogether.
Any thoughts?

Answer 1

EDIT: In your precise case try this:
nested <- mtcars %>% nest (data = -c(cyl,carb))
regressions <-nested %>%
mutate(
fit = map(data, ~ lm(mpg ~ hp, data = .x)),
tidied = map(fit, tidy),
glanced = map(fit, glance),
augmented = map(fit, augment)
)
regressions %>%
unnest(glanced) # to get statistics of fits
regressions %>%
unnest(tidied) # to get coefficients of all fits
You can use dplyr in combination with broom as in this vignette. There is an exact example with mtcars:
data(mtcars)
mtcars <- as_tibble(mtcars) # to play nicely with list-cols
mtcars
## # A tibble: 32 x 11
## mpg cyl disp hp drat wt qsec vs am gear carb
## <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 21 6 160 110 3.9 2.62 16.5 0 1 4 4
## 2 21 6 160 110 3.9 2.88 17.0 0 1 4 4
## 3 22.8 4 108 93 3.85 2.32 18.6 1 1 4 1
## 4 21.4 6 258 110 3.08 3.22 19.4 1 0 3 1
## 5 18.7 8 360 175 3.15 3.44 17.0 0 0 3 2
## 6 18.1 6 225 105 2.76 3.46 20.2 1 0 3 1
## 7 14.3 8 360 245 3.21 3.57 15.8 0 0 3 4
## 8 24.4 4 147. 62 3.69 3.19 20 1 0 4 2
## 9 22.8 4 141. 95 3.92 3.15 22.9 1 0 4 2
## 10 19.2 6 168. 123 3.92 3.44 18.3 1 0 4 4
## # ... with 22 more rows
mtcars %>%
nest(-am) %>%
mutate(
fit = map(data, ~ lm(wt ~ mpg + qsec + gear, data = .x)), # S3 list-col
tidied = map(fit, tidy)
) %>%
unnest(tidied)
## # A tibble: 8 x 8
## am data fit term estimate std.error statistic p.value
## <dbl> <list> <list> <chr> <dbl> <dbl> <dbl> <dbl>
## 1 1 <tibble [13 x 10~ <lm> (Intercep~ 4.28 3.46 1.24 2.47e-1
## 2 1 <tibble [13 x 10~ <lm> mpg -0.101 0.0294 -3.43 7.50e-3
## 3 1 <tibble [13 x 10~ <lm> qsec 0.0398 0.151 0.264 7.98e-1
## 4 1 <tibble [13 x 10~ <lm> gear -0.0229 0.349 -0.0656 9.49e-1
## 5 0 <tibble [19 x 10~ <lm> (Intercep~ 4.92 1.40 3.52 3.09e-3
## 6 0 <tibble [19 x 10~ <lm> mpg -0.192 0.0443 -4.33 5.91e-4
## 7 0 <tibble [19 x 10~ <lm> qsec 0.0919 0.0983 0.935 3.65e-1
## 8 0 <tibble [19 x 10~ <lm> gear 0.147 0.368 0.398 6.96e-1
What if you want not just the tidy output, but the augment and glance outputs as well, while still performing each regression only once? Since we’re using list-columns, we can just fit the model once and use multiple list-columns to store the tidied, glanced and augmented outputs.
regressions <- mtcars %>%
nest(-am) %>%
mutate(
fit = map(data, ~ lm(wt ~ mpg + qsec + gear, data = .x)),
tidied = map(fit, tidy),
glanced = map(fit, glance),
augmented = map(fit, augment)
)
regressions %>%
unnest(tidied)
## # A tibble: 8 x 10
## am data fit term estimate std.error statistic p.value glanced augmented
## <dbl> <lis> <lis> <chr> <dbl> <dbl> <dbl> <dbl> <list> <list>
## 1 1 <tib~ <lm> (Int~ 4.28 3.46 1.24 2.47e-1 <tibbl~ <tibble ~
## 2 1 <tib~ <lm> mpg -0.101 0.0294 -3.43 7.50e-3 <tibbl~ <tibble ~
## 3 1 <tib~ <lm> qsec 0.0398 0.151 0.264 7.98e-1 <tibbl~ <tibble ~
## 4 1 <tib~ <lm> gear -0.0229 0.349 -0.0656 9.49e-1 <tibbl~ <tibble ~
## 5 0 <tib~ <lm> (Int~ 4.92 1.40 3.52 3.09e-3 <tibbl~ <tibble ~
## 6 0 <tib~ <lm> mpg -0.192 0.0443 -4.33 5.91e-4 <tibbl~ <tibble ~
## 7 0 <tib~ <lm> qsec 0.0919 0.0983 0.935 3.65e-1 <tibbl~ <tibble ~
## 8 0 <tib~ <lm> gear 0.147 0.368 0.398 6.96e-1 <tibbl~ <tibble ~
regressions %>%
unnest(glanced)
## # A tibble: 2 x 16
## am data fit tidied r.squared adj.r.squared sigma statistic p.value df
## <dbl> <lis> <lis> <list> <dbl> <dbl> <dbl> <dbl> <dbl> <int>
## 1 1 <tib~ <lm> <tibb~ 0.833 0.778 0.291 15.0 7.59e-4 4
## 2 0 <tib~ <lm> <tibb~ 0.625 0.550 0.522 8.32 1.70e-3 4
## # ... with 6 more variables: logLik <dbl>, AIC <dbl>, BIC <dbl>,
## # deviance <dbl>, df.residual <int>, augmented <list>
regressions %>%
unnest(augmented)
## # A tibble: 32 x 16
## am data fit tidied glanced wt mpg qsec gear .fitted .se.fit
## <dbl> <lis> <lis> <list> <list> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 1 <tib~ <lm> <tibb~ <tibbl~ 2.62 21 16.5 4 2.73 0.209
## 2 1 <tib~ <lm> <tibb~ <tibbl~ 2.88 21 17.0 4 2.75 0.152
## 3 1 <tib~ <lm> <tibb~ <tibbl~ 2.32 22.8 18.6 4 2.63 0.163
## 4 1 <tib~ <lm> <tibb~ <tibbl~ 2.2 32.4 19.5 4 1.70 0.137
## 5 1 <tib~ <lm> <tibb~ <tibbl~ 1.62 30.4 18.5 4 1.86 0.151
## 6 1 <tib~ <lm> <tibb~ <tibbl~ 1.84 33.9 19.9 4 1.56 0.156
## 7 1 <tib~ <lm> <tibb~ <tibbl~ 1.94 27.3 18.9 4 2.19 0.113
## 8 1 <tib~ <lm> <tibb~ <tibbl~ 2.14 26 16.7 5 2.21 0.153
## 9 1 <tib~ <lm> <tibb~ <tibbl~ 1.51 30.4 16.9 5 1.77 0.191
## 10 1 <tib~ <lm> <tibb~ <tibbl~ 3.17 15.8 14.5 5 3.15 0.157
## # ... with 22 more rows, and 5 more variables: .resid <dbl>, .hat <dbl>,
## # .sigma <dbl>, .cooksd <dbl>, .std.resid <dbl>