Behold the painfully simple case and the error(s). Comments inline.
library(flexsurv)
#> Loading required package: survival
library(tidyverse)
library(magrittr)
#>
#> Attaching package: 'magrittr'
#> The following object is masked from 'package:purrr':
#>
#> set_names
#> The following object is masked from 'package:tidyr':
#>
#> extract
set.seed(2019)
train_data <- tribble(
~wait_time, ~called_yet, ~time_queued,
131.282999992371, 0, 1570733365.28,
358.296000003815, 1, 1570733421.187,
1352.13999986649, 1, 1570733540.923,
1761.61400008202, 0, 1570733941.343,
1208.25300002098, 0, 1570734327.11,
522.296999931335, 1, 1570734376.953,
241.75, 0, 1570734659.44,
143.156999826431, 0, 1570734809.673,
1202.79999995232, 1, 1570734942.907,
614.640000104904, 1, 1570735526.567
)
# Base survival works fine!
survival_model <- survreg(Surv(wait_time, called_yet) ~ time_queued,
data = train_data,
dist = "weibull")
survival_model
#> Call:
#> survreg(formula = Surv(wait_time, called_yet) ~ time_queued,
#> data = train_data, dist = "weibull")
#>
#> Coefficients:
#> (Intercept) time_queued
#> 4.533765e+05 -2.886352e-04
#>
#> Scale= 0.518221
#>
#> Loglik(model)= -40.2 Loglik(intercept only)= -40.5
#> Chisq= 0.5 on 1 degrees of freedom, p= 0.48
#> n= 10
# flexsurvreg can't even get a valid initializer for time_queued, even though
# the doc says it takes the mean of the data
flexsurv_model <- flexsurvreg(Surv(wait_time, called_yet) ~ time_queued,
data = train_data,
dist = "weibull")
#> Error in flexsurvreg(Surv(wait_time, called_yet) ~ time_queued, data = train_data, : Initial value for parameter 2 out of range
# Maybe the low variance of the predictor here is the problem? So let's up the
# variance just to see
train_data %<>% mutate_at("time_queued", subtract, 1.57073e9)
train_data
#> # A tibble: 10 x 3
#> wait_time called_yet time_queued
#> <dbl> <dbl> <dbl>
#> 1 131. 0 3365.
#> 2 358. 1 3421.
#> 3 1352. 1 3541.
#> 4 1762. 0 3941.
#> 5 1208. 0 4327.
#> 6 522. 1 4377.
#> 7 242. 0 4659.
#> 8 143. 0 4810.
#> 9 1203. 1 4943.
#> 10 615. 1 5527.
# Now it initializes, so that's different... but now it won't converge!
flexsurv_model <- flexsurvreg(Surv(wait_time, called_yet) ~ time_queued,
data = train_data,
dist = "weibull")
#> Warning in flexsurvreg(Surv(wait_time, called_yet) ~ time_queued, data
#> = train_data, : Optimisation has probably not converged to the maximum
#> likelihood - Hessian is not positive definite.
Created on 2019-10-19 by the reprex package (v0.3.0)
I mainly wanted to use flexsurv for its better plotting options and more standard shape & scale definitions - and the ancillary parameters are very attractive too - but now I'm mainly just wondering if I'm doing something really wrong, and flexsurv is trying to tell me not to trust my base survival model either.
Marco Sandri pointed out that recentering fixes it; however, recentering without rescaling only guarantees initialization, and still results in no convergence if the variance is very large. I'm considering this a bug since survival has no problem with the exact same model with the exact same values. Created an issue here.
Related
I am trying to understand how to apply step_pca to preprocess my data. Suppose I want to build a K-Nearest Neighbor classifier to the iris dataset. For the sake of simplicity, I will not split the original iris dataset into train and test. I will assume iris is the train dataset and I have some other observations as my test dataset.
I want to apply three transformations to the predictors in my train dataset:
Center all predictor variables
Scale all predictor variables
PCA transform all predictor variables and keep a number of them that explains, at least, 80% of my data variance
So this is what I have:
library(tidymodels)
iris_rec <-
recipe(Species ~ .,
data = iris) %>%
# center/scale
step_center(-Species) %>%
step_scale(-Species) %>%
# pca
step_pca(-Species, threshold = 0.8) %>%
# apply data transformation
prep()
iris_rec
#> Recipe
#>
#> Inputs:
#>
#> role #variables
#> outcome 1
#> predictor 4
#>
#> Training data contained 150 data points and no missing data.
#>
#> Operations:
#>
#> Centering for Sepal.Length, Sepal.Width, Petal.Length, Petal.... [trained]
#> Scaling for Sepal.Length, Sepal.Width, Petal.Length, Petal.... [trained]
#> PCA extraction with Sepal.Length, Sepal.Width, Petal.Length, Petal.W... [trained]
Created on 2022-10-13 with reprex v2.0.2
Ok, so far, so good. All the transformations are applied to my dataset. When I prepare my train dataset using juice, everything goes as expected:
# transformed training set
iris_train_t <- juice(iris_rec)
iris_train_t
#> # A tibble: 150 × 3
#> Species PC1 PC2
#> <fct> <dbl> <dbl>
#> 1 setosa -2.26 -0.478
#> 2 setosa -2.07 0.672
#> 3 setosa -2.36 0.341
#> 4 setosa -2.29 0.595
#> 5 setosa -2.38 -0.645
#> 6 setosa -2.07 -1.48
#> 7 setosa -2.44 -0.0475
#> 8 setosa -2.23 -0.222
#> 9 setosa -2.33 1.11
#> 10 setosa -2.18 0.467
#> # … with 140 more rows
Created on 2022-10-13 with reprex v2.0.2
So, I have two predictors based on PCA (PC1 and PC2) and my response variable. However, when I proceed with my modelling, I get an error: all the models I test fail, as you can see below:
# cross validation
set.seed(2022)
iris_train_cv <- vfold_cv(iris_train_t, v = 5)
# tuning
iris_knn_tune <-
nearest_neighbor(
neighbors = tune(),
weight_func = tune(),
dist_power = tune()
) %>%
set_engine("kknn") %>%
set_mode("classification")
# grid search
iris_knn_grid <-
grid_regular(neighbors(range = c(3, 9)),
weight_func(),
dist_power(),
levels = c(22, 2, 2))
# workflow creation
iris_wflow <-
workflow() %>%
add_recipe(iris_rec) %>%
add_model(iris_knn_tune)
# model assessment
iris_knn_fit_tune <-
iris_wflow %>%
tune_grid(
resamples = iris_train_cv,
grid = iris_knn_grid
)
#> x Fold1: preprocessor 1/1:
#> Error in `check_training_set()`:
#> ! Not all variables in the recipe are present in the supplied training...
#> x Fold2: preprocessor 1/1:
#> Error in `check_training_set()`:
#> ! Not all variables in the recipe are present in the supplied training...
#> x Fold3: preprocessor 1/1:
#> Error in `check_training_set()`:
#> ! Not all variables in the recipe are present in the supplied training...
#> x Fold4: preprocessor 1/1:
#> Error in `check_training_set()`:
#> ! Not all variables in the recipe are present in the supplied training...
#> x Fold5: preprocessor 1/1:
#> Error in `check_training_set()`:
#> ! Not all variables in the recipe are present in the supplied training...
#> Warning: All models failed. Run `show_notes(.Last.tune.result)` for more
#> information.
# cv results
collect_metrics(iris_knn_fit_tune)
#> Error in `estimate_tune_results()`:
#> ! All of the models failed. See the .notes column.
#> Backtrace:
#> ▆
#> 1. ├─tune::collect_metrics(iris_knn_fit_tune)
#> 2. └─tune:::collect_metrics.tune_results(iris_knn_fit_tune)
#> 3. └─tune::estimate_tune_results(x)
#> 4. └─rlang::abort("All of the models failed. See the .notes column.")
Created on 2022-10-13 with reprex v2.0.2
I am suspecting my problem is with the formula I defined on my iris_rec recipe. The formula there is
Species ~ ., data = iris
which means
Species ~ Sepal.Length + Sepal.Width + Petal.Length + Petal.Width, data = iris
However, when I run my models, the predictor variables are PC1 and PC2, so I guess the formula should be
Species ~ ., data = iris_train_t
or
Species ~ PC1 + PC2, data = iris_train_t
How can I inform my model that my variables and dataset changed? All the others step_* I used on my tidymodels have worked, but I am struggling specifically with step_pca.
Two things that are confusing.
First, you don't need to prep() or juice() a recipe before using it in a model or workflow. The tuning and resampling functions will be doing that within each resample.
You can prep() and juice() if you want the training set processed to troubleshoot, visualize, or otherwise explore. But you don’t need to otherwise.
Second, the recipe is basically a replacement for the formula. It knows what the predictors and outcomes are so there is rarely the need to use an additional formula on top of that.
(The exception is for models that require special formulas but otherwise no).
Here is updated code for you:
library(tidymodels)
iris_rec <-
recipe(Species ~ .,
data = iris) %>%
# center/scale
step_center(-Species) %>%
step_scale(-Species) %>%
# pca
step_pca(-Species, threshold = 0.8)
set.seed(2022)
iris_train_cv <- vfold_cv(iris, v = 5) #<- changes here
# tuning
iris_knn_tune <-
nearest_neighbor(
neighbors = tune(),
weight_func = tune(),
dist_power = tune()
) %>%
set_engine("kknn") %>%
set_mode("classification")
# grid search
iris_knn_grid <-
grid_regular(neighbors(range = c(3, 9)),
weight_func(),
dist_power(),
levels = c(22, 2, 2))
# workflow creation
iris_wflow <-
workflow() %>%
add_recipe(iris_rec) %>%
add_model(iris_knn_tune)
# model assessment
iris_knn_fit_tune <-
iris_wflow %>%
tune_grid(
resamples = iris_train_cv,
grid = iris_knn_grid
)
show_best(iris_knn_fit_tune, metric = "roc_auc")
#> # A tibble: 5 × 9
#> neighbors weight_func dist_power .metric .estima…¹ mean n std_err .config
#> <int> <chr> <dbl> <chr> <chr> <dbl> <int> <dbl> <chr>
#> 1 9 rectangular 1 roc_auc hand_till 0.976 5 0.00580 Prepro…
#> 2 7 triangular 1 roc_auc hand_till 0.975 5 0.00688 Prepro…
#> 3 9 triangular 2 roc_auc hand_till 0.975 5 0.00571 Prepro…
#> 4 8 triangular 1 roc_auc hand_till 0.975 5 0.00655 Prepro…
#> 5 9 triangular 1 roc_auc hand_till 0.975 5 0.00655 Prepro…
#> # … with abbreviated variable name ¹.estimator
Created on 2022-10-13 with reprex v2.0.2
I was getting acquainted with tidymodels by reading the book and this line in section 9.2 kept me thinking about retransformation.
It is best practice to analyze the predictions on the transformed
scale (if one were used) even if the predictions are reported using
the original units.
But I found it confusing that the examples in the book use a log transformation on the outcome, but they do not use a recipe for this (the recipe has not been introduced at this point, but later when they introduce recipe, still they do not use step_log for the outcome but just for the predictors). So I wanted to try that and found something puzzling, illustrated with the reprex below:
# So let's use most of the code from the examples in the book
library(tidymodels)
tidymodels_prefer()
set.seed(501)
# data budget
data(ames)
ames_split <- initial_split(ames, prop = 0.80, strata = Sale_Price)
ames_train <- training(ames_split)
ames_test <- testing(ames_split)
ames_folds <- vfold_cv(ames_train, v = 10, strata = Sale_Price)
# IJALM
lm_model <-
linear_reg(penalty = 0) |>
set_engine("glmnet")
# And use a recipe,
# but Instead of manually transforming the outcome like this ...
# `ames <- ames %>% mutate(Sale_Price = log10(Sale_Price))`
# let's include the outcome transformation into the recipe
simple_ames <-
recipe(
Sale_Price ~ Neighborhood + Gr_Liv_Area + Year_Built + Bldg_Type,
data = ames_train
) |>
step_log(Gr_Liv_Area, base = 10) |>
step_dummy(all_nominal_predictors()) |>
step_log(Sale_Price, base = 10, skip = TRUE)
lm_wflow <-
workflow() |>
add_model(lm_model) |>
add_recipe(simple_ames)
lm_res <- fit_resamples(
lm_wflow,
resamples = ames_folds,
control = control_resamples(save_pred = TRUE, save_workflow = TRUE),
metrics = metric_set(rmse)
)
collect_metrics(lm_res)
#> # A tibble: 1 x 6
#> .metric .estimator mean n std_err .config
#> <chr> <chr> <dbl> <int> <dbl> <chr>
#> 1 rmse standard 197264. 10 1362. Preprocessor1_Model1
# Now, I wanted to double-check how rmse was calculated
# It should be the mean of the rmse for each fold
# (individual values stored in this list lm_res$.metrics,
# with one element for each fold)
# and each rmse should have been calculated with the predictions of each fold
# (stored in lm_res$.predictions)
# So, this rmse corresponding to the first fold
lm_res$.metrics[[1]]
#> # A tibble: 1 x 4
#> .metric .estimator .estimate .config
#> <chr> <chr> <dbl> <chr>
#> 1 rmse standard 196074. Preprocessor1_Model1
# Should have been calculated with this data
lm_res$.predictions[[1]]
#> # A tibble: 236 x 4
#> .pred .row Sale_Price .config
#> <dbl> <int> <int> <chr>
#> 1 5.09 2 126000 Preprocessor1_Model1
#> 2 4.92 33 105900 Preprocessor1_Model1
#> 3 5.06 34 125500 Preprocessor1_Model1
#> 4 5.18 44 121500 Preprocessor1_Model1
#> 5 5.14 51 127000 Preprocessor1_Model1
#> 6 5.13 53 114000 Preprocessor1_Model1
#> 7 4.90 57 84900 Preprocessor1_Model1
#> 8 5.13 62 113000 Preprocessor1_Model1
#> 9 5.02 74 83500 Preprocessor1_Model1
#> 10 5.02 76 88250 Preprocessor1_Model1
#> # ... with 226 more rows
#> # i Use `print(n = ...)` to see more rows
# But here's the issue!
# The predictions are in the log-scale while the observed values
# are in the original units.
# This is just a quick-check to make sure the rmse reported above
# (calculated by yardstick) does in fact involve mixing-up the log-scale
# (predictions) and the original units (observed values)
yhat <- lm_res$.predictions[[1]]$.pred
yobs <- lm_res$.predictions[[1]]$Sale_Price
sqrt(mean((yhat - yobs)^2))
#> [1] 196073.6
# So, apparently, for cross-validation tidymodels does not `bake` the folds
# with the recipe to calculate the metrics
And here’s where I got it (at least I think so), after spending half
an hour writing this reprex. So, to not feel I wasted my time, I decided
to post it anyway, and put what I think is going on as an answer.
Perhaps someone finds it useful, because it was not evident to me at the
first time. Or perhaps someone can explain if there is something else going on.
Created on 2022-08-07 by the reprex package (v2.0.1)
It was basically your fault. You explicitly told tidymodels not to bake() this step. The line below was the culprit, in particular, the skip = TRUE part.
step_log(Sale_Price, base = 10, skip = TRUE)
Hence, tidymodels will not include that step in baking the folds before making the predictions and you end up with the log-scale of the predictions mixed-up with the untransformed outcome variable. This is perhaps one of the examples they had in mind when they wrote in the documentation that:
Care should be taken when using skip = TRUE as it may affect the
computations for subsequent operations.
You probably decided to skip that step, because the outcome variable may not be available on a new dataset for which you want predictions, and then the process would fail. But it basically messes up metrics for cross-validation. So, better not to skip the step and deal other way with that problem.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 1 year ago.
Improve this question
Is there an R package or function for tuning logistic regression hyperparameters similar to what can be done in Python? As far as I know, the glm function has no hyperparameters available to tune, even though there are several different hyperparameters for logistic regression. Is this just something that is not possible to do in R as it is in Python? Does the glm function just set them at fixed, default values and not allow the user to try different values? Is there a way to alter the original code of the glm function to specify different hyperparameters?
As far as I know, there are no tunable hyperparameters in glm, but there are other logistic regression functions where hyperparameters are tunable.
The tidymodels package is very convenient for machine learning using R. It streamlines hyperparameter tuning for various data preprocessing (e.g. PCA, ...) and modelling approaches (glm and many others).
You can tune the hyperparameters of a logistic regression using e.g. the glmnet method (engine), where penalty (lambda) and mixture (alpha) can be tuned.
Specify logistic regression model using tidymodels
library(tidymodels)
library(glmnet)
# show tunable hyperparameters for various logistic regression functions
show_model_info("logistic_reg")
#> Information for `logistic_reg`
#> modes: unknown, classification
#>
#> engines:
#> classification: glm, glmnet, keras, LiblineaR, spark, stan
#>
#> arguments:
#> glmnet:
#> penalty --> lambda
#> mixture --> alpha
#> LiblineaR:
#> penalty --> cost
#> mixture --> type
#> spark:
#> penalty --> reg_param
#> mixture --> elastic_net_param
#> keras:
#> penalty --> penalty
#>
#> [fit and prediction modules omitted to be more concise]
mod <- logistic_reg(mode = "classification", engine = "glmnet",
penalty = tune(), mixture = tune())
tune hyperparameters using tidymodels
## specify recipe: model formula and preprocessing steps (if any)
rec <- recipe(Group ~ Class1 + Class2, data = data)
## specify workflow (recipe and model specification)
w <- workflow(preprocessor = rec, spec = mod)
## specify resamples for cross-validation
set.seed(28483)
r <- vfold_cv(data = data, v = 10, repeats = 1, strata = "Group")
## specify tuning grid (hyperparameter search space)
g <- expand_grid(penalty = c(0,1,2),
mixture = seq(0,1,by=0.2))
## tune logistic regression model
fit_tune <- tune_grid(w, resamples = r, grid = g)
collect and plot tuning results
tune.met <- collect_metrics(fit_tune)
tune.met
#> # A tibble: 36 × 8
#> penalty mixture .metric .estimator mean n std_err .config
#> <dbl> <dbl> <chr> <chr> <dbl> <int> <dbl> <chr>
#> 1 0 0 accuracy binary 0.785 10 0.0299 Preprocessor1_Model01
#> 2 0 0 roc_auc binary 0.877 10 0.0244 Preprocessor1_Model01
#> 3 1 0 accuracy binary 0.785 10 0.0299 Preprocessor1_Model02
#> 4 1 0 roc_auc binary 0.875 10 0.0251 Preprocessor1_Model02
#> 5 2 0 accuracy binary 0.785 10 0.0299 Preprocessor1_Model03
#> 6 2 0 roc_auc binary 0.875 10 0.0251 Preprocessor1_Model03
#> 7 0 0.2 accuracy binary 0.785 10 0.0299 Preprocessor1_Model04
#> 8 0 0.2 roc_auc binary 0.876 10 0.0245 Preprocessor1_Model04
#> 9 1 0.2 accuracy binary 0.775 10 0.0271 Preprocessor1_Model05
#> 10 1 0.2 roc_auc binary 0.847 10 0.0265 Preprocessor1_Model05
#> # … with 26 more rows
ggplot(tune.met, aes(x = mixture, y = mean, colour = factor(penalty))) +
geom_line() +
facet_wrap(~.metric) +
theme_bw()
# show best
fit_tune %>% show_best(metric = "accuracy")
#> # A tibble: 5 × 8
#> penalty mixture .metric .estimator mean n std_err .config
#> <dbl> <dbl> <chr> <chr> <dbl> <int> <dbl> <chr>
#> 1 0 0 accuracy binary 0.785 10 0.0299 Preprocessor1_Model01
#> 2 1 0 accuracy binary 0.785 10 0.0299 Preprocessor1_Model02
#> 3 2 0 accuracy binary 0.785 10 0.0299 Preprocessor1_Model03
#> 4 0 0.2 accuracy binary 0.785 10 0.0299 Preprocessor1_Model04
#> 5 0 0.4 accuracy binary 0.785 10 0.0299 Preprocessor1_Model07
toy data used
library(tidyverse)
#set.seed(1)
x1 = rnorm(100, mean = 45, sd = 5); x2 = rnorm(100, mean = 50, sd = 3);
z1 = rnorm(100, mean = 13, sd = 1.3); z2 = rnorm(100, mean = 15, sd = 2.3)
Y0 = rep("No", 100);Y1 = rep("Yes",100)
a = c(x2,x1); b = c(z2,z1); c = c(Y0,Y1);
data = tibble(Group = factor(c), Class1 = a, Class2 = b)
Created on 2021-09-21 by the reprex package (v2.0.1)
I know that in tidymodels you can set a custom tunable parameter space by interacting directly with the workflow object as follows:
library(tidymodels)
model <- linear_reg(
mode = "regression",
engine = "glmnet",
penalty = tune()
)
rec_cars <- recipe(mpg ~ ., data = mtcars)
wkf <- workflow() %>%
add_recipe(rec_cars) %>%
add_model(model)
wkf_new_param_space <- wkf %>%
parameters() %>%
update(penalty = penalty(range = c(0.9, 1)))
but sometimes it makes more sense to do this right at the moment I specify a recipe or a model.
Someone knows a way to achieve this?
The parameter ranges are inherently separated from the model specification and recipe specification in tidymodels. When you set tune() you are giving a signal to the tune function that this parameter will take multiple values and should be tuned over.
So as a short answer, you can not specify ranges of parameters when you specify a recipe or a model, but you can create the parameters object right after as you did.
In the end, you need the parameter set to construct the grid values that you are using for hyperparameter tuning, and you can create those gid values in at least 4 ways.
The first way is to do it the way you are doing it, by pulling the needed parameters out of the workflow and modifying them when needed.
The second way is to create a parameters object that will match the parameters that you will need to use. This option and the remaining require you to make sure that you create values for all the parameters you are tuning.
The Third way is to skip the parameters object altogether and create the grid with your grid_*() function and dials functions.
The fourth way is to skip dials functions altogether and create the data frame yourself. I find tidyr::crossing() an useful replacement for grid_regular(). This way is a lot easier when you are working with integer parameters and parameters that don't benefit from transformations.
library(tidymodels)
model <- linear_reg(
mode = "regression",
engine = "glmnet",
penalty = tune()
)
rec_cars <- recipe(mpg ~ ., data = mtcars)
wkf <- workflow() %>%
add_recipe(rec_cars) %>%
add_model(model)
# Option 1: using parameters() on workflow
wkf_new_param_space <- wkf %>%
parameters() %>%
update(penalty = penalty(range = c(-5, 5)))
wkf_new_param_space %>%
grid_regular(levels = 10)
#> # A tibble: 10 × 1
#> penalty
#> <dbl>
#> 1 0.00001
#> 2 0.000129
#> 3 0.00167
#> 4 0.0215
#> 5 0.278
#> 6 3.59
#> 7 46.4
#> 8 599.
#> 9 7743.
#> 10 100000
# Option 2: Using parameters() on list
my_params <- parameters(
list(
penalty(range = c(-5, 5))
)
)
my_params %>%
grid_regular(levels = 10)
#> # A tibble: 10 × 1
#> penalty
#> <dbl>
#> 1 0.00001
#> 2 0.000129
#> 3 0.00167
#> 4 0.0215
#> 5 0.278
#> 6 3.59
#> 7 46.4
#> 8 599.
#> 9 7743.
#> 10 100000
# Option 3: Use grid_*() with dials objects directly
grid_regular(
penalty(range = c(-5, 5)),
levels = 10
)
#> # A tibble: 10 × 1
#> penalty
#> <dbl>
#> 1 0.00001
#> 2 0.000129
#> 3 0.00167
#> 4 0.0215
#> 5 0.278
#> 6 3.59
#> 7 46.4
#> 8 599.
#> 9 7743.
#> 10 100000
# Option 4: Create grid values manually
tidyr::crossing(
penalty = 10 ^ seq(-5, 5, length.out = 10)
)
#> # A tibble: 10 × 1
#> penalty
#> <dbl>
#> 1 0.00001
#> 2 0.000129
#> 3 0.00167
#> 4 0.0215
#> 5 0.278
#> 6 3.59
#> 7 46.4
#> 8 599.
#> 9 7743.
#> 10 100000
Created on 2021-08-17 by the reprex package (v2.0.1)
seems that this is an old question but I am having a hard time trying to insert this approach (option 1) in my workflow.
How is supposed to continue?
wkf_new_param_space is used as grid or as object in tuning model?
model_tuned <-
tune::tune_grid(
object = wkf_new_param_space, ?
resamples = cv_folds,
grid = wkf_new_param_space, ?
metrics = model_metrics,
control = tune::control_grid(save_pred = TRUE, save_workflow = TRUE)
)
b
DF1 = read.csv("Nerlove.3.csv",header=TRUE)
head(DF1, n=5)
split = round(nrow(DF1) * 0.60)
train = (DF1[1:split, ])
test = (DF1[(split + 1):nrow(DF1), ])
model = lm(output ~ ., train)
summary(model)
plot(train$cost, train$output, ylab = "Output", xlab = "Cost",main = "....")
abline(model, col=2)
c
plot(test$cost, test$output, ylab = "Output", xlab = "Cost",main = "....")
model1 = lm(output ~ ., test)
abline(model, col=2)
prediction = predict(model, test)
plot(prediction, main = "....")
abline(model1, col=2)
summary(model1)
d
library(stats)
X_0 = data.frame(cost = test$cost)
FI_mean = predict(model, newdata = X_0, interval="confidence", level = 0.95)
FI_ind = predict(model,newdata = X_0, interval = "prediction")
plot(test$cost, test$output, ylab = "Output", xlab = "Cost",main = "....")
abline(model, col=2)
min = test$cost
max = test$cost
newx = seq(min,max)
matlines(newx, FI_mean[,2:3], col = "blue", lty=2)
I need to plot the Confidence interval result I found around the regression line, but I'm getting an error. can anybody please help me to fix this. Thanks
This is the link for my data. I have edited it and only using the cost and output data in my dataframe
You are doing it wrong because you are using all the variables while developing the linear model by the command model = lm(output ~ ., train). But during plotting, you are using cost vs. output plotting (as in case of b and c) and in case of d, you are trying to predict using only one variable i.e. cost. Regression plot should be made between observed output vs. predicted output. For that, you can use the following code
library(lattice)
library(mosaic)
#> Loading required package: dplyr
#>
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#>
#> filter, lag
#> The following objects are masked from 'package:base':
#>
#> intersect, setdiff, setequal, union
#> Loading required package: ggformula
#> Loading required package: ggplot2
#> Loading required package: ggstance
#>
#> Attaching package: 'ggstance'
#> The following objects are masked from 'package:ggplot2':
#>
#> geom_errorbarh, GeomErrorbarh
#>
#> New to ggformula? Try the tutorials:
#> learnr::run_tutorial("introduction", package = "ggformula")
#> learnr::run_tutorial("refining", package = "ggformula")
#> Loading required package: mosaicData
#> Loading required package: Matrix
#> Registered S3 method overwritten by 'mosaic':
#> method from
#> fortify.SpatialPolygonsDataFrame ggplot2
#>
#> The 'mosaic' package masks several functions from core packages in order to add
#> additional features. The original behavior of these functions should not be affected by this.
#>
#> Note: If you use the Matrix package, be sure to load it BEFORE loading mosaic.
#>
#> Attaching package: 'mosaic'
#> The following object is masked from 'package:Matrix':
#>
#> mean
#> The following object is masked from 'package:ggplot2':
#>
#> stat
#> The following objects are masked from 'package:dplyr':
#>
#> count, do, tally
#> The following objects are masked from 'package:stats':
#>
#> binom.test, cor, cor.test, cov, fivenum, IQR, median, prop.test,
#> quantile, sd, t.test, var
#> The following objects are masked from 'package:base':
#>
#> max, mean, min, prod, range, sample, sum
DF1 = read.csv("Nerlove.csv",header=TRUE)
head(DF1, n=5)
#> cost output pl sl pk sk pf sf
#> 1 0.082 2 2.09 0.3164 183 0.4521 17.9 0.2315
#> 2 0.661 3 2.05 0.2073 174 0.6676 35.1 0.1251
#> 3 0.990 4 2.05 0.2349 171 0.5799 35.1 0.1852
#> 4 0.315 4 1.83 0.1152 166 0.7857 32.2 0.0990
#> 5 0.197 5 2.12 0.2300 233 0.3841 28.6 0.3859
split = round(nrow(DF1) * 0.60)
train = (DF1[1:split, ])
test = (DF1[(split + 1):nrow(DF1), ])
model = lm(output ~ ., train)
summary(model)
#>
#> Call:
#> lm(formula = output ~ ., data = train)
#>
#> Residuals:
#> Min 1Q Median 3Q Max
#> -409.65 -112.20 -4.61 94.20 430.76
#>
#> Coefficients:
#> Estimate Std. Error t value Pr(>|t|)
#> (Intercept) 1182.296 1861.876 0.635 0.527
#> cost 146.231 6.127 23.866 < 2e-16 ***
#> pl 14.897 80.142 0.186 0.853
#> sl -2471.312 1930.034 -1.280 0.204
#> pk 1.477 1.011 1.460 0.148
#> sk -869.468 1874.585 -0.464 0.644
#> pf -13.701 2.365 -5.794 1.08e-07 ***
#> sf -947.958 1861.490 -0.509 0.612
#> ---
#> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 169.3 on 87 degrees of freedom
#> Multiple R-squared: 0.9111, Adjusted R-squared: 0.904
#> F-statistic: 127.4 on 7 and 87 DF, p-value: < 2.2e-16
#Calibration plotting
pred_cal <- predict(model, newdata=train)
df_cal <- data.frame(Observed=train$output, Predicted=pred_cal)
xyplot(Predicted ~ Observed, data = df_cal, pch = 19, panel=panel.lmbands,
band.lty = c(conf =2, pred = 1))
#Validation plottig
pred_val <- predict(model, newdata=test)
df_val <- data.frame(Observed=test$output, Predicted=pred_val)
xyplot(Predicted ~ Observed, data = df_val, pch = 19, panel=panel.lmbands,
band.lty = c(conf =2, pred = 1))
Created on 2020-01-07 by the reprex package (v0.3.0)