The call mF#X should return fixed effect design matrix (necessary for marginal and conditional R^2 in glmm).
However it does not work.
Is even such matrix listed in model structure?
I appreciate any suggestions.
str(mF)
Formal class 'glmerMod' [package "lme4"] with 13 slots
..# resp :Reference class 'glmResp' [package "lme4"] with 11 fields
.. ..$ Ptr :<externalptr>
.. ..$ mu : num [1:480] 0.168 0.356 0.168 0.356 0.284 ...
.. ..$ offset : num [1:480] 0 0 0 0 0 0 0 0 0 0 ...
.. ..$ sqrtXwt: num [1:480] 0.373 0.479 0.373 0.479 0.451 ...
.. ..$ sqrtrwt: num [1:480] 2.68 2.09 2.68 2.09 2.22 ...
.. ..$ weights: num [1:480] 1 1 1 1 1 1 1 1 1 1 ...
.. ..$ wtres : num [1:480] -0.449 -0.744 -0.449 1.344 -0.629 ...
.. ..$ y : num [1:480] 0 0 0 1 0 0 0 1 0 0 ...
.. ..$ eta : num [1:480] -1.603 -0.591 -1.603 -0.591 -0.927 ...
.. ..$ family :List of 11
.. .. ..$ family : chr "binomial"
.. .. ..$ link : chr "logit"
.. .. ..$ linkfun :function (mu)
.. .. ..$ linkinv :function (eta)
.. .. ..$ variance :function (mu)
.. .. ..$ dev.resids:function (y, mu, wt)
.. .. ..$ aic :function (y, n, mu, wt, dev)
.. .. ..$ mu.eta :function (eta)
.. .. ..$ validmu :function (mu)
.. .. ..$ valideta :function (eta)
.. .. ..$ simulate :function (object, nsim)
.. .. ..- attr(*, "class")= chr "family"
.. ..$ n : num [1:480] 1 1 1 1 1 1 1 1 1 1 ...
.. ..and 41 methods, of which 29 are possibly relevant:
.. .. aic, allInfo, allInfo#lmResp, copy#envRefClass, devResid, fam,
.. .. initialize, initialize#lmResp, initializePtr, Laplace, link, muEta, ptr,
.. .. ptr#lmResp, resDev, setOffset, setResp, setTheta, setWeights, sqrtWrkWt,
.. .. theta, updateMu, updateMu#lmResp, updateWts, variance, wrkResids,
.. .. wrkResp, wrss, wtWrkResp
..# Gp : int [1:3] 0 60 72
..# call : language lme4::glmer(formula = Colour ~ Treatment + Habitat + (1 | Population) + (1 | Container), data = Data, family = "binomial", control = structure(list( ...
..# frame :'data.frame': 480 obs. of 5 variables:
.. ..$ Colour : int [1:480] 0 0 0 1 0 0 0 1 0 0 ...
.. ..$ Treatment : Factor w/ 2 levels "Cont","Exp": 1 2 1 2 1 2 1 2 1 2 ...
.. ..$ Habitat : Factor w/ 2 levels "A","B": 1 1 1 1 2 2 2 2 1 1 ...
.. ..$ Population: int [1:480] 1 1 1 1 1 1 1 1 1 1 ...
.. ..$ Container : int [1:480] 2 2 2 2 2 2 2 2 4 4 ...
.. ..- attr(*, "terms")=Classes 'terms', 'formula' length 3 Colour ~ Treatment + Habitat + (1 + Population) + (1 + Container)
.. .. .. ..- attr(*, "variables")= language list(Colour, Treatment, Habitat, Population, Container)
.. .. .. ..- attr(*, "factors")= int [1:5, 1:4] 0 1 0 0 0 0 0 1 0 0 ...
.. .. .. .. ..- attr(*, "dimnames")=List of 2
.. .. .. .. .. ..$ : chr [1:5] "Colour" "Treatment" "Habitat" "Population" ...
.. .. .. .. .. ..$ : chr [1:4] "Treatment" "Habitat" "Population" "Container"
.. .. .. ..- attr(*, "term.labels")= chr [1:4] "Treatment" "Habitat" "Population" "Container"
.. .. .. ..- attr(*, "order")= int [1:4] 1 1 1 1
.. .. .. ..- attr(*, "intercept")= int 1
.. .. .. ..- attr(*, "response")= int 1
.. .. .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv>
.. .. .. ..- attr(*, "predvars")= language list(Colour, Treatment, Habitat, Population, Container)
.. .. .. ..- attr(*, "dataClasses")= Named chr [1:5] "numeric" "factor" "factor" "numeric" ...
.. .. .. .. ..- attr(*, "names")= chr [1:5] "Colour" "Treatment" "Habitat" "Population" ...
.. .. .. ..- attr(*, "predvars.fixed")= language list(Colour, Treatment, Habitat)
.. ..- attr(*, "formula")=Class 'formula' length 3 Colour ~ Treatment + Habitat + (1 | Population) + (1 | Container)
.. .. .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv>
..# flist :List of 2
.. ..$ Container : Factor w/ 60 levels "2","4","6","8",..: 1 1 1 1 1 1 1 1 2 2 ...
.. ..$ Population: Factor w/ 12 levels "1","2","3","4",..: 1 1 1 1 1 1 1 1 1 1 ...
.. ..- attr(*, "assign")= int [1:2] 1 2
..# cnms :List of 2
.. ..$ Container : chr "(Intercept)"
.. ..$ Population: chr "(Intercept)"
..# lower : num [1:2] 0 0
..# theta : num [1:2] 0.0761 1.0537
..# beta : num [1:3] -1.251 1.012 0.676
..# u : num [1:72] -0.05 0.0254 -0.05 0.1008 -0.05 ...
..# devcomp:List of 2
.. ..$ cmp : Named num [1:11] 27.3 8.5 438.1 10.6 448.7 ...
.. .. ..- attr(*, "names")= chr [1:11] "ldL2" "ldRX2" "wrss" "ussq" ...
.. ..$ dims: Named int [1:14] 480 480 3 477 2 72 1 1 0 2 ...
.. .. ..- attr(*, "names")= chr [1:14] "N" "n" "p" "nmp" ...
..# pp :Reference class 'merPredD' [package "lme4"] with 18 fields
.. ..$ Lambdat:Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
.. .. .. ..# i : int [1:72] 0 1 2 3 4 5 6 7 8 9 ...
.. .. .. ..# p : int [1:73] 0 1 2 3 4 5 6 7 8 9 ...
.. .. .. ..# Dim : int [1:2] 72 72
.. .. .. ..# Dimnames:List of 2
.. .. .. .. ..$ : NULL
.. .. .. .. ..$ : NULL
.. .. .. ..# x : num [1:72] 0.0761 0.0761 0.0761 0.0761 0.0761 ...
.. .. .. ..# factors : list()
.. ..$ LamtUt :Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
.. .. .. ..# i : int [1:960] 0 60 0 60 0 60 0 60 0 60 ...
.. .. .. ..# p : int [1:481] 0 2 4 6 8 10 12 14 16 18 ...
.. .. .. ..# Dim : int [1:2] 72 480
.. .. .. ..# Dimnames:List of 2
.. .. .. .. ..$ : NULL
.. .. .. .. ..$ : NULL
.. .. .. ..# x : num [1:960] 0.0284 0.3935 0.0365 0.5046 0.0284 ...
.. .. .. ..# factors : list()
.. ..$ Lind : int [1:72] 1 1 1 1 1 1 1 1 1 1 ...
.. ..$ Ptr :<externalptr>
.. ..$ RZX : num [1:72, 1:3] 0.124 0.125 0.124 0.125 0.124 ...
.. ..$ Ut :Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
.. .. .. ..# i : int [1:960] 0 60 0 60 0 60 0 60 0 60 ...
.. .. .. ..# p : int [1:481] 0 2 4 6 8 10 12 14 16 18 ...
.. .. .. ..# Dim : int [1:2] 72 480
.. .. .. ..# Dimnames:List of 2
.. .. .. .. ..$ : chr [1:72] "2" "4" "6" "8" ...
.. .. .. .. ..$ : NULL
.. .. .. ..# x : num [1:960] 0.373 0.373 0.479 0.479 0.373 ...
.. .. .. ..# factors : list()
.. ..$ Utr : num [1:72] -0.094 -0.018 -0.094 0.058 -0.094 ...
.. ..$ V : num [1:480, 1:3] 0.373 0.479 0.373 0.479 0.451 ...
.. ..$ VtV : num [1:3, 1:3] 92.7 0 0 48.3 48.3 ...
.. ..$ Vtr : num [1:3] 15.738 0.879 3.455
.. ..$ X : num [1:480, 1:3] 1 1 1 1 1 1 1 1 1 1 ...
.. .. ..- attr(*, "dimnames")=List of 2
.. .. .. ..$ : chr [1:480] "1" "2" "3" "4" ...
.. .. .. ..$ : chr [1:3] "(Intercept)" "TreatmentExp" "HabitatB"
.. .. ..- attr(*, "assign")= int [1:3] 0 1 2
.. .. ..- attr(*, "contrasts")=List of 2
.. .. .. ..$ Treatment: chr "contr.treatment"
.. .. .. ..$ Habitat : chr "contr.treatment"
.. ..$ Xwts : num [1:480] 0.373 0.479 0.373 0.479 0.451 ...
.. ..$ Zt :Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
.. .. .. ..# i : int [1:960] 0 60 0 60 0 60 0 60 0 60 ...
.. .. .. ..# p : int [1:481] 0 2 4 6 8 10 12 14 16 18 ...
.. .. .. ..# Dim : int [1:2] 72 480
.. .. .. ..# Dimnames:List of 2
.. .. .. .. ..$ : chr [1:72] "2" "4" "6" "8" ...
.. .. .. .. ..$ : NULL
.. .. .. ..# x : num [1:960] 1 1 1 1 1 1 1 1 1 1 ...
.. .. .. ..# factors : list()
.. ..$ beta0 : num [1:3] 0 0 0
.. ..$ delb : num [1:3] 0 0 0
.. ..$ delu : num [1:72] -0.05 0.0254 -0.05 0.1008 -0.05 ...
.. ..$ theta : num [1:2] 0.0761 1.0537
.. ..$ u0 : num [1:72] 0 0 0 0 0 0 0 0 0 0 ...
.. ..and 42 methods, of which 30 are possibly relevant:
.. .. b, beta, CcNumer, copy#envRefClass, initialize, initializePtr,
.. .. installPars, L, ldL2, ldRX2, linPred, P, ptr, RX, RXdiag, RXi, setBeta0,
.. .. setDelb, setDelu, setTheta, solve, solveU, sqrL, u, unsc, updateDecomp,
.. .. updateL, updateLamtUt, updateRes, updateXwts
..# optinfo:List of 7
.. ..$ optimizer: chr "Nelder_Mead"
.. ..$ control :List of 3
.. .. ..$ xst : num [1:5] 0.02 0.02 0.0721 0.0421 0.0418
.. .. ..$ xt : num [1:5] 1.00e-05 1.00e-05 3.60e-05 2.11e-05 2.09e-05
.. .. ..$ verbose: int 0
.. ..$ derivs :List of 2
.. .. ..$ gradient: num [1:5] 4.64e-05 9.60e-05 -6.73e-05 -6.46e-05 -1.26e-04
.. .. ..$ Hessian : num [1:5, 1:5] 2.644 -0.247 0.162 -1.023 -0.625 ...
.. ..$ conv :List of 2
.. .. ..$ opt : num 0
.. .. ..$ lme4: list()
.. ..$ feval : num 232
.. ..$ warnings : list()
.. ..$ val : num [1:5] 0.0761 1.0537 -1.2511 1.0118 0.676
You can use the function model.matrix to obtain the fixed-effects design matrix:
model.matrix(fit)
where fit is the object returned by glmer.
Related
I was wondering if there was a way to retrieve the data from a model built from the BART package in R?
It seems to be possible using other bart packages, such as dbarts... but I can't seem to find a way to get the original data back from a BART model. For example, if I create some data and run a BART and dbarts model, like so:
library(BART)
library(dbarts)
# create data
df <- data.frame(
x = runif(100),
y = runif(100),
z = runif(100)
)
# create BART
BARTmodel <- wbart(x.train = df[,1:2],
y.train = df[,3])
# create dbarts
DBARTSmodel <- bart(x.train = df[,1:2],
y.train = df[,3],
keeptrees = TRUE)
Using the keeptrees option in dbarts allows me to retrieve the data using:
# retrieve data from dbarts
DBARTSmodel$fit$data#x
However, there doesn't seem to be any type of similar option when using BART. Is it even possible to retrieve the data from a BART model?
The Value: section of ?wbart suggests it doesn't return the input as part of the output, and none of the function arguments for wbart suggest that this can be changed.
Furthermore, if you look at the output of str, you can see that it's not present.
library(BART)
library(dbarts)
# create data
df <- data.frame(
x = runif(100),
y = runif(100),
z = runif(100)
)
# create BART
BARTmodel <- wbart(x.train = df[,1:2],
y.train = df[,3])
# create dbarts
DBARTSmodel <- bart(x.train = df[,1:2],
y.train = df[,3],
keeptrees = TRUE)
str(BARTmodel)
#> List of 13
#> $ sigma : num [1:1100] 0.258 0.262 0.295 0.278 0.273 ...
#> $ yhat.train.mean: num [1:100] 0.584 0.457 0.505 0.54 0.403 ...
#> $ yhat.train : num [1:1000, 1:100] 0.673 0.62 0.433 0.711 0.634 ...
#> $ yhat.test.mean : num(0)
#> $ yhat.test : num[1:1000, 0 ]
#> $ varcount : int [1:1000, 1:2] 109 114 111 118 115 114 115 110 114 117 ...
#> ..- attr(*, "dimnames")=List of 2
#> .. ..$ : NULL
#> .. ..$ : chr [1:2] "x" "y"
#> $ varprob : num [1:1000, 1:2] 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 ...
#> ..- attr(*, "dimnames")=List of 2
#> .. ..$ : NULL
#> .. ..$ : chr [1:2] "x" "y"
#> $ treedraws :List of 2
#> ..$ cutpoints:List of 2
#> .. ..$ x: num [1:100] 0.0147 0.0245 0.0343 0.0442 0.054 ...
#> .. ..$ y: num [1:100] 0.0395 0.0491 0.0586 0.0681 0.0776 ...
#> ..$ trees : chr "1000 200 2\n1\n1 0 0 0.01185590432\n3\n1 1 30 -0.01530736435\n2 0 0 0.01064412946\n3 0 0 0.02413784284\n3\n1 0 "| __truncated__
#> $ proc.time : 'proc_time' Named num [1:5] 1.406 0.008 1.415 0 0
#> ..- attr(*, "names")= chr [1:5] "user.self" "sys.self" "elapsed" "user.child" ...
#> $ mu : num 0.501
#> $ varcount.mean : Named num [1:2] 115 110
#> ..- attr(*, "names")= chr [1:2] "x" "y"
#> $ varprob.mean : Named num [1:2] 0.5 0.5
#> ..- attr(*, "names")= chr [1:2] "x" "y"
#> $ rm.const : int [1:2] 1 2
#> - attr(*, "class")= chr "wbart"
Whereas the output of str() for the bart output, while long, does contain the input:
str(DBARTSmodel)
#> List of 11
#> $ call : language bart(x.train = df[, 1:2], y.train = df[, 3], keeptrees = TRUE)
#> $ first.sigma : num [1:100] 0.289 0.311 0.268 0.253 0.242 ...
#> $ sigma : num [1:1000] 0.288 0.307 0.248 0.257 0.293 ...
#> $ sigest : num 0.295
#> $ yhat.train : num [1:1000, 1:100] 0.715 0.677 0.508 0.51 0.827 ...
#> $ yhat.train.mean: num [1:100] 0.583 0.456 0.504 0.544 0.404 ...
#> $ yhat.test : NULL
#> $ yhat.test.mean : NULL
#> $ varcount : int [1:1000, 1:2] 128 118 120 142 130 145 145 150 138 138 ...
#> ..- attr(*, "dimnames")=List of 2
#> .. ..$ : NULL
#> .. ..$ : chr [1:2] "x" "y"
#> $ y : num [1:100] 0.8489 0.0817 0.4371 0.8566 0.0878 ...
#> $ fit :Reference class 'dbartsSampler' [package "dbarts"] with 5 fields
#> ..$ pointer:<externalptr>
#> ..$ control:Formal class 'dbartsControl' [package "dbarts"] with 18 slots
#> .. .. ..# binary : logi FALSE
#> .. .. ..# verbose : logi TRUE
#> .. .. ..# keepTrainingFits: logi TRUE
#> .. .. ..# useQuantiles : logi FALSE
#> .. .. ..# keepTrees : logi TRUE
#> .. .. ..# n.samples : int 1000
#> .. .. ..# n.burn : int 100
#> .. .. ..# n.trees : int 200
#> .. .. ..# n.chains : int 1
#> .. .. ..# n.threads : int 1
#> .. .. ..# n.thin : int 1
#> .. .. ..# printEvery : int 100
#> .. .. ..# printCutoffs : int 0
#> .. .. ..# rngKind : chr "default"
#> .. .. ..# rngNormalKind : chr "default"
#> .. .. ..# rngSeed : int NA
#> .. .. ..# updateState : logi TRUE
#> .. .. ..# call : language bart(x.train = df[, 1:2], y.train = df[, 3], keeptrees = TRUE)
#> ..$ model :Formal class 'dbartsModel' [package "dbarts"] with 9 slots
#> .. .. ..# p.birth_death : num 0.5
#> .. .. ..# p.swap : num 0.1
#> .. .. ..# p.change : num 0.4
#> .. .. ..# p.birth : num 0.5
#> .. .. ..# node.scale : num 0.5
#> .. .. ..# tree.prior :Formal class 'dbartsCGMPrior' [package "dbarts"] with 3 slots
#> .. .. .. .. ..# power : num 2
#> .. .. .. .. ..# base : num 0.95
#> .. .. .. .. ..# splitProbabilities: num(0)
#> .. .. ..# node.prior :Formal class 'dbartsNormalPrior' [package "dbarts"] with 0 slots
#> list()
#> .. .. ..# node.hyperprior:Formal class 'dbartsFixedHyperprior' [package "dbarts"] with 1 slot
#> .. .. .. .. ..# k: num 2
#> .. .. ..# resid.prior :Formal class 'dbartsChiSqPrior' [package "dbarts"] with 2 slots
#> .. .. .. .. ..# df : num 3
#> .. .. .. .. ..# quantile: num 0.9
#> ..$ data :Formal class 'dbartsData' [package "dbarts"] with 10 slots
#> .. .. ..# y : num [1:100] 0.8489 0.0817 0.4371 0.8566 0.0878 ...
#> .. .. ..# x : num [1:100, 1:2] 0.152 0.666 0.967 0.248 0.668 ...
#> .. .. .. ..- attr(*, "dimnames")=List of 2
#> .. .. .. .. ..$ : NULL
#> .. .. .. .. ..$ : chr [1:2] "x" "y"
#> .. .. .. ..- attr(*, "drop")=List of 2
#> .. .. .. .. ..$ x: logi FALSE
#> .. .. .. .. ..$ y: logi FALSE
#> .. .. .. ..- attr(*, "term.labels")= chr [1:2] "x" "y"
#> .. .. ..# varTypes : int [1:2] 0 0
#> .. .. ..# x.test : NULL
#> .. .. ..# weights : NULL
#> .. .. ..# offset : NULL
#> .. .. ..# offset.test : NULL
#> .. .. ..# n.cuts : int [1:2] 100 100
#> .. .. ..# sigma : num 0.295
#> .. .. ..# testUsesRegularOffset: logi NA
#> ..$ state :List of 1
#> .. ..$ :Formal class 'dbartsState' [package "dbarts"] with 6 slots
#> .. .. .. ..# trees : int [1:1055] 0 18 -1 0 49 -1 -1 0 60 -1 ...
#> .. .. .. ..# treeFits : num [1:100, 1:200] -0.02252 0.00931 0.00931 0.02688 0.00931 ...
#> .. .. .. ..# savedTrees: int [1:2340360] 0 797997482 1070928224 1 -402902351 1070268808 -1 -1094651769 -1081938039 -1 ...
#> .. .. .. ..# sigma : num 0.297
#> .. .. .. ..# k : num 2
#> .. .. .. ..# rng.state : int [1:18] 0 1078575104 0 1078575104 -1657977906 1075613906 0 1078558720 277209871 -1068236140 ...
#> .. ..- attr(*, "runningTime")= num 0.477
#> .. ..- attr(*, "currentNumSamples")= int 1000
#> .. ..- attr(*, "currentSampleNum")= int 0
#> .. ..- attr(*, "numCuts")= int [1:2] 100 100
#> .. ..- attr(*, "cutPoints")=List of 2
#> .. .. ..$ : num [1:100] 0.0147 0.0245 0.0343 0.0442 0.054 ...
#> .. .. ..$ : num [1:100] 0.0395 0.0491 0.0586 0.0681 0.0776 ...
#> ..and 40 methods, of which 26 are possibly relevant:
#> .. copy#envRefClass, getLatents, getPointer, getTrees, initialize, plotTree,
#> .. predict, printTrees, run, sampleNodeParametersFromPrior,
#> .. sampleTreesFromPrior, setControl, setCutPoints, setData, setModel,
#> .. setOffset, setPredictor, setResponse, setSigma, setState, setTestOffset,
#> .. setTestPredictor, setTestPredictorAndOffset, setWeights,
#> .. show#envRefClass, storeState
#> - attr(*, "class")= chr "bart"
You can achieve what you are looking for using bartModelMatrix() function form BART package.
This function it will determinate the number of cutpoints necessary for each column.
In this way, you'll have so many columns as variables you have in your df.
In your example you're only insterested in x and y, so you'll only care for the first and second column from bartModelMatrix() matrix obtained.
So for the example you gave:
# create data
df <- data.frame(
x = runif(100),
y = runif(100),
z = runif(100),
)
# create BART
BARTmodel <- wbart(x.train = df[,1:2],
y.train = df[,3])
# create dbarts
DBARTSmodel <- bart(x.train = df[,1:2],
y.train = df[,3],
keeptrees = TRUE)
BARTmatrix <- bartModelMatrix(df)
BARTmatrix <- BARTmatrix[,1:2]
BARTmatrix == DBARTSmodel$fit$data#x
Hope that helped you
I obtain 0 fitted values when using nnet function as below:
nnet(realized1~ann_t1+ann_t2+ann_t3, data=df2_input, size=2)
Output is
> str(nnet(realized1~ann_t1+ann_t2+ann_t3, data=df2_input, size=2))
# weights: 11
initial value 134.845214
final value 0.147077
converged
List of 18
$ n : num [1:3] 3 2 1
$ nunits : int 7
$ nconn : num [1:8] 0 0 0 0 0 4 8 11
$ conn : num [1:11] 0 1 2 3 0 1 2 3 0 4 ...
$ nsunits : int 7
$ decay : num 0
$ entropy : logi FALSE
$ softmax : logi FALSE
$ censored : logi FALSE
$ value : num 0.147
$ wts : num [1:11] 14.815 -69.862 0.244 -0.456 -5.638 ...
$ convergence : int 0
$ fitted.values: num [1:800, 1] 0 0 0 0 0 0 0 0 0 0 ...
..- attr(*, "dimnames")=List of 2
.. ..$ : chr [1:800] "1601" "1602" "1603" "1604" ...
.. ..$ : NULL
$ residuals : num [1:800, 1] 0.004267 0.000401 0.002404 0.022561 0.001354 ...
..- attr(*, "dimnames")=List of 2
.. ..$ : chr [1:800] "1601" "1602" "1603" "1604" ...
.. ..$ : NULL
$ call : language nnet.formula(formula = realized1 ~ ann_t1 + ann_t2 + ann_t3, data = df2_input, size = 2)
$ terms :Classes 'terms', 'formula' language realized1 ~ ann_t1 + ann_t2 + ann_t3
.. ..- attr(*, "variables")= language list(realized1, ann_t1, ann_t2, ann_t3)
.. ..- attr(*, "factors")= int [1:4, 1:3] 0 1 0 0 0 0 1 0 0 0 ...
.. .. ..- attr(*, "dimnames")=List of 2
.. .. .. ..$ : chr [1:4] "realized1" "ann_t1" "ann_t2" "ann_t3"
.. .. .. ..$ : chr [1:3] "ann_t1" "ann_t2" "ann_t3"
.. ..- attr(*, "term.labels")= chr [1:3] "ann_t1" "ann_t2" "ann_t3"
.. ..- attr(*, "order")= int [1:3] 1 1 1
.. ..- attr(*, "intercept")= int 1
.. ..- attr(*, "response")= int 1
.. ..- attr(*, ".Environment")=<environment: R_GlobalEnv>
.. ..- attr(*, "predvars")= language list(realized1, ann_t1, ann_t2, ann_t3)
.. ..- attr(*, "dataClasses")= Named chr [1:4] "numeric" "numeric" "numeric" "numeric"
.. .. ..- attr(*, "names")= chr [1:4] "realized1" "ann_t1" "ann_t2" "ann_t3"
$ coefnames : chr [1:3] "ann_t1" "ann_t2" "ann_t3"
$ xlevels : Named list()
- attr(*, "class")= chr [1:2] "nnet.formula" "nnet"
Input data frame is as below:
> str(df2_input)
'data.frame': 800 obs. of 4 variables:
$ realized1: num 0.004267 0.000401 0.002404 0.022561 0.001354 ...
$ ann_t1 : num -4.4 -4.19 -4.3 -4.48 -4.6 ...
$ ann_t2 : num 0 -0.002996 -0.000298 -0.001964 -0.01963 ...
$ ann_t3 : num 0 -0.01662 0.00165 -0.01089 -0.10887 ...
Why do I get zero predictions and how can I fix it?
Thanks a lot.
I have done many bayesian models using the MCMCglmm package in R, like this one:
model=MCMCglmm(scale(lifespan)~scale(weight)*scale(littersize),
random=~idv(DNA1)+idv(DNA2),
data=df,
family="gaussian",
prior=prior1,
thin=50,
burnin=5000,
nitt=50000,
verbose=F)
summary(model)
post.mean l-95% CI u-95% CI eff.samp pMCMC
(Intercept) 11.23327 8.368 13.73756 6228 <2e-04 ***
weight -1.63770 -2.059 -1.23457 6600 <2e-04 ***
littersize 0.40960 0.024 0.80305 6600 0.0415 *
weight:littersize -0.33411 -0.635 -0.04406 5912 0.0248 *
I would like to plot the resulting interaction (weight:littersize) with ggeffects or sjPlots packages, like this:
plot_model(model,
type = "int",
terms = c("scale(lifespan)", "scale(weight)", "scale(littersize)"),
mdrt.values = "meansd",
ppd = TRUE)
But I obtain the next output:
`scale(weight)` was not found in model terms. Maybe misspelled?
`scale(littersize)` was not found in model terms. Maybe misspelled?
Error in terms.default(model) : no terms component nor attribute
Además: Warning messages:
1: Some model terms could not be found in model data. You probably need to load the data into the environment.
2: Some model terms could not be found in model data. You probably need to load the data into the environment.
Data is already loaded. I tried to write terms differently without the "scale(x)" term, and changed the model too to deal with equal terms, but I am still getting this error message. I am also open to plot this interaction with different packages.
My model str(model) is:
>str(model)
List of 20
$ Sol : 'mcmc' num [1:6600, 1:4] -0.814 1.215 -2.119 -0.125 -1.648 ...
..- attr(*, "dimnames")=List of 2
.. ..$ : NULL
.. ..$ : chr [1:4] "(Intercept)" "scale(weight)" "scale(littersize)" "scale(weight):scale(littersize)"
..- attr(*, "mcpar")= num [1:3] 7e+04 4e+05 5e+01
$ Lambda : NULL
$ VCV : 'mcmc' num [1:6600, 1:3] 1.094 0.693 1.58 0.645 1.161 ...
..- attr(*, "dimnames")=List of 2
.. ..$ : NULL
.. ..$ : chr [1:3] "phylo." "haplo." "units"
..- attr(*, "mcpar")= num [1:3] 7e+04 4e+05 5e+01
$ CP : NULL
$ Liab : NULL
$ Fixed :List of 3
..$ formula:Class 'formula' language scale(lifespan) ~ scale(weight) * scale(littersize)
.. .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv>
..$ nfl : int 4
..$ nll : num 0
$ Random :List of 5
..$ formula:Class 'formula' language ~idv(phylo) + idv(haplo)
.. .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv>
..$ nfl : num [1:2] 1 1
..$ nrl : int [1:2] 92 92
..$ nat : num [1:2] 0 0
..$ nrt : int [1:2] 1 1
$ Residual :List of 6
..$ formula :Class 'formula' language ~units
.. .. ..- attr(*, ".Environment")=<environment: 0x0000025ba05f8938>
..$ nfl : num 1
..$ nrl : int 92
..$ nrt : int 1
..$ family : chr "gaussian"
..$ original.family: chr "gaussian"
$ Deviance : 'mcmc' num [1:6600] -262.6 -137.3 -203.6 -83.6 -29.1 ...
..- attr(*, "mcpar")= num [1:3] 7e+04 4e+05 5e+01
$ DIC : num -158
$ X :Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
.. ..# i : int [1:368] 0 1 2 3 4 5 6 7 8 9 ...
.. ..# p : int [1:5] 0 92 184 276 368
.. ..# Dim : int [1:2] 92 4
.. ..# Dimnames:List of 2
.. .. ..$ : chr [1:92] "1.1" "2.1" "3.1" "4.1" ...
.. .. ..$ : chr [1:4] "(Intercept)" "scale(weight)" "scale(littersize)" "scale(weight):scale(littersize)"
.. ..# x : num [1:368] 1 1 1 1 1 1 1 1 1 1 ...
.. ..# factors : list()
$ Z :Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
.. ..# i : int [1:16928] 0 1 2 3 4 5 6 7 8 9 ...
.. ..# p : int [1:185] 0 92 184 276 368 460 552 644 736 828 ...
.. ..# Dim : int [1:2] 92 184
.. ..# Dimnames:List of 2
.. .. ..$ : NULL
.. .. ..$ : chr [1:184] "phylo1.NA.1" "phylo2.NA.1" "phylo3.NA.1" "phylo4.NA.1" ...
.. ..# x : num [1:16928] 0.4726 0.0869 0.1053 0.087 0.1349 ...
.. ..# factors : list()
$ ZR :Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
.. ..# i : int [1:92] 0 1 2 3 4 5 6 7 8 9 ...
.. ..# p : int [1:93] 0 1 2 3 4 5 6 7 8 9 ...
.. ..# Dim : int [1:2] 92 92
.. ..# Dimnames:List of 2
.. .. ..$ : NULL
.. .. ..$ : chr [1:92] "units.1" "units.2" "units.3" "units.4" ...
.. ..# x : num [1:92] 1 1 1 1 1 1 1 1 1 1 ...
.. ..# factors : list()
$ XL : NULL
$ ginverse : NULL
$ error.term : int [1:92] 1 1 1 1 1 1 1 1 1 1 ...
$ family : chr [1:92] "gaussian" "gaussian" "gaussian" "gaussian" ...
$ Tune : num [1, 1] 1
..- attr(*, "dimnames")=List of 2
.. ..$ : chr "1"
.. ..$ : chr "1"
$ meta : logi FALSE
$ y.additional: num [1:92, 1:2] 0 0 0 0 0 0 0 0 0 0 ...
- attr(*, "class")= chr "MCMCglmm"
Thank you.
Try to scale your predictors before fitting the model, i.e.
df$lifespan <- as.vecor(scale(df$lifespan))
Or better, use effectsize::standardize(), which does not create a matrix for a one-dimensial vector when scaling your variables:
df <- effectsize::standardize(df, select = c("lifespan", "weight", "littersize"))
Then you can call your model like this:
model <- MCMCglmm(lifespan ~ weight * littersize,
random=~idv(DNA1)+idv(DNA2),
data=df,
family="gaussian",
prior=prior1,
thin=50,
burnin=5000,
nitt=50000,
verbose=F)
Does this work?
i'm current construct product recommendation using R recommenderlab, after compute AR recommender, i'm hoping to understand the association rules, but i couldn't found any why to extract the complete association rules from the recommender object.
Below is the sample dataset
m <- matrix(sample(c(0,1), 50, replace=TRUE), nrow=5, ncol=10,
dimnames=list(users=paste("u", 1:5, sep=''),
items=paste("i", 1:10, sep='')))
Convert matrix into binaryRatingMatrix
b <- as(m, "binaryRatingMatrix")
create a user-based CF recommender using training data
r <- Recommender(getData(scheme, "train"), "AR")
Looking into the AR recommender object r#model$rule_base i found "rule_base"
Formal class 'Recommender' [package "recommenderlab"] with 5 slots
..# method : chr "AR"
..# dataType: chr "binaryRatingMatrix"
..# ntrain : int 5
..# model :List of 9
.. ..$ description : chr "AR: rule base"
.. ..$ rule_base :Formal class 'rules' [package "arules"] with 4 slots
.. .. .. ..# lhs :Formal class 'itemMatrix' [package "arules"] with 3 slots
.. .. .. .. .. ..# data :Formal class 'ngCMatrix' [package "Matrix"] with 5 slots
.. .. .. .. .. .. .. ..# i : int [1:145] 1 1 1 1 2 0 0 0 5 5 ...
.. .. .. .. .. .. .. ..# p : int [1:80] 0 1 2 3 4 5 6 7 8 9 ...
.. .. .. .. .. .. .. ..# Dim : int [1:2] 10 79
.. .. .. .. .. .. .. ..# Dimnames:List of 2
.. .. .. .. .. .. .. .. ..$ : NULL
.. .. .. .. .. .. .. .. ..$ : NULL
.. .. .. .. .. .. .. ..# factors : list()
.. .. .. .. .. ..# itemInfo :'data.frame': 10 obs. of 1 variable:
.. .. .. .. .. .. ..$ labels: chr [1:10] "i1" "i2" "i3" "i4" ...
.. .. .. .. .. ..# itemsetInfo:'data.frame': 0 obs. of 0 variables
.. .. .. ..# rhs :Formal class 'itemMatrix' [package "arules"] with 3 slots
.. .. .. .. .. ..# data :Formal class 'ngCMatrix' [package "Matrix"] with 5 slots
.. .. .. .. .. .. .. ..# i : int [1:79] 6 4 9 3 8 4 9 3 6 3 ...
.. .. .. .. .. .. .. ..# p : int [1:80] 0 1 2 3 4 5 6 7 8 9 ...
.. .. .. .. .. .. .. ..# Dim : int [1:2] 10 79
.. .. .. .. .. .. .. ..# Dimnames:List of 2
.. .. .. .. .. .. .. .. ..$ : NULL
.. .. .. .. .. .. .. .. ..$ : NULL
.. .. .. .. .. .. .. ..# factors : list()
.. .. .. .. .. ..# itemInfo :'data.frame': 10 obs. of 1 variable:
.. .. .. .. .. .. ..$ labels: chr [1:10] "i1" "i2" "i3" "i4" ...
.. .. .. .. .. ..# itemsetInfo:'data.frame': 0 obs. of 0 variables
.. .. .. ..# quality:'data.frame': 79 obs. of 4 variables:
.. .. .. .. ..$ support : num [1:79] 0.2 0.2 0.2 0.2 0.4 0.4 0.4 0.4 0.4 0.4 ...
.. .. .. .. ..$ confidence: num [1:79] 1 1 1 1 1 1 1 1 1 1 ...
.. .. .. .. ..$ lift : num [1:79] 1.67 1.25 1.25 1.25 1.67 ...
.. .. .. .. ..$ count : num [1:79] 1 1 1 1 2 2 2 2 2 2 ...
.. .. .. ..# info :List of 4
.. .. .. .. ..$ data : symbol data
.. .. .. .. ..$ ntransactions: int 5
.. .. .. .. ..$ support : num 0.1
.. .. .. .. ..$ confidence : num 0.8
.. ..$ support : num 0.1
.. ..$ confidence : num 0.8
.. ..$ maxlen : num 3
.. ..$ sort_measure : chr "confidence"
.. ..$ sort_decreasing: logi TRUE
.. ..$ apriori_control:List of 1
.. .. ..$ verbose: logi FALSE
.. ..$ verbose : logi FALSE
..# predict :function (model, newdata, n = 10, data = NULL, type = c("topNList", "ratings", "ratingMatrix"), ...)
Question: how can i extract the association rules from recommender object as data frame?
Get association rules dataframe with columns (LHS, RHS, support, confidence, lift, count)
you can use
#Convert rules into data frame
rules3 = as(rules, "data.frame")
I would like to extract some results from a lmermod object
require(lme4)
(fm1 <- lmer(Yield ~ 1|Batch, Dyestuff))
This produces
Linear mixed model fit by REML ['lmerMod']
Formula: Yield ~ 1 | Batch
Data: Dyestuff
REML criterion at convergence: 319.6543
Random effects:
Groups Name Std.Dev.
Batch (Intercept) 42.00
Residual 49.51
Number of obs: 30, groups: Batch, 6
Fixed Effects:
(Intercept)
1527
In particular, I would like to extract the standard devations of the 2 random effects, ie 42.00 and 49.51. I guess there may be a built-in method to do this, but I couldn't find it quickly so I thought I should be able to find it by using str(fm1), which produces this:
Formal class 'lmerMod' [package "lme4"] with 13 slots
..# resp :Reference class 'lmerResp' [package "lme4"] with 9 fields
.. ..$ Ptr :<externalptr>
.. ..$ mu : num [1:30] 1510 1510 1510 1510 1510 ...
.. ..$ offset : num [1:30] 0 0 0 0 0 0 0 0 0 0 ...
.. ..$ sqrtXwt: num [1:30] 1 1 1 1 1 1 1 1 1 1 ...
.. ..$ sqrtrwt: num [1:30] 1 1 1 1 1 1 1 1 1 1 ...
.. ..$ weights: num [1:30] 1 1 1 1 1 1 1 1 1 1 ...
.. ..$ wtres : num [1:30] 35.1 -69.9 -69.9 10.1 70.1 ...
.. ..$ y : num [1:30] 1545 1440 1440 1520 1580 ...
.. ..$ REML : int 1
.. ..and 26 methods, of which 14 are possibly relevant:
.. .. allInfo, copy#envRefClass, initialize, initialize#lmResp,
.. .. initializePtr, initializePtr#lmResp, objective, ptr, ptr#lmResp,
.. .. setOffset, setResp, setWeights, updateMu, wrss
..# Gp : int [1:2] 0 6
..# call : language lmer(formula = Yield ~ 1 | Batch, data = Dyestuff)
..# frame :'data.frame': 30 obs. of 2 variables:
.. ..$ Yield: num [1:30] 1545 1440 1440 1520 1580 ...
.. ..$ Batch: Factor w/ 6 levels "A","B","C","D",..: 1 1 1 1 1 2 2 2 2 2 ...
.. ..- attr(*, "terms")=Classes 'terms', 'formula' length 3 Yield ~ 1 + Batch
.. .. .. ..- attr(*, "variables")= language list(Yield, Batch)
.. .. .. ..- attr(*, "factors")= int [1:2, 1] 0 1
.. .. .. .. ..- attr(*, "dimnames")=List of 2
.. .. .. .. .. ..$ : chr [1:2] "Yield" "Batch"
.. .. .. .. .. ..$ : chr "Batch"
.. .. .. ..- attr(*, "term.labels")= chr "Batch"
.. .. .. ..- attr(*, "order")= int 1
.. .. .. ..- attr(*, "intercept")= int 1
.. .. .. ..- attr(*, "response")= int 1
.. .. .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv>
.. .. .. ..- attr(*, "predvars")= language list(Yield, Batch)
.. .. .. ..- attr(*, "dataClasses")= Named chr [1:2] "numeric" "factor"
.. .. .. .. ..- attr(*, "names")= chr [1:2] "Yield" "Batch"
.. .. .. ..- attr(*, "predvars.fixed")= language list(Yield)
.. ..- attr(*, "formula")=Class 'formula' length 3 Yield ~ 1 | Batch
.. .. .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv>
..# flist :List of 1
.. ..$ Batch: Factor w/ 6 levels "A","B","C","D",..: 1 1 1 1 1 2 2 2 2 2 ...
.. ..- attr(*, "assign")= int 1
..# cnms :List of 1
.. ..$ Batch: chr "(Intercept)"
..# lower : num 0
..# theta : num 0.848
..# beta : num 1527
..# u : num [1:6] -20.755 0.461 33.669 -27.212 66.877 ...
..# devcomp:List of 2
.. ..$ cmp : Named num [1:10] 9.15 1.88 61495.41 9590.84 71086.25 ...
.. .. ..- attr(*, "names")= chr [1:10] "ldL2" "ldRX2" "wrss" "ussq" ...
.. ..$ dims: Named int [1:12] 30 30 1 29 1 6 1 1 0 1 ...
.. .. ..- attr(*, "names")= chr [1:12] "N" "n" "p" "nmp" ...
..# pp :Reference class 'merPredD' [package "lme4"] with 18 fields
.. ..$ Lambdat:Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
.. .. .. ..# i : int [1:6] 0 1 2 3 4 5
.. .. .. ..# p : int [1:7] 0 1 2 3 4 5 6
.. .. .. ..# Dim : int [1:2] 6 6
.. .. .. ..# Dimnames:List of 2
.. .. .. .. ..$ : NULL
.. .. .. .. ..$ : NULL
.. .. .. ..# x : num [1:6] 0.848 0.848 0.848 0.848 0.848 ...
.. .. .. ..# factors : list()
.. ..$ LamtUt :Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
.. .. .. ..# i : int [1:30] 0 0 0 0 0 1 1 1 1 1 ...
.. .. .. ..# p : int [1:31] 0 1 2 3 4 5 6 7 8 9 ...
.. .. .. ..# Dim : int [1:2] 6 30
.. .. .. ..# Dimnames:List of 2
.. .. .. .. ..$ : NULL
.. .. .. .. ..$ : NULL
.. .. .. ..# x : num [1:30] 0.848 0.848 0.848 0.848 0.848 ...
.. .. .. ..# factors : list()
.. ..$ Lind : int [1:6] 1 1 1 1 1 1
.. ..$ Ptr :<externalptr>
.. ..$ RZX : num [1:6, 1] 1.98 1.98 1.98 1.98 1.98 ...
.. ..$ Ut :Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
.. .. .. ..# i : int [1:30] 0 0 0 0 0 1 1 1 1 1 ...
.. .. .. ..# p : int [1:31] 0 1 2 3 4 5 6 7 8 9 ...
.. .. .. ..# Dim : int [1:2] 6 30
.. .. .. ..# Dimnames:List of 2
.. .. .. .. ..$ : chr [1:6] "A" "B" "C" "D" ...
.. .. .. .. ..$ : NULL
.. .. .. ..# x : num [1:30] 1 1 1 1 1 1 1 1 1 1 ...
.. .. .. ..# factors : list()
.. ..$ Utr : num [1:6] 6384 6481 6634 6354 6787 ...
.. ..$ V : num [1:30, 1] 1 1 1 1 1 1 1 1 1 1 ...
.. ..$ VtV : num [1, 1] 30
.. ..$ Vtr : num 45825
.. ..$ X : num [1:30, 1] 1 1 1 1 1 1 1 1 1 1 ...
.. .. ..- attr(*, "dimnames")=List of 2
.. .. .. ..$ : chr [1:30] "1" "2" "3" "4" ...
.. .. .. ..$ : chr "(Intercept)"
.. .. ..- attr(*, "assign")= int 0
.. ..$ Xwts : num [1:30] 1 1 1 1 1 1 1 1 1 1 ...
.. ..$ Zt :Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
.. .. .. ..# i : int [1:30] 0 0 0 0 0 1 1 1 1 1 ...
.. .. .. ..# p : int [1:31] 0 1 2 3 4 5 6 7 8 9 ...
.. .. .. ..# Dim : int [1:2] 6 30
.. .. .. ..# Dimnames:List of 2
.. .. .. .. ..$ : chr [1:6] "A" "B" "C" "D" ...
.. .. .. .. ..$ : NULL
.. .. .. ..# x : num [1:30] 1 1 1 1 1 1 1 1 1 1 ...
.. .. .. ..# factors : list()
.. ..$ beta0 : num 0
.. ..$ delb : num 1527
.. ..$ delu : num [1:6] -20.755 0.461 33.669 -27.212 66.877 ...
.. ..$ theta : num 0.848
.. ..$ u0 : num [1:6] 0 0 0 0 0 0
.. ..and 42 methods, of which 30 are possibly relevant:
.. .. b, beta, CcNumer, copy#envRefClass, initialize, initializePtr,
.. .. installPars, L, ldL2, ldRX2, linPred, P, ptr, RX, RXdiag, RXi,
.. .. setBeta0, setDelb, setDelu, setTheta, solve, solveU, sqrL, u, unsc,
.. .. updateDecomp, updateL, updateLamtUt, updateRes, updateXwts
..# optinfo:List of 7
.. ..$ optimizer: chr "bobyqa"
.. ..$ control :List of 1
.. .. ..$ iprint: int 0
.. ..$ derivs :List of 2
.. .. ..$ gradient: num 1.61e-07
.. .. ..$ Hessian : num [1, 1] 14.1
.. ..$ conv :List of 2
.. .. ..$ opt : int 0
.. .. ..$ lme4: list()
.. ..$ feval : int 16
.. ..$ warnings : list()
.. ..$ val : num 0.848
...but this leaves me none the wiser.
lme4 supplies VarCorr for extracting variance and correlation components:
varcor <- VarCorr(fm1)
as.data.frame(varcor)
# grp var1 var2 vcov sdcor
#1 Batch (Intercept) <NA> 1764.05 42.0006
#2 Residual <NA> <NA> 2451.25 49.5101