I am trying to evaluate the sem model from a dataset, some of the data are in likert scale i.e from 1-5. and some of the data are COUNTS generated from the computer log for some of the activity.
Whereas while performing the fits the laveen is giving me the error as:
lavaan WARNING: some observed variances are (at least) a factor 1000 times larger than others; use varTable(fit) to investigate
To mitigate this warning I want to scale some of the variables. But couldn't understand the way for doing that.
Log_And_SurveyResult <- read_excel("C:/Users/Aakash/Desktop/analysis/Log-And-SurveyResult.xlsx")
model <- '
Reward =~ REW1 + REW2 + REW3 + REW4
ECA =~ ECA1 + ECA2 + ECA3
Feedback =~ FED1 + FED2 + FED3 + FED4
Motivation =~ Reward + ECA + Feedback
Satisfaction =~ a*MaxTimeSpentInAWeek + a*TotalTimeSpent + a*TotalLearningActivityView
Motivation ~ Satisfaction'
fit <- sem(model,data = Log_And_SurveyResult)
summary(fit, standardized=T, std.lv = T)
fitMeasures(fit, c("cfi", "rmsea", "srmr"))
I want to scale some of the variables like MaxTimeSpentInAWeek and TotalTimeSpent
Could you please help me figure out how to scale the variables? Thank you very much.
As Elias pointed out, the difference in the magnitude between the variables is huge and it is suggested to scale the variables.
The warning gives a hint and inspecting varTable(fit) returns summary information about the variables in a fitted lavaan object.
Rather than running scale() separately on each column, you could use apply() on a subset or on your whole data.frame:
## Scale the variables in the 4th and 7h column
Log_And_SurveyResult[, c(4, 7)] <- apply(Log_And_SurveyResult[, c(4, 7)], 2, scale)
## Scale the whole data.frame
Log_And_SurveyResult <- apply(Log_And_SurveyResult, 2, scale)
You can just use scale(MaxTimeSpentInAWeek). This will scale your variable to mean = 0 and variance = 1. E.g:
Log_And_SurveyResult$MaxTimeSpentInAWeek <-
scale(Log_And_SurveyResult$MaxTimeSpentInAWeek)
Log_And_SurveyResult$TotalTimeSpent <-
scale(Log_And_SurveyResult$TotalTimeSpent)
Or did I misunderstand your question?
I am currently calculating a multi-level analysis with 32 countries (country variable "CNTRY3"). The dependent variable is the willingness to pay for environmental protection "WTP" (scale 1 - 5, centred). I have included four random slopes in the third step (Random Intercept Random Slope) of the multi-level analysis (I would like to vary these based on my theory):
RINC_ALL_z -> Income per person (z-standardized)
social_trust (scale 1 - 5, centered) -> trust in other people
political_trust (scale 1 - 5, centred) -> trust in politics
EC_cen (scale 1 - 5, centered) -> Environmental awareness
.
modell.3a <- lmer(WTP ~ RINC_ALL_z + social_trust_cen + political_trust_cen +
Men + lowest_degree + middle_degree + requirement_university +
uncompleted_university + university_degree + AGE_cen + urban +
EC_cen + (RINC_ALL_z + social_trust_cen + political_trust_cen +
EC_cen|CNTRY3), data=ISSP2010_1)
Then the error message appears:
convergence code: 0 Model failed to converge with max|grad| =
0.00527884 (tol = 0.002, component 1)
Now I was able to research that if you calculate the model with the optimizer "BOBYQA", for example, these "convergence warnings" can be bypassed. And indeed, if I calculate the model like this, then no convergence warning appears any more:
modell.3b <- lmer(WTP ~ RINC_ALL_z + social_trust_cen + political_trust_cen +
Men + lowest_degree + middle_degree + requirement_university +
uncompleted_university + university_degree + AGE_cen + urban +
EC_cen + (RINC_ALL_z + social_trust_cen + political_trust_cen +
EC_cen|CNTRY3), data=ISSP2010_1, control = lmerControl(optimizer = "bobyqa",
optCtrl=list(maxfun=1e5)))
So I read in this very interesting article that if you use one optimizer, you should compare it with all other available optimizers to find out if certain optimizers influence the parameters of the regression. No sooner said than done (with the variable environmental awareness), I replicated the graphs of the article, but unfortunately the optimizers are not in one column (loglikelihood / t-value) as in the article. I have attached the two pictures here. My interpretation would be, regarding the log-likelihood comparison and the t-value comparison, that the majority (5) of the optimizers are in one column (including my used "BOBYQA" ) and only 2 optimizers differ from the majority of optimizers, so my used optimizer should not influence the parameters, right?
Loglikelihood comparison
T-Value Comparison
The first question would be what such an optimizer does (you just read that you should use optimizers to avoid convergence issues)?
The second question I have is, would you agree with this interpretation of the two diagrams?
I would be very happy about an answer, I have been thinking about this for several days... :-(
Many greetings
Joern
Please again accept my apologies for my little knowledge in R. I'm, trying to get better! I'm a biologist and my statistical knowledge is sadly low
I have the following data set:
Perc_Reacting,Pulses,IndMutant,Proportion
93,1,1,0.93
81,2,1,0.81
73,3,1,0.73
64,4,1,0.64
73,5,1,0.73
68,6,1,0.68
64,7,1,0.64
65,8,1,0.65
50,9,1,0.5
68,10,1,0.68
57,11,1,0.57
50,12,1,0.5
62,13,1,0.62
44,14,1,0.44
54,15,1,0.54
56,16,1,0.56
50,17,1,0.5
42,18,1,0.42
42,19,1,0.42
29,20,1,0.29
96,1,0,0.96
100,2,0,1
92,3,0,0.92
96,4,0,0.96
92,5,0,0.92
92,6,0,0.92
84,7,0,0.84
96,8,0,0.96
91,9,0,0.91
82,10,0,0.82
86,11,0,0.86
82,12,0,0.82
91,13,0,0.91
85,14,0,0.85
83,15,0,0.83
70,16,0,0.7
74,17,0,0.74
64,18,0,0.64
68,19,0,0.68
78,20,0,0.78
The first and last rows are the same, one expressed in % an the other in a 1-0 proportion
I need to run a Beta regression model, but when I try to create the model an error jumps:
model.beta<-betareg(C_elegans$Proportion~C_elegans$Pulses)
Error in betareg(C_elegans$Proportion ~ C_elegans$Pulses) :
invalid dependent variable, all observations must be in (0, 1)
Could you help me to create a beta regression model for this data and how to make relevant plots to show it fits good?
Also I need to propose a linear regression model for this data, can anyone let me know how you think it could be done better?
Here are the results of fitting the last three columns to a flat surface plane equation "Proportion = a + (b * Pulses) + (c * IndMutant)" with parameters a = 1.0468289473684214E+00, b = -1.8650375939849695E-02, and c = -2.5850000000000006E-01 yielding R-squared = 0.876 and RMSE = 0.064.
(here "absolute error" means "not relative error")
I'm trying to use the gmnl package to build a latent class choice model. I'd like to constrain a coefficient ($50 discount for seeing a GP, i.e., class.1.Discount$50Discount as below) to be positive (means better than no discount provided). However, an error message pops up:
Error in constrOptim2(theta = c(class.1.MedProviderGP = 0.513279040827413, : initial value not the feasible region
Coefficients:
Estimate Std. Error z-value Pr(>|z|)
class.1.MedProviderGP -0.2171971 0.1563375 -1.3893 0.1647466
class.1.FacilityAllAvailable 2.1752389 0.2011631 10.8133 < 2.2e-16 ***
class.1.Discount$50Discount -0.1849808 0.1874791 -0.9867 0.3238024
class.1.DiscountEntire_for_50 -0.1649310 0.2735538 -0.6029 0.5465622
class.1.DiscountEntire_for_100 -0.3601621 0.2389666 -1.5072 0.1317684
class.1.DiscountEntire_for_200 -0.4968307 0.3203479 -1.5509 0.1209234
Meanwhile I am able to add other constrains such as entire discount > $50 discount. No error message for these constraints. Is there any way to make the constrain $50 discount > 0? Or does this mean it is not feasible to set this coefficient to be positive?
Thank you for the clarification. It appears that this is caused by a starting value problem. You can specify a vector of starting values as follows:
# Fill in with the correct values.
param_start <- c()
# And then add it to your model call with the start option
LC3.constraint <- gmnl(ConceptChoice + MedProvider + Facility + Discount + Wait + Payment| 0 | 0 | 0 |
Age40above + EmployPart + EmployOther + TravelDiff + PerceptionHigh + GPCompetenceLow,
data = Scenario, model = "lc", Q = 2, panel = T,
method = "bfgs", constraints = list(ineqA = A, ineqB = B),
start = param_start)
This will allow you to specify a set of starting values that satisfies your constraints and should ensure that the estimation does not fail at starting values.
And now a little unsolicited advice. You are trying to estimate unobserved heterogeneity in your sample by estimating the probability that a patient's choices can be described by one class or the other. With maximum likelihood, the model will choose parameters that maximizes the likelihood of the data. Imposing constraints such as this based on your intuition as a researcher, will lead to inefficient estimation and may even lead to problems with identification. Why not let the data tell you if people perceived the discount in this way? If it doesn't show up in your data even when you increase the number of classes, maybe people did not think about it like this?
I am trying to learn hierarchical models in R and I have generated some sample data for myself. I am having trouble with the correct syntax for coding a multilevel regression problem.
I generated some data for salaries in a Business school. I made the salaries depend linearly on the number of years of employment and the total number of publications by the faculty member. The faculty are in various departments and I made the base salary(intercept) different for each department and also the yearly hike(slopes) different for each department. This way, I have the intercept (base salary) and slope(w.r.t experience in number of years) of the salary depend on the nested level (department) and slope w.r.t another explanatory variable (Publications) not depend on the nested level. What would be the correct syntax to model this in R?
here's my data
Data <-data.frame(Sl_No = c(1:40),
+ Dept = as.factor(sample(c("Mark","IT","Fin"),40,replace = TRUE)),
+ Years = round(runif(40,1,10)))
pubs <-round(Data$Years*runif(40,1,3))
Data$Pubs <- pubs
lookup_table<-data.frame(Dept = c("Mark","IT","Fin","Strat","Ops"),
+ base = c(100000,140000,150000,150000,120000),
+ slope = c(6000,5000,3000,2000,4000))
Data <- merge(Data,lookup_table,by = 'Dept')
salary <-Data$base+Data$slope*Data$Years+Data$Pubs*10000+rnorm(length(Data$Dept))*10000
Data$base<-NULL
Data$slope<-NULL
I have tried the following:
1)
multilevel_model<-lmer(Salary~1|Dept+Pubs+Years|Dept, data = Data)
Error in model.matrix.default(eval(substitute(~foo, list(foo = x[[2]]))), :
model frame and formula mismatch in model.matrix()
2)
multilevel_model<-lmer(`Salary`~ Dept + `Pubs`+`Years`|Dept , data = Data)
boundary (singular) fit: see ?isSingular
I want to see the estimates of the salary intercept and yearly hike by Dept and the estimate of the effect of publication as a standalone (pooled). Right now I am not getting the code to work at all.
I know the base salary and the yearly hike by dept and the effect of a publication (since I generated it).
Dept base Slope
Fin 150000 3000
Mark 100000 6000
Ops 120000 4000
IT 140000 5000
Strat 150000 2000
Every publication increases the salary by 10,000.
ANSWER:
Thanks to #Ben 's answer here I think the correct model is
multilevel_model<-lmer(Salary~(1|Dept)+ Pubs +(0+Years|Dept), data = Data)
This gives me the following fixed effects by running
summary(multilevel_model)
Fixed effects:
Estimate Std. Error t value
(Intercept) 131667.4 10461.0 12.59
Pubs 10235.0 550.8 18.58
Correlation of Fixed Effects:
Pubs -0.081
The Department level coefficients are as follows:
coef(multilevel_model)
$Dept
Years (Intercept) Pubs
Fin 3072.5133 148757.6 10235.02
IT 5156.6774 136710.7 10235.02
Mark 5435.8301 102858.3 10235.02
Ops 3433.1433 118287.1 10235.02
Strat 963.9366 151723.1 10235.02
These are pretty good estiamtes of the original values. Now I need to learn to assess "how good" they are. :)
(1)
multilevel_model<-lmer(`Total Salary`~ 1|Dept +
`Publications`+`Years of Exp`|Dept , data = sample_data)
I can't immediately diagnose why this gives a syntax error, but parentheses are generally recommended around random-effect terms because the | operator has high precedence in formulas. Thus the response/right-hand side (RHS) formula
~ (1|Dept) + (`Publications`+`Years of Exp`|Dept)
might work, except that it would be problematic because both terms contain the same intercept term: if you wanted to do this you'd probably need
~ (1|Dept) + (0+`Publications`+`Years of Exp`|Dept)
(2)
~ Dept + `Publications`+`Years of Exp`|Dept
It doesn't really make any sense to put the same variable (Dept) on both the left- and right-hand sides of the bar.
You should probably use
~ pubs + years_exp + (1 + years_exp|Dept)
Since in principle the effect of publication could vary across departments, the maximal model would be
~ pubs + years_exp + (1 + pubs + years_exp|Dept)
It rarely makes sense to include a random effect without its corresponding fixed effect.
Note that you may get singular fits even if you have the right model; see the ?isSingular man page.
if the 18 observations listed above represent your whole data set, it's very likely too small to fit the maximal model successfully. Rule of thumb is that you need 10-20 observations per parameter estimated, and the maximal model has (intercept + 2 fixed-effect params + (3*4)/2=6 random-effect parameters) = 9 parameters. (Since it's simulated, you can easily simulate a big data set ...)
I'd recommend renaming variables in your data frame so you don't have to fuss with backtick-protecting variable names with spaces in them ...
The GLMM FAQ has more on model specification