Calculate intraclass correlation by group in R - r

I need some programming/statistic help.
I have a database with multiple groups (variable "group"). The members of each group rated some items (in our example-dataset the variables "var1", "var2" and "var3").
I would like to get the intraclass variance for each group. In particular i would like to calculate the r*wg(j), ICC(1) and ICC(2).
I looked for a solution but the icc function in r expect to have the raters (my team members) as columns and not as row. I could find a way to do it by creating a subset for every group and then transposing every dataset but I believe there is an easier solution.
Thanks to anyone who can help me with this.
group <- c(1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 3, 4, 4, 4, 4)
var1 <- c(4, 5, 4, 2, 3, 4, 5, 3, 5, 8, 4, 3, 4, 4, 5)
var2 <- c(2, 3, 4, 2, 4, 4, 5, 6, 6, 9, 3, 3, 2, 5, 4)
var3 <- c(4, 5, 6, 2, 3, 6, 7, 6, 7, 8, 5, 6, 3, 3, 6)
df <- data.frame(group, var1, var2, var3)

Related

R: item name missing in the plot legend

With this code I get the plot I want
d <- density(mydata$item1)
plot(d)
This code is the same, but omits N/As. And there is a flaw in the plot's legend. As you can see, it doesn't tell what item is plotted, (x = .)
Can you tell where is the matter and how to fix it? Thank you for your help.
My data
structure(list(item1 = c(5, 5, 5, 5, 4, 4, 2, 1, 3, 4, 4, 3,
2, 5, 2, 4, 4, 3, 6, 5, 3, 2, 5, 3, 3, 1, 3, 5, 1, 3, 2, 6, 3,
5, 4, 4, 3, 5, 6, 3, 2, 6, 6, 5, 2, 2, 2, 3, 3, 3), item2 = c(5,
4, 5, 1, 2, 2, 3, 2, 2, 2, 2, 3, 2, 5, 1, 4, 4, 3, 3, 5, 3, 2,
4, 4, 3, 4, 4, 3, 7, NA, 2, 4, 2, 4, 2, 3, 5, 3, 5, 3, 2, 6,
6, 7, 2, 3, 2, 3, 1, 4), item3 = c(5, 5, 6, 7, 3, 4, 5, 2, 2,
6, 4, 2, 5, 7, 1, 2, 4, 5, 6, 6, 5, 2, 6, 5, 6, 4, 6, 4, 6, 4,
6, 5, 5, 6, 6, 6, 5, 6, 7, 5, 5, 7, 7, 6, 2, 6, 6, 6, 5, 3)), row.names = c(NA,
-50L), class = c("tbl_df", "tbl", "data.frame"))
Use the main = argument inside plot to make the title say whatever you want it to.
Data$item2 %>%
na.omit() %>%
density() %>%
plot(main = 'Density of Data$item2')
you had a little typo in your code as the density() call was piped into a plot call refering to the variable it was been written to ... this might have resulted in the strange plot.
In general the density() function won't work with NA values acording to the documentation so you have to set the argument na.rm = TRUE as the default is FALSE for the plot to work correctly... also as #AllanCameron pointed out in an earlier answer you can set the plot title manually.
d <- density(mydata$item2, na.rm = TRUE)
plot(d)
Possibly you can substitute, interpolate or impute the NA values so that you do not have to remove them for the denstiy() call. Though this obviously depends on your data, context and goals.

R: trying to omit missing values before plotting

Any idea why omitting N/A does not work with this code?
d <- density(Data$item2) %>%
na.omit()
I get the error Error in density.default(Data$item2) : 'x' contains missing values
This didn't work either
d <- Data %>% na.omit() %>%
density(Data$item2)
My data
structure(list(item1 = c(5, 5, 5, 5, 4, 4, 2, 1, 3,
4, 4, 3, 2, 5, 2, 4, 4, 3, 6, 5, 3, 2, 5, 3, 3, 1, 3, 5, 1, 3,
2, 6, 3, 5, 4, 4, 3, 5, 6, 3, 2, 6, 6, 5, 2, 2, 2, 3, 3, 3),
item2 = c(5, 4, 5, 1, 2, 2, 3, 2, 2, 2, 2, 3, 2,
5, 1, 4, 4, 3, 3, 5, 3, 2, 4, 4, 3, 4, 4, 3, 7, NA, 2, 4,
2, 4, 2, 3, 5, 3, 5, 3, 2, 6, 6, 7, 2, 3, 2, 3, 1, 4)), row.names = c(NA,
-50L), class = c("tbl_df", "tbl", "data.frame"))
I also tried to omit all the N/A in the beginning with this code, but it did not solve the problem
Data <- read_excel("C:/location/Data.xlsx") %>%
na.omit()
So, how to do this? Thanks for your help!
You need to remove the NA values from your data, not from the density object.
Data$item2 %>%
na.omit() %>%
density() %>%
plot()
Alternatively, use the na.rm = TRUE argument in density:
Data$item2 %>%
density(na.rm = TRUE) %>%
plot()
You can use:
`d <- Data %>% na.omit
density(d$item2)`.

Plots are not stored in list during loop

I have a problem similar to what is found here. I have a loop which runs through some modelling for different pairs of variables. Probably should not have used loops to go through them, but right now that is too late. Then I want to create a plot for each run. At first nothing showed before looking at that post. Looking at the post and implementing the best answer i could at least print the plots, but they still were not stored. The idea is to generate the plots, and then use grid.arrange to plot them together. Could someone show how to fix it? Here is some random data and the loop from example:
col1 <- c(2, 4, 1, 2, 5, 1, 2, 0, 1, 4, 4, 3, 5, 2, 4, 3, 3, 6, 5, 3, 6, 4, 3, 4, 4, 3, 4,
2, 4, 3, 3, 5, 3, 5, 5, 0, 0, 3, 3, 6, 5, 4, 4, 1, 3, 3, 2, 0, 5, 3, 6, 6, 2, 3)
col2 <- c(2, 4, 4, 0, 4, 4, 4, 4, 1, 4, 4, 3, 5, 0, 4, 5, 3, 6, 5, 3, 6, 4, 4, 2, 4, 4, 4,
1, 1, 2, 2, 3, 3, 5, 0, 3, 4, 2, 4, 5, 5, 4, 4, 2, 3, 5, 2, 6, 5, 2, 4, 6, 3, 3)
col3 <- c(2, 5, 4, 1, 4, 2, 3, 0, 1, 3, 4, 2, 5, 1, 4, 3, 4, 6, 3, 4, 6, 4, 1, 3, 5, 4, 3,
2, 1, 3, 2, 2, 2, 4, 0, 1, 4, 4, 3, 5, 3, 2, 5, 2, 3, 3, 4, 2, 4, 2, 4, 5, 1, 3)
data2 <- data.frame(col1,col2,col3)
data2[,1:3] <- lapply(data2[,1:3], as.factor)
colnames(data2)<- c("A","B","C")
myplots <- vector('list', ncol(data2))
for (i in seq_along(data2)) {
message(i)
myplots[[i]] <- local({
i <- i
p1 <- ggplot(data2, aes(x = data2[[i]])) +
geom_histogram(fill = "lightgreen") +
xlab(colnames(data2)[i])
print(p1)
})
}
I tried to change print to return, but to no avail. I get the plots printed in the View window in Rstudio, but the plots are not stored at all.
You can use the following code -
library(ggplot2)
myplots <- vector('list', ncol(data2))
for (i in seq_along(data2)) {
myplots[[i]] <- ggplot(data2, aes(x = .data[[colnames(data2)[i]]])) +
geom_histogram(fill = "lightgreen")
}
However, using lapply would be easier.
myplots <- lapply(names(data2), function(x)
ggplot(data2, aes(x = .data[[x]])) + geom_histogram(fill = "lightgreen"))
Plot the list of plots with grid.arrange.
gridExtra::grid.arrange(grobs = myplots)
data
A <- c(2, 4, 1, 2, 5, 1, 2, 0, 1, 4, 4, 3, 5, 2, 4, 3, 3, 6, 5, 3, 6, 4, 3, 4, 4, 3, 4,
2, 4, 3, 3, 5, 3, 5, 5, 0, 0, 3, 3, 6, 5, 4, 4, 1, 3, 3, 2, 0, 5, 3, 6, 6, 2, 3)
B <- c(2, 4, 4, 0, 4, 4, 4, 4, 1, 4, 4, 3, 5, 0, 4, 5, 3, 6, 5, 3, 6, 4, 4, 2, 4, 4, 4,
1, 1, 2, 2, 3, 3, 5, 0, 3, 4, 2, 4, 5, 5, 4, 4, 2, 3, 5, 2, 6, 5, 2, 4, 6, 3, 3)
C <- c(2, 5, 4, 1, 4, 2, 3, 0, 1, 3, 4, 2, 5, 1, 4, 3, 4, 6, 3, 4, 6, 4, 1, 3, 5, 4, 3,
2, 1, 3, 2, 2, 2, 4, 0, 1, 4, 4, 3, 5, 3, 2, 5, 2, 3, 3, 4, 2, 4, 2, 4, 5, 1, 3)
data2 <- data.frame(A,B,C)
Does this work for you?, With patchwork and purrr::reduce we can club these graphs to stack(horizontal or vertical) with each other. You can also use slashes(/) instead of plus(+) in reduce to make it appended vertically instead of horizontally. If you want to plot histogram you should have continuous data , In case you do want to plot counts for discrete data you should try geom_bar. If you do want to check for geom_bar then you need to convert the columns into factors. I am not so sure what plot you want to carry out, I am assuming that you have continuous data and you want to carry out histogram here. Please let me know if it doesn't work in your scenario.
library(tidyverse)
library(patchwork)
data2 <- data.frame(col1, col2, col3) ## No conversion of factors
nm <- names(data2)
g1 <- reduce(map2(data2,nm, ~ggplot(data2,aes(x =.x )) + geom_histogram(fill = "yellow4") + labs(x=.y, y = 'count')), `+`)
print(g1)
Or with slashes:
g2 <- reduce(map2(data2,nm, ~ggplot(data2,aes(x =.x )) + geom_histogram(fill = "yellow4") + labs(x=.y, y = 'count')), `/`)
print(g2)
Or if you want to have for loops then probably you can do this as well, you already have intialised myplots so not adding it here:
for (i in seq_along(data2)) {
myplots[[i]] <-
ggplot(data2, aes(x = data2[[i]])) +
geom_histogram(fill = "lightgreen") +
xlab(colnames(data2)[i])
}
Explanation:
Now you can use reduce with your myplots to arrange them, Note here myplots should be containing your 3 plots :
reduce(myplots, `+`)
for arranging it.
The map2 and reduce is similar solution, with map2 you are getting 3 plots saved into a list, so 3 objects are returned from below code:
plots <- map2(data2,nm, ~ggplot(data2,aes(x =.x )) + geom_histogram(fill = "yellow4") + labs(x=.y, y = 'count'))
To add them (arrange) them all you have to do is to use patchwork like below:
plots[[1]] + plots[[2]] + plots[[3]], but then its quite cumbersome, so we use reduce to make it happen like below:
reduce(plots, `+`)
Also like I mentioned earlier you can use slash instead of plus to make the arrangement vertical than horizontal. with plot_layout option in patchwork, you can create more flexible plots. You can check here .
with gridExtra : gridExtra::grid.arrange(grobs = (myplots)), again instead of myplots, it can be any list that contain ggplot objects.

Selecting columns using ends_with helper and a vector of string names

I have a data frame, in wide format, with each column representing one questionnaire item for one particular version of a questionnaire for a particular time point (repeated measures design).
My data would look something like the following:
df <- data.frame(id = c(1:5), t1_QOL_child_Q1 = c(5, 3, 6, 2, 7), t1_QOL_child_Q2 = c(5, 2, 3, 7, 1), t1_QOL_child_Q3 = c(7, 7, 6, 2, 5), t1_QOL_child_joy = c(9,9, 5, 3, 6), t1_QOL_teen_Q1 = c(5, 3, 6, 2, 7), t1_QOL_teen_Q2 = c(5, 2, 3, 7, 1), t1_QOL_teen_Q3 = c(7, 7, 6, 2, 5), t1_QOL_teen_joy = c(5, 7, 4, 7, 9), t1_QOL_adult_Q1 = c(5, 3, 6, 2, 7), t1_QOL_adult_Q2 = c(5, 2, 3, 7, 1), t1_QOL_adult_Q3 = c(7, 7, 6, 2, 5), t1_QOL_adult_joy = c(6, 5, 3, 3, 2), t2_QOL_child_Q1 = c(5, 3, 6, 2, 7), t2_QOL_child_Q2 = c(5, 2, 3, 7, 1), t2_QOL_child_Q3 = c(7, 7, 6, 2, 5), t2_QOL_child_joy = c(9,9, 5, 3, 6), t2_QOL_teen_Q1 = c(5, 3, 6, 2, 7), t2_QOL_teen_Q2 = c(5, 2, 3, 7, 1), t2_QOL_teen_Q3 = c(7, 7, 6, 2, 5), t2_QOL_teen_joy = c(5, 7, 4, 7, 9), t2_QOL_adult_Q1 = c(5, 3, 6, 2, 7), t2_QOL_adult_Q2 = c(5, 2, 3, 7, 1), t2_QOL_adult_Q3 = c(7, 7, 6, 2, 5), t2_QOL_adult_joy = c(6, 5, 3, 3, 2))
For example, column t1_QOL_child_Q1 would mean Question 1 (Q1) of the child version (child) of Quality of Life (QOL) questionnaire, with time point 1 (t1) data.
I want to select only subscales/columns whose suffix are labelled differently. In the sample data above, it would be the columns ending with "joy".
I have over 3000 columns and many more suffixes and it would be a pain to use the following:
select(df, ends_with("joy"), ends_with(<another suffix>), ends_with(<another suffix>))
I have thought of putting all the potential suffixes in a string vector, and use the vector as an input to the ends_with function, but ends_with could only take a single string instead of a vector of strings.
I have searched on Stackoverflow and found a solution that could accommodate a small vector of strings, which is the following:
select(df, sapply(vector_of_strings, starts_with))
However, I have too many suffixes in my vector of strings and the following error message resulted from it: Error: sapply(vector_of_strings, ends_with) must resolve to integer column positions, not a list
Help appreciated. Thanks!
We can use a single matches with multiple patterns separated by | to match substrings at the end ($) of the string
df %>%
select(matches("(joy|Q2)$"))

Is it possible to limit forecasts made by bsts to positive values only?

I am learning to use various forecasting packages available in R, and came across bsts(). The data I deal with is a time series of demands.
data=c(27, 2, 7, 7, 9, 4, 3, 3, 3, 9, 6, 2, 6, 2, 3, 8, 6, 1, 3, 8, 4, 5, 8, 5, 4, 4, 6, 1, 6, 5, 1, 3, 0, 2, 6, 7, 1, 2, 6, 2, 8, 6, 1, 1, 3, 2, 1, 3, 1, 6, 3, 4, 3, 7, 3, 4, 1, 7, 5, 6, 3, 4, 3, 9, 2, 1, 7, 2, 2, 9, 4, 5, 3, 4, 2, 4, 4, 8, 6, 3, 9, 2, 9, 4, 1, 3, 8, 1, 7, 7, 6, 0, 1, 4, 8, 9, 2, 5)
ts.main=ts(data, start=c(1910,1), frequency=12)
ss <- AddLocalLinearTrend(list(), y=ts.main)
ss <- AddSeasonal(ss, y=as.numeric(ts.temp), nseasons=12)
model <- bsts(as.numeric(ts.temp),
state.specification = ss,
niter = 1000)
pred <- predict(model, horizon = 12)
Is there way I can restrict pred$mean from becoming negative?
Since your data are a time series of counts, you need to take that into account rather than assume Gaussian errors; for some discussion on this and elaboration of some approaches, see for example Brandt et al 2000 and Brandt and Williams 2001. Luckily, the bsts package has a built-in functionality for this, the family option (see pages 24 to 26 of the documentation).
So, you can just do this
model <- bsts(as.numeric(ts.main),
state.specification = ss,
family = 'poisson',
niter = 1000)
so that the bsts() function correctly considers the data as counts, which will solve your issue, since the draws from the posterior predictive distribution will then be non-negative by definition.

Resources