I have 3 cluster pair plot with "Av. Mon. Hrs","Sat. Lvl","Last Eval", and found a matrix graph by below code.
library("ggplot2") # Expanded plotting functionality over "lattice" package
x<-cbind(HR_left$average_montly_hours,HR_left$satisfaction_level,HR_left$last_evaluation)
kmfit<-kmeans(x,3,nstart=25)
# Find the best 3 clusters using 25 random sets of (distinct) rows in x as initial centres.
pairs(x,col=(kmfit$cluster), labels=c("Av. Mon. Hrs","Sat. Lvl","Last Eval."))
It says
Cluster 1: The pairs plot characterised this cluster as working low
average monthly hours of employees, middle satisfaction range and a
low last evaluation.
Cluster 2: From the pairs plot, this cluster is
characterised by high monthly hours, very low satisfaction and high
evaluation.
Cluster 3: From the pairs plot, this cluster is
characterised by high monthly hours, high satisfaction and high
evaluation.
But I don't understand the pairplot graphs about how they interpretative of these three findings.
library(readr)
HR_comma_sep <- read_csv("https://stluc.manta.uqcloud.net/mdatascience/public/datasets/HumanResourceAnalytics/HR_comma_sep.csv")
HR_left<-HR_comma_sep[HR_comma_sep$left==1,]
library("ggplot2") # Expanded plotting functionality over "lattice" package
x<-cbind(HR_left$average_montly_hours,HR_left$satisfaction_level,HR_left$last_evaluation)
kmfit<-kmeans(x,3,nstart=25)
# Find the best 3 clusters using 25 random sets of (distinct) rows in x as initial centres.
pairs(x,col= (kmfit$cluster),labels=c("Av. Mon. Hrs","Sat. Lvl","Last Eval."))
The number of "monthly hours" is at a very different scale than the other two variables, thus is skewing the clustering. The difference in "hours worked" is dominating the differences in the other two variables.
Normalize each column by dividing by the mean, the range or finding the z-score.
Original Code:
library(readr)
HR_comma_sep <- read_csv("https://stluc.manta.uqcloud.net/mdatascience/public/datasets/HumanResourceAnalytics/HR_comma_sep.csv")
HR_left<-HR_comma_sep[HR_comma_sep$left==1,]
library("ggplot2")
x_org<-cbind(HR_left$average_montly_hours,
HR_left$satisfaction_level,
HR_left$last_evaluation)
kmfit<-kmeans(x_org, 3, nstart = 25)
pairs(x_org,col= (kmfit$cluster),labels=c("Av. Mon. Hrs","Sat. Lvl","Last Eval."))
Repeating the calculation using scaled values:
x_scaled<-cbind(scale(HR_left$average_montly_hours),
scale(HR_left$satisfaction_level),
scale(HR_left$last_evaluation))
kmfit<-kmeans(x_scaled, 3)
pairs(x_org,col= (kmfit$cluster),labels=c("Av. Mon. Hrs","Sat. Lvl","Last Eval."))
Using just the raw values, the clustering based on difference in the "monthly hours", The top plot shows 2 clusters (black and green) merged together and not clearly distinct.
After scaling the values and repeating the clustering, 3 clearly differentiated clusters are now clearly shown (bottom image).
Related
I have several cumulative incidence curves showing the incidence of an outcome after an exposure, plotted on the same graph, stratified by age at exposure.
The longest follow-up period is 25 years, but in the oldest group, the follow-up does not last any longer than 15 years (as they are all dead by then).
When I plot the cum inc curves, the curve for the oldest group 'flatlines' after 15y, but does not disappear until the 25y point, when all the other curves also disappear (i.e the plot ends).
Is there a way to stop a cumulative incidence curve being plotted when there is nobody at risk, even if other curves for different groups on the same graph go on for longer than the point you wish to stop plotting at for that one problematic curve?
I did this with the surv_fit function in R.
I'm using DTWCLUST package in r for multivariate time series clustering. Here's my code.
data("uciCT")
mvc <- tsclust(CharTrajMV, k = 4L, distance = "gak", seed = 390L)
plot(mvc)
The CharTrajMV data set has 100 observations with 3 variables. As I understand, clusters are determined based on 3 variables as opposed to univariate time series clustering.
Each cluster graph shows several similarly patterned time series (observations) belonging to that cluster. How is this graph drawn? There are 3 time series variables used for clustering, how does one pattern graph come out? I mean the input is 3-dimentional(variables) dataset, but the output is 1-dimentional.
Moreover, I can get the 3 variables's centroid for each cluster (using mvc#centroids)
plot(mvc, labels = list(nudge_x = -10, nudge_y = 1), type="centroids")
this code shows only one centroid for each cluster. Can I get 3 variables' centroid graphs for each cluster with plot option? or is this right approach?
This is covered in the documentation. Plotting so many different series in separate panes would get very congested, so, for multivariate plots, the variables are appended one after the other, and you get vertical dotted lines to see the place where that happened, maybe injecting some missing values in some places to account for differences in length. This does mean the x axis isn't so meaningful anymore, but it's only meant to be a quick visualization aid.
I’am trying to classify bivariate point patterns into groups using spatstat. The patterns are derived from the whole slide images of lymph nodes with cancer. I’ve trained a neural network to recognize cells of three types (cancer “LP”, immune cells “bcell” and all other cells). I do not wish to analyse all other cells but use them to construct a polygonal window in the shape of the lymph node. Thus, the patterns to be analysed are immune cells and cancer cells in polygonal windows. Each pattern can have several 10k cancer cells and up to 2mio immune cells. The patterns are of the type “Small World Model” as there is no possibility of points laying outside the window.
My classification should be based on the position of the cancer cells in relation to the immune cells. E.g. most cancer cells are laying on the “islands” of immune cells but in some cases cancer cells are (seemingly) uniformly dispersed and there are only a few immune cells. In addition, the patterns are not always uniform across the node. As I’m rather new to spatial statistics I developed a simple and crude method to classify the patterns. Here in short:
I calculated a kernel density of the immune cells with sigma=80 because this looked “nice” for me. Den<-density(split(cells)$"bcell",sigma=80,window= cells$window) (Should I have used e.g. sigma=bw.scott instead?)
Then I created a tessellation image by dividing density range in 3 parts (here again, I experimented with the breaks to get some “good looking results”).
rangesDenMax<-2*range(Den)[2]/3
rangesDenMin<-range(Den)[2]/3
map.breaks<-c(-Inf,rangesDenMin,rangesDenMax,Inf)
map.cuts <- cut(Den, breaks = map.breaks, labels = c("Low B-cell density","Medium B-cell density", "High B-cell density"))
map.quartile <- tess(image = map.cuts,window=cells$window)
tessImage<-map.quartile
Here are some examples of the plots of the tessellations with the cancer cell overlay (white dots). The lymph node on the left has a typical uniformly distributed “islands” of immune cells while the node on the right has only a few dense spots of immune cells and cancer cells not restricted to those spots:
heat map: immune cell kernel density, white dots: cancer cells
Then I measured a silly number of variables, which should give me a clue of how the cancer cells are distributed across the tessellation tiles (the calculation code is trivial so I post only the description of my variables):
LPlwB<-c() # proportion of cancer cells in low-b-cell-area
LPmdB<-c() # proportion of cancer cells in medium-b-cell-area
LPhiB<-c() # proportion of cancer cells in high-b-cell-area
AlwB<-c() # proportion of the low-b-cell area
AmdB<-c() # proportion of the medium-b-cell area
AhiB<-c() # proportion of the high-b-cell area
LPm1<-c() # mean distance to the 1st neighbour
LPm2<-c() # mean distance to the 2nd neighbour
LPm3<-c() # mean distance to the 3d neighbour
LPsd1<-c() # standard deviation of the mean distance to the 1st neighbour
LPsd2<-c() # standard deviation of the mean distance to the 2nd neighbour
LPsd3<-c() # standard deviation of the mean distance to the 3d neighbour
meanQ<-c() # mean quadratcount (I visually chose the quadrat size to be not too large and not too small)
sdevQ<-c() # standard deviation of the mean quadratcount
hiSAT<-c() # realised cancer cells saturation in high b-cell-area (number of cells observed divided by a number of cells, which could be fitted into the area considering the observed min distance between the cells)
mdSAT<-c() # realised cancer cells saturation in medium b-cell-area
lwSAT<-c() # realised cancer cells saturation in low b-cell-area
ll<-c() # Proportion LP neighbours of LP (contingency table count divided by total points)
lb<-c() # Proportion b-cell neighbours of LP
bl<-c() # Proportion b-cell neighbours of b-cells
bb<-c() # Proportion LP neighbours of b-cells
I z-scaled the variables, inspected them on a PCA-plot (the vectors pointed in different directions like needles of a sea urchin) and performed a hierarchical cluster analysis. I choose k by calculating fviz_nbclust(scaled_variables, hcut, method = "silhouette"). After dividing the dendrogram into k clusters and checking the cluster stability, I ended up with my groups, which seemed to make sense as cases with “islands” were separated from the "more dispersed" ones.
However, given the possibilities of the spatstat package I strongly feel like hitting nails into the wall with a smartphone.
It seems you are trying to quantify the way in which the cancer cells are positioned relative to the immune cells. You could do this by something like
Cancer <- split(cells)[["LP"]]
Immune <- split(cells)[["bcell"]]
Dimmune <- density(Immune, sigma=80)
f <- rhohat(Cancer, Dimmune)
plot(f)
Then f is a function that indicates the intensity (number per unit area) of cancer cells as a function of the density of immune cells. The plot shows the density of cancer cells on the vertical axis, against the density of immune cells on the horizontal axis.
If the graph of this function is flat, it means that the cancer cells are not paying attention to the density of immune cells. If the graph is steeply declining it means that cancer cells tend to avoid immune cells.
I suggest you first look at the plot of f for some example datasets to decide whether f has any ability to discriminate between spatial arrangements that you think should be classified as different. If so then you can use as.data.frame to extract the values of f and then use classical discriminant analysis (etc) to classify the slide images into groups.
Instead of density(Immune) you could use any other summary of the immune cells.
For example D <- distfun(Immune) would give you the distance to the nearest immune cell, and then f would compute the density of cancer cells as a function of the distance to nearest immune cell. And so on.
I have a dataset containing 1599 observations and 10 attributes on which iIneed to do kmeans clustering. I have done the kmeans with 6 clusters and I can see the cluster centers, size, etc. and which observation lies in which cluster. Now, I need to plot these results such that I have in a single plot the following information: On x-axis, I want 1 of the 10 attributes of my original data, on y-axis I want another attribute and in the plot, I want all 1599 observations, but I want them in 6 different colors for each cluster they belong. So, I will have 10C2 = 45 plots. Basically, this should give me the information that cluster 1 is high/medium/low in terms of a particular attribute while cluster 2 is so and so.....for all 6 clusters.
I tried the function plotcluster from fpc package but from what I understood, it maps the data into 2D, using PCA, and then plots the clusters in terms of 2 dimensions which are different from the original attributes. So now when I will say cluster 1 is low, in dim1, it wouldn't really make much sense.
Is there a function to do what I want, or should I just append the '$cluster' information from the kmeans output with my original data and try to plot taking 2 columns from my data at a time using the basic function plot()?
I suggest one solution, probably not the simplest one (with a for loop) but it seems to answer what you need:
df=mtcars
df$cluster = factor( kmeans(df, centers=6)$clust )
mycomb <- combn(1:ncol(df), 2)
for (xy in 1:45 ) {
plot(x=df[, mycomb[1,xy]],
y=df[, mycomb[2,xy]],
col=as.numeric(df$clust),
xlab=names(df)[mycomb[1,xy]],
ylab=names(df)[mycomb[2,xy]])
}
I can not find this information in the reference literature [1]
1)how adaptative.density() (package spatstat) manage duplicated spatial points. I have duplicated points exactly in the same position because I am combining measurements from different years, and I am expecting that the density curve is higher in those areas but I am not sure about it.
2) is the default value of f in adaptative.density() f=0 or f=1?
My guess is that it is f=0, so it is doing an adaptive estimate by calculating the intensity estimate at every location equal to the average intensity (number of points divided by window area)
Thank you for your time and input!
The default value of f is 0.1 as you can see from the "Usage" section in the help file.
The function subsamples the point pattern with this selection probability and uses the resulting pattern to generate a Dirichlet tessellation (if there are duplicated points here they are ignored). The other fraction of points (1-f) is used to estimate the intensity by the number of points in each tile of the tessellation divided by the corresponding area (here duplicated points count equally to the total count in the tile).