R: Identifying outliers in 2D gaussian distribution - r

Question

I have a two dimensional Gaussian distribution, and I am trying to identify outliers. This is not in the sense of outlier removals, but rather to identify samples that are the most dissimilar to the bulk.
http://imgur.com/hlOqjig
Do you have a suggestion how this is best done for this data? I have tried to fit a normal distribution on both dimensions and to calculate p-values for all data points, and then to identify the outliers as the data points with the lowest p-values. I, however, get the following result:
http://imgur.com/a/w6SAz
This is the code for calculating P-values:
library(fitdistrplus)
norm_pvalue <- function(input_dist, input_values) {
# Fitting normal distribution
fit <- fitdist(input_dist, "norm")
# Calculating p-values
p_values <- unlist(lapply(input_values, function(x) dnorm(x = x, mean= fit$estimate[['mean']], sd= fit$estimate[['sd']])))
return(p_values)
}
I would like the solution to be generalisable.

Answer 1

Without the data, it is hard to respond in any detail. However, you might want to check out the latest version of the package assertr, noted here: http://www.onthelambda.com/2017/03/20/data-validation-with-the-assertr-package/.
I really like its workflow, which is very generalisable.
For example, if you're looking to inspect data from a column (col) within a dataframe (df), you'd use something like:
library(assertr)
library(magrittr)
df %>% insist(within_n_sds(2), col)
This final function would then notify you of all outliers (i.e. those points that are more than two standard deviations from the mean). The package also includes plenty of different measures for assessing outliers.
In your case, the column in question would probably be based on the residuals from the best-fit line of PC1 and PC2:
PCA.lm = lm(PC2 ~ PC1, data=df)
PCA.res = resid(PCA.lm)
I hope that helps you.

Answer 2

I just ended up using the stat_ellipse of ggplot2 for identifying the outliers. I used a confidence level of 0.999.
This function extracts points outside the ellipsoid and takes a ggplot and the layer in which the ellipsoid is plotted.
# Function for identifying points outside ellipse
outside_ellipse <- function(ggplot, ellipsoid_layer_number) {
# Extracting components
build <- ggplot_build(ggplot)$data
points <- build[[1]]
ell <- build[[ellipsoid_layer_number]]
# Finding points are inside the ellipse, and add this to the data
df <- data.frame(points[1:2],
in.ell = as.logical(point.in.polygon(points$x, points$y, ell$x, ell$y)))
# Plot the result
ggplot(df, aes(x, y)) +
geom_point(aes(col = in.ell)) +
stat_ellipse()
# Returning indices of outliers
return(which(df$in.ell == FALSE))
}
Here I plot my data with the ellipsoid option, and extract the points outside the ellipsoid and add their information to the dataframe.
# Saving plot with confidence ellipsoid
plotData <- ggplot(pc_df, aes(PC1, PC2)) + geom_point() + stat_ellipse(level = 0.999)
# Identifying points outside ellipsoid
outside <- outside_ellipse(plotData, 2)
pc_df$in_ellipsoid <- rep(FALSE, dim(pc_df)[1])
pc_df$in_ellipsoid[outside] <- TRUE