Add a line from different result to boxplot graph in ggplot2 - r

I have a dataframe (df1) that contains 3 columns (y1, y2, x). I managed to plot a boxplot graph between y1, x and y2, x. I have another dataframe (df2) which contains two columns A, x. I want to plot a line graph (A,x) and add it to the boxplot. Note the variable x in both dataframes is the axis access, however, it has different values. I tried to combine and reshape both dataframes and plot based on the factor(x)... I got 3 boxplots in one graph. I need to plot df2 as line and df1 as boxplot in one graph.
df1 <- structure(list(Y1 = c(905L, 941L, 744L, 590L, 533L, 345L, 202L,
369L, 200L, 80L, 200L, 80L, 50L, 30L, 60L, 20L, 30L, 30L), Y2 = c(774L,
823L, 687L, 545L, 423L, 375L, 249L, 134L, 45L, 58L, 160L, 60L,
20L, 40L, 20L, 26L, 19L, 27L), x = c(10L, 10L, 10L, 20L, 20L,
20L, 40L, 40L, 40L, 50L, 50L, 50L, 70L, 70L, 70L, 90L, 90L, 90L
)), .Names = c("Y1", "Y2", "x"), row.names = c(NA, -18L), class = "data.frame")
df2 <- structure(list(Y3Line = c(384L, 717L, 914L, 359L, 241L, 265L,
240L, 174L, 114L, 165L, 184L, 96L, 59L, 60L, 127L, 54L, 31L,
44L), x = c(36L, 36L, 36L, 56L, 56L, 56L, 65L, 65L, 65L, 75L,
75L, 75L, 85L, 85L, 85L, 99L, 99L, 99L)), .Names = c("A",
"x"), row.names = c(NA, -18L), class = "data.frame")
df_l <- melt(df1, id.vars = "x")
ggplot(df_l, aes(x = factor(x), y =value, fill=variable )) +
geom_boxplot()+
# here I'trying to add the line graph from df2
geom_line(data = df2, aes(x = x, y=A))
Any suggestions?

In the second dataset you have three y values per x value, do you want to draw seperate lines per x value or the mean per x value? Both are shown below. The trick is to first change the x variables in both datasets to factors that contain all the levels of both variables.
df1 <-structure(list(Y1 = c(905L, 941L, 744L, 590L, 533L, 345L, 202L,
369L, 200L, 80L, 200L, 80L, 50L, 30L, 60L, 20L, 30L, 30L), Y2 = c(774L,
823L, 687L, 545L, 423L, 375L, 249L, 134L, 45L, 58L, 160L, 60L,
20L, 40L, 20L, 26L, 19L, 27L), x = c(10L, 10L, 10L, 20L, 20L,
20L, 40L, 40L, 40L, 50L, 50L, 50L, 70L, 70L, 70L, 90L, 90L, 90L
)), .Names = c("Y1", "Y2", "x"), row.names = c(NA, -18L), class = "data.frame")
df2 <- structure(list(Y3Line = c(384L, 717L, 914L, 359L, 241L, 265L,
240L, 174L, 114L, 165L, 184L, 96L, 59L, 60L, 127L, 54L, 31L,
44L), x = c(36L, 36L, 36L, 56L, 56L, 56L, 65L, 65L, 65L, 75L,
75L, 75L, 85L, 85L, 85L, 99L, 99L, 99L)), .Names = c("A",
"x"), row.names = c(NA, -18L), class = "data.frame")
library(ggplot2)
library(reshape2)
df_l <- melt(df1, id.vars = "x")
allLevels <- levels(factor(c(df_l$x,df2$x)))
df_l$x <- factor(df_l$x,levels=(allLevels))
df2$x <- factor(df2$x,levels=(allLevels))
Line per x category:
ggplot(data=df_l,aes(x = x, y =value))+geom_line(data=df2,aes(x = factor(x), y =A)) +
geom_boxplot(aes(fill=variable ))
Connected means of x categories:
ggplot(data=df2,aes(x = factor(x), y =A)) +
stat_summary(fun.y=mean, geom="line", aes(group=1)) +
geom_boxplot(data=df_l,aes(x = x, y =value,fill=variable ))

Related

Boxplot outlier removal causing problems with NbClust

I have a dataset from which I had to remove outliers. I used the boxplot method to remove my outliers however, I feel this method has changed the structure of my data from a table like structure to just a list. I am trying to use NbClust to get a prediction on the amount of clusters I should use. I also applied z-score scaling before attempting to use NbClust. I am really new to R and I am not sure how to change it back and/or if this is the reason the error is occurring with NbClust
The data also showed as "846 obs. of 18 variables" before outlier removal
to "List of 18" after outlier removal (Shown in the Global Environment panel)
Error: Error in t(jeu) %*% jeu :
requires numeric/complex matrix/vector arguments
I think the correct thing is to change it into a data frame but I am not too sure how to do correctly do this.
Data before outlier removal using boxplot method:
After outliers removed using boxplot method:
Reproduceable example
library(reshape2)
library(NbClust)
vehData <-
structure(
list(
Samples = 1:6,
Comp = c(95L, 91L, 104L, 93L, 85L,
107L),
Circ = c(48L, 41L, 50L, 41L, 44L, 57L),
D.Circ = c(83L,
84L, 106L, 82L, 70L, 106L),
Rad.Ra = c(178L, 141L, 209L, 159L,
205L, 172L),
Pr.Axis.Ra = c(72L, 57L, 66L, 63L, 103L, 50L),
Max.L.Ra = c(10L,
9L, 10L, 9L, 52L, 6L),
Scat.Ra = c(162L, 149L, 207L, 144L, 149L,
255L),
Elong = c(42L, 45L, 32L, 46L, 45L, 26L),
Pr.Axis.Rect = c(20L,
19L, 23L, 19L, 19L, 28L),
Max.L.Rect = c(159L, 143L, 158L, 143L,
144L, 169L),
Sc.Var.Maxis = c(176L, 170L, 223L, 160L, 241L, 280L),
Sc.Var.maxis = c(379L, 330L, 635L, 309L, 325L, 957L),
Ra.Gyr = c(184L,
158L, 220L, 127L, 188L, 264L),
Skew.Maxis = c(70L, 72L, 73L,
63L, 127L, 85L),
Skew.maxis = c(6L, 9L, 14L, 6L, 9L, 5L),
Kurt.maxis = c(16L,
14L, 9L, 10L, 11L, 9L),
Kurt.Maxis = c(187L, 189L, 188L, 199L,
180L, 181L),
Holl.Ra = c(197L, 199L, 196L, 207L, 183L, 183L),
Class = c("van", "van", "saab", "van", "bus", "bus")
),
row.names = c(NA,
6L), class = "data.frame")
#Remove outliers
removeOutliers <- function(data) {
OutVals <- boxplot(data)$out
remOutliers <- sapply(data, function(x) x[!x %in% OutVals])
return (remOutliers)
}
# Scale data -> same as scale() function
z_score <- function(x){
return ((x - mean(x))/sd(x))
}
vehDataRemove1 <- vehData[, -1]
vehDataRemove2 <- vehDataRemove1[,-19]
vehData <- vehDataRemove2
vehClass <- vehData$Class
#Begin removing outliers
removeOutliers1 <- removeOutliers(vehData)
removeOutliers2 <- removeOutliers(removeOutliers1)
removeOutliers3 <- removeOutliers(removeOutliers2)
removeOutliers4 <- removeOutliers(removeOutliers3)
cleanVehicleData <- removeOutliers4
cl_vehDataScale <- lapply(cleanVehicleData, z_score)
set.seed(26)
clusterNo <- NbClust(cl_vehDataScale, distance="euclidean", min.nc=2, max.nc=10,
method="kmeans", index="all")

calculate sum of values in dataframe based on values in other columns

I have a dataframe in R in which values correspond to value estimates and their margin of error (MoE).
Column names consist of a pattern, an indicator character (e = estimate, m = margin of error) and an ID that matches estimate and margin of error.
So, the column names look like "XXXe1, XXXm1, XXXe2, XXXm2, ...".
Goal
I am trying to create a function to (for each row)
Calculate the sum of the estimates. (That is pretty straightforward.)
Calculate the aggregated margin of error. This is the square root of the sum of the squares of each MoE.
Condition: the MoE of estimates marked as 0 should only be added once.
Examples:
In row 20, the aggregated MoE should only be sqrt(123^2).
In row 13, B01001e4 and B01001e5 are 0, so their MoE is only counted once.
So far, I have done the following to build a function that does this:
estimate_aggregator <- function(DF_to_write_on, New_column_name, source_df, pattern){
subset_df <- source_df[, grepl(pattern, names(source_df))] # I subset all the columns named with the pattern, regardless of whether they are estimate or margin of error
subset_df_e <- source_df[, grepl(paste0(pattern, "e"), names(source_df))] # I create a table with only the estimated values to perform the sum
DF_to_write_on[paste0(New_column_name, "_e")]<- rowSums(subset_df_e) # I write a new column in the new DF with the rowSums of the estimates values, having calculated the new estimate
return(DF)
}
What I am missing: a way to write in the new dataframe the result of selecting the XXXmYY values of those columns that have no 0 value in their corresponding estimate. If there is one or more 0 in the estimates, then I should include the MoE 123 in the calculation only once.
What would be the cleanest way to achieve this? I see that my struggle is on dealing with several columns at once and the fact that the values on the XXXeYY columns determine the selection of the XXXmYY ones.
Expected output
row1: DF_to_write_on[paste0(New_column_name,"_m") <- sqrt(176^2 + 117^2+22^2 + 123^2)
row2: DF_to_write_on[paste0(New_column_name,"_m") <- sqrt(123^2)
B01001e1 B01001m1 B01001e2 B01001m2 B01001e3 B01001m3 B01001e4 B01001m4 B01001e5 B01001m5
15 566 176 371 117 14 22 0 123 0 123
20 0 123 0 123 0 123 0 123 0 123
Data
structure(list(B01001e1 = c(1691L, 2103L, 975L, 2404L, 866L,
2140L, 965L, 727L, 1602L, 1741L, 948L, 1771L, 1195L, 1072L, 566L,
1521L, 2950L, 770L, 1624L, 0L), B01001m1 = c(337L, 530L, 299L,
333L, 264L, 574L, 227L, 266L, 528L, 498L, 320L, 414L, 350L, 385L,
176L, 418L, 672L, 226L, 319L, 123L), B01001e2 = c(721L, 1191L,
487L, 1015L, 461L, 1059L, 485L, 346L, 777L, 857L, 390L, 809L,
599L, 601L, 371L, 783L, 1215L, 372L, 871L, 0L), B01001m2 = c(173L,
312L, 181L, 167L, 170L, 286L, 127L, 149L, 279L, 281L, 152L, 179L,
193L, 250L, 117L, 234L, 263L, 155L, 211L, 123L), B01001e3 = c(21L,
96L, 70L, 28L, 33L, 90L, 12L, 0L, 168L, 97L, 72L, 10L, 59L, 66L,
14L, 0L, 35L, 47L, 14L, 0L), B01001m3 = c(25L, 71L, 73L, 26L,
33L, 79L, 18L, 123L, 114L, 79L, 59L, 15L, 68L, 99L, 22L, 123L,
31L, 37L, 20L, 123L), B01001e4 = c(30L, 174L, 25L, 91L, 4L, 27L,
30L, 43L, 102L, 66L, 54L, 85L, 0L, 16L, 0L, 26L, 34L, 27L, 18L,
0L), B01001m4 = c(26L, 148L, 30L, 62L, 9L, 27L, 25L, 44L, 82L,
52L, 46L, 48L, 123L, 21L, 123L, 40L, 33L, 32L, 27L, 123L), B01001e5 = c(45L,
44L, 7L, 46L, 72L, 124L, 45L, 34L, 86L, 97L, 0L, 83L, 0L, 30L,
0L, 66L, 0L, 23L, 33L, 0L), B01001m5 = c(38L, 35L, 12L, 37L,
57L, 78L, 36L, 37L, 62L, 97L, 123L, 50L, 123L, 42L, 123L, 59L,
123L, 31L, 49L, 123L)), .Names = c("B01001e1", "B01001m1", "B01001e2",
"B01001m2", "B01001e3", "B01001m3", "B01001e4", "B01001m4", "B01001e5",
"B01001m5"), row.names = c(NA, 20L), class = "data.frame")
From your description it sounds like your desired output should have 2 columns, the row sum of the estimate, and the function of the row margins of errors using the logic you describe. Here is one (somewhat roundabout) solution to that problem.
I saved your data as df.
# Isolate estimate and MoE dataframes
df_e <- df[,grepl('e', names(df))]
df_m <- df[,grepl('m', names(df))]
# Temporary matrix used to isolate 0 values for MoE, count number of zero occurances, and convert those MoE values to NA
mat <- df_e == 0
mat <- t(apply(mat, 1, cumsum))
df_m[mat > 1] = NA
# Combine with estimate row sum
output_df <- data.frame(
e = rowSums(df[,grepl('e', names(df))]),
m = apply(df_m, 1, function(x) sqrt(sum(x^2, na.rm = T)))
)
head(output_df)
e m
1 2508 382.4173
2 3608 637.5061
3 1564 358.5178
4 3584 380.3512
5 1436 320.9595
6 3440 651.4031

Exact number of every value next to the bar [duplicate]

This question already has answers here:
How to put labels over geom_bar in R with ggplot2
(4 answers)
Closed 5 years ago.
Having a dataset like this:
df <- structure(list(word = structure(c(1L, 12L, 23L, 34L, 43L, 44L,
45L, 46L, 47L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 13L,
14L, 15L, 16L, 17L, 18L, 19L, 20L, 21L, 22L, 24L, 25L, 26L, 27L,
28L, 29L, 30L, 31L, 32L, 33L, 35L, 36L, 37L, 38L, 39L, 40L, 41L,
42L), .Label = c("word1", "word10", "word11", "word12", "word13",
"word14", "word15", "word16", "word17", "word18", "word19", "word2",
"word20", "word21", "word22", "word23", "word24", "word25", "word26",
"word27", "word28", "word29", "word3", "word30", "word31", "word32",
"word33", "word34", "word35", "word36", "word37", "word38", "word39",
"word4", "word40", "word41", "word42", "word43", "word44", "word45",
"word46", "word47", "word5", "word6", "word7", "word8", "word9"
), class = "factor"), frq = c(1975L, 1665L, 1655L, 1469L, 1464L,
1451L, 1353L, 1309L, 1590L, 1545L, 1557L, 1556L, 1130L, 1153L,
1151L, 1150L, 1144L, 1141L, 1115L, 194L, 195L, 135L, 135L, 130L,
163L, 167L, 164L, 159L, 153L, 145L, 143L, 133L, 133L, 153L, 153L,
150L, 119L, 115L, 115L, 115L, 114L, 113L, 113L, 113L, 115L, 102L,
101L)), .Names = c("word", "frq"), class = "data.frame", row.names = c(NA,
-47L))
With this command lines I produce a bar plot graph
dat2 = transform(df,word = reorder(word,frq))
df2 <- head(dat2, 10)
p = ggplot(df2, aes(x = word, y = frq)) + geom_bar(stat = "identity", fill = "yellow")
p2 <- p +coord_flip()
How is it possible to have the number of frq in the end of every bar?
I would use annotate..
p2 + annotate(geom = "text",x = df2$word, y= df2$frq, label = df2$frq)

Plotting a Dataframe in R

I have a dataframe of the form
Region Name 3-15 4-15 5-15 ... 3-16
Name1 30 82 56 ... 32
Name2 65 23 38 ... 11
... ... ... ... ... ...
Name18 87 33 11 ... 51
The first column being the names of regions and the other columns being recorded events over time (monthly by column)
I'd like to plot the recorded monthly values over time with respect to their associated name. Specifically, a different line for each Named region with a differentiated colour. Any advice would be appreciated, a lot of the plotting functions for data frames seem to function on frames of a different format.
dput() data:
dataframe <- structure(list("LSOA Name" = c("Lancaster 001", "Lancaster 002",
"Lancaster 003", "Lancaster 004", "Lancaster 005", "Lancaster 006",
"Lancaster 008", "Lancaster 009", "Lancaster 010", "Lancaster 011",
"Lancaster 013", "Lancaster 014", "Lancaster 015", "Lancaster 016",
"Lancaster 017", "Lancaster 018", "Lancaster 019", "Lancaster 020"
), "3-15" = c(49L, 16L, 17L, 28L, 21L, 197L, 57L, 143L, 78L,
121L, 67L, 223L, 41L, 86L, 66L, 27L, 40L, 77L), "4-15" = c(63L,
11L, 26L, 29L, 19L, 203L, 69L, 154L, 82L, 125L, 62L, 198L, 44L,
99L, 64L, 26L, 42L, 99L), "5-15" = c(67L, 10L, 20L, 30L, 10L,
194L, 62L, 186L, 61L, 110L, 75L, 273L, 29L, 126L, 92L, 34L, 41L,
88L), "6-15" = c(58L, 8L, 18L, 36L, 29L, 198L, 62L, 167L, 83L,
110L, 59L, 254L, 26L, 99L, 73L, 17L, 30L, 109L), "7-15" = c(53L,
29L, 27L, 23L, 38L, 188L, 56L, 149L, 90L, 129L, 37L, 226L, 32L,
119L, 57L, 14L, 30L, 96L), "8-15" = c(44L, 9L, 25L, 28L, 29L,
237L, 69L, 171L, 78L, 108L, 45L, 261L, 22L, 103L, 68L, 33L, 35L,
108L), "9-15" = c(59L, 12L, 18L, 35L, 19L, 230L, 45L, 128L, 74L,
144L, 56L, 223L, 26L, 90L, 51L, 27L, 23L, 120L), "10-15" = c(45L,
26L, 31L, 23L, 25L, 195L, 53L, 155L, 74L, 120L, 58L, 276L, 38L,
92L, 72L, 25L, 40L, 123L), "11-15" = c(31L, 11L, 33L, 15L, 19L,
188L, 52L, 127L, 66L, 102L, 50L, 241L, 26L, 74L, 72L, 26L, 35L,
68L), "12-15" = c(34L, 22L, 21L, 22L, 17L, 205L, 80L, 150L, 73L,
109L, 50L, 228L, 29L, 57L, 59L, 14L, 45L, 93L), "1-16" = c(20L,
9L, 25L, 21L, 11L, 199L, 46L, 124L, 65L, 117L, 40L, 224L, 28L,
88L, 43L, 22L, 18L, 94L), "2-16" = c(54L, 11L, 29L, 20L, 11L,
164L, 44L, 117L, 70L, 85L, 46L, 192L, 23L, 89L, 50L, 27L, 29L,
86L), "3-16" = c(53L, 11L, 24L, 26L, 19L, 203L, 45L, 144L, 66L,
109L, 47L, 213L, 15L, 120L, 59L, 15L, 33L, 127L)), .Names = c("LSOA Name",
"3-15", "4-15", "5-15", "6-15", "7-15", "8-15", "9-15", "10-15",
"11-15", "12-15", "1-16", "2-16", "3-16"), row.names = c(NA,
-18L), class = "data.frame")
A typical way of plotting lines by groups in ggplot is to shift the data to long format, where one column identifies the group, and the other columns identify the x and y axis values.
This example shifts your data into long format with three columns: LSOAName, month_col, and values_col. It adds a day value onto the month-year, and converts that column to a date. Then it plots a line for each group.
I've renamed your dataframe d, because dataframe could be easily misinterpreted as the function data.frame().
# load libraries
library(magrittr)
library(dplyr)
library(tidyr)
library(ggplot2)
# rename dataframe so it doesn't look so much like the base function
d <- dataframe
# remove spaces in column names
names(d) <- gsub(" ", "", names(d))
# shift data from wide to long and then
# add a day value and convert day-month-year to date class
d %<>% gather(month_col, values_col, -LSOAName) %>%
mutate(month_col = as.Date(paste0("1-", month_col), "%d-%m-%y"))
# plot using ggplot2
ggplot(d, aes(x = month_col, y = values_col, colour = LSOAName)) +
geom_line()
Edit
%<>% is found in the magrittr package. It is a compound pipe assignment operator. While %>% returns the result of a pipeline, %<>% assigns the result back to the left side object.
Instead of writing
d <- d %>% [pipeline]
you can assign the results to d by writing
d %<>% [pipeline]

How to union tables in R?

I am trying to union two tables. Every month I get new data coming in. It will be handy for me to add the new data to the existing dataframe. I am not seeking to merge them as they are the same variables.
A little example as follow: M and N have the same dimension. I would like to combine M and N together
Thanks in advance
M <- structure(list(ID= c(56L, 67L, 68L, 73L, 77L, 87L), Mary = c(73L,
82L, 80L, 78L, 79L, 80L), Dave = c(45L, 42L, 51L, 46L, 60L, 54L
), Anne = c(78L, 85L, 92L, 83L, 77L, 89L), Bob = c(51L, 49L,
58L, 54L, 62L, 68L)), .Names = c("ID", "Mary", "Dave", "Anne",
"Bob"), class = "data.frame", row.names = c(NA, -6L))
N <- structure(list(ID= c(53L, 22L, 21L, 73L, 727L, 27L), Mary = c(72L,
82L, 80L, 78L, 79L, 80L), Dave = c(45L, 42L, 51L, 46L, 62L, 54L
), Anne = c(78L, 85L, 92L, 22L, 77L, 89L), Bob = c(52L, 49L,
58L, 54L, 62L, 628L)), .Names = c("ID", "Mary", "Dave", "Anne",
"Bob"), class = "data.frame", row.names = c(NA, -6L))
This might be all you need:
MN <- rbind(M, N)
If the two data.frames have different columns, then I would recommend this instead:
library(plyr)
MN <- rbind.fill(M, N)
Finally, if you need to remove duplicates:
MN <- MN[!duplicated(MN),]

Resources