I am using R programming language. Suppose I have the following data ("my_data"):
student first_run second_run third_run fourth_run fifth_run sixth_run seventh_run eight_run ninth_run tenth_run
1 student1 19.70847 21.79771 16.49083 19.51691 13.97987 14.60733 13.89703 15.24651 20.75679 18.44020
2 student2 11.22369 15.36253 16.90215 20.20724 15.90227 15.14539 13.74945 18.30090 19.55124 17.24132
3 student3 15.93649 17.03599 14.20214 13.17548 14.70327 15.49697 13.08945 19.94142 22.41674 17.37958
4 student4 16.18733 15.13197 14.79481 16.75177 14.51287 17.71816 13.45054 14.25553 19.89091 18.88981
5 student5 18.71084 18.85453 17.15864 19.38880 15.68862 18.39169 15.26428 16.04526 18.92532 16.62409
6 student6 19.75246 12.74605 18.52214 17.92626 14.48501 17.20780 13.10512 12.46502 20.68583 15.87711
7 student7 14.75144 23.82376 18.51366 20.77424 14.22155 16.08186 12.95981 12.67820 20.12166 15.66006
8 student8 17.06516 15.63075 13.72026 15.02068 14.21098 15.99414 14.64818 16.15603 21.74607 17.07382
9 student9 20.27611 12.44592 12.26502 15.13456 14.61552 18.72192 15.11129 17.60746 18.83831 17.55257
10 student10 17.70736 16.21620 14.10861 17.20014 16.59376 19.50027 13.05073 15.80002 18.09781 18.34313
I want to add 2 columns to this data:
my_mean : the mean of each row
my_median: the median of each row
I tried the following code in R:
my_data$median = apply(my_data, 1, median, na.rm=T)
my_data$mean = apply(my_data, 1, mean, na.rm=T)
But I don't think this code is correct. For instance, when using this code, the median of the second row of data is returned as "16.90215"
But when I manually take the median of this row:
median(11.22369 , 15.36253 , 16.90215 , 20.20724, 15.90227 , 15.14539 , 13.74945 , 18.30090 , 19.55124 , 17.24132)
I get an answer of
11.22
Can someone please show me what I am doing wrong?
Thanks
The calculation is incorrect i.e. the first argument of median is 'x' which can be a vector. The second argument is na.rm, followed by variadic arguments .... So, when write 11.22369, 15.36253, the 'x' is taken as 11.22369 and that is the value returned. Instead, it should be a vector by concatenation c
median(c(11.22369 , 15.36253 , 16.90215 , 20.20724, 15.90227 , 15.14539 , 13.74945 , 18.30090 , 19.55124 , 17.24132))
[1] 16.40221
Also, based on the OP's data, the first column should be dropped which is character or factor
apply(my_data[-1], 1, median, na.rm=TRUE)
1 2 3 4 5 6 7 8 9 10
17.46551 16.40221 15.71673 15.65965 17.77517 16.54246 15.87096 15.81245 16.34356 16.89695
The second row is used in the manual calculation
library(dplyr)
df %>%
rowwise() %>%
mutate(median = median(c_across(where(is.numeric))),
mean = mean(c_across(where(is.numeric))))
c_across and rowwise were created for this type of situation. Most verbs work column-wise. To change this behavior pipe to rowwise first.
c_across will then combine all values in a row that are numeric (hence where(is.numeric) into a numeric vector and then mean or median can be applied.
Note: You will likely want to pipe the output to ungroup since rowwise creates a rowwise grouped data frame.
Here is an alternative using pmap along with passing all the arguments simultaneously thus using ellipsis i.e. .... The output is needed to be unnested with unnest_wider from tidyr:
library(tidyr)
library(dplyr)
library(purrr)
df %>%
mutate(res = pmap(across(where(is.numeric)),
~ list(median = median(c(...)),
avg = mean(c(...))))) %>%
unnest_wider(res)
output:
student first_run second_run third_run fourth_run fifth_run sixth_run seventh_run eight_run ninth_run tenth_run median avg
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 student1 19.7 21.8 16.5 19.5 14.0 14.6 13.9 15.2 20.8 18.4 17.5 17.4
2 student2 11.2 15.4 16.9 20.2 15.9 15.1 13.7 18.3 19.6 17.2 16.4 16.4
3 student3 15.9 17.0 14.2 13.2 14.7 15.5 13.1 19.9 22.4 17.4 15.7 16.3
4 student4 16.2 15.1 14.8 16.8 14.5 17.7 13.5 14.3 19.9 18.9 15.7 16.2
5 student5 18.7 18.9 17.2 19.4 15.7 18.4 15.3 16.0 18.9 16.6 17.8 17.5
6 student6 19.8 12.7 18.5 17.9 14.5 17.2 13.1 12.5 20.7 15.9 16.5 16.3
7 student7 14.8 23.8 18.5 20.8 14.2 16.1 13.0 12.7 20.1 15.7 15.9 17.0
8 student8 17.1 15.6 13.7 15.0 14.2 16.0 14.6 16.2 21.7 17.1 15.8 16.1
9 student9 20.3 12.4 12.3 15.1 14.6 18.7 15.1 17.6 18.8 17.6 16.3 16.3
10 student10 17.7 16.2 14.1 17.2 16.6 19.5 13.1 15.8 18.1 18.3 16.9 16.7
You could definitely benefit from the speed of matrixStats library.
matrixStats::rowMedians(as.matrix(d[-1]))
# [1] 17.46551 16.40221 15.71673 15.65965 17.77517 16.54246 15.87096 15.81245 16.34356 16.89695
matrixStats::rowMeans2(as.matrix(d[-1]))
# [1] 17.44417 16.35862 16.33775 16.15837 17.50521 16.27728 16.95862 16.12661 16.25687 16.66180
stopifnot(all.equal(matrixStats::rowMedians(as.matrix(d[-1])),
as.numeric(apply(d[-1], 1, median, na.rm=T))))
stopifnot(all.equal(matrixStats::rowMeans2(as.matrix(d[-1])),
as.numeric(apply(d[-1], 1, mean, na.rm=T))))
Data:
d <- structure(list(student = c("student1", "student2", "student3",
"student4", "student5", "student6", "student7", "student8", "student9",
"student10"), first_run = c(19.70847, 11.22369, 15.93649, 16.18733,
18.71084, 19.75246, 14.75144, 17.06516, 20.27611, 17.70736),
second_run = c(21.79771, 15.36253, 17.03599, 15.13197, 18.85453,
12.74605, 23.82376, 15.63075, 12.44592, 16.2162), third_run = c(16.49083,
16.90215, 14.20214, 14.79481, 17.15864, 18.52214, 18.51366,
13.72026, 12.26502, 14.10861), fourth_run = c(19.51691, 20.20724,
13.17548, 16.75177, 19.3888, 17.92626, 20.77424, 15.02068,
15.13456, 17.20014), fifth_run = c(13.97987, 15.90227, 14.70327,
14.51287, 15.68862, 14.48501, 14.22155, 14.21098, 14.61552,
16.59376), sixth_run = c(14.60733, 15.14539, 15.49697, 17.71816,
18.39169, 17.2078, 16.08186, 15.99414, 18.72192, 19.50027
), seventh_run = c(13.89703, 13.74945, 13.08945, 13.45054,
15.26428, 13.10512, 12.95981, 14.64818, 15.11129, 13.05073
), eight_run = c(15.24651, 18.3009, 19.94142, 14.25553, 16.04526,
12.46502, 12.6782, 16.15603, 17.60746, 15.80002), ninth_run = c(20.75679,
19.55124, 22.41674, 19.89091, 18.92532, 20.68583, 20.12166,
21.74607, 18.83831, 18.09781), tenth_run = c(18.4402, 17.24132,
17.37958, 18.88981, 16.62409, 15.87711, 15.66006, 17.07382,
17.55257, 18.34313)), class = "data.frame", row.names = c("1",
"2", "3", "4", "5", "6", "7", "8", "9", "10"))
Related
I have two data frames in R
df1 <- data.frame(Name = c("RIS_001", "RIS_002", "RIS_003", "RIS_004", "RIS_005")) %>%
mutate(Value = c(5, 3, 8, 6, 9))
df2 <- data.frame(Prod = c("RIS_010", "RIS_011", "RIS_012", "RIS_013", "RIS_014", "RIS_015", "RIS_016", "RIS_017")) %>%
mutate(Value = c(54, 87, 92, 48, 66, 35, 12, 18))
I want to create two new data frames from them as in below images. How to accomplish this in R?
new_df1, concatenation of Row & Column or Column & Row separated by a special character "|"
new_df2, mean of each intersecting row & column
Ex: Mean for RIS_001 & RIS_010 = mean(5 + 54) = 29.5
Appreciate any help. Thank you!
You can outer for the first df:
o <- outer(df1$Name, df2$Prod, paste, sep = " | ")
rownames(o) <- df1$Name
colnames(o) <- df2$Prod
And for the second df:
m <- outer(df1$Value, df2$Value, \(x, y) (x + y) / 2)
rownames(m) <- df1$Name
colnames(m) <- df2$Prod
RIS_010 RIS_011 RIS_012 RIS_013 RIS_014 RIS_015 RIS_016 RIS_017
RIS_001 29.5 46.0 48.5 26.5 35.5 20.0 8.5 11.5
RIS_002 28.5 45.0 47.5 25.5 34.5 19.0 7.5 10.5
RIS_003 31.0 47.5 50.0 28.0 37.0 21.5 10.0 13.0
RIS_004 30.0 46.5 49.0 27.0 36.0 20.5 9.0 12.0
RIS_005 31.5 48.0 50.5 28.5 37.5 22.0 10.5 13.5
I might be mis-interpreting your question, but this seems to me to be what you are asking?
library(tidyverse)
#vals sep by bar
outer(
pivot_wider(df1, names_from = Name, values_from = Value),
pivot_wider(df2, names_from = Prod, values_from = Value),
\(x, y) paste(x, y, sep = "|")
)
#> RIS_010 RIS_011 RIS_012 RIS_013 RIS_014 RIS_015 RIS_016 RIS_017
#> RIS_001 "5|54" "5|87" "5|92" "5|48" "5|66" "5|35" "5|12" "5|18"
#> RIS_002 "3|54" "3|87" "3|92" "3|48" "3|66" "3|35" "3|12" "3|18"
#> RIS_003 "8|54" "8|87" "8|92" "8|48" "8|66" "8|35" "8|12" "8|18"
#> RIS_004 "6|54" "6|87" "6|92" "6|48" "6|66" "6|35" "6|12" "6|18"
#> RIS_005 "9|54" "9|87" "9|92" "9|48" "9|66" "9|35" "9|12" "9|18"
#vals mean
outer(
pivot_wider(df1, names_from = Name, values_from = Value),
pivot_wider(df2, names_from = Prod, values_from = Value),
\(x, y) (as.numeric(x)+as.numeric(y))/2
)
#> RIS_010 RIS_011 RIS_012 RIS_013 RIS_014 RIS_015 RIS_016 RIS_017
#> RIS_001 29.5 46.0 48.5 26.5 35.5 20.0 8.5 11.5
#> RIS_002 28.5 45.0 47.5 25.5 34.5 19.0 7.5 10.5
#> RIS_003 31.0 47.5 50.0 28.0 37.0 21.5 10.0 13.0
#> RIS_004 30.0 46.5 49.0 27.0 36.0 20.5 9.0 12.0
#> RIS_005 31.5 48.0 50.5 28.5 37.5 22.0 10.5 13.5
I'm trying to create a function that summarizes several vectors and the prompt is
Write a function data_summary which takes three inputs:\
`dataset`: A data frame\
`vars`: A character vector whose elements are names of columns from dataset which the user wants summaries for\
`group.name`: A length one character vector which gives the name of the column from dataset which contains the factor which will be used as a grouping variable
\`var.names`: A character vector of the same length as vars which gives the names that the user would like used as the entries under “Variable” in the resulting output. This should be set equal to vars by default, so the default behavior is to use the column names from dataset.
The output of the function should be a data frame with the following structure:
Column names of the data frame will be:\
`Variable`\
`Missing`\
The `first` level of the factor group.name\
The `second` level of the factor group.name\
…\
The `kth` level of the factor group.name\
`p-value`
I've set up the code already,
data_summary <- function(dataset,vars,group.name,var.names) {
}
but I'm unsure how to proceed because I do not understand what this is trying to accomplish and what the output should look like. There is an example that shows
#data_summary<-function(dataset, vars,group.name, var.name){}
#example
#data_summary(titanic4, c("survived", "female", "age", "sibsp", "parch", "fare", "cabin"), "pclass")
#data_summary(titanic4, c("survived", "female", "age", "sibsp", "parch", "fare", "cabin"), "pclass", c("Survival rate", "% Female", "Age", "# siblings/spouses aboard", "# children/parents aboard", "Fare ($)", "Cabin"))
But it really did not help me outside of inputting the arguments for the function.
You can use dplyr package for this function. Also I don't know by which functions you want summarise your dataframe, so I use all functions which summary function returns from base package.
My data:
> NewSKUMatrix
# A tibble: 268,918 x 4
LagerID FilialID CSBID Price
<int> <int> <int> <dbl>
1 233 2578 1005 38.3
2 333 2543 NA 61.0
3 334 2543 NA 15.0
4 335 2543 NA 11.0
5 337 2301 NA 71.0
6 338 2031 NA 37.0
7 338 2044 NA 35.0
8 338 2054 NA 36.0
9 338 2060 NA 37.0
10 338 2063 NA 36.0
# ... with 268,908 more rows
Function:
data_summary <- function(data,
variables,
values,
names = NULL) {
if (is.null(x = names)) {
names <- variables
}
data %>%
group_by_at(.vars = variables) %>%
summarise_at(
.vars = values,
.funs = list(
Min. = min,
`1st Qu.` = ~ quantile(x = ., probs = 0.25),
Median = median,
Mean = mean,
`3rd Qu.` = ~ quantile(x = ., probs = 0.75),
Max. = max
)
) %>%
rename_at(.vars = variables,
.funs = ~ names)
}
Output:
data_summary(NewSKUMatrix,
c('LagerID'),
c('Price'),
c('SKU'))
# A tibble: 32,454 x 7
SKU Min. `1st Qu.` Median Mean `3rd Qu.` Max.
<int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 17 39.0 39.0 39.0 39.0 39.0 39.0
2 18 120. 120. 120. 121. 120. 140.
3 21 289. 289. 289. 289. 289. 289.
4 24 37.0 37.0 37.0 45.2 45.2 70.0
5 25 14.0 14.0 14.0 14.0 14.0 14.0
6 55 30.9 30.9 30.9 30.9 30.9 30.9
7 117 26.9 26.9 26.9 26.9 26.9 26.9
8 118 24.8 24.9 24.9 25.1 25.1 25.7
9 119 24.8 24.8 24.9 25.1 25.3 25.7
10 158 104. 108. 108. 107. 108. 108.
# ... with 32,444 more rows
I have a dataframe with 6 features like this:
X1 X2 X3 X4 X5 X6
Modern Dog 9.7 21.0 19.4 7.7 32.0 36.5
Golden Jackal 8.1 16.7 18.3 7.0 30.3 32.9
Chinese Wolf 13.5 27.3 26.8 10.6 41.9 48.1
Indian Wolf 11.5 24.3 24.5 9.3 40.0 44.6
Cuon 10.7 23.5 21.4 8.5 28.8 37.6
Dingo 9.6 22.6 21.1 8.3 34.4 43.1
I want to draw a line plot like this:
I'm trying this:
plot(df$X1, type = "o",col = "red", xlab = "Month", ylab = "Rain fall")
lines(c(df$X2, df$X3, df$X4, df$X5, df$X6), type = "o", col = "blue")
But it's only plotting a single variable. I'm sorry if this question is annoying, i'm totally new to R and i just don't know how to get this done. I would really appreciate any help on this.
Thanks in advance
The easiest way would be to convert your dataset to a long format (e.g. by using the gather function in the tidyr package), and then plotting using the group aesthetic in ggplot.
I recreate your dataset, assuming your group variable is named "Group":
df <- read.table(text = "
Group X1 X2 X3 X4 X5 X6
Modern_Dog 9.7 21.0 19.4 7.7 32.0 36.5
Golden_Jackal 8.1 16.7 18.3 7.0 30.3 32.9
Chinese_Wolf 13.5 27.3 26.8 10.6 41.9 48.1
Indian_Wolf 11.5 24.3 24.5 9.3 40.0 44.6
Cuon 10.7 23.5 21.4 8.5 28.8 37.6
Dingo 9.6 22.6 21.1 8.3 34.4 43.1 ",
header = TRUE, stringsAsFactors = FALSE)
Then convert the dataset to long format and plot:
library(tidyr)
library(ggplot2)
df_long <- df %>% gather(X1:X6, key = "Month", value = "Rainfall")
ggplot(df_long, aes(x = Month, y = Rainfall, group = Group, shape = Group)) +
geom_line() +
geom_point() +
theme(legend.position = "bottom")
See also the answers here: Group data and plot multiple lines.
I am an R beginner user and I face the following problem. I have the following data frame:
distance speed
1 61.0 36.4
2 51.4 35.3
3 42.2 34.2
4 33.4 32.8
5 24.9 31.3
6 17.5 28.4
7 11.5 24.1
8 7.1 19.4
9 3.3 16.9
10 0.5 15.5
11 4.4 15.1
12 8.5 15.5
13 13.1 17.3
14 18.8 20.5
15 25.7 24.1
16 33.3 26.3
17 41.0 27.0
18 48.7 27.7
19 56.6 28.4
20 64.8 29.2
21 73.6 31.7
22 83.3 34.2
23 93.4 35.3
The column distance represents the distance of a following object over a specific point and the column speed the object's speed. As you can see the object is getting closer to the point and then it is getting away. I am trying to make its speed profile. I tried the following code but it didn't give me the plot I want (because I want to show how its speed is changing when the moving object moves closer and past the reference point)
ggplot(speedprofile, aes(x = distance, y = speed)) + #speedprofile is the data frame
geom_line(color = "red") +
geom_smooth() +
geom_vline(xintercept = 0) # the vline is the reference line
The plot is the following:
Then, I tried to set the first 10 distances as negative manually which are prior to zero (0). So I get a plot closer to that I want:
But there is a problem. The distance can't be defined as negative.
To sum up, the expected plot is the following (and I am sorry for the quality).
Do you have any ideas on how to solve this?
Thank you in advance!
You can do something like this to auto-compute the change point (to know when the distance should be negative) and then set the axis labels to be positive.
Your data (in case anyone needs it to answer):
read.table(text="distance speed
61.0 36.4
51.4 35.3
42.2 34.2
33.4 32.8
24.9 31.3
17.5 28.4
11.5 24.1
7.1 19.4
3.3 16.9
0.5 15.5
4.4 15.1
8.5 15.5
13.1 17.3
18.8 20.5
25.7 24.1
33.3 26.3
41.0 27.0
48.7 27.7
56.6 28.4
64.8 29.2
73.6 31.7
83.3 34.2
93.4 35.3", stringsAsFactors=FALSE, header=TRUE) -> speed_profile
Now, compute the "real" distance (negative for approaching, positive for receding):
speed_profile$real_distance <- c(-1, sign(diff(speed_profile$distance))) * speed_profile$distance
Now, compute the X axis breaks ahead of time:
breaks <- scales::pretty_breaks(10)(range(speed_profile$real_distance))
ggplot(speed_profile, aes(real_distance, speed)) +
geom_smooth(linetype = "dashed") +
geom_line(color = "#cb181d", size = 1) +
scale_x_continuous(
name = "distance",
breaks = breaks,
labels = abs(breaks) # make all the labels for the axis positive
)
Provided fonts are working well on your system you could even do:
labels <- abs(breaks)
labels[(!breaks == 0)] <- sprintf("%s\n→", labels[(!breaks == 0)])
ggplot(speed_profile, aes(real_distance, speed)) +
geom_smooth(linetype = "dashed") +
geom_line(color = "#cb181d", size = 1) +
scale_x_continuous(
name = "distance",
breaks = breaks,
labels = labels,
)
My data contains x axis points and y value for each x axis point. The x axis points are not evenly distributed. I need to visualize how the x axis points are clustered and how does the y value appears for such clusters. To see how the x values are clustered I can plot density plot on x value, however it does not reflect the y values at that cluster.
for example- if 100 points (lets say) on x axis are very close to each other and all has positive y value I want my plot go up at that point, if those 100 points has negative y value I want my plot go down the zero line in plot, if those 100 points has both positive and negative y values I want my plot be around zero point. Similarly, even if the those 100 points all has positive value, if they are scattered along long distance I want the plot be near the zero line.
In short, density of x points and its y value both matters to me and I want to plot smooth line. Could anyone help me with this?(stat_smooth did not do the work as it makes my plot almost straight line)
here are my x and y axis values (I did not know how to insert table here)
x axis values
x_value
86645
87018
987522
989433
989934
991055
995476
9987548
9987885
9988511
9988522
9991975
9992246
9992428
9993646
9993668
9994285
9994309
9994317
9994425
9994437
9994581
9994856
9994878
9995045
9995072
9995103
9995142
9995153
9995521
9996329
9996568
9997122
9997269
9997277
9997282
9998216
9999596
9999838
10001799
10004506
10007993
10008597
10009002
10009022
10009225
10009530
10009657
10010526
10012288
10012897
10012899
10012901
10014614
10014903
10015001
10015039
10015059
10015340
10015342
10016761
10018152
10020062
10024053
10024058
10024284
10024318
10025853
10026758
10028903
10029674
10029835
10030862
10031185
10031737
10033603
10035054
10035100
10036294
10036678
10036691
10036698
10036783
10037234
10037289
10037388
10039332
10039431
10042426
10042469
10042471
10043156
10043218
10043225
10045396
10045986
10046533
10046604
10047066
10047179
10047865
10048106
10048136
10048873
10049328
10049724
10049961
10049974
10050014
10050020
10050039
10050041
10050450
10050451
10050558
10050561
10051330
10051336
10052228
Y axix values:
y_value
16.7
14.3
10.5
18.2
20.0
16.7
14.3
10.4
27.3
22.2
11.1
-18.2
-10.1
-13.3
-26.4
-13.3
-15.4
14.3
15.4
11.7
26.7
18.2
64.7
21.2
20.0
11.8
-17.9
25.0
14.2
20.0
18.2
12.5
12.5
10.5
11.1
12.5
14.3
-20.0
12.5
-20.0
16.7
13.3
18.2
20.0
30.0
20.0
11.8
-18.8
20.0
20.0
12.5
18.8
13.3
-15.4
18.2
18.9
28.6
20.0
12.5
16.1
15.4
10.5
13.3
29.7
23.1
18.2
14.3
12.5
12.5
16.7
11.1
20.0
18.2
18.2
13.2
13.3
11.8
15.4
14.3
23.8
18.2
33.3
18.2
-12.5
12.5
23.1
21.7
14.3
16.7
11.1
16.7
12.5
11.1
12.5
18.2
12.5
11.0
20.0
18.2
15.8
10.5
10.2
10.5
14.3
11.8
25.0
13.8
16.4
16.7
-18.2
18.2
16.7
18.2
18.2
11.8
12.5
14.3
17.9
10.5
Note: In what follows, I've combined your x and y data into a data frame df with columns x and y.
Looking at a simple scatter plot, it appears that your data is grouped more or less into five clusters:
with(df,plot(x,y))
To see the distribution in both the x and y-direction you need a 2-dimensional kernal density estimate, which is available in package MASS. You can then plot this in 3 dimensions (with the density as z) using the rgl package.
library(MASS) # for kde2d(...)
library(rgl) # for open3d(...) and surface3d(...)
dens <- kde2d(df$x,df$y)
zlim <- range(dens$z)
palette <- rev(heat.colors(10))
col <- palette[9*(dens$z-zlim[1])/diff(zlim) + 1] # assign colors to heights for each point
with(dens,open3d(scale=c(x=1/diff(range(x)),y=1/diff(range(y)),z=1/diff(range(z)))))
with(dens,surface3d(x,y,z, color=col))
title3d(xlab="X",ylab="Y")