R - Skip columns in pmax command if they do not exist - r

I'd like to use the pmax command to create a new column. My code Looks like this:
Master <- Master %>%
mutate(RAM = pmax(RAM1, RAM2, RAM3, RAM4, RAM5, RAM6, RAM7, RAM8, RAM9, RAM10,
RAM11, RAM12, RAM13, RAM14, RAM15, RAM16, RAM17, RAM18,
RAM19, RAM20, RAM21, RAM22, RAM23, RAM24, RAM25, RAM26,
RAM27, RAM28, RAM29, RAM30, RAM31, RAM32, RAM33, RAM34,
RAM35, RAM36, RAM37, RAM38, RAM39, RAM40, RAM41, RAM42,
RAM43, RAM44, RAM45, RAM46, RAM47, RAM48, RAM49, RAM50,
RAM51, RAM52, RAM53, RAM54, RAM55, RAM56, RAM57, RAM58,
RAM59, RAM60, RAM61, RAM62, RAM63, RAM64, RAM65, RAM66,
RAM67, RAM68, RAM69, RAM70, RAM71, RAM72, RAM73, RAM74,
RAM75, RAM76, RAM77, RAM78, RAM79, RAM80, RAM81, RAM82,
RAM83, RAM84, RAM85, RAM86, RAM87, RAM88, RAM89, RAM90,
RAM91, RAM92, na.rm =T))
In my current data base, however, only the columns RAM1 to RAM8 exist. In this case, I want R to skip all the other columns mentioned in the Statement and to only use column RAM1 to RAM8 (it is okay if R displays an error message, but I don't want the program to interrupt running the code).
Any ideas how to do so?
Thanks!

One way to do this would be as follows:
Set up some data to make a reproducible example
set.seed(0)
Master <- data.frame(Other=100,RAM1=1:10, RAM2=1:10, RAM3=1:10, RAM4=1:10,
RAM5=1:10, RAM6=1:10, RAM7=1:10, RAM8=rnorm(10)+5)
Master[5,5] <- NA
Select required columns of the dataframe:
Master[colnames(Master) %in% paste0("RAM",1:92)]
Use do.call to run pmax using the selected columns as arguments, and adding the argument na.rm=TRUE
Master$RAM <- do.call(pmax, c(Master[colnames(Master) %in% paste0("RAM",1:92)], na.rm=TRUE))
Sample output:
Master
# Other RAM1 RAM2 RAM3 RAM4 RAM5 RAM6 RAM7 RAM8 RAM
#1 100 1 1 1 1 1 1 1 6.262954 6.262954
#2 100 2 2 2 2 2 2 2 4.673767 4.673767
#3 100 3 3 3 3 3 3 3 6.329799 6.329799
#4 100 4 4 4 4 4 4 4 6.272429 6.272429
#5 100 5 5 5 NA 5 5 5 5.414641 5.414641
#6 100 6 6 6 6 6 6 6 3.460050 6.000000
#7 100 7 7 7 7 7 7 7 4.071433 7.000000
#8 100 8 8 8 8 8 8 8 4.705280 8.000000
#9 100 9 9 9 9 9 9 9 4.994233 9.000000
#10 100 10 10 10 10 10 10 10 7.404653 10.000000

Related

Create a new column using the names of existing columns [duplicate]

This question already has answers here:
For each row return the column name of the largest value
(10 answers)
Closed 12 months ago.
From a data frame in this structure
1 2 3
0.2393876 0.3388785 0.4217339
0.2984463 0.3948705 0.3066832
0.3042033 0.3321453 0.3636514
0.2290906 0.2796392 0.4912702
0.1971198 0.3273682 0.4755120
0.2328518 0.3991225 0.3680256
0.3031909 0.3790213 0.3177878
0.2734835 0.3233044 0.4032121
0.3190916 0.3256503 0.3552581
0.2600000 0.3281130 0.4118871
How is it possible to compare the value in the three columns, find the highest and add the name of the column in a new variable. Example output:
1 2 3 highestcolumn
0.2393876 0.3388785 0.4217339 3
0.2984463 0.3948705 0.3066832 2
0.3042033 0.3321453 0.3636514 3
Use which.max and apply on each row.
df$highestcolumn <- apply(df, 1, which.max)
> df
X1 X2 X3 mean
1 0.2393876 0.3388785 0.4217339 3
2 0.2984463 0.3948705 0.3066832 2
3 0.3042033 0.3321453 0.3636514 3
4 0.2290906 0.2796392 0.4912702 3
5 0.1971198 0.3273682 0.4755120 3
6 0.2328518 0.3991225 0.3680256 2
7 0.3031909 0.3790213 0.3177878 2
8 0.2734835 0.3233044 0.4032121 3
9 0.3190916 0.3256503 0.3552581 3
10 0.2600000 0.3281130 0.4118871 3
We may use max.col
df$highestcolumn <- max.col(df, "first")
-output
> df
1 2 3 highestcolumn
1 0.2393876 0.3388785 0.4217339 3
2 0.2984463 0.3948705 0.3066832 2
3 0.3042033 0.3321453 0.3636514 3
4 0.2290906 0.2796392 0.4912702 3
5 0.1971198 0.3273682 0.4755120 3
6 0.2328518 0.3991225 0.3680256 2
7 0.3031909 0.3790213 0.3177878 2
8 0.2734835 0.3233044 0.4032121 3
9 0.3190916 0.3256503 0.3552581 3
10 0.2600000 0.3281130 0.4118871 3

Strange behaviour gtools:mixedorder combined with dplyr::arrange?

I noticed an unexpected outcome while using dplyr::arrange together with gtools::mixedorder.
Consider:
library(tidyverse)
test <- data.frame(V1 = c("all13_LG1", "all13_LG10", "all13_LG11",
"all13_LG12", "all13_LG13", "all13_LG14", "all13_LG15", "all13_LG16",
"all13_LG2", "all13_LG3", "all13_LG4", "all13_LG5", "all13_LG6",
"all13_LG7", "all13_LG8", "all13_LG9"),
V2 = c(rep(1:16)))
test2 <- test %>% arrange(gtools::mixedorder(V1))
test3 <- test %>% slice(gtools::mixedorder(V1))
In test2 the 1st column is sorted: "all13_LG1", "all13_LG3", "all13_LG4", "all13_LG5", "all13_LG6", "all13_LG7", "all13_LG8", "all13_LG9", "all13_LG10", "all13_LG11", "all13_LG12", "all13_LG13", "all13_LG14", "all13_LG15", "all13_LG16", "all13_LG2"
Whereas in test3, the columns are sorted as one would expect when using gtools:mixedorder
Why is this happening when I combine arrange and mixedtools? Is this a bug?
Many thanks,
Anneke
To use results of mixedorder in arrange you need to order the results.
library(dplyr)
test %>% arrange(order(gtools::mixedorder(V1)))
# V1 V2
#1 all13_LG1 1
#2 all13_LG2 9
#3 all13_LG3 10
#4 all13_LG4 11
#5 all13_LG5 12
#6 all13_LG6 13
#7 all13_LG7 14
#8 all13_LG8 15
#9 all13_LG9 16
#10 all13_LG10 2
#11 all13_LG11 3
#12 all13_LG12 4
#13 all13_LG13 5
#14 all13_LG14 6
#15 all13_LG15 7
#16 all13_LG16 8
We can slice with mixedsort and match
library(dplyr)
test %>%
slice(match(gtools::mixedsort(V1), V1))
-output
V1 V2
1 all13_LG1 1
2 all13_LG2 9
3 all13_LG3 10
4 all13_LG4 11
5 all13_LG5 12
6 all13_LG6 13
7 all13_LG7 14
8 all13_LG8 15
9 all13_LG9 16
10 all13_LG10 2
11 all13_LG11 3
12 all13_LG12 4
13 all13_LG13 5
14 all13_LG14 6
15 all13_LG15 7
16 all13_LG16 8

Extracting data from dataframe using different dataframe without headers (R)

I have a gridded data as a data-frame that has daily temperatures (in K) for 30 years. I need to extract data for days that matches another data-frame and keep the first and second columns (lon and lat).
Data example:
gridded data from which I need to remove days that do not match days in the second data (df2$Dates)
>head(Daily.df)
lon lat 1991-05-01 1991-05-02 1991-05-03 1991-05-04 1991-05-05 1991-05-06 1991-05-07 1991-05-08 1991-05-09
1 5.000 60 278.2488 280.1225 280.3909 279.4138 276.6809 276.2085 276.6250 277.7930 276.9693
2 5.125 60 278.2514 280.1049 280.3789 279.4395 276.7141 276.2467 276.6571 277.8264 277.0225
3 5.250 60 278.2529 280.0871 280.3648 279.4634 276.7437 276.2849 276.6918 277.8608 277.0740
4 5.375 60 278.2537 280.0687 280.3488 279.4858 276.7691 276.3238 276.7289 277.8960 277.1232
5 5.500 60 278.2537 280.0493 280.3319 279.5066 276.7909 276.3633 276.7688 277.9313 277.1701
6 5.625 60 278.2539 280.0294 280.3143 279.5264 276.8090 276.4042 276.8111 277.9666 277.2147
1991-05-10 1991-05-11 1991-05-12 1991-05-13 1991-05-14 1991-05-15 1991-05-16 1991-05-17 1991-05-18 1991-05-19
1 276.9616 277.3436 273.3149 274.4931 274.6967 275.6298 272.2511 271.5413 271.7289 271.7964
2 276.9689 277.2988 273.3689 274.5399 274.6801 275.6307 272.2214 271.4445 271.6410 271.7023
3 276.9720 277.2533 273.4225 274.5811 274.6646 275.6241 272.1858 271.3391 271.5424 271.5989
4 276.9716 277.2080 273.4726 274.6146 274.6507 275.6109 272.1456 271.2274 271.4340 271.4872
5 276.9689 277.1632 273.5163 274.6382 274.6380 275.5917 272.1022 271.1121 271.3168 271.3693
6 276.9645 277.1190 273.5507 274.6501 274.6263 275.5672 272.0571 270.9955 271.1919 271.2469
1991-05-20 1991-05-21 1991-05-22 1991-05-23 1991-05-24 1991-05-25 1991-05-26 1991-05-27 1991-05-28 1991-05-29
1 272.2633 268.0039 268.5981 269.4139 267.7836 265.8771 263.5669 266.1666 269.7285 272.5083
2 272.2543 268.0218 268.5847 269.4107 267.7886 265.8743 263.5125 266.1031 269.6471 272.4676
3 272.2434 268.0369 268.5716 269.4089 267.7910 265.8669 263.4592 266.0332 269.5697 272.4217
4 272.2308 268.0507 268.5597 269.4090 267.7925 265.8559 263.4066 265.9581 269.4987 272.3714
5 272.2164 268.0642 268.5505 269.4112 267.7936 265.8425 263.3546 265.8797 269.4355 272.3175
6 272.2005 268.0793 268.5451 269.4154 267.7962 265.8276 263.3039 265.7997 269.3818 272.2614
1991-05-30 1991-05-31 1991-06-01 1991-06-02 1991-06-03 1991-06-04 1991-06-05 1991-06-06 1991-06-07 1991-06-08
1 274.2950 273.4715 274.5197 274.7548 273.8259 272.4433 274.1811 274.4135 274.3999 276.0327
2 274.2205 273.4638 274.5292 274.8316 273.8658 272.4700 274.1992 274.4426 274.4650 276.0698
3 274.1421 273.4549 274.5373 274.9027 273.9028 272.4980 274.2160 274.4781 274.5309 276.1012
4 274.0609 273.4452 274.5438 274.9665 273.9365 272.5273 274.2322 274.5211 274.5969 276.1255
5 273.9784 273.4353 274.5482 275.0216 273.9660 272.5576 274.2481 274.5725 274.6617 276.1417
6 273.8960 273.4253 274.5508 275.0668 273.9912 272.5887 274.2649 274.6334 274.7239 276.1487
1991-06-09 1991-06-10 1991-06-11 1991-06-12 1991-06-13 1991-06-14 1991-06-15 1991-06-16 1991-06-17 1991-06-18
1 276.5216 277.1812 277.8093 278.3013 278.5323 278.5403 277.9563 278.3461 275.8296 273.8277
2 276.5531 277.1925 277.8261 278.3409 278.4956 278.5317 277.9148 278.3234 275.8167 273.8302
3 276.5861 277.2065 277.8457 278.3748 278.4503 278.5181 277.8654 278.2939 275.8057 273.8358
4 276.6204 277.2239 277.8684 278.4029 278.3988 278.4996 277.8080 278.2583 275.7966 273.8427
5 276.6564 277.2466 277.8945 278.4253 278.3423 278.4759 277.7429 278.2171 275.7888 273.8504
6 276.6938 277.2753 277.9242 278.4414 278.2834 278.4472 277.6715 278.1714 275.7819 273.8570
1991-06-19 1991-06-20 1991-06-21 1991-06-22 1991-06-23 1991-06-24 1991-06-25 1991-06-26 1991-06-27 1991-06-28
1 275.1738 274.6805 275.6100 274.8936 273.5818 273.2099 273.1788 271.2747 273.2458 276.9931
2 275.1808 274.7123 275.7043 274.9494 273.5861 273.1770 273.2280 271.2435 273.2662 276.9822
3 275.1859 274.7478 275.7993 275.0009 273.5956 273.1439 273.2730 271.2133 273.2803 276.9678
4 275.1891 274.7879 275.8941 275.0467 273.6107 273.1106 273.3130 271.1840 273.2886 276.9502
5 275.1902 274.8337 275.9870 275.0857 273.6318 273.0777 273.3472 271.1556 273.2918 276.9307
6 275.1891 274.8864 276.0776 275.1168 273.6589 273.0454 273.3752 271.1285 273.2905 276.9101
1991-06-29 1991-06-30
1 272.0784 273.5677
2 272.0577 273.5973
3 272.0339 273.6237
4 272.0075 273.6476
5 271.9794 273.6701
6 271.9500 273.6925
Second data I'm using for extracting (using Dates variable)
>head(df2)
Dates Temp Wind.S Wind.D
1 5/1/1991 18 4 238
2 5/2/1991 18 8 93
3 5/4/1991 22 8 229
4 5/6/1991 21 4 81
5 5/7/1991 21 8 192
6 5/9/1991 17 8 32
7 5/13/1991 22 8 229
8 5/18/1991 21 4 81
9 6/2/1991 21 8 192
10 6/7/1991 17 8 32
The header of the final data I'm looking for is something like this:
>head(df3)
lon lat 1991-05-01 1991-05-02 1991-05-04 1991-05-06 1991-05-09 1991-05-13
Example data following the format of yours
Daily.df <- data.frame(lon=1:5,lat=1:5,A=1:5,B=1:5,C=1:5,D=1:5)
colnames(Daily.df) <- c("lon","lat","1991-05-01","1991-05-02","1991-05-03","1991-05-04")
lon lat 1991-05-01 1991-05-02 1991-05-03 1991-05-04
1 1 1 1 1 1 1
2 2 2 2 2 2 2
3 3 3 3 3 3 3
4 4 4 4 4 4 4
5 5 5 5 5 5 5
df2 <- data.frame(Dates = c("5/1/1991","5/2/1991","5/4/1991"))
Dates
1 5/1/1991
2 5/2/1991
3 5/4/1991
Using lubridate to convert df2$Dates into the right format, make a vector of the column names you want to keep (thesedates) including lon and lat. Then use select_at to keep those columns.
library(lubridate)
library(dplyr)
thesedates <- c("lon","lat",as.character(mdy(df2$Dates)))
new.df <- Daily.df %>%
select_at(vars(thesedates))
Output
lon lat 1991-05-01 1991-05-02 1991-05-04
1 1 1 1 1 1
2 2 2 2 2 2
3 3 3 3 3 3
4 4 4 4 4 4
5 5 5 5 5 5
If you want to have a long data set to match, I would think you need to first convert the dates in df2 into the proper format and then wrangle the data into wide format.
Step 1 - Convert dates into correct format
df2$Dates <- as.Date(df2$Dates, format = "%m/%d/%Y")
Step 2 - convert to wide format
library(tidyr)
spread(df2, Dates, data)

Replacing each value in a vector with its rank number for a data.frame

In this hypothetical scenario, I have performed 5 different analyses on 13 chemicals, resulting in a score assigned to each chemical within each analysis. I have created a table as follows:
---- Analysis1 Analysis2 Analysis3 Analysis4 Analysis5
Chem_1 3.524797844 4.477695034 4.524797844 4.524797844 4.096698498
Chem_2 2.827511555 3.827511555 3.248136118 3.827511555 3.234398548
Chem_3 2.682144761 3.474646298 3.017780505 3.682144761 3.236152242
Chem_4 2.134137304 2.596921333 2.95181339 2.649076603 2.472875191
Chem_5 2.367736454 3.027814219 2.743137896 3.271122346 2.796607809
Chem_6 2.293110565 2.917318708 2.724156207 3.293110565 2.530967343
Chem_7 2.475709113 3.105794018 2.708222528 3.475709113 3.088819908
Chem_8 2.013451822 2.259454085 2.683273938 2.723554966 2.400976121
Chem_9 2.345123123 3.050074893 2.682845391 3.291851228 2.700844104
Chem_10 2.327658894 2.848729452 2.580415233 3.327658894 2.881490893
Chem_11 2.411243882 2.98131398 2.554456095 3.411243882 3.109205453
Chem_12 2.340778276 2.576860244 2.549707035 3.340778276 3.236545826
Chem_13 2.394698249 2.90682524 2.542599327 3.394698249 3.12936843
I would like to create columns corresponding to each analysis which contain the rank position for each chemical. For instance, under Analysis1,Chem_1 would have value "1", Chem_2 would have value "2", Chem_3 would have value "4", Chem_7 would have value "4", Chem_11 would have value "5", and so on.
We can use dense_rank from dplyr
library(dplyr)
df %>%
mutate_each(funs(dense_rank(-.)))
In base R, we can do
df[] <- lapply(-df, rank, ties.method="min")
In data.table, we can use
library(data.table)
setDT(df)[, lapply(-.SD, frank, ties.method="dense")]
To avoid the copies from multiplying with -, as #Arun mentioned in the comments
lapply(.SD, frankv, order=-1L, ties.method="dense")
You can also do this in base R:
cbind("..." = df[,1], data.frame(do.call(cbind,
lapply(df[,-1], order, decreasing = T))))
... Analysis1 Analysis2 Analysis3 Analysis4 Analysis5
1 Chem_1 1 1 1 1 1
2 Chem_2 2 2 2 2 12
3 Chem_3 3 3 3 3 3
4 Chem_4 7 7 4 7 2
5 Chem_5 11 9 5 11 13
6 Chem_6 13 5 6 13 11
7 Chem_7 5 11 7 12 7
8 Chem_8 9 6 8 10 10
9 Chem_9 12 13 9 6 5
10 Chem_10 10 10 10 9 9
11 Chem_11 6 4 11 5 6
12 Chem_12 4 12 12 8 4
13 Chem_13 8 8 13 4 8
If I'm not mistaken, you want to have the column-wise rank of your table. Here is my solution:
m=data.matrix(df) # converts data frame to matrix, convert your data to matrix accordingly
apply(m, 2, function(c) rank(c)) # increasingly
apply(m, 2, function(c) rank(-c)) # decreasingly
However, I believe you could solve it by yourself with the help of the answers to this question
Get rank of matrix entries?

How to put information obtained by cast function of reshape package back in my original data frame in R

I have a data.frame in panel format (country-year) and I need to calculate the mean of a variable by country and at each five years. So I just used the 'cast' function from 'reshape' package and it worked. Now I need to put this information(the mean by quinquennium) in the old data.frame, so I can run some regressions. How can I do that? Below I provide an example to ilustrate what I want:
set.seed(2)
fake= data.frame(y=rnorm(20), x=rnorm(20), country=rep(letters[1:2], each=10), year=rep(1:10,2), quinquenio= rep(rep(1:2, each=5),2))
fake.m = melt.data.frame(fake, id.vars=c("country", "year", "quinquenio"))
cast(fake.m, country ~ quinquenio, mean, subset=variable=="x", na.rm=T)
Now, everything is fine and I get what I wantted: the mean of x and y, by country and by quinquennial years. Now, I would like to put them back in the data.frame fake, like this:
y x country year quinquenio mean.x
1 -0.89691455 2.090819205 a 1 1 0.8880242
2 0.18484918 -1.199925820 a 2 1 0.8880242
3 1.58784533 1.589638200 a 3 1 0.8880242
4 -1.13037567 1.954651642 a 4 1 0.8880242
5 -0.08025176 0.004937777 a 5 1 0.8880242
6 0.13242028 -2.451706388 a 6 2 -0.2978375
7 0.70795473 0.477237303 a 7 2 -0.2978375
8 -0.23969802 -0.596558169 a 8 2 -0.2978375
9 1.98447394 0.792203270 a 9 2 -0.2978375
10 -0.13878701 0.289636710 a 10 2 -0.2978375
11 0.41765075 0.738938604 b 1 1 0.2146461
12 0.98175278 0.318960401 b 2 1 0.2146461
13 -0.39269536 1.076164354 b 3 1 0.2146461
14 -1.03966898 -0.284157720 b 4 1 0.2146461
15 1.78222896 -0.776675274 b 5 1 0.2146461
16 -2.31106908 -0.595660499 b 6 2 -0.8059598
17 0.87860458 -1.725979779 b 7 2 -0.8059598
18 0.03580672 -0.902584480 b 8 2 -0.8059598
19 1.01282869 -0.559061915 b 9 2 -0.8059598
20 0.43226515 -0.246512567 b 10 2 -0.8059598
I appreciate any tip in the right direction. Thanks in advance.
ps.: the reason I need this is that I'll run a regression with quinquennial data, and for some variables (like per capita income) I have information for all years, so I decided to average them by 5 years.
I'm sure there's an easy way to do this with reshape, but my brain defaults to plyr first:
require(plyr)
ddply(fake, c("country", "quinquenio"), transform, mean.x = mean(x))
This is quite hackish, but one way to use reshape building off your earlier work:
zz <- cast(fake.m, country ~ quinquenio, mean, subset=variable=="x", na.rm=T)
merge(fake, melt(zz), by = c("country", "quinquenio"))
though I'm positive there has to be a better solution.
Here's a more old school approach using tapply, ave, and with
fake$mean.x <- with(fake, unlist(tapply(x, list(country, quinquenio), ave)))

Resources