I have a large data.frame with several variables like "89+2" (all two-digit integer + one-digit integer) and I'm trying to quickly convert to numeric variables. Realistically, either just eliminating the second numeric OR performing the calculation and adding them together would work... Bit of an R newbie. Any help appreciated.
example:
df$LM = c("91+2", "89+3", "88+2")
Looking for
df$LM_num = c(91, 89, 88)
or
df$LM_num = c(93, 92, 90)
We can use separate
library(tidyr)
library(dplyr)
separate(df, LM, into = c("LM_num1", "LM_num2"), convert = TRUE) %>%
mutate(LM_num = LM_num1 + LM_num2)
Or with parse_number
library(readr)
df$LM_num <- parse_number(df$LM)
Or another option is eval(parse
df$LM_num <- sapply(df$LM, function(x) eval(parse(text = x)))
Assuming df given reproducibly in the Note at the end use the first line below to get the first number or the second line to get the sum. They both read the LM column as if it were a file splitting on + creating a two column data frame. The first line extracts the first column whereas the second line adds the two columns. No packages are used.
transform(df, LM_num = read.table(text = LM, sep = "+")[[1]])
transform(df, LM_num = rowSums(read.table(text = LM, sep = "+")))
Note
df <- data.frame(LM = c("91+2", "89+3", "88+2"))
Another option would be:
x <- '92+3'
sum(as.numeric(strsplit(x, split = '+',fixed = TRUE)[[1]]))
In case of having a data.frame:
df <- data.frame(LM = c("91+2", "89+3", "88+2"))
df$sum <- sapply(seq_len(nrow(df)),
function(i) sum(as.numeric(strsplit(df$LM, split = '+', fixed = TRUE)[[i]])))
# LM sum
# 1 91+2 93
# 2 89+3 92
# 3 88+2 90
Related
My data is imported into R as a list of 60 tibbles each with 13 columns and 8 rows. I want to detect outliers defined as 2*sd by comparing each value in column "2" to the mean of all values of column "2" in the same row.
I know that I am on a wrong path with these lines, as I am not comparing the single values
lapply(list, function(x){
if(x$"2">(mean(x$"2")) + (2*sd(x$"2"))||x$"2"<(mean(x$"2")) - (2*sd(x$"2"))) {}
})
Also I was hoping to replace all values that are thus identified as outliers by the corresponding mean calculated from the 60 values in the same position as the outlier while keeping everything else, but I am also quite unsure how to do that.
Thank you!
you haven't added an example of your code so I've made a quick and simple example to demonstrate my answer. I think this would be much more straightforward logic if you first combine the list of tibbles into a single tibble. This allows you to do everything you want in a simple dplyr pipe, ultimately identifying outliers by 1's in the 'outlier' column:
library(tidyverse)
tibble1 <- tibble(colA = c(seq(1,20,1), 150),
colB = seq(0.1,2.1,0.1),
id = 1:21)
tibble2 <- tibble(colA = c(seq(101,120,1), -150),
colB = seq(21,41,1),
id = 1:21)
# N.B. if you don't have an 'id' column or equivalent
# then it makes it a lot easier if you add one
# The 'id' column is essentially shorthand for an index
tibbleList <- list(tibble1, tibble2)
joinedTibbles <- bind_rows(tibbleList, .id = 'tbl')
res <- joinedTibbles %>%
group_by(id) %>%
mutate(meanA = mean(colA),
sdA = sd(colA),
lowThresh = meanA - 2*sdA,
uppThresh = meanA + 2*sdA,
outlier = ifelse(colA > uppThresh | colA < lowThresh, 1, 0))
I'm relatively new to R and I have looked for an answer for my problem but didn't find one. I want to compare two dataframes.
library(dplyr)
library(gtools)
v1 <- LETTERS[1:10]
combinations_from_4_letters <- (as.data.frame(combinations(n = 10, r = 4, v = v1),
stringsAsFactors = FALSE))
combinations_from_4_letters$group <- rep(1:15, each = 14)
combinations_from_2_letters <- (as.data.frame(combinations(n = 10, r = 2, v = v1),
stringsAsFactors = FALSE))
Dataframe 'combinations_from_4_letters' contains all combinations that can be made from 10 letters without repetitions and permutations. The combinations are binned into groups from 1-15. I want to find out how often pairs of the 10 letters (saved in dataframe 'combinations_from_2_letters') are found in each group (basically a frequency table). I started doing a complicated loop looping through both dataframes but I think there must be a more 'R' solution to it, similar to comparing a dataframe and a vector like:
combinations_from_4_letters %in% combinations_from_2_letters[i,])
Thank you in advance for your help!
I recommend an approach like the following:
# adding dummy column for a complete cross-join
combinations_from_4_letters = combinations_from_4_letters %>%
mutate(ones = 1)
combinations_from_2_letters = combinations_from_2_letters %>%
mutate(ones = 1)
joined = combinations_from_2_letters %>%
inner_join(combinations_from_4_letters, by = "ones") %>%
# comparison goes here
mutate(within = ifelse(comb2 %in% comb4, 1, 0)) %>%
group_by(comb2) %>%
summarise(freq = sum(within))
You'll probably need to modify to ensure it matches the exact column names and your comparison condition.
Key ideas:
adding filler column so we have a complete cross-join
mutate a new indicator column for whether the two letter pair is within the four letter pair
sum indicators on the two letter pair
I have variables with names such as r1a r3c r5e r7g r9i r11k r13g r15i etc. I am trying to select variables which starts with r5 - r12 and create a dataframe in R.
The best code that I could write to get this done is,
data %>% select(grep("r[5-9][^0-9]" , names(data), value = TRUE ),
grep("r1[0-2]", names(data), value = TRUE))
Given my experience with regular expressions span a day, I was wondering if anyone could help me write a better and compact code for this!
Here's a regex that gets all the columns at once:
data %>% select(grep("r([5-9]|1[0-2])", names(data), value = TRUE))
The vertical bar represents an 'or'.
As the comments have pointed out, this will fail for items such as r51, and can also be shortened. Instead, you will need a slightly longer regex:
data %>% select(matches("r([5-9]|1[0-2])([^0-9]|$)"))
Suppose that in the code below x represents your names(data). Then the following will do what you want.
# The names of 'data'
x <- scan(what = character(), text = "r1a r3c r5e r7g r9i r11k r13g r15i")
y <- unlist(strsplit(x, "[[:alpha:]]"))
y <- as.numeric(y[sapply(y, `!=`, "")])
x[y > 4]
#[1] "r5e" "r7g" "r9i" "r11k" "r13g" "r15i"
EDIT.
You can make a function with a generalization of the above code. This function has three arguments, the first is the vector of variables names, the second and the third are the limits of the numbers you want to keep.
var_names <- function(x, from = 1, to = Inf){
y <- unlist(strsplit(x, "[[:alpha:]]"))
y <- as.integer(y[sapply(y, `!=`, "")])
x[from <= y & y <= to]
}
var_names(x, 5)
#[1] "r5e" "r7g" "r9i" "r11k" "r13g" "r15i"
Remove the non-digits, scan the remainder in and check whether each is in 5:12 :
DF <- data.frame(r1a=1, r3c=2, r5e=3, r7g=4, r9i=5, r11k=6, r13g=7, r15i=8) # test data
DF[scan(text = gsub("\\D", "", names(DF)), quiet = TRUE) %in% 5:12]
## r5e r7g r9i r11k
## 1 3 4 5 6
Using magrittr it could also be written like this:
library(magrittr)
DF %>% .[scan(text = gsub("\\D", "", names(.)), quiet = TRUE) %in% 5:12]
## r5e r7g r9i r11k
## 1 3 4 5 6
JMP has a "split table" platform:
http://www.jmp.com/support/help/Split_Columns.shtml
Here is the image for it:
The "split by" becomes part of the column headers.
The "split columns" are the columns spread out.
The "group" are retained columns.
I have looked at a few links/pages and can't seem to get this right in R. Right now I have to kluge it into a macro in JMP.
Links that didn't help me include:
Use dplyr's group_by to perform split-apply-combine
https://www.rstudio.com/wp-content/uploads/2015/02/data-wrangling-cheatsheet.pdf
Split a column of a data frame to multiple columns
I need to split a table of ~20k rows and ~30 columns, along one of the columns (integers between 0 and 13), to being ~1400 rows with ~25 split into 350.
An inelegant, but repeatable, example is splitting this cars table
according to this:
Yields this:
How do I do this and retain the ~5 non-split columns using an R library like tidyr or dplyr?
Using reshape, it's not too terrible to do one split column at a time. You could then merge the model and engine.disp together. For your real example, you could just change the lists in aggregate and formula in cast.
x <- read.csv('http://web.pdx.edu/~gerbing/data/cars.csv',stringsAsFactors = F)
names(x) <- tolower(names(x))
agg <- aggregate(list(model = x$model),list(origin = x$origin,cylinders = x$cylinders,year = x$year),FUN = paste,collapse = ',')
require(reshape)
output <- cast(data = agg,formula = origin + cylinders ~ year,value = 'model')
Edit:
I haven't checked all possible cases, but this function should work similar to the split tables, or at least give you a good start.
x <- read.csv('http://web.pdx.edu/~gerbing/data/cars.csv',stringsAsFactors = F)
names(x) <- tolower(names(x))
jmpsplitcol <- function(data,splitby,splitcols,group){
require(reshape)
require(tidyr)
aggsplitlist <- data[ ,names(data) %in% c(splitby,group)]
aggsplitlist <- lapply(aggsplitlist,`[`)
agg <- aggregate(list(data[ ,names(data) %in% splitcols]),aggsplitlist,FUN = paste,collapse = ',')
newgat <- gather_(data = agg,key = 'splitcolname','myval',splitcols)
castformula <- as.formula(paste(paste(group,collapse = ' + '),'~','splitcolname','+',splitby))
output <- cast(data = newgat,formula = castformula,value = 'myval')
output
}
res <- jmpsplitcol(x,c('year'),c('engine.disp','model'),c('origin','cylinders'))
head(res2)
I create a data frame which now I want to separate one new column by split the ":" in first column.
data frame:
unc.edu.0057f9f7-779b-4914-8290-abbad2a0d81e.2556919.rsem.genes.normalized_results:ASL|435 214.4421
unc.edu.0057f9f7-779b-4914-8290-abbad2a0d81e.2556919.rsem.genes.normalized_results:ASS1|445 2863.8055
unc.edu.0057f9f7-779b-4914-8290-abbad2a0d81e.2556919.rsem.genes.normalized_results:OTC|5009 0
unc.edu.050c2191-b96c-41e7-abdb-e52cbe82f268.2456235.rsem.genes.normalized_results:ASL|435 332.7522
unc.edu.050c2191-b96c-41e7-abdb-e52cbe82f268.2456235.rsem.genes.normalized_results:ASS1|445 3322.629
unc.edu.050c2191-b96c-41e7-abdb-e52cbe82f268.2456235.rsem.genes.normalized_results:OTC|5009 0
desired output:
unc.edu.0057f9f7-779b-4914-8290-abbad2a0d81e.2556919.rsem.genes.normalized_results ASL|435 214.4421
unc.edu.0057f9f7-779b-4914-8290-abbad2a0d81e.2556919.rsem.genes.normalized_results ASS1|445 2863.8055
unc.edu.0057f9f7-779b-4914-8290-abbad2a0d81e.2556919.rsem.genes.normalized_results OTC|5009 0
unc.edu.050c2191-b96c-41e7-abdb-e52cbe82f268.2456235.rsem.genes.normalized_results ASL|435 332.7522
unc.edu.050c2191-b96c-41e7-abdb-e52cbe82f268.2456235.rsem.genes.normalized_results ASS1|445 3322.629
unc.edu.050c2191-b96c-41e7-abdb-e52cbe82f268.2456235.rsem.genes.normalized_results OTC|5009 0
I have tried
strsplit(df$V1, split = "\\:")
but Error in strsplit(t$V1, split = "\:") : non-character argument come out. Thank you.
The error is because we have a variable of class factor. Convert it to character and it should work
lst <- strsplit(as.character(df$V1), split = ":", fixed = TRUE)
If we need to create two columns, one easy way is with read.table
df1 <- read.table(text = as.character(df$V1), sep=":", stringsAsFactors=FALSE)
Or using separate from tidyr
library(tidyr)
separate(df1, V1, into = c("V1", "V2"))
tidyr::separate(data = df, col = V1, into = c('a', 'b'), sep = ':')