Consider the following example
> data_text <- data.frame(text = c('where', 'are', 'you'),
blob = c('little', 'nice', 'text'))
> data_text
# A tibble: 3 x 2
text blob
<chr> <chr>
1 where little
2 are nice
3 you text
I want to print the rows that contain the regex text (that is, row 3)
Problem is, I have hundreds of columns and I dont know which one contains this string. str_detect only work with one column at a time...
How can I do that using the stringr package?
Thanks!
With stringr and dplyr you can do this.
You should use filter_all from dplyr >= 0.5.0.
I have extended the data to have a better look on the result:
library(dplyr)
library(stringr)
data_text <- data.frame(text = c('text', 'where', 'are', 'you'),
one_more_text = c('test', 'test', 'test', 'test'),
blob = c('wow', 'little', 'nice', 'text'))
data_text %>%
filter_all(any_vars(str_detect(., 'text')))
# output
text one_more_text blob
1 text test wow
2 you test text
You can treat the data.frame as a list and use purrr::map to check each column, which can then be reduced into a logical vector that filter can handle. Alternatively, purrr::pmap can iterate over all the columns in parallel:
library(tidyverse)
data_text <- data_frame(text = c('where', 'are', 'you'),
blob = c('little', 'nice', 'text'))
data_text %>% filter(map(., ~.x == 'text') %>% reduce(`|`))
#> # A tibble: 1 x 2
#> text blob
#> <chr> <chr>
#> 1 you text
data_text %>% filter(pmap_lgl(., ~any(c(...) == 'text')))
#> # A tibble: 1 x 2
#> text blob
#> <chr> <chr>
#> 1 you text
matches = apply(data_text,1,function(x) sum(grepl("text",x)))>0
result = data_text[matches,]
No other packages required. Hope this helps!
Related
I would like to make a string basing on ids from other columns where the real value sits in a dictionary.
Ideally, this would look like:
library(tidyverse)
region_dict <- tibble(
id = c("reg_id1", "reg_id2", "reg_id3"),
name = c("reg_1", "reg_2", "reg_3")
)
color_dict <- tibble(
id = c("col_id1", "col_id2", "col_id3"),
name = c("col_1", "col_2", "col_3")
)
tibble(
region = c("reg_id1", "reg_id2", "reg_id3"),
color = c("col_id1", "col_id2", "col_id3"),
my_string = str_c(
"xxx"_,
region_name,
"_",
color_name
))
#> # A tibble: 3 x 3
#> region color my_string
#> <chr> <chr> <chr>
#> 1 reg_id1 col_id1 xxx_reg_1_col_1
#> 2 reg_id2 col_id2 xxx_reg_2_col_2
#> 3 reg_id3 col_id3 xxx_reg_3_col_3
Created on 2021-03-01 by the reprex package (v0.3.0)
I know of dplyr's recode() function but I can't think of a way to use it the way I want.
I also thought about first using left_join() and then concatenating the string from the new columns. This is what would work but doesn't seem pretty to me as I would get columns that I'd need to remove later. In the real dataset I have 5 variables.
I'll be glad to read your ideas.
This may also be solved with a fuzzyjoin, but based on the similarity in substring, it would make sense to remove the prefix substring from the 'id' columns of each data and do a left_join, then create the 'my_string' by pasteing the columns together
library(stringr)
library(dplyr)
region_dict %>%
mutate(id1 = str_remove(id, '.*_')) %>%
left_join(color_dict %>%
mutate(id1 = str_remove(id, '.*_')), by = 'id1') %>%
transmute(region = id.x, color = id.y,
my_string = str_c('xxx_', name.x, '_', name.y))
-output
# A tibble: 3 x 3
# region color my_string
# <chr> <chr> <chr>
#1 reg_id1 col_id1 xxx_reg_1_col_1
#2 reg_id2 col_id2 xxx_reg_2_col_2
#3 reg_id3 col_id3 xxx_reg_3_col_3
I'm trying to read an array out of a JSON structure with tidyjson as I'm trying to fasten up my code.
My input data is of the structure
json <- "{\"key1\":\"test\",\"key2\":[\"abc\",\"def\"]}"
I want my output to be a data frame where key1 is one column and key2 is the second column in which all elements of the array are pasted together and separated by ";".
I tried something like
result <- json %>% spread_values(a = jstring("key1"), b = paste0(jstring("key2"), collapse = ";"))
I really have no idea how to get the array out of the JSON in the spread_values function.
I got what I want with
key2 <- json %>% enter_object("key2")
attributes(key2)$JSON %>% unlist() %>% paste0(collapse = ";")
but as I don't have unique keys I can't join it to the rest of my data and I think there must be a better way.
I'm glad you got something working! In case anyone else happens upon this question, there are definitely many ways to accomplish this task!
One is to use tidyjson to gather the data into a tall structure, then summarize:
library(tidyjson)
library(dplyr)
json <- "{\"key1\":\"test\",\"key2\":[\"abc\",\"def\"]}"
myj <- tidyjson::as.tbl_json(json)
myj %>%
# make the data tall
spread_values(key1 = jstring(key1)) %>%
enter_object("key2") %>%
gather_array("idx") %>%
append_values_string("key2") %>%
# now summarize
group_by(key1) %>%
summarize(key2 = paste(key2, collapse = ";"))
#> # A tibble: 1 x 2
#> key1 key2
#> <chr> <chr>
#> 1 test abc;def
Created on 2021-10-29 by the reprex package (v0.3.0)
Another way is to grab the json data directly with json_get_column() and mutate that:
library(tidyjson)
library(dplyr)
json <- "{\"key1\":\"test\",\"key2\":[\"abc\",\"def\"]}"
myj <- tidyjson::as.tbl_json(json)
myj %>%
spread_values(key1 = jstring(key1)) %>%
enter_object("key2") %>%
json_get_column("array") %>%
mutate(key2 = purrr::map_chr(array, ~ paste(.x, collapse = ";"))) %>%
as_tibble() %>% # drop tbl_json structure
select(key1, key2)
#> # A tibble: 1 x 2
#> key1 key2
#> <chr> <chr>
#> 1 test abc;def
Created on 2021-10-29 by the reprex package (v0.3.0)
I have two data sets. One has a lengthy combinations of text keys attached to identifying data:
set.seed(123)
library(tidyverse)
id <- paste0("V", sample((1000:9999), size = 5))
text <- c("ARROW", "ARROWHEAD", "OTHERARROW", "OTHER", "HEADOTHER")
df <- tibble(id, text)
df
id text
<chr> <chr>
1 V3588 ARROW
2 V8093 ARROWHEAD
3 V4679 OTHERARROW
4 V8944 OTHER
5 V9460 HEADOTHER
The other looks up those keys and assigns them shorter values:
original <- c("ARROW", "HEAD", "OTHER")
revised <- c("A", "H", "O")
lookup <- tibble(original, revised)
lookup
original revised
<chr> <chr>
1 ARROW A
2 HEAD H
3 OTHER O
My desired output is to make df1, a data frame that replaces text with a combination of revised codes:
correctText <- c("A", "AH", "AO", "O", "HO")
df1 <- tibble(id, correctText)
df1
id correctText
<chr> <chr>
1 V3588 A
2 V8093 AH
3 V4679 AO
4 V8944 O
5 V9460 HO
What is the most efficient way to do this (base R or dplyr only)? Right now I'm doing it with a for loop, but it's too slow for the Shiny app where I want to implement it.
We can use str_replace with a named vector
library(dplyr)
library(stringr)
df %>%
mutate(text = str_replace_all(text, set_names(revised, original)))
# A tibble: 5 x 2
# id text
# <chr> <chr>
#1 V3462 A
#2 V3510 AH
#3 V9717 OA
#4 V3985 O
#5 V2841 HO
stringr functions are based on stringi and should be very efficient
With base R only (and dplyr, since you use a tibble, so I threw mutate in as well), you could use this function:
multisub <- function(target, output, string) {
replacement.list <- apply(cbind(target, output), 1, as.list)
mygsub <- function(l, x) gsub(pattern = l[1], replacement = l[2], x, perl=TRUE)
Reduce(mygsub, replacement.list, init = string, right = TRUE)
}
df %>% mutate(text = multisub(original, revised, text))
#> # A tibble: 5 x 2
#> id text
#> <chr> <chr>
#> 1 V3462 A
#> 2 V3510 AH
#> 3 V9717 OA
#> 4 V3985 O
#> 5 V2841 HO
# base R oly:
data.frame(id=id, text = multisub(original, revised, text), stringsAsFactors = FALSE)
Performance-wise it is pretty competitive with the stringr solution.
I have a large spreadsheet with 3200 observations that has a list of genes in a column. The column however has a bunch of junk that I don't need (example below). How can I use stringr to remove the unnecessary junk and leave only the gene name?
Example: The gene names are TEM-126 and ykkD.
gb|AY628199|+|203-1064|ARO:3000988|TEM-126
gb|AL009126|+|1376854-1377172|ARO:3003064|ykkD
If your gene names are always at the tail of your strings, you can try the code below
> gsub(".*\\|","",v)
[1] "TEM-126" "ykkD"
DATA
v <- c("gb|AY628199|+|203-1064|ARO:3000988|TEM-126",
"gb|AL009126|+|1376854-1377172|ARO:3003064|ykkD")
Using stringr:
str_split_fixed(genes, '\\|', n = 6)[, 6]
As you said you have those names in a column and it seems that the gene name is the last "word", you could easily do that using just two packages from tidyverse, dplyr and stringr.
library(dplyr)
library(stringr)
df <- tibble::tribble(
~Text,
"gb|AY628199|+|203-1064|ARO:3000988|TEM-126",
"gb|AL009126|+|1376854-1377172|ARO:3003064|ykkD"
)
df %>%
mutate(gene = word(Text, start = -1, end = -1, sep = "\\|"))
#> # A tibble: 2 x 2
#> Text gene
#> <chr> <chr>
#> 1 gb|AY628199|+|203-1064|ARO:3000988|TEM-126 TEM-126
#> 2 gb|AL009126|+|1376854-1377172|ARO:3003064|ykkD ykkD
If you have a vector genevec of gene names, you can vectorise the function:
stringr::str_split(pattern="\\|", string=genevec, simplify=T)[,6]
library(NLP)
library(tm)
library(tidytext)
library(tidyverse)
library(topicmodels)
library(dplyr)
library(stringr)
library(purrr)
library(tidyr)
#sample dataset
tags <- c("product, productdesign, electronicdevice")
web <- c("hardware, sunglasses, eyeware")
tags2 <- data_frame(tags, web, stringsAsFactors = FALSE)
#tokenize the words
toke <- tags2 %>%
unnest_tokens(word, tags)
toke
#create a dummy variable
toke2 <- toke%>% mutate(
product = ifelse(str_detect(word, "^product$"), "1", "0"))
#unnest the toke
nested_toke <- toke2 %>%
nest(word) %>%
mutate(text = map(data, unlist),
text = map_chr(text, paste, collapse = " "))
nested_toke %>%
select(text)
When I nest the column of tokenized words after creating the dummy variable based on the string "product" it seems to be inserting "product" into a new row below the original row where "product" was located.
product underlined should be in the row above
When you add a new column after unnesting, you have to think about what to do with it if you want to nest again. Let's work through it and see what we're talking about.
library(tidyverse)
tags <- c("product, productdesign, electronicdevice")
web <- c("hardware, sunglasses, eyeware")
tags2 <- data_frame(tags, web)
library(tidytext)
tidy_tags <- tags2 %>%
unnest_tokens(word, tags)
tidy_tags
#> # A tibble: 3 x 2
#> web word
#> <chr> <chr>
#> 1 hardware, sunglasses, eyeware product
#> 2 hardware, sunglasses, eyeware productdesign
#> 3 hardware, sunglasses, eyeware electronicdevice
So that is your data set unnested, converted to a tidy form. Next, let's add the new column that detects whether the word "product" is in the word column.
tidy_product <- tidy_tags %>%
mutate(product = ifelse(str_detect(word, "^product$"),
TRUE,
FALSE))
tidy_product
#> # A tibble: 3 x 3
#> web word product
#> <chr> <chr> <lgl>
#> 1 hardware, sunglasses, eyeware product T
#> 2 hardware, sunglasses, eyeware productdesign F
#> 3 hardware, sunglasses, eyeware electronicdevice F
Now think about what your options are for nesting again. If you nest again without taking into account the new column (nest(word)) the structure has a NEW COLUMN and will have to make a NEW ROW to account for the two different values that can take. You could instead do something like nest(word, product) but then the TRUE/FALSE values will end up in your text string. If you are wanting to get back to the original text format, you need to remove the new column you created, because having it there changes the relationships between rows and columns.
nested_product <- tidy_product %>%
select(-product) %>%
nest(word) %>%
mutate(text = map(data, unlist),
text = map_chr(text, paste, collapse = ", "))
nested_product
#> # A tibble: 1 x 3
#> web data text
#> <chr> <list> <chr>
#> 1 hardware, sunglasses, eyeware <tibble [3 × 1]> product, productdesign, …
Created on 2018-02-22 by the reprex package (v0.2.0).