I have Lookup_DF which contains dictionary to refer strings and Raw_file which has combination of strings, Lookup_DF is having Types to populate in Result data frame based on Items in raw files.
Item1=c("Banana","Toamto","Potato","Palak")
Item2=c("","Orange","Onion","Mango")
Type1=c("Fruit","Vegetable","Vegetable","Leaves")
Type2=c("","Fruit","Vegetable","Fruit")
DF1=data.frame(Item1,Item2,Type1,Type2)
Items=c("Onion,Potato,Ginger","Tomato","Banana","Palak,Mango","Onion,Capsicum","Orange,Sweet_potato")
Raw_file=data.frame(Items)
Result_Type1=c("Vegetable","Vegetable","Fruit","Leaves","","")
Result_Type2=c("Vegetable","","","Fruit","Vegetable","Fruit")
Result=data.frame(Items,Result_Type1,Result_Type2)
My Output data frame would look like Result.
I tried something with str_detect in case statement but not able to get it. Can someone help me out on this.
Maybe you can do a join between these two tables (similar to your other question).
First would put DF1 in long format. For Raw_file, use separate_rows to have a single item in each row before the join.
library(tidyverse)
DF1_long <- DF1 %>%
pivot_longer(cols = everything(),
names_to = c(".value", "number"),
names_pattern = "(\\w+)(\\d+)$")
Raw_file %>%
mutate(value = Items) %>%
separate_rows(value) %>%
inner_join(DF1_long, by = c("value" = "Item")) %>%
group_by(Items) %>%
distinct(Items, number, .keep_all = TRUE) %>%
pivot_wider(id_cols = Items,
names_from = number,
values_from = Type,
names_prefix = "Result_Type")
Output
Items Result_Type2 Result_Type1
<chr> <chr> <chr>
1 Onion,Potato,Ginger Vegetable Vegetable
2 Tomato NA Vegetable
3 Banana NA Fruit
4 Palak,Mango Fruit Leaves
5 Onion,Capsicum Vegetable NA
6 Orange,Sweet_potato Fruit NA
Related
Does anyone know if there are any ways I can turn dataset from the left side to the right side in excel macro or other programming tools?
Use R code to manipulate dataset on the left side to the one at the right side
This doesn't exactly match what you are looking for, but further data analysis might be easier formatted this way
library(tidyverse)
df = read_csv("Downloads/test.csv")
df = df %>% group_by(Transaction, Item) %>% summarize(count_value = n())
result = df %>% pivot_wider(names_from = "Item", values_from = "count_value", values_fill = 0)
I pivot data this way:
library(tidyverse)
df <- data.frame(Transaction=c(1,2,2),
Item=c("Apple", "Banana", "Coconut"))
df
df <- df %>%
group_by(Transaction) %>%
mutate(ItemNumber=paste0("Item", row_number()))
df
df <- df %>%
pivot_wider(names_from = ItemNumber,
values_from = Item)
df
# Transaction Item1 Item2
#1 1 Apple NA
#2 2 Banana Coconut
Context
I have created a small sample dataframe to explain my problem. The original one is larger, as it has many more columns. But it is formatted in the same way.
df = data.frame(Case1.1.jpeg.text="the",
Case1.1.jpeg.text.1="big",
Case1.1.jpeg.text.2="DOG",
Case1.1.jpeg.text.3="10197",
Case1.2.png.text="framework",
Case1.3.jpg.text="BE",
Case1.3.jpg.text.1="THE",
Case1.3.jpg.text.2="Change",
Case1.3.jpg.text.3="YOUWANTTO",
Case1.3.jpg.text.4="SEE",
Case1.3.jpg.text.5="in",
Case1.3.jpg.text.6="theWORLD",
Case1.4.png.text="09.80.56.60.77")
The dataframe consists of output from a text detection ML model based on a certain number of input images.
The output format makes each word for each image a separate column, thereby creating a very wide dataset.
Desired Output
I am looking to create a cleaner version of it, with one column containing the image name (e.g. Case1.2.png) and the second with the concatenation of all possible words that the model finds in that particular image (the number of words varies from image to image).
result = data.frame(Case=c('Case1.1.jpeg','Case1.2.png','Case1.3.jpg','Case1.4.png'),
Text=c('thebigDOG10197','framework','BETHEChangeYOUWANTTOSEEintheWORLD','09.80.56.60.77'))
I have tried many approaches based on similar questions found on Stackoverflow, but none seem to give me the exact output I'm looking for.
Any help on this would be greatly appreciated.
library(tidyr)
library(dplyr)
df %>%
pivot_longer(cols = everything(),
names_pattern = "(.*)\\.(text.*)",
names_to = c("Case", NA)) %>%
group_by(Case) %>%
summarize(value = paste(value, collapse = ""), .groups = "drop")
Alternatively, this can be accomplished using just the pivot functions from tidyr:
library(tidyr)
library(stringr)
df %>%
pivot_longer(cols = everything(),
names_pattern = "(.*)\\.(text).*",
names_to = c("Case", "cols")) %>%
pivot_wider(id_cols = Case,
values_from = value,
names_from = cols,
values_fn = str_flatten)
Output
Case value
<chr> <chr>
1 Case1.1.jpeg thebigDOG10197
2 Case1.2.png framework
3 Case1.3.jpg BETHEChangeYOUWANTTOSEEintheWORLD
4 Case1.4.png 09.80.56.60.77
A possible solution:
library(tidyverse)
df %>%
pivot_longer(everything()) %>%
mutate(name = str_remove(name, "\\.text\\.*\\d*")) %>%
group_by(name) %>%
summarise(text = str_c(value, collapse = ""))
#> # A tibble: 4 x 2
#> name text
#> <chr> <chr>
#> 1 Case1.1.jpeg thebigDOG10197
#> 2 Case1.2.png framework
#> 3 Case1.3.jpg BETHEChangeYOUWANTTOSEEintheWORLD
#> 4 Case1.4.png 09.80.56.60.77
An option in base R is stack the data into a two column data.frame with stack and then do a group by paste with aggregate
aggregate(cbind(Text = values) ~ Case, transform(stack(df),
Case = trimws(ind, whitespace = "\\.text.*")), FUN = paste, collapse = "")
Case Text
1 Case1.1.jpeg thebigDOG10197
2 Case1.2.png framework
3 Case1.3.jpg BETHEChangeYOUWANTTOSEEintheWORLD
4 Case1.4.png 09.80.56.60.77
You can use pivot_longer(everything()), manipulate the "Case" column, group, and paste together:
pivot_longer(df,everything(),names_to="Case") %>%
mutate(Case = str_remove_all(Case, ".text.*")) %>%
group_by(Case) %>% summarize(Text=paste(value, collapse=""))
Output:
Case Text
<chr> <chr>
1 Case1.1.jpeg thebigDOG10197
2 Case1.2.png framework
3 Case1.3.jpg BETHEChangeYOUWANTTOSEEintheWORLD
4 Case1.4.png 09.80.56.60.77
I got help with this questions a while ago:
How to replace multiple values in a string depending on a list of keys
Now I need to take into account that some keys are not to be "translated". So in this case I want the key1-4 should be translated into code1-4. I want it to be able to handle keys that arent in the key_code translation. If I add a key that is missing from the keycode, say keyx, somewhere where there is already another valid key value, I can just filter the NA that appears when joining with the key_codes. But if I have an id which has only the keyx value, that whole row dissapears and I want to keep it (it can show up as NA for example). Any ideas on how to solve that?
library(dplyr)
library(tidyr)
library(stringr)
values = tibble(id = 1:4, values = c("key1;keyx", "key3;key4;key1", "key2;key1", "keyx"))
key_code = tibble(key = c("key1", "key2", "key3", "key4"), code = c("code1", "code2", "code3", "code4"))
values %>%
separate_rows(values) %>%
left_join(key_code, by = c("values" = "key")) %>%
group_by(id) %>%
filter(!is.na(code)) %>%
summarise(code = str_c(code, collapse=";"))
We could use an if/else condition to check if all the elements in the 'code' are NA, then return NA or else to paste the non-NA elements
library(dplyr)
library(tidyr)
library(stringr)
values %>%
separate_rows(values) %>%
left_join(key_code, by = c("values" = "key")) %>%
group_by(id) %>%
summarise(code = if(all(is.na(code))) NA_character_ else
str_c(str_replace_na(code, ""), collapse=";"), .groups = 'drop')
-output
# A tibble: 4 x 2
# id code
# <int> <chr>
#1 1 code1;
#2 2 code3;code4;code1
#3 3 code2;code1
#4 4 <NA>
I have a dataframe with the following factor variable:
> head(example.df)
path
1 C:/Users/My PC/pinkhipppos/tinyhorsefeet/location1/categoryA/eyoshdzjow_random_image.txt
(made up dirs).
I want to split into separate columns based on a delimiter: /.
I can do this using
library(tidyverse)
example.df <- example.df %>%
separate(path,
into=c("dir",
"ok",
"hello",
"etc...",
"finally...",
"location",
"category",
"filename"),
sep="/")
Although, I am only interested in the last two dirs and the file name or the last 3 results from the separate function. As parent directories (higher than location) may change. My desired output would be:
> head(example.df)
location category filename
1 location1 categoryA eyoshdzjow_random_image.txt
Reproducible:
example.df <- as.data.frame(
c("C:/Users/My PC/pinkhipppos/tinyhorsefeet/location1/categoryA/eyoshdzjow_random_image.txt",
"C:/Users/My PC/pinkhipppos/tinyhorsefeet/location2/categoryB/jdugnbtudg_random_image.txt")
)
colnames(example.df)<-"path"
One way in base R is to split string at "/" and select last 3 elements from each list.
as.data.frame(t(sapply(strsplit(as.character(example.df$path), "/"), tail, 3)))
# V1 V2 V3
#1 location1 categoryA eyoshdzjow_random_image.txt
#2 location2 categoryB jdugnbtudg_random_image.txt
Using tidyverse, we can get the data in long format, select last 3 entries in each row and get the data in wide format.
library(tidyverse)
example.df %>%
mutate(row = row_number()) %>%
separate_rows(path, sep = "/") %>%
group_by(row) %>%
slice((n() - 2) : n()) %>%
mutate(cols = c('location', 'category', 'filename')) %>%
pivot_wider(names_from = cols, values_from = path) %>%
ungroup() %>%
select(-row)
# A tibble: 2 x 3
# location category filename
# <chr> <chr> <chr>
#1 location1 categoryA eyoshdzjow_random_image.txt
#2 location2 categoryB jdugnbtudg_random_image.txt
Or similar concept as base R but using tidyverse
example.df %>%
mutate(temp = map(str_split(path, "/"), tail, 3)) %>%
unnest_wider(temp, names_repair = ~paste0("dir", seq_along(.) - 1)) %>%
select(-dir0)
Simple question,
I've provided two different data frames below with code/output, why does one work and the other doesn't? Having trouble understanding the Key/Value inputs (when they need to be explicitly defined/and what it means to just have them as strings in the input).
library(tidyverse)
dat <- data.frame(one = c("x", "x", "x"), two = c("x", "", "x"),
three = c("", "", ""), type = c("chocolate", "vanilla", "strawberry"))
dat %>%
na_if("") %>%
gather("Key", "Val", -type,na.rm=TRUE) %>%
rowid_to_column %>%
spread(Key, Val,fill = "") %>%
select(-1) # works well
dat %>%
na_if("") %>%
gather("Key", "Val", -type,na.rm=TRUE)
Error: Strings must match column names. Unknown columns: Val
Extra Credit: if someone could explain the effect of rowit_to_column & spread(), that'd be helpful.
Perhaps I'm missing something, but I can't reproduce your error.
dat %>%
na_if("") %>% # Replace "" with NA
gather("Key", "Val", -type, na.rm = TRUE) %>% # wide -> long
rowid_to_column() %>% # Sequentially number rows
spread(Key, Val, fill = "") %>% # long -> wide
select(-1) # works well # remove row number
# type one two
#1 chocolate x
#2 vanilla x
#3 strawberry x
#4 chocolate x
#5 strawberry x
dat %>%
na_if("") %>% # Replace "" with NA
gather("Key", "Val", -type, na.rm = TRUE) # wide -> long
# type Key Val
#1 chocolate one x
#2 vanilla one x
#3 strawberry one x
#4 chocolate two x
#6 strawberry two x
Explanation:
na_if("") replaces "" entries with NA.
gather("Key", "Val", -type, na.rm = TRUE) turns a wide table into a long "key-value" table, by storing entries in all columns except type in two columns Key (i.e. the column name) and Val (i.e. the entry). na.rm = TRUE removes rows with NA values.
rowid_to_column sequentially numbers the rows.
spread(Key, Val, fill = "") turns a long "key-value" table into a wide table, with as many columns as there are unique keys in Key. Entries are taken from column Val, if an entry is missing it's filled with "".
select(-1) removes the first column.