I have a similar problem than the following, but the solution presented in the following link does not work for me:
tidyr spread does not aggregate data
I have a df in the following structure:
UndesiredIndex DesiredIndex DesiredRows Result
1 x1A x1 A 50,32
2 x1B x2 B 7,34
3 x2A x1 A 50,33
4 x2B x2 B 7,35
Using the code below:
dftest <- bd_teste %>%
select(-UndesiredIndex) %>%
spread(DesiredIndex, Result)
I expected the following result:
DesiredIndex A B
A 50,32 50,33
B 7,34 7,35
Although, I keep getting the following result:
DesiredIndex x1 x2
1 A 50.32 NA
2 B 7.34 NA
3 A NA 50.33
4 B NA 7.35
PS: Sometimes I force the column UndesiredIndex out with select(-UndesiredIndex), but I keep getting the following message:
Adding missing grouping variables: UndesiredIndex
Might be something easy to stack those rows, but I'm new to R and have been trying so hard to solve this but without success.
Thanks in advance!
We group by DesiredIndex, create a sequence column and then do the spread:
library(tidyverse)
df1 %>%
select(-UndesiredIndex) %>%
group_by(DesiredIndex) %>%
mutate(new = LETTERS[row_number()]) %>%
ungroup %>%
select(-DesiredIndex) %>%
spread(new, Result)
# A tibble: 2 x 3
# DesiredRows A B
# <chr> <chr> <chr>
#1 A 50,32 50,33
#2 B 7,34 7,35
Data
df1 <- structure(
list(
UndesiredIndex = c("x1A", "x1B", "x2A", "x2B"),
DesiredIndex = c("x1", "x2", "x1", "x2"),
DesiredRows = c("A", "B", "A", "B"),
Result = c("50,32", "7,34", "50,33", "7,35")
),
class = "data.frame",
row.names = c("1", "2", "3", "4")
)
Shorter, but more theoretically round-about.
Data
(Thanks to #akrun!)
df1 <- structure(
list(
UndesiredIndex = c("x1A", "x1B", "x2A", "x2B"),
DesiredIndex = c("x1", "x2", "x1", "x2"),
DesiredRows = c("A", "B", "A", "B"),
Result = c("50,32", "7,34", "50,33", "7,35")
),
class = "data.frame",
row.names = c("1", "2", "3", "4")
)
This is a great technique for concatenating rows.
df1 %>%
group_by(DesiredRows) %>%
summarise(Result = paste(Result, collapse = "|")) %>% #<Concatenate rows
separate(Result, into = c("A", "B"), sep = "\\|") #<Separate by '|'
#> # A tibble: 2 x 3
#> DesiredRows A B
#> <chr> <chr> <chr>
#> 1 A 50,32 50,33
#> 2 B 7,34 7,35
Created on 2018-08-06 by the reprex package (v0.2.0).
Related
After years of using your advices to another users, here is my for now unsolvable issue...
I have a dataset with thousands of rows and hundreds of column, that have one column with a possible value in common. Here is a subset of my dataset :
ID <- c("A", "B", "C", "D", "E")
Dose <- c("1", "5", "3", "4", "5")
Value <- c("x1", "x2", "x3", "x2", "x3")
mat <- cbind(ID, Dose, Value)
What I want is to assign a unique value to the rows that have the "Value" column in common, like that :
ID <- c("A", "B", "C", "D", "E")
Dose <- c("1", "5", "3", "4", "5")
Value <- c("153254", "258634", "896411", "258634", "896411")
Code <- c("1", "2", "3", "2", "3")
mat <- cbind(ID, Dose, Value, Code)
Does anyone have an idea that could help me a little ?
Thanks !
We may use match here
library(dplyr)
mat %>%
mutate(Code = match(Value, unique(Value)))
-output
ID Dose Value Code
1 A 1 153254 1
2 B 5 258634 2
3 C 3 896411 3
4 D 4 258634 2
5 E 5 896411 3
data
mat <- data.frame(ID, Dose, Value)
You should consider using a data.frame:
mat <- data.frame(ID, Dose, Value)
Using dplyr you could create the desired output:
library(dplyr)
mat %>%
group_by(Value) %>%
mutate(Code = cur_group_id()) %>%
ungroup()
This returns
# A tibble: 5 x 4
ID Dose Value Code
<chr> <chr> <chr> <int>
1 A 1 153254 1
2 B 5 258634 2
3 C 3 896411 3
4 D 4 258634 2
5 E 5 896411 3
I want to fill df2 with information from df1.
df1 as below
ID Mutation
1 A
2 B
2 C
3 A
df2 as below
ID A B C
1
2
3
For example, if mutation A is found in ID 1, then I want it in df2 it marked as "Y".
So the df2 result should be
ID A B C
1 Y
2 Y Y
3 Y
I have hundreds of IDs and more than 20 mutations. How can I efficiently achieve this in R? Thanks!
Using data.table you can try
setDT(df)
df2 <- dcast(df,formula = ID~Mutation )
df2[, c("A", "B", "C") := lapply(.SD, function(x) ifelse(is.na(x), " ", "Y")), ID]
df2
#Output
ID A B C
1: 1 Y
2: 2 Y Y
3: 3 Y
Create a new column with value 'Y' and cast the data in wide format.
library(dplyr)
library(tidyr)
df %>%
mutate(value = 'Y') %>%
pivot_wider(names_from = Mutation, values_from = value, values_fill = '')
# ID A B C
# <int> <chr> <chr> <chr>
#1 1 "Y" "" ""
#2 2 "" "Y" "Y"
#3 3 "Y" "" ""
data
df <- structure(list(ID = c(1L, 2L, 2L, 3L), Mutation = c("A", "B",
"C", "A")), class = "data.frame", row.names = c(NA, -4L))
I have an output data frame as below. But I would like to rearrange to achieve the result in df2. Is there a way for me to arrange or group it?
df>
a_test1 b_test1 c_test1 a_test2 b_test2 c_test2
Test Test1 Test1 Test1 Test2 Test2 Test2
Result 10 9 4 4 3 1
df2>
a b c
Test1 10 9 4
Test2 4 3 1
dat <- data.frame(a_test1 = 10,
b_test1 = 9,
c_test1 = 4,
a_test2 = 4,
b_test2 = 3,
c_test2 = 1)
You can achieve this with this code:
library(tidyverse)
dat %>%
pivot_longer(cols = everything(),
names_sep = "_",
names_to = c("prefix", "suffix")) %>%
pivot_wider(names_from = prefix)
which gives:
# A tibble: 2 x 4
suffix a b c
<chr> <dbl> <dbl> <dbl>
1 test1 10 9 4
2 test2 4 3 1
UPDATE:
TO asked if it would still work with different column names that contain several underscores as separator:
dat2 <- data.frame(a_test1_10 = 10,
b_test1_10 = 9,
c_test1_10 = 4,
a_test2_10 = 4,
b_test2_10 = 3,
c_test2_10 = 1)
pivot_spec <- data.frame(.name = colnames(dat2),
.value = c("a", "b", "c", "a", "b", "c"),
test_group = c("test1", "test1", "test1", "test2", "test2", "test2"))
This pivot_spec looks like:
.name .value test_group
1 a_test1_10 a test1
2 b_test1_10 b test1
3 c_test1_10 c test1
4 a_test2_10 a test2
5 b_test2_10 b test2
6 c_test2_10 c test2
and then ou can just continue pivoting. Actually, the whole pivoting now looks much cleaner and you don't need to combine a pivot_longer with a pivot_wider.
dat2 %>%
pivot_longer_spec(pivot_spec)
which gives:
# A tibble: 2 x 4
test_group a b c
<chr> <dbl> <dbl> <dbl>
1 test1 10 9 4
2 test2 4 3 1
As you can see, createing this pivot_spec template makes the whole thing extremely flexible. The .name column contains all your required data columns, the .value column contains the new column names and maps the old column names to the new ones. And the test_group (you can choose whatever name you like) column determines the rows that would be created and which original column should appear in which column.
You can reshape the dat, filter rows and turn column into rownames.
tidyr::pivot_longer(df,
cols = everything(),
names_to = c('.value', 'col'),
names_sep = '_') %>%
dplyr::filter(!grepl('Test', a)) %>%
type.convert(as.is = TRUE) %>%
tibble::column_to_rownames('col')
# a b c
#test1 10 9 4
#test2 4 3 1
data
df <- structure(list(a_test1 = c("Test1", "10"), b_test1 = c("Test1",
"9"), c_test1 = c("Test1", "4"), a_test2 = c("Test2", "4"), b_test2 = c("Test2",
"3"), c_test2 = c("Test2", "1")), class = "data.frame", row.names = c("Test",
"Result"))
I have a data frame like below:
how do I remove na and use below value to go up?
Thanks
id name.america name.europe name.asia
1 a <NA> <NA>
2 <NA> b <NA>
3 <NA> <NA> c
4 d <NA> <NA>
Change to:
id name.america name.europe name.asia
1 a b c
2 d
We can loop through the columns and remove the NA, then make the lengths of the list elements same by appending NA at the end after getting the max length of the list element. Based on that, subset the 'id' column of the dataset and append with the output
lst <- lapply(df1[-1], na.omit)
lst1 <- lapply(lst, `length<-`, max(lengths(lst)))
out <- data.frame(lst1)
out1 <- cbind(id = df1$id[seq_len(nrow(out))], out)
out1
# id name.america name.europe name.asia
#1 1 a b c
#2 2 d <NA> <NA>
If we need NA to be changed to blanks ("") - not recommended
out1[is.na(out1)] <- ""
data
df1 <- structure(list(id = 1:4, name.america = c("a", NA, NA, "d"),
name.europe = c(NA, "b", NA, NA), name.asia = c(NA, NA, "c",
NA)), class = "data.frame", row.names = c(NA, -4L))
tidyverse-based solution
require(tidyverse)
df1 %>%
gather(key = "name", value = "val", -id) %>%
na.omit() %>%
select(-id) %>%
group_by(name) %>%
mutate(id = 1:n()) %>%
spread(key = name, value = val)
Results
# A tibble: 2 x 4
id name.america name.asia name.europe
<int> <chr> <chr> <chr>
1 1 a c b
2 2 d NA NA
Notes
If desired you can re-order columns with select or that variable prior to transformation.
NAs are left as such. If desired, you can use tidyr::replace_na to insert some string or space. I would discourage you from doing that.
Data
Taken from #akrun's answer above.
df1 <- structure(
list(
id = 1:4,
name.america = c("a", NA, NA, "d"),
name.europe = c(NA, "b", NA, NA),
name.asia = c(NA, NA, "c",
NA)
),
class = "data.frame",
row.names = c(NA, -4L)
)
df1[, -1] <- lapply(df1[,-1], function(x) c(na.omit(x), rep("",length(x)-length(na.omit(x)))))
df1[1:max(colSums(!(df1[,-1]==""))),]
# id name.america name.europe name.asia
#1 1 a b c
#2 2 d
I have some poorly formatted data that I must work with. It contains two identifiers in the first two rows, followed by the data. The data looks like:
V1 V2 V3
1 Date 12/16/18 12/17/18
2 Equip a b
3 x1 1 2
4 x2 3 4
5 x3 5 6
I want to gather the data to make it tidy, but gathering only works when you have single column names. I've tried looking at spreading as well. The only solutions I've come up with are very hacky and don't feel right. Is there an elegant way to deal with this?
Here's what I want:
Date Equip metric value
1 12/16/18 a x1 1
2 12/16/18 a x2 3
3 12/16/18 a x3 5
4 12/17/18 b x1 2
5 12/17/18 b x2 4
6 12/17/18 b x3 6
This approach gets me close, but I don't know how to deal with the poor formatting (no header, no row names). It should be easy to gather if the formatting was proper.
> as.data.frame(t(df))
V1 V2 V3 V4 V5
V1 Date Equip x1 x2 x3
V2 12/16/18 a 1 3 5
V3 12/17/18 b 2 4 6
And here's the dput
structure(list(V1 = c("Date", "Equip", "x1", "x2", "x3"), V2 = c("12/16/18",
"a", "1", "3", "5"), V3 = c("12/17/18", "b", "2", "4", "6")), class = "data.frame", .Names = c("V1",
"V2", "V3"), row.names = c(NA, -5L))
Thanks for posting a nicely reproducible question. Here's some gentle tidyr/dplyr massaging.
library(tidyr)
df %>%
gather(key = measure, value = value, -V1) %>%
spread(key = V1, value = value) %>%
dplyr::select(-measure) %>%
gather(key = metric, value = value, x1:x3) %>%
dplyr::arrange(Date, Equip, metric)
#> Date Equip metric value
#> 1 12/16/18 a x1 1
#> 2 12/16/18 a x2 3
#> 3 12/16/18 a x3 5
#> 4 12/17/18 b x1 2
#> 5 12/17/18 b x2 4
#> 6 12/17/18 b x3 6
Updated for tidyr v1.0.0:
This is just a little bit cleaner syntax with the pivot functions.
df %>%
pivot_longer(cols = -V1) %>%
pivot_wider(names_from = V1) %>%
pivot_longer(cols = matches("x\\d"), names_to = "metric") %>%
dplyr::select(-name)
You can using reshape
library(reshape)
row.names(df) = df$V1
df$V1 = NULL
df = melt(data.frame(t(df)),id.var = c('Date','Equip'))
df[order(df$Date),]
Date Equip variable value
1 12/16/18 a x1 1
3 12/16/18 a x2 3
5 12/16/18 a x3 5
2 12/17/18 b x1 2
4 12/17/18 b x2 4
6 12/17/18 b x3 6
Here's another way starting from your approach using t(). We can replace the headers from the first row and then drop the first row, allowing just a single gather which might be more intuitive.
library(tidyverse)
df <- structure(list(V1 = c("Date", "Equip", "x1", "x2", "x3"), V2 = c(
"12/16/18",
"a", "1", "3", "5"
), V3 = c("12/17/18", "b", "2", "4", "6")), class = "data.frame", .Names = c(
"V1",
"V2", "V3"
), row.names = c(NA, -5L))
df %>%
t() %>%
`colnames<-`(.[1, ]) %>%
`[`(-1, ) %>%
as_tibble() %>%
gather("metric", "value", x1:x3) %>%
arrange(Date, Equip, metric)
#> # A tibble: 6 x 4
#> Date Equip metric value
#> <chr> <chr> <chr> <chr>
#> 1 12/16/18 a x1 1
#> 2 12/16/18 a x2 3
#> 3 12/16/18 a x3 5
#> 4 12/17/18 b x1 2
#> 5 12/17/18 b x2 4
#> 6 12/17/18 b x3 6
Created on 2018-04-20 by the reprex package (v0.2.0).