I have a huge data frame with 10000 rows and 2048 columns. I am trying to convert this into a multi-dimensional array in the dimensions of 10000[rows]x32x32x2.
Here each row data should be converted into 32x32x2
> For ex:
1 2 3 4 5 6 7 8 9 10 11
2 a b c d e f g h i j
3 q k l m j g t y u r
4 a e t i o p l m n s
5
>
> I want to create 10000 arrays with dimension of 32x32x2 here for ex
> 5x5x2
1 array
a b c d e
f g h i j
>
> 2nd array
q k l m j
g t y u r
I am fairly new to R building models.
Please can you anyone help me.
Related
I want to add a total row (as in the Excel tables) while writing my data.frame in a worksheet.
Here is my present code (using openxlsx):
writeDataTable(wb=WB, sheet="Data", x=X, withFilter=F, bandedRows=F, firstColumn=T)
X contains a data.frame with 8 character variables and 1 numeric variable. Therefore the total row should only contain total for the numeric row (it will be best if somehow I could add the Excel total row feature, like I did with firstColumn while writing the table to the workbook object rather than to manually add a total row).
I searched for a solution both in StackOverflow and the official openxslx documentation but to no avail. Please suggest solutions using openxlsx.
EDIT:
Adding data sample:
A B C D E F G H I
a b s r t i s 5 j
f d t y d r s 9 s
w s y s u c k 8 f
After Total row:
A B C D E F G H I
a b s r t i s 5 j
f d t y d r s 9 s
w s y s u c k 8 f
na na na na na na na 22 na
library(janitor)
adorn_totals(df, "row")
#> A B C D E F G H I
#> a b s r t i s 5 j
#> f d t y d r s 9 s
#> w s y s u c k 8 f
#> Total - - - - - - 22 -
If you prefer empty space instead of - in the character columns you can specify fill = "" or fill = NA.
Assuming your data is stored in a data.frame called df:
df <- read.table(text =
"A B C D E F G H I
a b s r t i s 5 j
f d t y d r s 9 s
w s y s u c k 8 f",
header = TRUE,
stringsAsFactors = FALSE)
You can create a row using lapply
totals <- lapply(df, function(col) {
ifelse(!any(!is.numeric(col)), sum(col), NA)
})
and add it to df using rbind()
df <- rbind(df, totals)
head(df)
A B C D E F G H I
1 a b s r t i s 5 j
2 f d t y d r s 9 s
3 w s y s u c k 8 f
4 <NA> <NA> <NA> <NA> <NA> <NA> <NA> 22 <NA>
This question already has answers here:
Find max per group and return another column
(4 answers)
Closed 5 years ago.
I have a dataframe df
df = data.frame(L = rep(letters[1:6], each = 2),
M = rep(letters[7:12]),
freq = sample(c(5, 10), replace = FALSE))
L M freq
1 a g 5
2 a h 10
3 b i 5
4 b j 10
5 c k 5
6 c l 10
7 d g 5
8 d h 10
9 e i 5
10 e j 10
11 f k 5
12 f l 10
I want to select the most frequent M for each L.
In this example the output would show:
h, j, l, h, j, l
Frequency is not necessarily every second value in the actual problem.
How can I do this easily?
I've tried a tapply approach, but get stuck here because this seems to only apply to variables and can't be used to subset a subset data frame. (This didn't result in anything close so I won't post the approach)
We can do
library(data.table)
setDT(df)[, .(M = M[which.max(freq)]), L]
# L M
#1: a h
#2: b j
#3: c l
#4: d h
#5: e j
#6: f l
Or order the 'freq' and select the first 'M' for each 'L'
setDT(df)[order(-freq), .(M = M[1]) , L]
Another solution using dplyr
df %>% group_by(L) %>% top_n(1, freq) %>% .$M
#### [1] h j l h j l
eventually transform into character at the end...
For the data frame below I want to perform kolmogorov-smirnov tests for multiple columns. Column ID is the record ID, A-D are factors consisting of 2 levels ('Other' and A,B,C,D respectively. My test variable is in column E.
Now I would like to perform 4 KS tests:
Distributions of E for column A (A vs O)
Distributions of E for column B (B vs O)
Distributions of E for column C (C vs O)
Distributions of E for column A (D vs O)
In reality, I have 80 columns, so I'm looking for a way to perform these 80 tests 'Simultaneously'
ID A B C D E
1 1 O B C O 1
2 2 O O O O 3
3 3 O O O D 2
4 4 A O C D 7
5 5 A B O O 12
6 6 O O O O 4
7 7 O B O O 8
I hope this solves your problem:
dat <- read.table("path/data.txt") # your data imported into my session.
cols <- c("A", "B", "C", "D") #these are the your columnss with categories. We leave the others out.
E <- dat$E # but save the E variable
lapply(cols, function(i){ # Evaluate E at each level of each column
x <- factor(dat[,i])
a <- E[x == levels(x)[1]]
b <- E[x == levels(x)[2]]
ks.test(a, b)
}) #you get a list with the results for each column
I have a data frame that has one value in each cell, but my last column is a list.
Example. Here there are 3 columns. X and Y columns have one value in each row. But column Z is actually a list. It can have multiple values in each cell.
X Y Z
1 a d h, i, j
2 b e j, k
3 c f l, m, n, o
I need to create this:
X Y Z
1 a d h
2 a d i
3 a d j
4 b e j
4 b e k
5 c f l
6 c f m
7 c f n
8 c f o
Can someone help me figure this out ? I am not sure how to use melt or dcast or any other function for this.
Thanks.
unnest from tidyr works
library(tidyr)
unnest(dat, Z)
I a dataframe which I have subsetted using normal indexing. Code below.
dframe <- dframe[1:10, c(-3,-7:-10)]
But when I write dframe$Symbol I get the output.
BABA ORCL LFC TSM ACT ABBV MA ABEV KMI UPS
3285 Levels: A AA AA^B AAC AAN AAP AAT AAV AB ABB ABBV ABC ABEV ABG ABM ABR ABR^A ABR^B ABR^C ABRN ABT ABX ACC ACCO ACE ACG ACH ACI ACM ACN ACP ACRE ACT ACT^A ACW ADC ADM ADPT ADS ADT ADX AEB AEC AED AEE AEG AEH AEK AEL AEM AEO AEP AER AES AES^C AET AF AF^C ... ZX
I'm wondering what is happening here. Does the dframe dataframe only contain 10 rows or still all rows, but only outputs 10 rows?
Thanks
That's just the way factors work. When you subset a factor, it preserves all levels, even those that are no longer represented in the subset. For example:
f1 <- factor(letters);
f1;
## [1] a b c d e f g h i j k l m n o p q r s t u v w x y z
## Levels: a b c d e f g h i j k l m n o p q r s t u v w x y z
f2 <- f1[1:10];
f2;
## [1] a b c d e f g h i j
## Levels: a b c d e f g h i j k l m n o p q r s t u v w x y z
To answer your question, it's actually slightly tricky to append all missing levels to a factor. You have to combine the existing factor data with all missing indexes (here I'm referring to the integer indexes that the factor class internally uses to map the actual factor data to its levels vector, which is stored as an attribute on the factor object), and then rebuild a factor (using the original levels) from that combined data. Below I demonstrate this, now randomizing the subset taken from f1 to demonstrate that order does not matter:
set.seed(1); f3 <- sample(f1,10);
f3;
## [1] g j n u e s w m l b
## Levels: a b c d e f g h i j k l m n o p q r s t u v w x y z
factor(c(f3,setdiff(1:nlevels(f3),as.integer(f3))),labels=levels(f3));
## [1] g j n u e s w m l b a c d f h i k o p q r t v x y z
## Levels: a b c d e f g h i j k l m n o p q r s t u v w x y z