I'm trying to export a Julia dataframe into a CSV and it contains nothing values. To simiplify let's say I have the following dataframe:
using DataFrames, CSV
df = DataFrame(A = [nothing,2,3], B = [4,5,nothing])
When I try to export, I get the following error:
df |> CSV.write("df.csv")
ArgumentError: `nothing` should not be printed; use `show`, `repr`, or custom output instead.
Should I convert the nothing values to something else like missing? If so, how would I code this?
You can use something function to convert nothing to whatever you want (e.g. missing) like this:
something.(df, missing) |> CSV.write("aaa.txt")
The current reasoning behind the design not to support nothing when writing a CSV file is given here.
I prefer the syntax of the CSV transform argument:
CSV.write("filename.csv", df, transform=(col, val) -> something(val, missing))
From the CSV.write documentation
Posting this as a separate answer, hopefully to distinguish comments and gather clarifications if I am missing something important.
Related
This is the code I am trying to run:
data_table<-data_table%>%
merge(new_table, by = 'Sample ID')%>%
mutate(Normalized_value = ((1.8^(data_table$Ubb - data_table$Ct_adj))*10000))
I want to first add the new column ("Ubb") from "new_table" and then add a calculated column using that new column. However, I get an error saying that Ubb column does not exist. So it's not performing merge before running mutate? When I separate the functions everything works fine:
data_table<-data_table%>%
merge(new_table, by = 'Sample ID')
data_table<-data_table%>%
mutate(Normalized_value = ((1.8^(data_table$Ubb - data_table$Ct_adj))*10000))
I would like to keep everything together just for style, but I'm also just curious, shouldn't R perform merge first and then mutate? How does order of operation during piping work?
Thank you!
you dont need to refer to column name with $ sign. i.e. use Normalized_value = ((1.8^(Ubb - Ct_adj))*10000)
because it is merged now. with $ sign I believe R, even though does the merge, has original data_table still in memory. because the assignment operator did not work yet. the assignment will take place after all operations.
Try running the code like this:
data_table<-data_table%>%
merge(new_table, by = 'Sample ID')%>%
mutate(Normalized_value = ((1.8^(Ubb - Ct_adj))*10000))
Notice that I'm not using the table name with the $ within the pipe. Your way is telling the mutate column to look at a vector. Maybe it's having some trouble understanding the length of that vector when used with the merge. Just call the variable name within the pipe. It's easiest to think of the %>% as 'and then': data_table() and then merge() and then mutate(). You might also want to think about a left_join instead of a merge.
I'm hoping to produce a pipe-friendly function where a user specifies the "name of choice" for a new column produced by the function as one of the function arguments.
In the function below, I'd like name_for_elective to be something that the user can set at will, and afterwards, the user could expect that there will be a new column in their data with the name that they provided here.
I've looked at https://dplyr.tidyverse.org/reference/dplyr_data_masking.html, the mutate() function documentation, searched here, and tried working with https://dplyr.tidyverse.org/reference/rename.html, but to no avail.
elective_open<-function(.data,name_for_elective,course,tiebreaker){
name_for_elective<-rlang::ensym(name_for_elective)
course<-rlang::ensym(course)
tiebreaker<-rlang::ensym(tiebreaker)
.data%>%
mutate(!!name_for_elective =ifelse(!!tiebreaker==max(!!tiebreaker),1,0))%>%mutate(!!name_for_elective=ifelse(!!name_for_elective==0,!!course[!!name_for_elective==1],""))%>%
filter(!(!!course %in% !!name_for_elective))
}
I've included this example function because there are several references to the desired new column name, and I'm unsure if the context in which the reference is made changes syntax.
As you can see, I was hoping !!name_for_elective would let me name our new column, but no. I've played with {{}}, not using rlang::ensym, and still haven't got this figured out.
Any solution would be greatly appreciated.
This: Use dynamic variable names in `dplyr` may be helpful, but I can't seem to figure out how to extend this to work in the case where multiple references are made to the name argument.
Example data, per a good suggestion by #MrFlick, takes the form below:
dat<-tibble(ID=c("AA","BB","AA","BB","AA","BB"),Class=c("A_Class","B_Class","C_Class","D_Class","E_Class","F_Class"),
randomNo=c(.75,.43,.97,.41,.27,.38))
The user could then run something like:
dat2<-dat%>%
elective_open(MyChosenName,Class,randomNo)
A desired result, if using the function a single time, would be:
desired_result_1<-tibble(ID=c("AA","BB","AA","BB"),
Class=c("A_Class","D_Class","E_Class","F_Class"),
randomNo=c(.75,.41,.27,.38),
MyChosenName=c("C_Class","B_Class"))
The goal would be to allow this function to be used again if so desired, with a different name specified.
In the case where a user runs:
dat3<-dat%>%
elective_open(MyChosenName,Class,randomNo)%>%
mutate(Just_Another_One=1)%>%
elective_open(SecondName,Class,randomNo)
The output would be:
desired_result_2<-tibble(ID=c("AA","BB"),
Class=c("E_Class","F_Class"),
randomNo=c(.27,.38),
MyChosenName=c("C_Class","B_Class"),
Just_Another_One=c(1,1),
SecondName=c("A_Class","D_Class"))
In reality, there may be any number of records with the same ID, and any number of Class-es.
In this case you can just stick to using the embrace {{}} option for your variables. If you want to dynamically create column names, you're going to still need to use :=. The difference here is that you can use the glue-style syntax with the embrace operator to get the name of the symbol. This works with the data provided.
elective_open <- function(.data, name_for_elective, course, tiebreaker){
.data%>%
mutate("{{name_for_elective}}" := ifelse({{tiebreaker}}==max({{tiebreaker}}),1,0)) %>%
mutate("{{name_for_elective}}" := ifelse({{name_for_elective}}==0,{{course}}[{{name_for_elective}}==1],"")) %>%
filter(!({{course}} %in% {{name_for_elective}}))
}
I have a list of identifiers as follows:
url_num <- c('85054655', '85023543', '85001177', '84988480', '84978776', '84952756', '84940316', '84916976', '84901819', '84884081', '84862066', '84848942', '84820189', '84814935', '84808144')
And from each of these I'm creating a unique variable:
for (id in url_num){
assign(paste('test_', id, sep = ""), FUNCTION GOES HERE)
}
This leaves me with my variables which are:
test_8505465, test_85023543, etc, etc
Each of them hold the correct output from the function (I've checked), however my next step is to combine them into one big vector which holds all of these created variables as a seperate element in the vector. This is easy enough via:
c(test_85054655,test_85023543,test_85001177,test_84988480,test_84978776,test_84952756,test_84940316,test_84916976,test_84901819,test_84884081,test_84862066,test_84848942,test_84820189,test_84814935,test_84808144)
However, as I update the original 'url_num' vector with new identifiers, I'd also have to come down to the above chunk and update this too!
Surely there's a more automated way I can setup the above chunk?
Maybe some sort of concat() function in the original for-loop which just adds each created variable straight into an empty vector right then and there?
So far I've just been trying to list all the variable names and somehow get the output to be in an acceptable format to get thrown straight into the c() function.
for (id in url_num){
cat(as.name(paste('test_', id, ",", sep = "")))
}
...which results in:
test_85054655,test_85023543,test_85001177,test_84988480,test_84978776,test_84952756,test_84940316,test_84916976,test_84901819,test_84884081,test_84862066,test_84848942,test_84820189,test_84814935,test_84808144,
This is close to the output I'm looking for but because it's using the cat() function it's essentially a print statement and its output can't really get put anywhere. Not to mention I feel like this method I've attempted is wrong to begin with and there must be something simpler I'm missing.
Thanks in advance for any help you guys can give me!
Troy
I have (fbodata) 'data.frame': 6181090 obs. of 41 variables:
I want to subset it and save the portion that pertains to a specific subset (like a zip). My approach seems to work when it is not in a function, but I ultimately want to use sapply.
nmakedir <- function(item, ccol) {
snipped a bunch of code that works
trim<- fbodata[ which(paste(ccol)==item),]
trim%>% drop_na(paste(ccol))
trim<- droplevels(trim
save(trim, file = paste(item, "rda", sep="."))
}
The line that doesn't work is one where I creating the subset with which. If i hardcode the line using fbodata$zip instead of paste(ccol) it works fine. Eventually, I plan to call it with something like:
sapply(unique(fbodata$zip),zip, FUN = nmakedir)
I appreciate any clues, I have been on this for a good long while.
A few things going on:
ccol is a string. paste(ccol) is the same string. You never need to call paste with only one argument. (You can use paste to coerce non-strings to strings, but in that case you should use as.character() to be clear.)
Keeping in mind that ccol is a string, what is fbodata$zip? It's a column! What is the equivalent using ccol and brackets? fbodata[[ccol]] or fbodata[, ccol]. You can use either of those interchangeably with fbodata$zip. So, this bad line
fbodata[ which(paste(ccol)==item),]
# should be this:
fbodata[which(fbodata[[ccol]] == item), ]
drop_na, like most dplyr functions, expects (quoting from the help) "bare variable names", not strings. Also from the help, "See Also: drop_na_ for a version that uses regular evaluation and is suitable for programming with". In this case, I don't think you need to do anything more than replace drop_na with drop_na_.
You are missing a right parenthesis on your droplevels command.
There might be more, but this is much as I can see without any sample data. Your sapply call looks funny to me because I thought zip is supposed to be a column name, but when you call sapply(unique(fbodata$zip),zip, FUN = nmakedir) it needs to be an object in your global environment. I would think sapply(unique(fbodata$zip), 'zip', FUN = nmakedir) makes more sense, but without a reproducile example there's no way to know.
It also seems like you're coding your own version of split. I would probably start this off with fbo_split = split(fbodata, fbodata$zip) and then use lapply to drop_na_, droplevels, and save, but maybe your snipped code makes that a less good idea.
I think this won't be very complex, but I'm unable to figure this out.
I have in Matlab a 17x1 struct object with 6 fields, named photolist. I only want to export the name field , to use in R.
photolist.name gives me the list I need, but when I want to store it in a variable:
name = photolist.name
I only get the first value, same for
name = getfield(photo_list, 'name')
and while
name = [photolist.name]
gives me all values, it does so in one long string without spaces.
using
save('temp.mat', 'photolist')
gives me something I can import to R, but then I need to go multiple nested layers deep to get the values I need, which is a workaround but not very satisfying.
How do i save just the .name fields to a variable?
Found it, was already answered here
names = extractfield(photolist, 'name')
And another way to get the same result is:
names = {photolist.name}