I have a dataframe with a picture below, it contains a list of dataframes in the 2nd column, the column name is content. I also have a column called racenames in the 3rd column that I'd like to put inside of my csvs while running through the code. I can't figure out a way to get the list of dataframes to write to a csv in a loop or anything.
The code below works at writing a csv for the first dataframe in the content column, but I would like to write all of the dataframes at the same time so that I don't need to manually change the numbers/names for hours. All of the data has been scraped in one of my prior loops.
write.csv(ARCA_separate[[2]][[1]], file = "C:\\Users\\bubba\\Desktop\\RStuff\\Scraping\\ARCA 2012 Season\\*racename*.csv")
Here is what the data I'm working with looks like. The dataframe is called ARCA_separate.
How do I write all of the csvs and grab the corresponding racename in the same row to put into my csv name?
You can try this:
purrr::walk2(ARCA_separate$content, ARCA_separate$racename, function (x, y) write_csv(x, paste0("C:\\Users\\bubba\\Desktop\\RStuff\\Scraping\\ARCA 2012 Season\\", y, ".csv")))
Related
Using R, I want to pick last row data only from over 2000 csv in the same directory
and make single dataframe.
Directory = "C:\data
File name, for example '123456_p' (6 number digit)
Each csv has different number of rows, but has the same number of columns (10 columns)
I know the tail and list function, but over 2000 dataframes, inputting manually is time wasting.
Is there any way to do this with loop through R?
As always, I really appreciate your help and support
There are four things you need to do here:
Get all the filenames we want to read in
Read each in and get the last row
Loop through them
Bind them all together
There are many options for each of these steps, but let's use purrr for the looping and binding, and base-R for the rest.
Get all the filenames we want to read in
You can do this with the list.files() function.
filelist = list.files(pattern = '.csv')
will generate a vector of filenames for all CSV files in the working directory. Edit as appropriate to specify the pattern further or target a different directory.
Read each in and get the last row
The read.csv() function can read in each file (if you want it to go faster, use data.table::fread() instead), and as you mentioned tail() can get the last row. If you build a function out of this it will be easier to loop over, or change the process if it turns out you need another step of cleaning.
read_one_file = function(x) {
tail(read.csv(x), 1)
}
Loop through them
Bind them all together
You can do both of these steps at once with map_df() in the purrr package.
library(purrr)
final_data = map_df(filelist, read_one_file)
I have created a dataset that consists of 574 Rows and 85 Columns. The data type is a list. I want to export this data to CSV as I want to perform some analysis. I tried converting List to Dataframe using dataFrame <- as.data.frame(Data) command. I also looked out for other commands but was not able to convert the list to dataframe, or any other format. My goal is to export the data to a CSV file.
This image is a preview of the dataset:
This image shows that data type is list of dimension 574*85:
You can try this "write.csv" function on your list.
write.csv(list,"a.csv")
it will automatically save in your working directory.
Provide a list format of yours. I'm not sure below answer is useful for you or not.
If your list is as below
all_data_list = [[1,2,3],[1,4,5],[1,5,6],...]
you have to do:
df = pd.DataFrame(all_data_list)
I recently began using R with no prior coding experience after I was transferred to a new department and I want to understand how some R functions work. I have this written code:
read.csv("something.csv",header=TRUE)$DATE123
The csv file contains a time series with header that begins with DATE in A1 cell.
How does R classify that A column is DATE123? is it because of the header=true and $?
As explained in the comments, header=TRUE indicates that the first row of your files are column names. Thus, every object in that row will be a column with that name. In your case, there is probably a field in the first row of your csv file that is called DATE123.
A data frame consists of rows and columns. Each column in the data frame can be accessed by the $ sign. If the name of the data frame is df and one of the columns is named DATE123, then you can extract all data from that column by using the following command:
df$DATE123
Apologies if this is a trivial question. I saw others like it such as: How can I turn a part of the filename into a variable when reading multiple text files into R? , but I still seem to be having some trouble...
I have been given 50000 .txt files. Each file contains a single observation (a single row of data) with exactly 12 variables (number of columns). The name of each .txt file is fairly regular. Specifically, each .txt file has a code at the end indicating the type of observation across three dimensions. An example of this code is 'VL-VL-NE' or 'VL-M-N' or 'H-H-L' (not including the apostrophes). Therefore, an example of a file name could be 'I-love-using-R-20_01_2016-VL-VL-NE.txt'.
My problem is that I want to include this code at the end of the .txt file in the actual vector itself when I import into R, i.e., I want to add three more variables (columns) at the end of the table corresponding to the three parts of code at the end of the file name.
Any help would be greatly appreciated.
Because you have exactly the same number of columns in each file, why don't you import them into R using a loop that looks for all .txt files in a particularly directory?
df <- c()
for (x in list.files(pattern="*.txt")) {
u<-read.csv(x, skip=6)
u$Label = factor(x) #A column that is the filename
df <- rbind(df,u)
}
You'll note that the file name itself becomes a column. Once everything is into R, it should be fairly easy to use a regex function to extract the exact elements you need from the file name column (df$Label).
I am currently working with a Toolbox, so panel data must be contained in CSV files for each country. I have a 60 country panel for 1980-2014 quarterly data in a single CSV file. Rather than copying it manually I would like to use a looping routine.
This is what I tried to do:
mydata<-read.csv("regression.csv")
value<-split(mydata, mydata$country, drop=FALSE)
As far I understand I need to use lapply to export the data into 60 CSV files.
Can anyone help me with this please?
We loop through the names of the list elements using lapply, get the first 7 characters of the name with substr and use that to create the file name in the write.csv
lapply(names(value), function(x) write.csv(value[[x]],
file=paste0(substr(x, 1,7), '.csv'), quote=FALSE, row.names=FALSE))