When i need to import multiple csv files I use:
Cluster <- lapply(dir(),read.csv)
Previously setting the working directoy of course, but somehow today it stopped working, and returning this error message:
Error in read.table(file = file, header = header, sep = sep, quote = quote, :
no lines available in input
The only unusual thing i did was setting the Java directory manually so that way rJava can be loaded.
Any idea what happen?
Related
I am trying to read this large text file (~3gb) into R, but unfortunately I am not being able to read fully load it. What happens is that I'm missing a lot of rows (I get a dataframe of ~700 thousand rows, while I know the file has at least 4-5million).
The code I was initially using was as follows:
df<-read.delim("file.txt",quote = "",comment.char = "")
However, besides noticing that R wasn't loading all the rows, I was also receiving this warning:
Warning message:
In scan(file = file, what = what, sep = sep, quote = quote, dec = dec, :
embedded nul(s) found in input
I searched for a bit online, and I found I could solve it by adding the skipNul = TRUE argument. When I included it in the read.delim function, the warning stopped showing, but my file keeps missing a lot of rows, and still returns the same number of rows as before.
I have loaded files of similar size in the past, so I'm not sure why this is happening.
If someone has any idea what might be causing the problem, I would be very thankful.
I am trying to read a csv file in R, but I am getting some errors.
This is what I have and also I have set the correct path
mydata <- read.csv("food_poisioning.csv")
But I am getting this error
Error in make.names(col.names, unique = TRUE) :
invalid multibyte string at '<ff><fe>Y'
In addition: Warning messages:
1: In read.table(file = file, header = header, sep = sep, quote = quote, :
line 1 appears to contain embedded nulls
2: In read.table(file = file, header = header, sep = sep, quote = quote, :
line 2 appears to contain embedded nulls
I believe I am getting this error because my csv file is actually not separated by comma, but it has spaces. This is what is looks like:
I tried using sep=" ", but it didn't work.
If you're having difficulty using read.csv() or read.table() (or writing other import commands), try using the "Import Dataset" button on the Environment panel in RStudio. It is useful especially when you are not sure how to specify the table format or when the table format is complex.
For your .csv file, use "From Text (readr)..."
A window will pop up and allow you to choose a file/URL to upload. You will see a preview of the data table after you select a file/URL. You can click on the column names to change the column class or even "skip" the column(s) you don't need. Use the Import Options to further manage your data.
Here is an example using CreditCard.csv from Vincent Arel-Bundock's Github projects:
You can also modify and/or copy and paste the code in Code Preview, or click Import to run the code when you are ready.
not a question but a problem somebody else might stumble upon. I handle some data in csvs each week that is put into zip files to save space.
Usually I can easily read the csvs in the zip file with this code:
connections = unz(zip_path,csv_file)
DAT_r = read.csv2(connections, sep=";", dec=",", header=TRUE, stringsAsFactors=TRUE,
encoding="latin1", fill=TRUE, check.names=FALSE)
Today however I got the misleading error:
Error in read.table(file = file, header = header, sep = sep, quote = quote) : no lines available in input
After tedious checking of the csvs I realised the zip file was larger than usual. Indeed the size was too big to read in which spawned the error. Splitting the zip in two files resolved the error
Cheers
I am trying to read a csv file from aws datalake using R.
I used the below code to read the data, unfortunately I am getting an error
Error in read.table(file = file, header = header, sep = sep, quote =
quote, : no lines available in input
I am using the below code,
aws.signature::use_credentials()
c<- get_object("s3://datalake-1/x-data/")
cobj<- rawToChar(c)
con<- textConnection(cobj)
data <- read.csv(con)
close(con)
data
It looks like the file is not present at the address/URI provided. Unable to reproduce this error so, maybe look for your CSV's correct location.
Apart from that I'd also put the read statement within tryCatch as referenced in an already existing answer linked here
I am using Rstudio with R 3.3.1 on Windows 7 and I have installed CITAN package. I am trying to import bibliography entries from a CSV file that I exported from Scopus (as it is, untouched), choosing to export all available information.
This is the error that I get:
example <- Scopus_ReadCSV("scopus.csv")
Error in Scopus_ReadCSV("scopus.csv") : Column not found: `Source'.
In addition: Warning messages:
1: In read.table(file = file, header = header, sep = sep, quote = quote, :
invalid input found on input connection 'scopus.csv'
2: In read.table(file = file, header = header, sep = sep, quote = quote, :
incomplete final line found by readTableHeader on 'scopus.csv'
Column `Source' is there when I open the file, so I do not know why it says 'not found'.
Eventually I came into the following conclusions:
The encoding of the CSV file as exported from Scopus was UTF-8-BOM, which does not seem to be recognized from R when using Scopus_readCSV("file.csv") or read.table("file.csv", header = TRUE, sep = ",", fileEncoding = "UTF-8").
Although it is used an encoding type for the file from Scopus, there can be found some "strange" non-english characters which are not readable from the read function in R. (Mainly found this problem in names with special characters)
Solutions for those issues:
Open the CSV file with a notepad application like the Notepad++ and save the file with UTF-8 encoding to become readable for R as UTF-8.
When running the read function in R you will notice that it stops reading (e.g. in the 40th out of 200 registries). See where exactly it stopped and this way you can find the special character, by opening the CSV with the notepad, and then you can erase/change it as you wish in order to not have the same issue in R again.
Another solution that worked for me:
Open the file in Google Sheets, then download it from there again as a *.csv-file. R opens it correctly afterwards.