Basically, there are a few excel reports that are emailed every morning to us. I download them directly to a specified drive then I wrangle them in R.
The thing is that I have to manually open each file and save them before running my script in R. If not this will happen:
When I go in and manually open and save the files and re-run my script. I get the correct results:
Firstly, do you guys know why this happens? and secondly, is there a function that will allow me to open these files and save them. I did try openxlsx. However, I still have to manually press the save button.
Here is the function I created to bring in the files:
store.FUN = function(x)
{
m = as.data.frame(read_excel(file))
names(m) = c(1:length(m))
m[1, 1] = str_sub(m[2, 1], 13)
m = bind_rows((m)[1, ], subset(m, (m)[1] == "Total Income"))
m[2, 1] = m[2, 2]
m = m[-c(2)]
return(m)
}
district_1.stores = sapply(store.file, store.FUN, simplify=FALSE) %>%
bind_rows(.id = "Store ID")
Thanks!
Edit: So it looks like the cells are formulated:
But, if I do nothing and only save the file and go back to R to perform the script, the numbers pull in just fine.
Here is an example of the excel file:
enter image description here
I took the time to post the issue on github for openxlsx.
Tl;dr: it's not a bug, it's a built-in "problem" when importing from/exporting to Excel and is true for all such packages. The developer suggests exactly what the TO did in case Excel sheets contain formula: open the file in Excel first, save it and only then import it into R. Which doesn't answer the TO's question (which was how to open and save an Excel file automatically through R), but I'm posting this answer nonetheless, because it adds some helpful context.
https://github.com/ycphs/openxlsx/issues/261 and https://github.com/ycphs/openxlsx/issues/188#issuecomment-832591241
Actually just found that I can use the "reticulate" package in R to run a python module for this purpose.
Thanks for your help everyone!
I am new to using the Jupyter notebook with R kernel.
I have R code written in two files Settings.ipynb and Main_data.ipynb.
My Settings.ipynb file has a lot of details. I am showing sample details below
Schema = "dist"
resultsSchema = "results"
sourceName = "hos"
dbms = "postgresql" #Should be "sql server", "oracle", "postgresql" or "redshift"
user <- "hos"
pw <- "hos"
server <- "localhost/hos"
port <- "9763"
I would like to source Settings file in Main_data code file.
When I was using R studio, it was easy as I just use the below
source('Settings.R')
But now in Main_data Jupyter Notebook with R kernel, when I write the below piece of code
source('Settings.R') # settings file is in same directory as main_data file
I get the below error
Error in source("Settings.R"): Settings.R:2:11: unexpected '['
1: {
2: "cells": [
^
Traceback:
1. source("Settings.R")
When I try the below, I get another error as shown below
source('Settings.ipynb')
Error in source("Settings.ipynb"): Settings.ipynb:2:11: unexpected '['
1: {
2: "cells": [
^
Traceback:
1. source("Settings.ipynb")
How can I source an R code and what is the right way to save it (.ipynb or .R format in a jupyter notebook (which uses R kernel)). Can you help me with this please?
updated screenshot
We could create a .INI file in the same working directory (or different) and use ConfigParser to parse all the elements. The .INI file would be
Settings.INI
[settings-info]
schema = dist
resultsSchema = results
sourceName = hos
dbms = postgresql
user = hos
pw = hos
server = localhost/hos
Then, we initialize a parser object, read the contents from the file. We could have multiple subheadings (here it is only 'settings-info') and extract the components using either [[ or $
library(ConfigParser)
props <- ConfigParser$new()
props <- props$read("Settings.INI")$data
props[["settings-info"]]$schema
From the Jupyter notebook
the 'Settings.INI' file
Trying to save a Jupyter notebook file in .R format will not work as the format is a bit messed up (due to the presence of things like { "cells" : [....". You can verify this by opening your .R file in Jupyter Notebook.
However, you can use a vim editor/R studio to create a .R file. This will allow you to have the contents as is without any format issues such as { "cells" : [....".
Later from another jupyter notebook, you can import/source the .R file created using vim editor/R studio. This resolved the issue for me.
In summary, don't use jupyter notebook to create .R file and source them using another jupyter notebook file.
Currently in RStudio keybinding CTRL/CMD + SHIFT + B perform the following job
devtools::install().
How can I change it to have this behaviour:
devtools::install(build_vignettes = TRUE)
If not, is it possible to create the entirely new keybinding for it?
You can execute anything by using an add-in. See https://rstudio.github.io/rstudioaddins/ for details. In summary, you create a small R package containing a function like
installWithVignettes <- function() devtools::install(build_vignettes = TRUE)
then set up a special file in the package inst/rstudio/addins.dcf that names this as an add-in. After that you can assign a shortcut to it.
This question already has answers here:
How to open CSV file in R when R says "no such file or directory"?
(16 answers)
Closed 2 years ago.
I'm new to R, and after researching this error extensively, I'm still not able to find a solution for it. Here's the code. I've checked my working directory, and made sure the files are in the right directory. Appreciate it. Thanks
pollutantmean <- function(directory, pollutant = "nitrate", id= 1:332)
{ if(grep("specdata",directory) ==1)
{
directory <- ("./specdata")
}
mean_polldata <- c()
specdatafiles <- as.character(list.files(directory))
specdatapaths <- paste(directory, specdatafiles, sep="")
for(i in id)
{
curr_file <- read.csv(specdatapaths[i], header=T, sep=",")
head(curr_file)
pollutant
remove_na <- curr_file[!is.na(curr_file[, pollutant]), pollutant]
mean_polldata <- c(mean_polldata, remove_na)
}
{
mean_results <- mean(mean_polldata)
return(round(mean_results, 3))
}
}
The error I'm getting is below:
Error in file(file, "rt") : cannot open the connection
file(file, "rt")
read.table(file = file, header = header, sep = sep, quote = quote,
dec = dec, fill = fill, comment.char = comment.char, ...)
read.csv(specdatapaths[i], header = T, sep = ",")
pollutantmean3("specdata", "sulfate", 1:10)
In addition: Warning message:
In file(file, "rt") :
cannot open file './specdata001.csv': No such file or directory
You need to change directory <- ("./specdata") to directory <- ("./specdata/")
Relative to your current working directory, you are looking for the file 001.csv, which is in your specdata directory.
This question is nearly impossible to answer without any context, since you have not provided us with the structure of your working directory here. Fortunately for you, I have already taken R Programming on Coursera, so I already did this homework question.
The reason why you see this error I guess is because RStudio lost the path of your working directory.
(1) Go to session...
(2) Set working directory...
(3) Choose directory...
--> Then you can see a window pops up.
--> Choose the folder where you store your data.
This is the way without any code that you change your working directory.
Hope this can help you.
Set your working directory one level/folder higher. For example, if it is already set as:
setwd("C:/Users/Z/Desktop/Files/RStudio/Coursera/specdata")
go up one level back and set it as:
setwd("C:/Users/Z/Desktop/Files/RStudio/Coursera")
In other words, do not make "specdata" folder as your working directory.
I just spent a lot of time trying to understand what was wrong on my code too...
And it seems to be simple if you are using windows.
When you name your file "blabla.txt" then windows name it "blabla.txt.txt"...
That's the same with .CSV files so windows create a file named "001.csv.csv" if you called it "001.csv"
So, when you create your .csv file, just rename it "001" and open it in R using read.table("/absolute/path/of/directory/with/required/001.csv")
It works for me.
close your R studio and run it again as an administrator. That did the magic for me. Hope it works for you and anyone going through this too.
One better check that can be done if you get such error while accessing a file is to use the file.exists("file_path/file_name") function. This function will return TRUE if the file is existing and accessible, else False.
If running on Windows try running R or R Studio as administrator to avoid Windows OS file system constraints.
Use setwd() to change to appropriate directory.
Use only filename to access any file in the working directory.
Navigate a folder above by using "../<filename>".
Got this error and found that RStudio on my Windows machine try to use \ as escape symbol, so had to replace it with \\ to deal with it.
Try file.exists function with your path, e.g.:
file.exists("D:\\R\\path_to_file.csv")
Error in file(file, "rt") :
I just faced the same error and resolved by removing spacing in address using paste0 instead of paste
filepath=paste0(directory,"/",filename[1],sep="")
I got this same error message and fixed it in the easiest way I could. I put my .csv file into a file folder on my desktop, opened desktop in the window next to console on RStudio, and then opened my file there, and checked the box next to my .csv file, then I used the "more" pull down menu at the top of this window to set this as my working directory...probably the easiest thing for SUPER beginners like me :)
This error also occurs when you try to use the result of getwd() directly in the path. The problem is the missingness of the "/" at the end. Try the following code:
projectFolder <- paste(getwd(), "/", sep = '')
The paste() is to concatenate the forward slash at the end.
I got my R code file from a friend and was not able to run read.csv command but If I copy the same command(read.csv ) to a new R script file, it ran fine.
Below command was not running in R code file shared by my friend, working directory,file name etc were all correct because If I created a new R script file and ran below command ,it worked.
df <- read.csv("file.csv",header=TRUE,stringsAsFactors = FALSE,strip.white =
TRUE,sep = ',')
issue/resolution:
I right clicked the R code file and unblocked the file and click save button and issue got resolved.
I your R code file is in Downloads folder in windows , then move to some other folder.
I had a same issue .I removed the extension from the file name.Example my file name was saved as xyz. csv. i saved it as xyz.
Error in file(file, "rt")
Created a .r file and saved it in Desktop together with a sample_10000.csv file.
Once trying to read it
heisenberg <- read.csv(file="sample_100000.csv")
was getting the same error as you
heisenberg <- read.csv(file="sample_10000")
Error in file(file, "rt") : cannot open the connection In addition: Warning message:
In file(file, "rt") : cannot open file 'sample_10000': No such file or
directory
I knew at least two ways to fix this, one using the absolute path and the other changing the working directory.
Absolute path
I fixed it adding the absolute path to the file, more precisely
heisenberg <- read.csv(file="C:/Users/tiago/Desktop/sample_100000.csv")
Working directory
This error shows up because RStudio has a specific working directory defined which isn't necessarily the place the .r file is at.
So, to fix using this approach I've gone to Session > Set Working Directory > Chose Directory (CTRL + Shift + H) and selected Desktop, where the .csv file was at. That way running the following command also worked
heisenberg <- read.csv(file="sample_100000.csv")
I was getting the same error when trying to import 10,000 files. I tried opening a single file with Excel and Excel gave me an error message: "the file path is too long".
The file was buried in 6 nested folders, which is a problem of itself. Moving all the files to a desktop folder solved the issue without having to change any code.
I came across a solution based on a few answers popped in the thread.
(windows 10)
setwd("D:/path to folder where the files are")
directory <- getwd()
myfile<- "a_file_in_the_folder.txt"
filepath=paste0(directory,"/",myfile[1],sep="")
table <- read.table(table, sep = "\t", header=T, row.names = 1, dec=",")
I have an R program that does a bunch of data analysis and outputs the results to a text file. Unfortunately when I call
system2("open", file_path_as_absolute(toFile));
The R interpreter states the following
'../output/HLA-A,B,C,DR,DP,DQ' not found
'GT2' not found
'vs' not found
'LT2_DRB1_output2of9.10.11.12.13.14.16.25.26.28.30.31.32.33.37.38.40.47.57.58.60
.67.70.71.73.74.77.78.85.86.txt' not found
I am assuming, based upon this error, that the file_path_as_absolute tokenizes the file name, but I am not sure how to disable this. I have also tried normalizePath(), but I get the same error.
Edit
The file itself is called
"HLA-A,B,C,DR,DP,DQ GT2 vs LT2_DRB1_output2of9.10.11.12.13.14.16.25.26.28.30.31.32.33.37.38.40.47.57.58.60.67.70.71.73.74.77.78.85.86.txt"
and is located in ../output
Here is the code I ran to open the file, which gave me the same error
openSesame <- paste0('"', file_path_as_absolute(toFile), '"');
system2("open", openSesame);
I can get the following code to work on my system, starting in a directory where sister directory ../output exists (I don't have open on my system so I used gedit as the external command):
fn <- "HLA-A,B,C,DR,DP,DQ GT2 vs LT2_DRB1_output2of9.10.11.12.13.14.16.25.26.28.30.31.32.33.37.38.40.47.57.58.60.67.70.71.73.74.77.78.85.86.txt"
prot <- function(x) paste0('"',x,'"')
writeLines(c("a","b"),con=paste0("../output/",fn))
n <- tools::file_path_as_absolute(paste0("../output/",fn))
file.show(n)
system2("gedit",prot(n))
Can you make a reproducible example along these lines that fails on your system? (Does file.show() work for you?)
(Posting as an answer rather than a comment for purposes of code formatting ...)