HANA RLANG temporary file cannot open the connection - r

I'm using SAP HANA together with R. I use the cSPADE implementation in R and therefore load the data into a temporary file.
However until know it worked fine so far but now I getting an error. It seems to be some problem with the temporary file.
Could not execute 'CALL "BETA_SYSTEM"."run"()' in 15.741 seconds .
SAP DBTech JDBC: [2048]: column store error: [2048] "BETA_SYSTEM"."run": line 21 col 2 (at pos 821): [2048] (range 3): column store error: [2048] "BETA_SYSTEM"."beta.procedures::selectSpadeData": line 33 col 4 (at pos 992): [2048] (range 3): column store error: search table error: [34082] Execution of R script failed.;Error in file(file, ifelse(append, "a", "w")) :
cannot open the connection
stack trace:
2: file(file, ifelse(append, "a", "w"))
1: write.table(spade_data, file = file, sep = " ", row.names = FALSE,
quote = FALSE, col.names = FALSE)
CREATE PROCEDURE BETA_SYSTEM.spadeR (
IN spade_data "SPADE_INPUT_T",
OUT result "SPADE_RESULT_T"
)
LANGUAGE RLANG
AS
BEGIN
library(Matrix)
library(arules)
library(arulesSequences)
file <- tempfile()
write.table(spade_data, file = file, sep = " ", row.names=FALSE, quote = FALSE, col.names=FALSE)
df_trans <- read_baskets(file, info=c("sequenceID", "eventID", "SIZE"))
as(df_trans, "data.frame")
err_res <- tryCatch({
s_res <- cspade(df_trans, parameter = list(support = 0), control = list(verbose = TRUE))
}, warning= function(war) {
}, error=function(err) {
}, finally={
}
)
if(length(err_res) == 0){
result<-data.frame(sequence="EMPTY",support=0)
} else {
result <- as(s_res, "data.frame")
}
END;

The error stack indicates that there is a problem with accessing the file. This happens in the R processes spawned by RSERV. So it's outside SAP HANA and I would propose to check whether the R process with the same working directory can actually perform the file operation.
Maybe the file system is full or a access permission has been changed!?

Related

Error in file(file, ifelse(append, "a", "w")) : cannot open the connection - how to fix it?

I already searched multiple times but can't understand why the msg pops up. Please help me out. My code is as follows:
(WD <- getwd())
if (!is.null(WD)) setwd(WD)
indata = paste0(WD,"/US_ind.rda","", collapse = NULL)
library(haven)
library(AER)
library(sandwich)
library(lmtest)
library(pracma)
....
regTableA<- stargazer(plm_base_gamma,plm_base_delta, plm_base_both,
font.size = "footnotesize",
out = "output/regTableA.tex",
column.sep.width = "2pt",
se = stderror,
title = "Zillow data")
At the last step, I want to export the table into tex form. But the error msg repeatedly pops up. How can I fix this?

ShortRead package error in function for writing large files in fastq format

Hello
I am practicing ShortRead package for analyzing fastq file format. A text book I am reading, write a function for writing large fastq files as below:
trim.file <- function (f1,
destination=sprintf("%s_filtered.fastq", fq)){
stream <- open(FastqStreamer(f1))
on.exit(close(stream))
repeat {
fq <- yield(stream)
if (length(fq) == 0){break}
fq <- fq[nFilter()(fq)]
fq <- trimTails(fq, 5, "A", successive = T)
fq <- fq[width(fq) > 80]
writeFastq(fq, destination, "a", compress = F)
}
}
trim.file(FastqFiles [1])
The FastqFiles [1] is the path/filename of the original fastq file that is created with this code FastqFiles <- list.files(path=fasrQDir, pattern = "*.fastq", full.names = T) and fasrQDir is the directory of my original fastq file.
When I run this code I encounter this error:
Error in h(simpleError(msg, call)) :
error in evaluating the argument 'file' in selecting a method for function 'writeFastq': no method for coercing this S4 class to a vector
In addition: Warning message:
In open.connection(con$con) :
Please let me know what is the solution.
Best wishes

How to open/read an ENVI file in R

I'm new at using GIS with R and I'm trying to open an ENVI file containing hyperspectral data following the suggestions from this post R how to read ENVI .hdr-file?, but I don't seem to be able to do so. I tried three different approaches but all of them failed. I also can't seem to find any other posts where my problem is described.
# install.packages("rgdal")
# install.packages("raster")
# install.packages("caTools")
library("rgdal")
library("raster")
library("caTools")
dirname <- "S:/LAB-cavender/4_Project_Folders/oakWilt/oak_wilt_image_analyses/R_input/6.15.2021 - Revisions/ENVI export/AISA/Resampled_flights"
filename <- file.path(dirname, "AISA_Flight_4_resampled")
file.exists(filename)
The first option that I tried was using file name only
x <- read.ENVI(filename)
But I got the following error message:
#Error in read.ENVI(filename) :
# read.ENVI: Could not open input file: S:/LAB-cavender/4_Project_Folders/oakWilt/oak_wilt_image_analyses/R_input/6.15.2021 - Revisions/ENVI export/AISA/Resampled_flights/AISA_Flight_4_resampled
#In addition: Warning message:
# In nRow * nCol * nBand : NAs produced by integer overflow
I tried then the second option which is using file name + header file name read using file.path
headerfile <- file.path(dirname, "AISA_Flight_4_resampled")
x <- read.ENVI(filename = filename,headerfile = headerfile)
Again, I got an error message that says:
#Error in read.ENVI(filename = filename, headerfile = headerfile) :
# read.ENVI: Could not open input header file: S:/LAB-cavender/4_Project_Folders/oakWilt/oak_wilt_image_analyses/R_input/6.15.2021 - Revisions/ENVI export/AISA/Resampled_flights/AISA_Flight_4_resampled
Finally, I tried the third option by using file name + header file name read using readLines
hdr_file <- readLines(con = "S:/LAB-cavender/4_Project_Folders/oakWilt/oak_wilt_image_analyses/R_input/6.15.2021 - Revisions/ENVI export/AISA/Resampled_flights/AISA_Flight_4_resampled.hdr")
x <- read.ENVI(filename = filename,headerfile = hdr_file)
But I got the error message:
#Error in read.ENVI(filename = filename, headerfile = hdr_file) :
# read.ENVI: Could not open input header file: ENVIdescription = { Spectrally Resampled File. Input number of bands: 63, output number of bands: 115. [Fri Jun 25 16:57:21 2021]}samples = 5187lines = 6111bands = 115header offset = 0file type = ENVI Standarddata type = 4interleave = bilsensor type = Unknownbyte order = 0map info = {UTM, 1.000, 1.000, 482828.358, 5029367.353, 7.5000000000e-001, 7.5000000000e-001, 15, North, WGS-84, units=Meters}coordinate system string = {PROJCS["UTM_Zone_15N",GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137.0,298.257223563]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-93.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]}default bands = {46,31,16}wavelength units = Nanometersdata ignore value = -9999.00000000e+000band names = { Resampled
# In addition: Warning message:
# In if (!file.exists(headerfile)) stop("read.ENVI: Could not open input header file: ", :
# the condition has length > 1 and only the first element will be used
Any help would be really appreciated!

Error when trying to write files using 'if' function with l_ply

I'm relatively new to using R in this way and I'm completely stuck with the following problem.
I'm attempting to save html pages for parliamentary debates to a local folder in order to carry out some scraping in the future. I've written the following function (relying on other snippets of code rather than entirely freestyle!) in order to construct the directory, strip the URL down to a more understandable format (e.g. "2010-12_academieshl.html"), and then, if the file does not exist, write the file to the specified folder. (At this point I'm aware that the use of gsub below is kind of clumsy!)
dlPages <- function(pageurl, folder ,handle) {
dir.create(folder, showWarnings = FALSE)
gsub_URL <- gsub("/stages.html", "", link_list)
gsub_URL <- gsub("http://services.parliament.uk/bills/", "", gsub_URL)
gsub_URL <- gsub("/", "_", gsub_URL)
page_name <- str_c(gsub_URL, ".html")
if (!file.exists(str_c(folder, "/", page_name))) {
content <- try(getURL(pageurl, curl = handle))
write(content, str_c(folder, "/", page_name))
Sys.sleep(1)
} }
I'm then using l_ply to run a list (link_list) of links over the function:
handle <- getCurlHandle()
l_ply(link_list, dlPages,
folder = "lords_bills_all",
handle = handle)
The following error message is being returned:
Error in file(file, ifelse(append, "a", "w")) : invalid 'description' argument
Along with the following warning messages:
In addition: Warning messages:
1: In if (!file.exists(str_c(folder, "/", page_name))) { :
the condition has length > 1 and only the first element will be used
2: In if (file == "") file <- stdout() else if (substring(file, 1L, :
the condition has length > 1 and only the first element will be used
3: In if (substring(file, 1L, 1L) == "|") { :
the condition has length > 1 and only the first element will be used
Can someone help me understand where I'm going wrong? Also, is it best to use an 'if' argument in this scenario or write a 'for' loop instead?
Thanks in advance.
Andy

Error trying to read a PDF using readPDF from the tm package

(Windows 7 / R version 3.0.1)
Below the commands and the resulting error:
> library(tm)
> pdf <- readPDF(PdftotextOptions = "-layout")
> dat <- pdf(elem = list(uri = "17214.pdf"), language="de", id="id1")
Error in file(con, "r") : cannot open the connection
In addition: Warning message:
In file(con, "r") :
cannot open file 'C:\Users\Raffael\AppData\Local\Temp
\RtmpS8Uql1\pdfinfo167c2bc159f8': No such file or directory
How do I solve this issue?
EDIT I
(As suggested by Ben and described here)
I downloaded Xpdf copied the 32bit version to
C:\Program Files (x86)\xpdf32
and the 64bit version to
C:\Program Files\xpdf64
The environment variables pdfinfo and pdftotext are referring to the respective executables either 32bit (tested with R 32bit) or to 64bit (tested with R 64bit)
EDIT II
One very confusing observation is that starting from a fresh session (tm not loaded) the last command alone will produce the error:
> dat <- pdf(elem = list(uri = "17214.pdf"), language="de", id="id1")
Error in file(con, "r") : cannot open the connection
In addition: Warning message:
In file(con, "r") :
cannot open file 'C:\Users\Raffael\AppData\Local\Temp\RtmpKi5GnL
\pdfinfode8283c422f': No such file or directory
I don't understand this at all because the function variable is not defined by tm.readPDF yet. Below you'll find the function pdf refers to "naturally" and to what is returned by tm.readPDF:
> pdf
function (elem, language, id)
{
meta <- tm:::pdfinfo(elem$uri)
content <- system2("pdftotext", c(PdftotextOptions, shQuote(elem$uri),
"-"), stdout = TRUE)
PlainTextDocument(content, meta$Author, meta$CreationDate,
meta$Subject, meta$Title, id, meta$Creator, language)
}
<environment: 0x0674bd8c>
> library(tm)
> pdf <- readPDF(PdftotextOptions = "-layout")
> pdf
function (elem, language, id)
{
meta <- tm:::pdfinfo(elem$uri)
content <- system2("pdftotext", c(PdftotextOptions, shQuote(elem$uri),
"-"), stdout = TRUE)
PlainTextDocument(content, meta$Author, meta$CreationDate,
meta$Subject, meta$Title, id, meta$Creator, language)
}
<environment: 0x0c3d7364>
Apparently there is no difference - then why use readPDF at all?
EDIT III
The pdf file is located here: C:\Users\Raffael\Documents
> getwd()
[1] "C:/Users/Raffael/Documents"
EDIT IV
First instruction in pdf() is a call to tm:::pdfinfo() - and there the error is caused within the first few lines:
> outfile <- tempfile("pdfinfo")
> on.exit(unlink(outfile))
> status <- system2("pdfinfo", shQuote(normalizePath("C:/Users/Raffael/Documents/17214.pdf")),
+ stdout = outfile)
> tags <- c("Title", "Subject", "Keywords", "Author", "Creator",
+ "Producer", "CreationDate", "ModDate", "Tagged", "Form",
+ "Pages", "Encrypted", "Page size", "File size", "Optimized",
+ "PDF version")
> re <- sprintf("^(%s)", paste(sprintf("%-16s", sprintf("%s:",
+ tags)), collapse = "|"))
> lines <- readLines(outfile, warn = FALSE)
Error in file(con, "r") : cannot open the connection
In addition: Warning message:
In file(con, "r") :
cannot open file 'C:\Users\Raffael\AppData\Local\Temp\RtmpquRYX6\pdfinfo8d419174450': No such file or direc
Apparently tempfile() simply doesn't create a file.
> outfile <- tempfile("pdfinfo")
> outfile
[1] "C:\\Users\\Raffael\\AppData\\Local\\Temp\\RtmpquRYX6\\pdfinfo8d437bd65d9"
The folder C:\Users\Raffael\AppData\Local\Temp\RtmpquRYX6 exists and holds some files but none is named pdfinfo8d437bd65d9.
Intersting, on my machine after a fresh start pdf is a function to convert an image to a PDF:
getAnywhere(pdf)
A single object matching ‘pdf’ was found
It was found in the following places
package:grDevices
namespace:grDevices [etc.]
But back to the problem of reading in PDF files as text, fiddling with the PATH is a bit hit-and-miss (and annoying if you work across several different computers), so I think the simplest and safest method is to call pdf2text using system as Tony Breyal describes here.
In your case it would be (note the two sets of quotes):
system(paste('"C:/Program Files/xpdf64/pdftotext.exe"',
'"C:/Users/Raffael/Documents/17214.pdf"'), wait=FALSE)
This could easily be extended with an *apply function or loop if you have many PDF files.

Resources