As I understand from googling, SQLLoader is able to receive wildcards in the INFILE parameter.
I'm trying to load a file like this:
INFILE 'E:\FOLDER\myfile*.txt'
But the SQLLoader fails with errors
500: Unable to open file
554: Error opening file
There is only one file that responses to the wildcard, which means that I'm not trying to load multiple files here.
Is there an option to achieve this without using batch to pass the file as parameter?
Thank you!
An asterisk used with INFILE means data to load will be in the control file itself.
INFILE *
...
begindata
1,testdata 1, etc
2,testdata 2, etc
It's not the same as a file-globbing asterisk. The best way is to supply the filename on the commandline via the DATA= parameter.
Related
So I have a collection on json files located on my local machine that I am reading in currently using the command
file <- tbl_df(ndjson::stream_in("path/to/file.json")
I have copied these files to a linux server (using WinSCP) and I want to stream them in to my R session just like I did in the above code with ndjson. When searching for ways to do this I came across one method using RCurl that looked like this
file <- scp(host = "hostname", "path/to/file.json", "pass", "user")
but that returned an error
Error in function (type, msg, asError = TRUE) : Authentication failure
but either way I want to avoid copying my passphrase into my Rscript as other will see this script. I also came across a method suggesting this
d <- read.table(pipe('ssh -l user host "cat path/to/file.json"'))
however this command returned the error
no lines available in input
and I believe read.table would cause me issues anyways. Does anyone know I way I could read new line delimited json files from a remote server into an R session? Thank you in advance! Let me know if I can make my question more clear.
I have a RM Server running on a VM (Ubuntu) on top of my Win10 machine.
I have a process to read a .csv file and write its contents on a MySQL database on a MySQL Server which also runs on the same VM.
The problem is that the read file operator does not seem to be able to find the file.
Scenario1.
When I try as location-name in the read csv operator ../data/myFile.csv
and run the process on Server I am getting Failed to execute initialization process: Error executing process /apps/myApp/process/task_read_csv_to_db: The file 'java.io.FileNotFoundException: /root/../data/myFile.csv (No such file or directory)' does not exist.
Scenario2.
When I try as location-name in the read csv operator /apps/myApp/data/myFile.csv
and run the process on Server I am getting Failed to execute initialization process: Error executing process /apps/myApp/process/task_read_csv_to_db: The file 'java.io.FileNotFoundException: /apps/myApp/data/myFile.csv (No such file or directory)' does not exist.
What is the right filepath that I should give to the Read CSV operator?
Just to update with the answer. After David's suggestion, I resulted in storing the .csv file outside of the /rapidminer-server-home/data/repository since every remote repository seems to be depicted with an integer instead of its original name, making the use of the actual full path of the file not usable.
I would say, the issue is that depending on the location of the JobAgent that is executing your process, the relative path might be varying.
Is /apps/myApp/data/myFile.csv the correct path to the file? If not, I would suggest to use the absolute path to the file. Hope this helps.
Best,
David
I need to delete files from an FTP site once I have processed them in R (parsing content). However, nothing I try seems to work.
this is what ive trying, and variations of.
library(RCurl)
curlPerform(url="sftp://user:password#sftplocation/folder/", quote="DELE filename.pdf")
curlPerform(url="ftp://xxx.xxx.xxx.xxx/", quote="DELE file.txt", userpwd = "user:pass")
Error is
Error in function (type, msg, asError = TRUE) : Unknown SFTP command
When I run the following code, I get a lovely list of all the files (which is used to download them). So I know the connection is working just great, and the parsing from the downloaded files works great!
curlPerform(url="sftp://user:password#sftplocation/folder/")
Thanks,
Siobhan
To delete over sftp, use rm instead of DELE - which looks like an ftp rather than an sftp command.
Then make sure you have the full file path. This works for me:
curlPerform(
url="sftp://me#host.example.com/",
.opts=list(
ssh.public.keyfile=pub,
ssh.private.keyfile=pri),
verbose=TRUE,
quote="rm /home/me/test/test.txt")
Note I've put my credentials in some key files so I don't put the password in plain text in the code.
I'm not convinced this is the best way to do it, since I can't stop it printing the contents of the URL... There's might be an option...
I am trying to use lua to access redis values from nginx. When i execute lua files on command line there everything is ok i am able to read and write values to redis. But i when try to execute the same files from nginx by accessing a location in which access_by_lua directive is written the following error logged in error log file
no field package.preload['socket']
no file '/home/sivag/redis/redis-lua/src/socket.lua'
no file 'src/socket.lua'
no file '/home/sivag/lua/socket.lua'
no file '/opt/openresty/lualib/socket.so'
no file './socket.so'
no file '/usr/local/lib/lua/5.1/socket.so'
no file '/opt/openresty/luajit/lib/lua/5.1/socket.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
What is the reason for this and how can i resolve this?
In my case I just needed to install the lua-socket package, as the socket library is not built into the default Lua installation like it is in some other languages.
You get this error because your code executes the command require("socket")
This command will search for a file with that name in several directories. If successful the content will be executed as Lua code. If it is not successful you'll end up with your error message.
In order to fix this you have to add the path containing the file either to the system variable LUA_PATH or you have to add it to the global table package.path befor you require the file.
Lua will replace ? with the name you give to require()
For example
package.path = package.path .. ";" .. thisPathContainsTheLuaFile .. "?.lua"
Please read:
http://www.lua.org/manual/5.3/manual.html#pdf-require
https://www.lua.org/pil/8.1.html
I want to import datasets in R that come from an FTP server. I am using FileZilla to manually see the files. Currently my data is in a xxxx.csv.gz file in the FTP server and a new file gets added once a day.
My issues are that I have tried using the following link as guidance and it doesn't seem to work well in my case:
Using R to download newest files from ftp-server
When I attempt the following code an error message comes up:
library(RCurl)
url <- "ftp://yourServer"
userpwd <- "yourUser:yourPass"
filenames <- getURL(url, userpwd = userpwd,
ftp.use.epsv = FALSE,dirlistonly = TRUE)
Error:
Error in function (type, msg, asError = TRUE) :
Failed to connect to ftp.xxxx.com port 21: Timed out
The reason why this happened is because under the credentials: it states that I should use Port: 22 for secure port
How do I modify my getURL function so that I can access Port: 22?
Also there is a directory after making this call that I need to get to in order to access the files.
For example purposes: let's say the directory is:
Directory: /xxx/xxxxx/xxxxxx
(I've also tried attaching this to the original URL callout and the same error message comes up)
Basically I want to get access to this directory, upload individual csv.gz files into R and then automatically call the following day's data.
The file names are:
XXXXXX_20160205.csv.gz
(The file names are just dates and each file will correspond to the previous day)
I guess the first step is to just make a connection to the files and download them and later down the road, automatically call the previous day's csv.gz file.
Any help would be great, thanks!