R -RMySQL- how to save more sql queries to file? - r

Question

I do some data analysis in R. On end of script I want save my results to file. I know there is more options how to do it, but they don't work properly. When I try sink() it works but it give me :
<MySQLResult:1,5,1>
host logname user time request_fline status
1 142.4.5.115 - - 2018-01-03 12:08:58 GET /phpmyadmin?</script><script>alert('<!--VAIBS-->');</script><script> HTTP/1.1 400
size_varchar referer agent ip_adress size_int cookie time_microsec filename
1 Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0 445 - 142.4.5.115 445 - 159 -
request_protocol keepalive request_method contents_of_foobar contents_of_notefoobar port child_id
<MySQLResult:1,5,1>
[1] host logname user time request_fline status
[7] size_varchar referer agent ip_adress size_int cookie
<0 rows> (or 0-length row.names)
which is totally unusable because I cant export that type of data. If I try write.table it give file with one row which is possible read but after one row R skript and give me error : Error in isOpen(file, "w") : invalid connection and when I try write.csv result is same. And when I try lapply it give me just empty file.
There is my code :
fileConn<-file("outputX.txt")
fileCon2<-file("outputX.csv")
sink("outputQuery.txt")
for (i in 1:length(awq)){
sql <- paste("SELECT * FROM mtable ORDER BY cookie LIMIT ", awq[i], ",1")
nb <- dbGetQuery(mydb, sql)
print (nb)
write.table(nb, file = fileConn, append = TRUE, quote = FALSE, sep = " ", eol = "\n", na = "NA", row.names = FALSE, col.names = FALSE)
write.csv(nb, file = fileCon2,row.names=FALSE, sep=" ")
lapply(nb, write, fileConn, append=TRUE, ncolumns=7)
writeLines(unlist(lapply(nb, paste, collapse=" ")))
}
sink()
close(fileConn)
close(fileCon2)
I am new in R, so I don't know what else should I try.What I want is 1 file where data will be print in form which is easy to read and export. For example tike this :
142.4.5.115 - - 2018-01-03 12:08:58 GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 400 Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0 445 - 142.4.5.115 445 - 145 - HTTP/1.1 0 GET - - 80 7216 ?/><!--VAIBS--> GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 - 0 /phpmyadmin - 354 0
142.4.5.115 - - 2018-01-03 12:10:23 GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 400 Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0 445 - 142.4.5.115 445 - 145 - HTTP/1.1 0 GET - - 80 7216 ?/><!--VAIBS--> GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 - 0 /phpmyadmin - 354 0
142.4.5.115 - - 2018-01-03 12:12:41 GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 400 Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0 445 - 142.4.5.115 445 - 145 - HTTP/1.1 0 GET - - 80 7216 ?/><!--VAIBS--> GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 - 0 /phpmyadmin - 354 0
142.4.5.115 - - 2018-01-03 12:15:29 GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 400 Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0 445 - 142.4.5.115 445 - 145 - HTTP/1.1 0 GET - - 80 7216 ?/><!--VAIBS--> GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 - 0 /phpmyadmin - 354 0
or this :
host,logname,user,time, request_fline status,size_varchar,referer agent,ip_adress,size_int,cookie,time_microsec,filename,request_protocol,keepalive,request_method,contents_of_foobar,contents_of_notefoobar port child_id
1 142.4.5.115 - - 2018-01-03 12:08:58 GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 400 Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0 445 - 142.4.5.115 445 - 145 - HTTP/1.1 0 GET - - 80 7216 ?/><!--VAIBS--> GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 - 0 /phpmyadmin - 354 0
2 142.4.5.115 - - 2018-01-03 12:10:23 GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 400 Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0 445 - 142.4.5.115 445 - 145 - HTTP/1.1 0 GET - - 80 7216 ?/><!--VAIBS--> GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 - 0 /phpmyadmin - 354 0
3 142.4.5.115 - - 2018-01-03 12:12:41 GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 400 Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0 445 - 142.4.5.115 445 - 145 - HTTP/1.1 0 GET - - 80 7216 ?/><!--VAIBS--> GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 - 0 /phpmyadmin - 354 0
4 142.4.5.115 - - 2018-01-03 12:15:29 GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 400 Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0 445 - 142.4.5.115 445 - 145 - HTTP/1.1 0 GET - - 80 7216 ?/><!--VAIBS--> GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1 - 0 /phpmyadmin - 354 0
or something similar. Best of all, will be some help how to write write.table in loop without error. But I will welcome any functional solution. Best what I have is :
sql <- paste("SELECT * FROM idsaccess ORDER BY cookie LIMIT ", awq[1], ",1")
nb <- dbGetQuery(mydb, sql)
write.table(nb, file = fileConn, append = TRUE, quote = FALSE, sep = " ", eol = "\n", na = "NA", row.names = FALSE, col.names = FALSE)
fileConn<-file("outputX1.txt")
sql <- paste("SELECT * FROM idsaccess ORDER BY cookie LIMIT ", awq[2], ",1")
nb <- dbGetQuery(mydb, sql)
write.table(nb, file = fileConn, append = true, quote = FALSE, sep = " ", eol = "\n", na = "NA", row.names = FALSE, col.names = FALSE)
But this give every query to own file. And I don't want have every query in own file. Any help ?

Answer 1

Simply concatenate all query dataframes into one large dataframe since they all share same structure, and then output to file in one call which is really the typical way to use write.table or its wrapper, write.csv:
Specifically, turn for loop:
for (i in 1:length(awq)){
sql <- paste("SELECT * FROM mtable ORDER BY cookie LIMIT ", awq[i], ",1")
nb <- dbGetQuery(mydb, sql)
}
Into lapply for a list of dataframes:
df_list <- lapply(1:length(awq), function(i) {
sql <- paste0("SELECT * FROM mtable ORDER BY cookie LIMIT ", awq[i], ",1")
nb <- dbGetQuery(mydb, sql)
})
Then, row bind with do.call to stack all dfs into a single dataframe and output to file:
final_df <- do.call(rbind, df_list)
write.table(final_df, file = "outputX.txt", append = true, quote = FALSE, sep = " ",
eol = "\n", na = "NA", row.names = FALSE, col.names = FALSE)