Cannot find function src_dbi in R - r

I am trying to use R to access postgresql db on Heroku, and I found that I can use src_dbi from dplyr package.
I have dplyr properly installed, but when I try to call src_dbi I get an error message:
Error in src_dbi(db_con) : could not find function "src_dbi"
This happens right when I run:
db <- src_dbi(db_con)
after poviding the credentials:
config <- run("heroku", c("config:get", "postgres://xxxxxxxxxxxxxx
", "-a", "prjectAlpha"))
pg <- httr::parse_url(config$stdout)
dbConnect(RPostgres::Postgres(),
dbname = "xxxxxxxxxx",
host = "xxxxxxxxx.amazonaws.com",
port = 5432,
user = "xxxxxx",
password = "xxxxxxxxxxxxxxxxx",
sslmode = "require"
) -> db_con
The idea is to be able to download a table and re-upload it after making a few changes with R.

My solution to access Heroku Postgresql from R:
library(dbplyr) #in case you have an error, run: system("defaults write org.R-project.R force.LANG en_US.UTF-8") from Rails console, then restart R.
library(processx)
library(RPostgres)
library(httr)
library(tidyverse)
library(dplyr)
config <- run("heroku", c("config:get", "postgres://xxxxxxxxxxxxxx
", "-a", "prjectAlpha"))
pg <- httr::parse_url(config$stdout)
dbConnect(RPostgres::Postgres(),
dbname = "xxxxxxxxxx",
host = "xxxxxxxxx.amazonaws.com",
port = 5432,
user = "xxxxxx",
password = "xxxxxxxxxxxxxxxxx",
sslmode = "require"
) -> db_con
Once the connection is set up:
db <- src_dbi(db_con)
Once the connection is established, check for the available tables
db
Now time to retrive data
A lot of examples only show treatment directly from the collection, but one might just be interested to read the tables. I have a table called "weather_records"
weather_records_local_df <- tbl(db_con, "weather_records")
df <- collect(weather_records_local_df)
Then do what you want with the data. Hope it helps.

Related

RPostgreSQL - SCRAM error when trying to connect to local database

I am trying to connect to my localhost postgres DB and I get the following error.
library("RPostgreSQL")
drv <- dbDriver("PostgreSQL")
connec <- dbConnect(drv, dbname = "dbnamehere", port = 5432,user = "some_username", password = "somepassword")
Error in postgresqlNewConnection(drv, ...) :
RPosgreSQL error: could not connect admin_sci4i#localhost:5432 on dbname "website": SCRAM authentication requires libpq version 10 or above
It seems related to authentication security but I am having a local DB.. Is there any way to do anything in pgAdmin 4 and avoid this error (even if it is less secure)?
Surprisingly I managed to connect using other packages
library("RODBC")
library("odbc")
library("RPostgres")
con <- dbCanConnect(RPostgres::Postgres(),dbname="dbnamehere",port = 5432,user = "some_username", password = "somepassword")
con # Checks connection is working
con1 = dbConnect(RPostgres::Postgres(),dbname="dbnamehere",port = 5432,user = "some_username", password = "somepassword")
con1 # Checks connect
dbListTables(con1) # See tables
A nice explanation and walkthrough here.

Error in .local(drv, ...) : Failed to connect to database: Error: Can't connect to MySQL server on 'xx.143.13.xxx' (0)

I am getting erro in .local(drv,...).
I have no idea how to fix this issue.
Can anyone help me out why showing such a error and how to fix this error?
code:
library(RMySQL)
mydb = dbConnect(MySQL(), user='XXXXXXX',
password='XXXXXXXX', dbname='XXXXXXXX',
host='##.143.13.XXX', port=XXXX)
Thank you.
Try using DBI and pool packages from R.
library(DBI)
library(pool)
pool <- dbPool(drv = RMySQL::MySQL(), dbname = "dbName", host = "localhost", username = "root", password = "psw", port = 3306, unix.sock = "/var/run/mysqld/mysqld.sock")
df <- dbGetQuery(pool, "SELECT * FROM tablename;")
Provide MySQL socket path of the machine at unix.sock(in ubuntu : mysql_config --socket)

R connection to postgresql requiring SSL

I'm trying to connect R to a postgresql database that requires SSL.
Here's the connection string that works for me when using PSQL:
postgresql://USERNAME:PASSWORD#HOSTNAME:PORT/DBNAME?ssl=true, replacing all the uppercase strings with appropriate values.
In R, I don't know how to handle the SSL parameter. When I try
conn = dbConnect(RPostgreSQL::PostgreSQL(), host="HOST", dbname="DBNAME", user="USERNAME", password="PASSWORD", port=PORT)
I get
Error in postgresqlNewConnection(drv, ...) :
RS-DBI driver: (could not connect USERNAME#HOST on dbname "DBNAME"
)
When I try conn = dbConnect(RPostgreSQL::PostgreSQL(), host="HOST", dbname="DBNAME", user="USERNAME", password="PASSWORD", port=PORT, ssl=TRUE)
I get
Error in postgresqlNewConnection(drv, ...) : unused argument (ssl = TRUE)
Connect to Postgres via SSL using R suggests adding extra info to the dbname parameter. I've tried dbname="DBNAME ssl=TRUE" which results in RS-DBI driver: (could not connect (null)#HOST on dbname "(null)" I get the same result with sslmode=allow and sslmode=require (as suggested by the above post).
The documentation for the PostgreSQL driver says, under "User Authentication", "The passed string can be empty to use all default parameters, or it can contain one or more parameter settings separated by comma. Each parameter setting is in the form parameter = "value". Spaces around the equal sign are optional." But I haven't been able to get it to accept any parameters other than the three shown in the function prototype.
I'm out of ideas; help appreciated.
You can try this :
Create a configuration file like "configuration.yml" and add your setup :
db:
host : "your_localhost"
dbname : "your_database_name?ssl=true"
user : "your_user_name"
port : 5432
password : "your_password"
Install this packages :
install.packages(yaml, dependencies = TRUE)
install.package(RPostgreSQL, dependencies = TRUE)
install.packages(DBI, dependencies = TRUE)
Run :
driver <- DBI:::dbDriver("PostgreSQL")
con <- do.call( RPostgreSQL:::dbConnect,
c(drv = driver, yaml:::yaml.load_file("configuration.yml")$db)
)
/!\ Note /!\ : don't forget to add this statement
*?ssl=true*
on the configuration.yml file on the dbname field.
Hope this help !

RPostgreSQL can't connect

I'm having trouble connecting my R client to redshift through the RPostgreSQL package, despite it working very easily through psql. I've tried downloading and sourcing the redshift-ssl-ca-cert.pem file, but this still doesn't seem to work. Any ideas what could be causing this? Here's my R code:
library("RPostgreSQL")
drv <- dbDriver("PostgreSQL")
host = 'host.com'
dbname = 'dbname'
port = 1234
password = 'password'
username = 'user'
redshift_cert = paste0(FILE_PATH, 'redshift-ssl-ca-cert.pem')
pg_dsn = paste0(
'dbname=', dbname, ' ',
'sslrootcert=', redshift_cert, ' ',
'sslmode=verify-full'
)
con <- dbConnect(drv, dbname=pg_dsn, host=host, port=port, password=password, user=username)
and I always get this error message
Error in postgresqlNewConnection(drv, ...) :
RS-DBI driver: (could not connect user#host.com on dbname "dbname"
)
Meanwhile this command using psql works perfectly
psql 'host=host.com dbname=dbname sslmode=require port=1234 user=user password=password'
I've also tried other variations of sslmode including require and allow for the R code, but nothing works. Would appreciate any ideas thanks!

How to send application_name to RPostgreSQL connection

I know that PostgreSQL database allows to keep track of application name for each connection and it looks like application_name variable should work with RPostgreSQL, but how exactly should I do this? Adding variable application_name = "test" to dbConnect doesn't work.
I'm not sure you can pass application_name='test' as an argument to dbConnect in RPostgreSQL (there is an optional options argument, but I couldn't figure out what kind of data it expects).
An alternative would be to run the following SQL query immediately after opening the connection:
SET application_name='test'
and it should work until you close the connection.
It's not well documented but something like this works:
drv <- dbDriver("PostgreSQL")
con <- dbConnect(drv,
dbname = "<my_db>",
host = "<my_host>",
[...],
options = "-c application_name=my_app_name"
)
It works with dbPool too (package 'pool'):
pool <- dbPool(
drv = dbDriver("PostgreSQL"),
dbname = "<my_db>",
host = "<my_host>",
[...]
minSize = 0,
maxSize = 3,
options = "-c application_name=my_app_name"
)
Check with pg_stat_activity
mydb=> SELECT state, usename, application_name FROM pg_stat_activity WHERE datname = 'mydb';
state | usename | application_name
--------+----------+------------------
active | postgres | psql
idle | myuser | my_app_name

Resources