Unable to connect Corda node to Postgres with SSL - corda

My Postgres DB in GCP (Google Cloud Platform) only accepts connections over SSL.
I tried the below inside my node.conf without any success:
dataSourceProperties {
dataSourceClassName = "org.postgresql.ds.PGSimpleDataSource"
dataSource.url = "jdbc:postgresql://db-private-ip:5432/my_node"
dataSource.ssl = true
dataSource.sslMode = verify-ca
dataSource.sslRootCert = "/opt/corda/db-certs/server-ca.pem"
dataSource.sslCert = "/opt/corda/db-certs/client-cert.pem"
dataSource.sslKey = "/opt/corda/db-certs/client-key.pem"
dataSource.user = my_node_db_user
dataSource.password = my_pass
}
I'm sure that the keys (sslMode, sslRootCert, sslCert, and sslKey) are acceptable in node.conf (even though they are not mentioned anywhere in Corda docs), because in the logs I didn't get any errors that those key are not recognized.
I get this error when I try to start the node:
[ERROR] 21:58:48+0000 [main] pool.HikariPool. - HikariPool-1 - Exception during pool initialization. [errorCode=zmhrwq, moreInformationAt=https://errors.corda.net/OS/4.3/zmhrwq]
[ERROR] 21:58:48+0000 [main] internal.NodeStartupLogging. - Could not connect to the database. Please check your JDBC connection URL, or the connectivity to the database.: Could not connect to the database. Please check your JDBC connection URL, or the connectivity to the database. [errorCode=18t70u2, moreInformationAt=https://errors.corda.net/OS/4.3/18t70u2]
I tried adding ?ssl=true to the end of the data source URL as suggested in (Azure Postgres Database requires SSL Connection from Corda) but that didn't fix the problem.
Also for the same values I'm able to use the psql client to connect my VM to the DB:
psql "sslmode=verify-ca sslrootcert=server-ca.pem sslcert=client-cert.pem sslkey=client-key.pem hostaddr=db-private-ip user=some-user dbname=some-pass"

Turns out the JDBC driver cannot read the key from a PEM file, it has to be converted to a DER file using:
openssl pkcs8 -topk8 -inform PEM -in client-key.pem -outform DER -nocrypt -out client-key.der
chmod 400 client-key.der
chown corda:corda client-key.der
More details here: https://github.com/pgjdbc/pgjdbc/issues/1364
So the correct config should look like this:
dataSourceProperties {
dataSourceClassName = "org.postgresql.ds.PGSimpleDataSource"
dataSource.url = "jdbc:postgresql://db-private-ip:5432/db-name"
dataSource.ssl = true
dataSource.sslMode = verify-ca
dataSource.sslRootCert = "/opt/corda/db-certs/server-ca.pem"
dataSource.sslCert = "/opt/corda/db-certs/client-cert.pem"
dataSource.sslKey = "/opt/corda/db-certs/client-key.der"
dataSource.user = db-user-name
dataSource.password = db-user-pass
}

Related

Airflow - MSSQL connection works on UI but fails in DAG run

On Airflow 2.2.4 (Postgres and Celery).
I have a connection created for Microsoft SQL Server. When I try to test the connection, UI asked me to enter API Auth user/password (Basic Auth) and showed up a green flash "Connection successfully tested"
But,
When I use the same connection ID in an Operator definition to run some SQL queries, I am getting an error as below.
pymssql._mssql.MSSQLDatabaseException: (20009, b'DB-Lib error message 20009, severity 9:\nUnable to connect: Adaptive Server is unavailable or does not exist (<IP>)\nNet-Lib error during Connection timed out (110)\n
DB-Lib error message 20009, severity 9:\nUnable to connect: Adaptive Server is unavailable or does not exist (<IP>)\n
Net-Lib error during Connection timed out (110)\n')
Here is what my code from my custom operator looks like,
def get_update_num(self):
conn = None
try:
query = f"""
UPDATE SOME_TABLE
SET COL_A = COL_A+1
OUTPUT INSERTED.COL_A
WHERE COL_ID='12345'
self.log.info(f"SQL = {query}")
conn_id = self.conn_id # This is equal to MSSQL_CONNECTION
hook = MsSqlHook(mssql_conn_id=conn_id)
conn = hook.get_conn()
hook.set_autocommit(conn, True)
cursor = conn.cursor()
cursor.execute(query)
row = cursor.fetchone()
self.log.info(f"row = {row}")
return row[0]
except Exception as e:
message = "Error: Could not run SQL"
raise AirflowException(message)
finally:
if not conn:
conn.close()
Any help would be much appreceated.

Connection reset by peer when trying to connect to Postgres via SSL using R

I'm trying to connect to a database via SSL using the code suggested here: https://github.com/ropensci/ssh/issues/13
I listed the dummy code below that shows how I enable the connection and try to query some data. The solution works great for 'smaller' queries.
However, when I try to get 'larger' data, the query fails and R gives back the following error:
System failure for: recv() from user (Connection reset by peer) follwed by a fetching error Failed to fetch row: SSL error: decryption failed or bad record mac (see output in code snipped)
Accordingly to the 1st error message, I suppose the error occurs served-sided ('reset by peer' --> What does "connection reset by peer" mean?).
Is that true or is there a way to fix this error on the client side (in R)?
ssh::ssh_read_key(file = ssh::ssh_home("id_rsa"), password = "rsa_password")
cmd <- "session <- ssh::ssh_connect('user#host:port');ssh::ssh_tunnel(session, port = 5432, target = '127.0.0.1:5432')"
pid <- sys::r_background(std_out = T, args = c("-e", cmd))
dbcon <-DBI::dbConnect(drv = RPostgres::Postgres(),
dbname = "db_name",
host = "127.0.0.1",
port = 5432,
user = "db_user",
password = "db_password",
base::list(sslmode="require"),
service = NULL)
# example of working query
res <- DBI::dbGetQuery(conn = dbcon, statement = "SELECT * FROM small_table") #
# example of non-working query (see R-otutput)
res <- DBI::dbGetQuery(conn = dbcon, statement = "SELECT * FROM large_table;") #
## R-output
# Tunneled 31897311 bytes...Fehler: System failure for: recv() from user (Connection reset by peer)
# Ausführung angehalten
# Fehler: Failed to fetch row: SSL error: decryption failed or bad record mac
# Warnmeldung:
# Disconnecting from unused ssh session. Please use ssh_disconnect()
"Connection reset by peer" means that whatever you have tried connecting to has responded in an RST flag, meaning that they have reset the connection.

Is it possible to use Oracle Wallet for node connection to its vault?

Is it possible to start an enterprise node, using an Oracle 12c backed up
vault configured via Oracle Wallet (i.e., configure the node.conf using only a
dataSource.url="jdbc:oracle:thin:#host:port:#SOME_ORACLE_WALLET_TNS" without
specifying any dataSource.username or dataSource.password parameters ) ?
In this case, please inform which additional oracle .jar files should be
added to the node drivers dir.
Oracle wallet is supported in Corda Enterprise. Below is a working configuration with Oracle Wallet, and has been tested with Oracle 11g and Oracle 12c.
Prerequisites
Oracle wallet is configured with auto login (-auto_login_local) for the node's database
Assuming that the node's database connection URL is configured in tnsnames.ora, with alias "db11g":
$ echo 'For JDBC URL ==> "jdbc:oracle:thin:#localhost:1521/xe" ==> below is an example of tnsnames.ora'
$ cat ~/oracle_experiment/tnsnames.ora
db11g =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = localhost)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = xe)
)
)
Wallet location is configured in sqlnet.ora:
$ cat ~/oracle_experiment/sqlnet.ora
WALLET_LOCATION =
(SOURCE =
(METHOD = FILE)
(METHOD_DATA =
(DIRECTORY = /Users/corda/oracle_wallet/)
)
)
SQLNET.WALLET_OVERRIDE = TRUE
SSL_CLIENT_AUTHENTICATION = FALSE
SSL_VERSION = 0
Sqlplus should be able to login without a password challenge:
sqlplus /#db11g
SQL*Plus: Release 12.2.0.1.0 Production on Tue Nov 27 15:17:00 2018
Copyright (c) 1982, 2017, Oracle. All rights reserved.
Last Successful login time: Tue Nov 27 2018 14:46:09 +08:00
Connected to:
Oracle Database 12c Standard Edition Release 12.1.0.2.0 - 64bit Production
SQL>
Necessary steps
Change the database-specific configuration from:
$ cat dbconfig_oracle11g.conf
dataSourceProperties = {
dataSourceClassName = "oracle.jdbc.pool.OracleDataSource"
dataSource.url = "jdbc:oracle:thin:#localhost:1521/xe"
dataSource.user = corda_es_user
dataSource.password = corda_es_passwd
}
To
$ cat dbconfig_oracle_wallet.conf
dataSourceProperties = {
dataSourceClassName = "oracle.jdbc.pool.OracleDataSource"
dataSource.url = "jdbc:oracle:thin:/#db11g"
dataSource.user=null
dataSource.password=null // user and password can't be ignored and can't be left blank.
}
Download and copy the following JARs into the node's drivers folder:
]$ ls <corda>/drivers/
ojdbc8.jar
osdt_cert.jar
osdt_core.jar
oraclepki.jar
Start the node with the oracle.net.wallet_location and oracle.net.tns_admin options:
]$ java -Doracle.net.wallet_location=/Users/corda/oracle_wallet/ -Doracle.net.tns_admin=/Users/corda/oracle_experiment/ -jar corda.jar
Refer to the SSL with JDBC blog for more details.

R connection to postgresql requiring SSL

I'm trying to connect R to a postgresql database that requires SSL.
Here's the connection string that works for me when using PSQL:
postgresql://USERNAME:PASSWORD#HOSTNAME:PORT/DBNAME?ssl=true, replacing all the uppercase strings with appropriate values.
In R, I don't know how to handle the SSL parameter. When I try
conn = dbConnect(RPostgreSQL::PostgreSQL(), host="HOST", dbname="DBNAME", user="USERNAME", password="PASSWORD", port=PORT)
I get
Error in postgresqlNewConnection(drv, ...) :
RS-DBI driver: (could not connect USERNAME#HOST on dbname "DBNAME"
)
When I try conn = dbConnect(RPostgreSQL::PostgreSQL(), host="HOST", dbname="DBNAME", user="USERNAME", password="PASSWORD", port=PORT, ssl=TRUE)
I get
Error in postgresqlNewConnection(drv, ...) : unused argument (ssl = TRUE)
Connect to Postgres via SSL using R suggests adding extra info to the dbname parameter. I've tried dbname="DBNAME ssl=TRUE" which results in RS-DBI driver: (could not connect (null)#HOST on dbname "(null)" I get the same result with sslmode=allow and sslmode=require (as suggested by the above post).
The documentation for the PostgreSQL driver says, under "User Authentication", "The passed string can be empty to use all default parameters, or it can contain one or more parameter settings separated by comma. Each parameter setting is in the form parameter = "value". Spaces around the equal sign are optional." But I haven't been able to get it to accept any parameters other than the three shown in the function prototype.
I'm out of ideas; help appreciated.
You can try this :
Create a configuration file like "configuration.yml" and add your setup :
db:
host : "your_localhost"
dbname : "your_database_name?ssl=true"
user : "your_user_name"
port : 5432
password : "your_password"
Install this packages :
install.packages(yaml, dependencies = TRUE)
install.package(RPostgreSQL, dependencies = TRUE)
install.packages(DBI, dependencies = TRUE)
Run :
driver <- DBI:::dbDriver("PostgreSQL")
con <- do.call( RPostgreSQL:::dbConnect,
c(drv = driver, yaml:::yaml.load_file("configuration.yml")$db)
)
/!\ Note /!\ : don't forget to add this statement
*?ssl=true*
on the configuration.yml file on the dbname field.
Hope this help !

oracle odbc driver configuration

I'm having a problem with configuring oracle odbc
the dialog page is blank
when I enter TNS name as : XE
I get the following error:
unable to connect SQLState=08004
my tnsnames file is:
KPI_SERVER=
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST =localhost)(PORT =1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = XE)
)
)
the connection is successful in SQL developer by the following data:
hostname: localhost
port number: 1521
service: XE
and the trns_admin variable is set to: C:\oracle_odbc\tnsnames
Path is set to: C:\oracle_odbc
what did I do wrong?
thank you for your time
We try to make order: open a command prompt,
launch echo %TNS_ADMIN% the result is C:\oracle_odbc\?
launch dir C:\oracle_odbc\, the result is tnsnames.ora?
launch type C:\oracle_odbc\tnsnames.ora the result is the content of "my tnsnames file is" section of your initial post?
If all the response are yes, can you retry to lauch 'sqlplus.exe dbuser/dbpassword#KPI_SERVER

Resources