I am coming across issues using the SSHHook class in a DAG.
The error is
File "/usr/local/airflow/.local/lib/python3.7/site-packages/paramiko/pkey.py", line 307, in _read_private_key_file
with open(filename, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: '-----BEGIN RSA PRIVATE KEY-----\n'
What I am doing is taking a PEM key from AWS Secrets Manager and writing it to a file-like object using StringIO.
secrets_manager_hook = SecretsManagerHook()
sm_client = secrets_manager_hook.get_conn()
secret = sm_client.get_secret_value(SecretId='<SECRET>')
pem_key_value = secret["SecretString"]
with StringIO(initial_value=pem_key_value) as pem_file:
ssh_hook=SSHHook(ssh_conn_id=None, remote_host=<HOST>, username='ec2-user', key_file=pem_file)
ssh_hook_conn=ssh_hook.get_conn()
ssh_hook.exec_ssh_client_command(ssh_client=ssh_hook_conn, command='echo Hello', get_pty=False)
My PEM key that I uploaded as just a plain text secret looks like this
-----BEGIN RSA PRIVATE KEY-----
######
...
...
######
-----END RSA PRIVATE KEY-----
If I print the secret value retrieved from the Boto3 client it is a similar output.
I'm not sure of what the error is related to. Do I explicitly need new line characters \n at the end of every line? How can I implement that? StringIO seems like it already has newline='\n' as a default parameter.
The key_file takes file path. It does not accept a file-like object.
The only way to provide in-memory key is using private_key "extra option" in connection specified by ssh_conn_id.
Related
i'm trying to connect esp32 to firestore but an error appear on the serial monitor
Token info: type = OAuth2.0 access token, status = error
Token error: code: -116, message: mbedTLS, mbedtls_pk_parse_key: PK - Invalid key tag or value
i'm using source code from https://github.com/mobizt/Firebase-ESP-Client/blob/main/examples/Firestore/ImportDocuments/ImportDocuments.ino
Most likely you've not provided the private key string correctly (TLS libraries are rather picky about that), or the private key is not in correct format.
E.g. this is how you generate a 2048 bit RSA keypair in PKCS8 format (which is what the sample code appears to use):
$ ssh-keygen -t rsa -b 2048 -m PKCS8 -f testkey
This is how you add the resulting private key as a multiline raw string literal in C++:
const char PRIVATE_KEY[] = R"(-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDhhA52G0UQGFk5
+WDGpWXgVHR6Bjpyq81HZ0SI9PQYTZa87b4qOysL8QkpeXiUR5Ltza1A5ceLERfy
2fpwhLIdHDmFN9dYQIN7ODf8dPRFVPT9ABegHDGzNWQJBuDe9ffVfF8DYeSlo7iq
vKUSeL2IpbTFn7Ccard68Wee3GueIm0GVi6V/HKhJYT4Cq6Fp8g/xVMdwAD6AFR8
8vBH1FERJn+6U89xdgewnjrw69jhlctziIdDxANedqTX/aEkchKcFhhmXehwsFfn
3gdToJBu0fClaTUvm0tYMRLaoUfAQ9t6J9TtPkA3ZZFzge7FZGl3lpFd6NOiiiQD
Fi9NLXx7AgMBAAECggEANNV1deIpqf2qWdS9AY0acxYWO8hdxSPO8h2Vhuisj00O
NbEVTluSWDfeR6M3xrWGDMSVBJFQ0JnunGzJKbdM61kDeNx+TSTAKQwgwbq8x8mB
pAxzfIVed3fYueysnk/OAmwi7m3u3Yef6hBMNCmpbew0j+A7xkmSNvF7r1JAvEkv
wnRy4GY9+39HHotF0T2BofdmxA0i5hS7NQc5AxJ+O9xVB4qoUCoPDXsOXfQoGURe
ugR7ANSONwIbnfSkcTanoMeqaQeSpLe+cNlBKTG+ofQyOsM2vSbPndUNx+CerLPe
Jya+/owBgYCTeOG+E+Bs05God/y/HuX5hUJVHSQHIQKBgQD7Z9XFf6Ecs8oGHx6g
D4qWNxLolB9Ls6W9MkXrrTcPMWQJIbYM2RxNLVjdArVLDT7yUHUH/KWx/Qrd9x5P
g8H8htO2vSHJRBpFbxL5wYudsh6G38GOs65OEcjKMWryzet44bRPe9zkULbfqiwy
5KOyi4VoUc63HeiKTJbHaTwWaQKBgQDloxmU22G7Vno7BhmReH4veZ+yV2jnX8Uc
cp4ciq7XqPZNUMKTskmrLl6Lw9I2xY8ahdP0kYWB6CKJr9DF8TN09n4NN2yLz89o
mHowHCdsyjE51l7lmkdHSxCOVt+Wt4vsoegZAydm8mtAfA0W63tgMDgKpCyqtn/H
vl6v08jHQwKBgQDwtO48doKvlIRtcAZGghrS0FBOVuPP+2VIzIWdrO72oPeCtsAz
dPxf+HcIxLsH7NrD7rzetMuAqyJoe16OCz0Ep6K1aEidB5OPv1WPw/wFNLF9uWb9
VpQhM23oCYC4kOhPiOq234UTJSSy6Z//6mHBZUyKISmmsgAfykzjCKL7YQKBgDNQ
qCXIhGVtpLDN4ybLyz7mqdYbjFiv4QdDYau1He8ZVbjfRCvnGmlDWmNXJgGRUjI2
F7A+WJw3+IfWHwwsozgO4QopCXuQu6N+90cK85Bu+mMXUWtzd1wqHRABv8MZTw5Y
+9mjB8oMgAUP84jcpFxRSiSunUCkuZc0JkqaQ/KrAoGAYxtl+555GpXK9eLfrVOG
02czqaXssGXIi31aGTICy7Esk8OgaG+pd4brcICeYE/CQ6gDjt6m/dHcw+DFmeFX
nVDHCIIDB9SeHM0Z3wZi/JruCtoB6vTxhA5lA9QeI4TpZvYXQkuerRUzRvhxyzkV
PrYun0MrVLxiiwezSTxlyGw=
-----END PRIVATE KEY-----
)";
My Postgres DB in GCP (Google Cloud Platform) only accepts connections over SSL.
I tried the below inside my node.conf without any success:
dataSourceProperties {
dataSourceClassName = "org.postgresql.ds.PGSimpleDataSource"
dataSource.url = "jdbc:postgresql://db-private-ip:5432/my_node"
dataSource.ssl = true
dataSource.sslMode = verify-ca
dataSource.sslRootCert = "/opt/corda/db-certs/server-ca.pem"
dataSource.sslCert = "/opt/corda/db-certs/client-cert.pem"
dataSource.sslKey = "/opt/corda/db-certs/client-key.pem"
dataSource.user = my_node_db_user
dataSource.password = my_pass
}
I'm sure that the keys (sslMode, sslRootCert, sslCert, and sslKey) are acceptable in node.conf (even though they are not mentioned anywhere in Corda docs), because in the logs I didn't get any errors that those key are not recognized.
I get this error when I try to start the node:
[ERROR] 21:58:48+0000 [main] pool.HikariPool. - HikariPool-1 - Exception during pool initialization. [errorCode=zmhrwq, moreInformationAt=https://errors.corda.net/OS/4.3/zmhrwq]
[ERROR] 21:58:48+0000 [main] internal.NodeStartupLogging. - Could not connect to the database. Please check your JDBC connection URL, or the connectivity to the database.: Could not connect to the database. Please check your JDBC connection URL, or the connectivity to the database. [errorCode=18t70u2, moreInformationAt=https://errors.corda.net/OS/4.3/18t70u2]
I tried adding ?ssl=true to the end of the data source URL as suggested in (Azure Postgres Database requires SSL Connection from Corda) but that didn't fix the problem.
Also for the same values I'm able to use the psql client to connect my VM to the DB:
psql "sslmode=verify-ca sslrootcert=server-ca.pem sslcert=client-cert.pem sslkey=client-key.pem hostaddr=db-private-ip user=some-user dbname=some-pass"
Turns out the JDBC driver cannot read the key from a PEM file, it has to be converted to a DER file using:
openssl pkcs8 -topk8 -inform PEM -in client-key.pem -outform DER -nocrypt -out client-key.der
chmod 400 client-key.der
chown corda:corda client-key.der
More details here: https://github.com/pgjdbc/pgjdbc/issues/1364
So the correct config should look like this:
dataSourceProperties {
dataSourceClassName = "org.postgresql.ds.PGSimpleDataSource"
dataSource.url = "jdbc:postgresql://db-private-ip:5432/db-name"
dataSource.ssl = true
dataSource.sslMode = verify-ca
dataSource.sslRootCert = "/opt/corda/db-certs/server-ca.pem"
dataSource.sslCert = "/opt/corda/db-certs/client-cert.pem"
dataSource.sslKey = "/opt/corda/db-certs/client-key.der"
dataSource.user = db-user-name
dataSource.password = db-user-pass
}
Given that PGP supports encryption using multiple public keys, how it can be achieved in mulesoft?
adding two keys in Public Key Ring File Name field on encryption config window for pgp encrypter tab results in error as org.mule.module.pgp.exception.MissingPGPKeyException: No key file found in: abc.gpg,test.pgp
Is it possible to add multiple public keys from this encrypt module or else how it can be done? mule runtime : 3.8.5
much appreciate any help. Thank you!
I solved this by using java component- invoking terminal and running gpg encrypt command from java runtime. I am just checking for OS first to build a command string to run its respective terminal
boolean isWindows = System.getProperty("os.name")
.toLowerCase().startsWith("windows");
/*gpg command options may vary as per your requirement. multiple --recipient option here is the way to encrypt with multiple public keys.
Using StringBuilder helps to build this string from input/dynamic values.
*/
String command = "gpg --pgp6 --armor --batch --output encryptedHelloWorld.pgp --trust-model always --recipient "<part of UserID1 (either name or emailId)>" --recipient "<part of UserID2>" --encrypt helloWorld.txt"
/*in case you need to change directory to where your file is to encrypt it from one command, you could append this
`"cd"+ <your path to file> "&" + command` ----> for windows
`"cd"+ <your path to file> ";" + command` ----> for linux
*/
public int executeCommand(String command) throws IOException, InterruptedException {
Process pr;
if (isWindows) {
String[] cmd = { "cmd.exe", "/c", command };
pr = Runtime.getRuntime().exec(cmd);
}
else {
String[] cmd = { "/bin/sh", "-c", command };
pr = Runtime.getRuntime().exec(cmd);
}
int exitStatus = pr.waitFor(); // this gives you value 0 if success or other than 0 which ties to error message
errorInputStream = pr.getErrorStream(); //streaming error message
return exitStatus;
}
I try to deploy a datapower inside IPC with the following parameter
{
"ibm-datapower-dev": {
"name": "ibm-datapower-dev",
"crypto.frontsideCert": "-----BEGIN CERTIFICATE REQUEST----- MIICijCCAXICAQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUx ITAfBgNVBAoMGEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDCCASIwDQYJKoZIhvcN AQEBBQADggEPADCCAQoCggEBAK2cJ7rtqGPsFoFNLjOQ08knGO429u+EdIEPiXvc nvumhIoSmOquUOzyIOqPCY35Wso2EjS97dH3Qa+X8Qo8L6FfmWDBn4C6RWVPxxh1 a2P0TM1eSzPP/bVU0yghgdgPkU6jYfA0OmnBN3KOHl39gc86cI8tXUxstnlACzv+ me1RKmvMfy+IE3C0qBvvPjhNjIU4vx9OJ/DaxMh2shJxHlt718mZTCy/fAqE5bc5 19Atxy7aPD9LFWQ6/mGbnLzH8l4MqrZYFkVkZx8h3uYt3eyN0co4FicBuGtgs97s VcpZz9tC2yFEi25Iql+AdGuaK+hMNU2pF5GdNaARlyLjemECAwEAAaAAMA0GCSqG SIb3DQEBCwUAA4IBAQBvGtz0xYQOX1SqTdutLLKSJdlUt3jin3NoEvvDKAHhmpnc DDMfamqM4wjXb1JlTQKSHAdgLB/KHMgJPlb4pn9pHVCsZ9cEwz0Qz0oGPyr/rmYC IZRfmImqKdDLDhVWyK8GLv6cYwQEkuRMzytXX8DMcJcWRXuJj3cuyVGQ7BKxSnl7 0U9pbUb8mNwcvJnZfsfL4AIQR2//NoTVI+XKXg9INoJXa4WaWXAYR4QlV1+Ow52P uGnBaCsCx+JbD+2hhbbGJNzPqG1ZaZUEvnQiqOTcVseeGg6uLZ4KAGaJ52ZVWcmT Lmt+TMkML8Ajm11mJnqJIclMXs61rsHVPHbjVl5q -----END CERTIFICATE REQUEST-----",
"crypto.frontsideKey": "-----BEGIN RSA PRIVATE KEY----- MIIEpAIBAAKCAQEArZwnuu2oY+wWgU0uM5DTyScY7jb274R0gQ+Je9ye+6aEihKY 6q5Q7PIg6o8JjflayjYSNL3t0fdBr5fxCjwvoV+ZYMGfgLpFZU/HGHVrY/RMzV5L M8/9tVTTKCGB2A+RTqNh8DQ6acE3co4eXf2Bzzpwjy1dTGy2eUALO/6Z7VEqa8x/ L4gTcLSoG+8+OE2MhTi/H04n8NrEyHayEnEeW3vXyZlMLL98CoTltznX0C3HLto8 P0sVZDr+YZucvMfyXgyqtlgWRWRnHyHe5i3d7I3RyjgWJwG4a2Cz3uxVylnP20Lb IUSLbkiqX4B0a5or6Ew1TakXkZ01oBGXIuN6YQIDAQABAoIBAEI3znuhDFpYgVj+ PEIU2oLd88dglZ1T8zAK4hCpXMZ1hBY587OHm3xU+jMdLbLGSlfYeec+DkjIu3gj Rx/1RuGRWioqkBEEe1WiMTlmB/kPBIBMl/iCO63/Va734NOtVGofLUr2UNVniiOa i4CkIfANxUMHNY/h41OqFt7iRInci3ILaHBwvLQLUc0ZHZcP1jHH/6zLxsx7ueap XbOFOrRgVt620HIDBtCntJBO24KN3TveKuLSrZPGU3G/dilOEA8kTdl9ftD4Qgbe Y4sZ0sz72WjHNSVOrdTNfyLKPOmbTs+qGW7SZWOWkDmvmLO5fu6fPWnjZqTikaUz ZWfuhJUCgYEA3x1+573AblgZGshKWFa33dYHvCOVjaqUfbNyvqWEA64vl/VBhOGU 8WUFAVlqX5tN1NxUF9KLF5Ed2iIRb8d1VUb4gTCs8BVX1rJN2VFzYIBwr66G/9bO KHAH2ABq7PgRWxEBvsvgbD6gwPAgWtx+KiSGaX1V3qvmASoz1Vj4wMcCgYEAxzK+ 1YngK9Jc4JgJMYqkfRHKwjBq99CzKJHKtXzaHVqcFmH2OqmwGWb4t0DyQsi5PKVM u4OFS3/vgSKpEwem94dB6OwuAioeX6pSkYW4qGLhOm9V9iPrjg/Q4v5+ym6iNMB+ 0CZdeR1TGI1qLFkv8ziSdN1K2Ycv2fYGSRpNE5cCgYEAgTG2M4C39e1DlDOYgTCt xWHTFslQJzk4RNVtQyHaoLrzSj21E7oAIgvJ6y8YnoXyeqiedTDwY9QfAhmqGRzZ P2kaKszBxz/EnkifNZCpi560Ibag63I57EZ69EPBprg6bI4bgZzStjtJoI8rXRHC aKq/vkPaPlcxl0kVvJuJZy8CgYEApqw7KQl4XLJjrDkKWD2dbFjawqkol7o0bhmu 9zREfJM4TzIgiO06v8Z9DTh2fJLfC3N0ROHDQm0FxZNuzNF2T18JMw+LX5xGVd15 wPGWlK5HrlFCJ/XePrvVGFnBVThE2MbIVPoE9DYpNT3+PKVTjbskMEyJOIH48/L7 R2eOXnkCgYBRCjh9ns9PSneKWGfz+ymlB0b4kCZBU69Q3ladgN50oxG6QmiccObT EXEeUl+XqhlhyR7MeNSz7IUugHSVKqpa2fGFTQ3Uk7k6QslSXHcJFl3HBjK/Ejcc H8zB9FPVe1gOqeK2HBnThC7zWOseGrBMWhyeH2cNdGXnw+dexWUTig== -----END RSA PRIVATE KEY-----",
"datapowerEnv.workerThreads": "4",
"image.pullPolicy": "IfNotPresent",
"image.repository": "ibmcom/datapower",
"image.tag": "7.6.0",
"patternName": "webApplicationProxy",
"replicaCount": "1",
"resources.limits.cpu": "4",
"resources.limits.memory": "8Gi",
"resources.requests.cpu": "2",
"resources.requests.memory": "4Gi",
"service.name": "datapower",
"service.type": "NodePort",
"webApplicationProxy.backendURL": "https://www.ibm.com",
"webApplicationProxy.containerPort": "8443"
}
}
and it failed (and no log info is available)
Just to complete info I generate the cert/key inside linux with the following command
generate key
openssl genrsa -out /tmp/hostname.key 2048
genrate cert
openssl req -new -key /tmp/hostname.key -out /tmp/hostname-2017.req
what is wrong ? where info/log is available to understand the error ?
The only way I can get the ICP GUI to accept my keys and certs is to strip out the header, footer, and line feeds. Only one continuous, encrypted string remains.
You should base64 encode the key and cert. They'll be used as a k8s secret, and those are required to be base64 encoded.
First, let's understand what these values are doing.
Each of the keys crypto.frontsideCert and crypto.frontsideKey map to values used in a Kubernetes secret. You can actually see how these values are substituted in the chart:
https://github.com/IBM/charts/blob/master/stable/ibm-datapower-dev/templates/secrets.yaml#L7-L8
And to provide a secret to Kubernetes, the value must be base-64 encoded. While the secret isn't really encrypted, it will be obscured from casual observation.
You can base64 encode these strings using base64 on your favorite linux distro:
cat /tmp/hostname.key | base64
# Outputs a large block of text, which typically ends in "==" cat
cat /tmp/hostname-2017.req | base64
# Outputs a large block of text, which typically ends in "=="
If you're ever concerned that the value in a secret isn't what you expect, you can always verify it:
echo "This is an obscured secret." | base64
VGhpcyBpcyBhbiBvYnNjdXJlZCBzZWNyZXQuCg==
echo VGhpcyBpcyBhbiBvYnNjdXJlZCBzZWNyZXQuCg== | base64 --decode
This is an obscured secret.
I've created public and private keys using PuttyGen then the public key was used to encrypt a message.
I am now trying to decrypt such message using the private key with the following command:
echo [my encrypted message] | openssl enc -d -base64 -A | openssl rsautl -decrypt -inkey ~/.ssh/private.ppk
but I get the error:
unable to load Private Key
6870300:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:707:Expecting: ANY PRIVATE KEY
The private key looks like this (actual key omitted):
PuTTY-User-Key-File-2: ssh-rsa
Encryption: none
Comment: rsa-key-20170724
Public-Lines: 6
[...]
Private-Lines: 14
[...]
Private-MAC: [...]
What's wrong?
Thanks to this guy I sorted it out.
I've reopened the private key with PuttyGen and exported as OpenSSH, then used this new file to decrypt the message.