I am using the S3cmd put with an -e flag to encrypt a file.
When I try to copy the file to Redshift (from S3) I get an error. I am using my passphrase converted to AES256 as my master key.
copy xxx
from 'xxx'
access_key_id 'xxx'
secret_access_key 'xxx'
master_symmetric_key 'xxx'
IGNOREHEADER 8
encrypted
delimiter ',';
I receive an error that looks like this:
ERROR: XX000: Failed writing body (0 != ) Cause: S3 object 'xxx' does not have 'x-amz-meta-x-amz-key' metadata
Related
I am using informatica, I have Singlestore DB which I am trying to connect.
I am able to login to singelstore DB using Singlestore ODBC Driver as below.
Singlestore version:8.0.5
SS ODBC Driver version: 1.1.1
Singlestore is self managed.
[abc#rnd-2 ~]$ isql SingleStore-server
+---------------------------------------+
| Connected! |
| |
| sql-statement |
| help [tablename] |
| quit |
| |
+---------------------------------------+
SQL> ^C
While I am trying to connect informatica with Singlestore using ODBC Connection, I am gettion error:
Message Code: WRT_8001
Message: Error connecting to database...
WRT_8001 [Session s_test Username dev DB Error -1
[DataDirect][ODBC lib] Driver Manager Message file not found. Please check for the value of InstallDir in your odbc.ini.
Database driver error...
Function Name : Connect
Database driver error...
Function Name : Connect
Database Error: Failed to connect to database using user [dev] and connection string [SingleStore-server].]Message Code: WRT_8001
Message: Error connecting to database...
WRT_8001 [Session s_test Username dev DB Error -1
[DataDirect][ODBC lib] Driver Manager Message file not found. Please check for the value of InstallDir in your odbc.ini.
Database driver error...
Function Name : Connect
Database driver error...
Function Name : Connect
Database Error: Failed to connect to database using user [dev] and connection string [SingleStore-server].]
My location of odbc.ini file: /etc/odbc.ini
odbc.ini
[SingleStore_server]
Description=SingleStore server
Driver=/home/abc/singlestore-connector-odbc-1.1.1-centos7-amd64/libssodbca.so
SERVER=<>
USER=<>
PASSWORD=<>
DATABASE=<>
PORT=<>
I added path in .bash_profile, but still getting same error:
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin
export PATH
export ODBCINI=/etc/odbc.ini
Pls let me know how to resolve this error.
Ref link: https://knowledge.informatica.com/s/article/577839?language=en_US
https://knowledge.informatica.com/s/article/Error-connecting-to-database-DataDirect-ODBC-lib-Driver-Manager-Message-file-not-found-Please-check-for-the-value-of-InstallDir-in-your-odbc-ini-while?language=en_US
https://docs.singlestore.com/managed-service/en/developer-resources/connect-with-application-development-tools/connect-with-odbc/the-singlestore-odbc-driver.html
Reg export ODBCINI=/etc/odbc.ini, I have seen Informatica always use their ODBC drivers. Can you please check if you have single store drivers available in /<INFA_HOME>/ODBCX.version/odbc.ini file? If yes, i highly recommend to use it.
If yes, please see if you can test the ODBC driver with Infa provided tool $INFA_HOME/tools/debugtools/ssgodbc -d dsn -u username -p password [-v] against your DB. This will ensure you have no issues with ODBC setup.
You can find all about this here link.
If no, then, pls make sure you have installed correct version single store ODBC drivers (32 or 64 bit) and Informatica user have RWX permission on them. Then,
Add the driver path to LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$HOME/server_dir:$ODBCHOME/lib;
set ODBCINI=/etc/odbc.ini
grant access - chmod 777 /etc/odbc.ini
see if the tool ssgodbc is able to establish connection.
Please see the following examples of integrating SingleStore data with Informatica:
JDBC - https://www.cdata.com/kb/tech/singlestore-jdbc-informatica-cloud.rst
ODBC - https://www.cdata.com/kb/tech/singlestore-odbc-informatica.rst
I am trying to configure remote github repo as the salt server root but it can't make the authentication successful with the pub/priv keypair. I have given the location of the keys in the /etc/salt/master file as well.
Below are the logs I am getting:
2018-11-05 01:48:32,197 [salt.utils.gitfs :1574][ERROR ][21391] Error occurred fetching gitfs remote 'git#[github-endpoint].git': failed to start SSH session: Unable to exchange encryption keys
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/salt/utils/gitfs.py", line 1552, in _fetch
fetch_results = origin.fetch(**fetch_kwargs)
File "/usr/lib64/python2.7/site-packages/pygit2/remote.py", line 405, in fetch
File "/usr/lib64/python2.7/site-packages/pygit2/errors.py", line 64, in check_error
GitError: failed to start SSH session: Unable to exchange encryption keys
I have checked the keypair and connection to the github endpoint.
I am able to sync the repo manually in the server.
I found with the same issue and I finally solved with the following steps:
I create a new ssh key: ssh-keygen -f gitfs_ssh -C 'test#example.com'
Then, I read that an empty line at the end of the private key could be fatal for libssh2, so I removed the empty lines at the bottom of the file (added by ssh-keygen at creation time) and then the new key began to work.
More info in this link
I am running meteor application in a localhost on port 3000 and I can't connect R with my MongoDB. ( I checked this code for MongoDB running on port 27017 without meteor - just pure database and it's working properly). Meteor create an own database which is calling meteor and inside are my collections ( including images in this sample).
library(RMongo)
mongo<- mongoDbConnect("meteor", host="127.0.0.1", port=3000) #error
#mongo<- mongoDbConnect("meteor", host="127.0.0.1", port=27017)# - that's work
output <- dbGetQuery(mongo, 'images', '{}')
print(output)
I have this error:
error in '.jcall(rmongo.object#javaMongo, "S", "dbGetQuery", collection, ':
com.mongodb.MongoException$Network: Read operation to server /127.0.0.1:3000 failed on database meteor
dbGetQuery ... dbGetQueryForKeys -> dbGetQueryForKeys -> .jcall -> .jcheck -> .Call
EDIT:
the same problem with any other R packages like mongolite
No suitable servers found (serverSelectionTryOnce set): [connection closed calling ismaster on 'localhost:3000']
In case that somebody else will have this problem:
type:
meteor mongo -U
to get URL address of your db, and now copy paste to host:""
I'm currently testing the CoreOS container runtime rocket and recreated a scenario to sign and distribute images via Meta Discovery which is based on the following guide. When I try to run a self-signed image using Meta Discovery I get the following error/output:
rkt: using image from local store for image name coreos.com/rkt/stage1-coreos:0.16.0
rkt: searching for app image rocket-example.eu/hellorocket
rkt: remote fetching from URL "https://rocket-example.eu/images/hellorocket.aci"
prefix: "rocket-example.eu/hellorocket"
key: "https://rocket-example.eu/pubkeys.gpg"
gpg key fingerprint is: 993C 033A 1556 CCDF 4321 EB17 8192 E9F7 DBD1 49AE
subkey fingerprint: 02BB E974 02CF 0676 28C8 424C DFB3 FED2 080B 7D76
RXXXX XXXXX (ACI signing key) <rXXXX.XXXXX#XXXXX.XX-XXXXX.de>
Key "https://rocket-example.eu/pubkeys.gpg" already in the keystore
rkt: downloading signature from https://rocket-example.eu/images/hellorocket.aci
Downloading signature: 0 B/1.75 MB
Downloading signature: 3.83 KB/1.75 MB
Downloading signature: 1.75 MB/1.75 MB
run: openpgp: invalid data: tag byte does not have MSB set
I'm using a VM running Ubuntu 15.10, rkt 0.16.0 and GnuPG 2.0.23. The images are provided by a local nginx server.
The created signature hellorocket.aci.asc looks like:
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAABAgAGBQJWqPPqAAoJEJtfmFGWacfBx0gH/i1EVAs2HJm7rOpp0WqbamFa
kC6vH1qs8Rvcagpkcar5ZAZFhC1oQVnF7oB7mvU4Ich3BOS0bBXCgef39oGxVXD6
HrHDB1FX1Q4hFMCnJgFNR4isPaaGy9Hm0uNjE8QxPWBtLgYW3zp5EwBRz3uRizQ7
+BY5Bm+cBIICENKcweTwIXlVgEFk8eFSnMyJ7NP56LbHbZWbb6gFywmz/5A4yJPJ
Qit/iT+FwSfU+xBMpNc2KEux46DfmfpBMippBtMh8wba7Unrjig3oV2Phyqe+UOL
Z6zJjg7dJiAxj7NOwzQRscUyXqmN1yXCF5Tj5ldOwMHXqdXVBw5/KzoTzk1Kl4w=
=9lM+
-----END PGP SIGNATURE-----
I connect to 8 different unix servers from Windows, using connection type 'SSH' in putty. I use the same username/password for each server.
Currently when I need to change passwords (every 60 days), I need to open putty, select the session I want to connect to, type my current password (in the putty window that opens), type "passwd", enter my current password, and then enter my new password.
Then I exit and repeat the process 7 times.
How can I convert this to an automated process where I simply need to supply a script/batch process with my old and new password?
Here is how I automated the process:
Download and install ActiveTCL Community Edition (download the 32 bit version, even if you are on 64 bit windows, as the 64 bit version does not have "Expect" which is what you need to run the automated script)
Open the tclsh85 executable that was created by the install
Run this command "teacup install Expect" (note, this is case sensitive. You may need to setup special http settings if you receive an error and/or are on vpn or using a proxy)
Download Putty's "plink.exe" and either place it in the bin directory of ActiveTCL (default install directory is "C:\Tcl\bin") or alter your "Path" environment variable to include the path to this executable (wherever you downloaded plink.exe). This is the command-line version of Putty which your script will use.
Anywhere on your drive, create a text file named "servers.txt" with a list of the servers (one per line). They should all share the same password, as the script will login to all of them with the same password (that you supply), and change the password to the one you supply.
In the same directory as "servers.txt" create a new text file called "ChangePassword.tcl" (or whatever you want to call it, but be sure its file type is "tcl"). Right click the file and edit in notepad (or whatever text editor you prefer) and paste this script in it.
package require Expect
exp_log_user 0
set exp::nt_debug 1
proc changepw {host user oldpass newpass} {
spawn plink $host
log_user 0
expect {
"login as: " { }
}
exp_send "$user\r"
expect "sword: "
exp_send "$oldpass\r"
expect "\$ "
exp_send "passwd\r"
expect "sword: "
exp_send "$oldpass\r"
expect "sword: "
exp_send "$newpass\r"
expect "sword: "
exp_send "$newpass\r"
set result $expect_out(buffer)
exp_send "exit\r"
return $result
}
label .userlbl -text "Username:"
label .oldpasslbl -text "\nOld Password: "
label .newpasslbl -text "\nNew Password: "
set username "username"
entry .username -textvariable username
set oldpassword "oldpassword"
entry .oldpassword -textvariable oldpassword
set newpassword "newpassword"
entry .newpassword -textvariable newpassword
button .button1 -text "Change Password" -command {
set fp [open "servers.txt" r]
set file_data [read $fp]
close $fp
set data [split $file_data "\n"]
foreach line $data {
.text1 insert end "Changing password for: $line\n"
set output [changepw $line $username $oldpassword $newpassword]
.text1 insert end "$output\n\n"
}
}
text .text1 -width 50 -height 30
pack .userlbl .username .oldpasslbl .oldpassword .newpasslbl .newpassword .button1 .text1
Save the script and then launch the ChangePassword.tcl file.
Here is a picture of what it looks like when you open the ChangePassword.tcl file:
The rest should be self explanatory. Note the program does not output when your password change was successful but it will tell you when it fails. Also note, this was my first tcl script (and first time using Expect) so the script is by no means "optimized" and could probably be improved but it gets the job done. Feel free to edit, or make suggestions/improvements.
Sounds like you want Expect, an extension of TCL that can mimic typing at a keyboard for a console application. See the examples for how to do this.
Now there is something you've written that worries me:
I connect to 8 different unix servers, using connection type 'SSH' in putty. I use the same username/password for each server.
Why aren't you using SSH keys for automating the logon?
Great article! Just elaborating on step-3. Please note the commands to provide Proxy server information in case "teacup install Expect" fails due to connectivity issue:
%teacup install Expect
Resolving Expect ... Not found in the archives.
...
Aborting installation, was not able to locate the requested entity.
child process exited abnormally
% teacup list teacup
0 entities found
Problems which occurred during the operation:
* http://teapot.activestate.com :
{connect failed connection refused} {can't read
"state(sock)": no such element in array while executing
"fileevent $state(sock) writable {}"} NONE
% teacup proxy "abcproxy.mycorp.com" 8080
Proxying through abcproxy.mycorp.com # 8080
% set http_proxy_user MyNetworkID
MyNetworkID
% set http_proxy_pass MyNetworkPassword
MyNetworkPassword
% teacup list teacup
entity name version platform
----------- ------ --------------- ----------
application teacup 8.5.16.0.298388 win32-ix86
----------- ------ --------------- ----------
1 entity found
% teacup install Expect
Resolving Expect ... [package Expect 5.43.2 win32-ix86 # http://teapot.activestate.com]
Resolving Tcl 8.4 -is package ... [package Tcl 8.6.1 _ ... Installed outside repository, probing dependencies]
Retrieving package Expect 5.43.2 win32-ix86 ...# http://teapot.activestate.com ...
Ok
Installing into C:/app/Tcl/lib/teapot
Installing package Expect 5.43.2 win32-ix86
%