I am trying to copy files from my edge node to HDFS using oozie. Many suggested to setup password less ssh to get this done.
Iam unable to login to oozie user as it is a service user.
Is there any other way other than password less ssh.
Thanks in advance.
Other than password less ssh there are two more options :
1. My preferred option : Use JSch java library and create a java application which will accept a shell script to be executed as argument. Using the JSch , it will perform ssh on the configured edge node and execute the shell script on the edge node. In the jsch, you can configure, the edgenode username and password. Use 'JCEKS' file to store the password.
Then add a Java Action in Oozie to run the java application created using JSch.
2. Use "/usr/bin/expect" library to create a shell script, which will perform ssh on edgenode and then run the configured shell script. More details are here Use expect in bash script to provide password to SSH command
Related
Can someone please give me an example for corporatePasswordStore that is mentioned here:
https://docs.corda.net/node-administration.html?fbclid=IwAR0gRwe5BtcWO0NymZVyE7_yMfthu2xxnU832vZHdbuv17S-wPXgb7iVZSs#id2
I've been doing a lot of research in the last few days on how to hide the plain passwords from node.conf; it's a new topic for me and this is what I came up with so far:
Create a priv/pub key with gpg2
Create a password store with pass (using the key that I generated earlier).
Store all the plain passwords from node.conf inside that password store.
Replace the plain passwords in node.conf with environment variables (e.g. keyStorePassword = ${KEY_PASS})
Create a script file (e.g. start_node.sh) that will do the following:
a. Set an environment variable to one of the passwords from the password store: export key_store_password=$(pass node.conf/keyStorePassword)
b. Start the node: java -jar corda.jar
c. Restart the gpg agent to clear the cached passwords, otherwise you can get any password from the store without passing the passphrase: gpgconf --reload gpg-agent
Pros:
Using the bash file start_node.sh allows to set many passwords as environment variables at once (e.g. keyStore, trustStore, db passwords, RPC user password)
Since we are running the bash file with bash start_node.sh and not source start_node.sh, the environment variable is not exposed to the parent process (i.e. you cannot read that environment variable value inside the terminal where you ran bash start_node.sh
History commands are not enabled by default inside bash scripts.
Cons:
You no longer can have a service that automatically starts on VM startup, because the start_node.sh script will ask for the passphrase for your gpg key that was used to encrypt the passwords inside the password store (i.e. it's an interactive script).
Am I over-complicating this? Do you have an easier approach? Is it even necessary to hide the plain passwords?
I'm using Corda open source so I can't use the Configuration Obfuscator (which is for Enterprise only): https://docs.corda.r3.com/tools-config-obfuscator.html#configuration-obfuscator (edited)
I wrote a detailed article here: https://blog.b9lab.com/enabling-corda-security-with-nodes-configuration-file-412ce6a4371c, which covers the following topics:
Enable SSL for database connection.
Enable SSL for RPC connection.
Enable SSL for Corda webserver.
Enable SSL for Corda standalone shell.
Hide plain text passwords.
Set permissions for RPC users.
I would like to copy a file from my computer to a remote server via SCP using R.
I have found 2 functions that appear to satisfy this partially.
1.
Using ssh.utils
ssh.utils::cp.remote(path.src="~/myfile.txt",
remote.dest="username#remote",
remote.src="", path.dest="~/temp", verbose=TRUE)
I've noticed that with this method, if I need to enter a password (when remote doesn't have my public key), the function produces an error.
2.
Using RCurl:
RCurl appears to have more robust functionality in the scp() function, but, from what I can tell, it is only for copying a file from a remote to my local machine. I would like to do the opposite.
Is there another way to use these functions or is there another function that would be able to copy a file from my local machine to a remote machine via SCP?
One approach to address the need to enter a password interactively is to use sshpass (see https://stackoverflow.com/a/13955428/6455166) in a call to system, e.g.
system('sshpass -p "password" scp ~/myfile.txt username#remote.com:/some/remote/path')
See the linked answer above for more details, including options to avoid embedding the password in the command.
I have an IBM WebSphere Application Server v8.5 (WAS) installed on Linux RedHat 6.
My question: how can I check on the following by command (if exist):
Is the application server running or not?
Is the web application that deployed on it running or not?
The database connectivity (using datasource) is connected successfully or not?
The easiest and quickest to check all these things is to use web administrative console available at http://yourHost:9060/ibm/console.
If you want to use command, then:
Is the application server running or not?
You can check that issuing serverStatus command (will check all servers):
%PROFILE_ROOT%/bin/serverStatus.sh -all
or for specific server:
%PROFILE_ROOT%/bin/serverStatus.sh serverName
the output will be something like:
C:\IBM\WebSphere\AppServer85\profiles\AppSrv02\bin>serverstatus server1
ADMU0116I: Tool information is being logged in file
C:\IBM\WebSphere\AppServer85\profiles\AppSrv02\logs\server1\serverSta
tus.log
ADMU0128I: Starting tool with the AppSrv02 profile
ADMU0500I: Retrieving server status for server1
ADMU0508I: The Application Server "server1" is STARTED
Is the web application that deployed on it running or not?
There is no direct command for this. You can use wsadmin script for that. A simple one could be like the one below, if it returns entry the application is running:
print AdminControl.completeObjectName('type=Application,name=myApplication,*')
For more details check this question How do I determine if an application is running using wsadmin?
The database connectivity (using datasource) is connected successfully or not?
There is no direct command for this. You can use wsadmin script for that also. Here is sample script:
ds = AdminConfig.getid('/DataSource:Default Datasource/')
AdminControl.testConnection(ds)
For more details check this page Testing data source connections using wsadmin scripting
The serverStatus.sh command is s..l..o..w.. If you want an answer today then there is a file in the logs folder with the process PID:
/opt/IBM/WebSphere/AppServer/profiles/AppSrv01/logs/servername/servername.pid
That file contains the PID of the server process. If it is running:
ps -p pid
Then the server is up.
1. ps -ef | grep dmgr
2. ps -ef | grep <application name>
Also grep SystemOut.log for e-business and verify latest timestamp.
Log into admin console, browse to DataSource, display from all scopes, select your datasource and then click test. As long as the nodeagent is running and has been restarted at least once since the Datasource config and credentials were added, then this test should be fairly accurate.
I have a command to drop postgresql database:
sudo -u myuser psql -c "drop database my_databse"
How I could execute this sentence from an aspnet website(c#)?
Is it possible to execute without introducing a password?
This app is running in another server:
I have a windows server running aspnet website.
I have an ubuntu server running postgresql database.
drop database my_databse is an SQL command. You can run it directly from nPgSQL. There is no need to run psql, and no point doing so.
If your usual user account doesn't have the access rights for it, you can make a new temporary connection in nPgSQL just for that command. You might find SET SESSION AUTHORIZATION or SET ROLE useful, too.
I want to copy files from local machine to remote server through SCP command. I was able to copy files through Cygwin. Now I want to achieve this thru Qt. How do I invoke 'SCP' command? If I make use of QProcess, then how will I get to know about password prompt?
As fara as I know, you will hit the same issue with scp prompting for the password whichever way you try to call the command (even if you try writing a bash script that calls scp, for instance)
Possible solution I'm aware of includes:
create a public/private key and distribute them so that you do not need to be prompted with the password. An interesting paper on the topic is here
create an expect script and invoke it with QProcess
Require your users to use public keys and your problem is solved: https://hkn.eecs.berkeley.edu/~dhsu/ssh_public_key_howto.html
scp, ssh, sftp, etc. deliberately make it hard to take a password other than directly from the user. Specifically, things like expect will not work. And if they did, they wouldn't be secure.