airflow errors out when trying to execute remote script through SSHHook - airflow

Question

With airflow, I am trying to execute a remote script through SSHHook. The script is simply like this
echo "this is a test"
Inside the remote machine, I can run it through "bash test".
I created an airflow script like this:
import airflow
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta
from airflow.contrib.operators.ssh_operator import SSHOperator
from airflow.contrib.hooks.ssh_hook import SSHHook
# add a new SSH connection using the WEB UI under the admin --> connections tab.
sshHook = SSHHook(ssh_conn_id="test_ssh")
# Following are defaults which can be overridden later on
default_args = {
'owner': 'tester',
'depends_on_past': False,
'start_date': datetime(2019,6,24),
'email': ['user123#gmail.com'],
'email_on_failure': True,
'email_on_retry': True,
'retries': 1,
'retry_delay': timedelta(minutes=1),
}
dag = DAG('test', default_args=default_args)
t1 = SSHOperator(
ssh_conn_id= sshHook,
task_id='test_20190624',
command='bash /home/tester/run_test',
dag=dag)
Then I got an error like this:
[2019-06-24 11:27:17,790] {ssh_operator.py:80} INFO - ssh_hook is not provided or invalid. Trying ssh_conn_id to create SSHHook.
[2019-06-24 11:27:17,792] {__init__.py:1580} ERROR - SSH operator error: 'SSHHook' object has no attribute 'upper'
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/contrib/operators/ssh_operator.py", line 82, in execute
timeout=self.timeout)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/contrib/hooks/ssh_hook.py", line 90, in __init__
conn = self.get_connection(self.ssh_conn_id)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/hooks/base_hook.py", line 80, in get_connection
conn = random.choice(cls.get_connections(conn_id))
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/hooks/base_hook.py", line 71, in get_connections
conn = cls._get_connection_from_env(conn_id)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/hooks/base_hook.py", line 63, in _get_connection_from_env
environment_uri = os.environ.get(CONN_ENV_PREFIX + conn_id.upper())
AttributeError: 'SSHHook' object has no attribute 'upper'

Answer 1

You should just directly use the SSH Connection ID or just use SSHHook. The problem here is you have mixed both.
1) Using SSHHook:
t1 = SSHOperator(
ssh_hook = sshHook,
task_id='test_20190624',
command='bash /home/tester/run_test',
dag=dag)
2) Using SSH Connection directly:
t1 = SSHOperator(
ssh_conn_id="test_ssh",
task_id='test_20190624',
command='bash /home/tester/run_test',
dag=dag)