I installed it and run a HivePartitionSensor to test hive meta store connection.It throw a exception than tell me can not connect to hive meta store
thrift.transport.TTransport.TTransportException: Could not connect to ...
,but I'm sure the host and port is right.I find that the port return from mysql is long type such as 9083L,so I change the source code in hive_hooks.py from
socket = TSocket.TSocket(ms.host, ms.port)
to
socket = TSocket.TSocket(ms.host, int(ms.port))
just cast ms.port to int,it works!
Does anyone came across this problem before?
Env:python2.7,airflow:v1-8-stable,mysql 5.7
Related
How do I get the connections made with flyway to postgres to use keepalives ?
I've tried appending ?tcpKeepAlive=true to the URL, that doesn't seem to work. (the url output while using the cli seems to omit that this is part of the URL, trying to set it as a jdbc.property. only throws an error)
what am I missing ?
note: my migration works with psql over tcp to a remote host without issue, but flyways dies reporting the connection as closed after ~ 10 minutes.
I am very new to aws IOT. I am getting the error:
iot_tls_connect L#143 TCP Connection Error.
I am trying to use the sample aws c code. I have replaced the header config with the config for my account. but none of the sample example go beyond this error. I am using Raspberry Pi.
Sounds like you have the correct certs/keys in place. Make sure the host name is just .iot.us-west-2.amazonaws.com and not https://A<...>.iot.us-west-2.amazonaws.com if you copy-paste it from the console.
I have CoreFTP configured for localhost and the next code:
JSch.setLogger(new MyJschLogger()); //class for console output
Session session = jsch.getSession("user", "localhost", 21);
Properties config = new Properties();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
session.setPassword("password");
session.connect();
when program achieves connect(), two messages appear at console output:
INFO: Connecting to localhost port 21
INFO: Connection established
...and nothing more happens. After some minutes, connection is closed by foreign host exception appears.
Why?
Thanks for all!
Port 21 is the normal port for FTP. JSch is only an SSH client, with support for SFTP in the ChannelSFTP class. JSch knows nothing about FTP (and SFTP is unrelated to FTP, other than by name and that it allows similar things).
You need to setup your server to use the SSH protocol (usually on port 22, but you can use any port, as long as you use the same port on the client). See the documentation - I think you have to check the SSH check box.
Also, if your code is nothing more than what you posted, then nothing more than connecting will happen. To transfer files, you will need to open a ChannelSFTP, and issue the right commands (e.g. call one or more of the put/get methods).
I also faced the similar issue:
"ERROR 2016-04-27 15:05:16,489 [CollectionThreadPool-0] com.dell.supportassist.collector.cli.executor.SSHExecutor: com.jcraft.jsch.JSchException: connection is closed by foreign host"
In my case, channel was getting closed randomly. And when we are trying to re-connect the channel then it was not re-connecting and failing.
This was happening due to looping logic while connecting, so I tried to connect the session without channel by calling method connectWithoutOpenChannel instead of connectinternal(). This resolved my issue.
So, I used the example on http://www.jcraft.com/jsch/examples/Sftp.java and I was trying to connect to ftp.secureftp-test.com.
That destination is a valid testing SFTP server (as mentioned in secureftp-test dot com/). I confirmed it by connecting to the server through nautilus. I also made sure that the password on my program was correct as well.
But, when I run the program and it hits session.connect(), it just says "INFO: Connection established" then it becomes quiet for a minute then it return:
"com.jcraft.jsch.JSchException: connection is closed by foreign host"
I am pretty sure that it got stuck in the while loop of the library but I do not know why. I tried it against my local ftp server and it also had the same problem.
During the quiet moment, I can type. But pressing enter does not send anything to the server.
Has anyone heard or seen the same problem?
Okay, it seems that secureftp-test.com is NOT an sftp server. That is ftps server.
What I did instead was creating my own sftp:
http://wiki.vpslink.com/Configuring_vsftpd_for_secure_connections_(TLS/SSL/SFTP)
Then run the example code but making sure that I have these lines before doing session.connect:
String knownHostsFilename = "/home/yourname/.ssh/known_hosts";
jsch.setKnownHosts(knownHostsFilename);
I am trying to connect to the Quartz scheduler remotely, so I can get a list of jobs that are scheduled to run. On the server application I used the below code:
NameValueCollection properties = new NameValueCollection();
properties.Add("quartz.scheduler.instanceName", "OlScheduler");
properties.Add("quartz.scheduler.instanceId", "Ol");
properties.Add("quartz.threadPool.type", "Quartz.Simpl.SimpleThreadPool, Quartz");
properties.Add("quartz.threadPool.threadCount", "5");
properties.Add("quartz.threadPool.threadPriority", "Normal");
properties.Add("quartz.scheduler.registryPort", "1099");
properties.Add("quartz.scheduler.rmi.export", "true");
properties.Add("quartz.scheduler.rmi.registryHost", "localhost");
properties.Add("quartz.scheduler.rmi.registryPort", "1099");
ISchedulerFactory sf = new StdSchedulerFactory(properties);
IScheduler sched = sf.GetScheduler();
and install and run it as a service, then to connect remotely to it from another application I used:
properties["quartz.scheduler.instanceName"] = "RemoteClient";
properties["quartz.threadPool.type"] = "Quartz.Simpl.SimpleThreadPool, Quartz";
properties["quartz.threadPool.threadCount"] = "5";
properties["quartz.threadPool.threadPriority"] = "Normal";
properties["quartz.scheduler.proxy"] = "true";
properties["quartz.scheduler.proxy.address"] = "http://170.20.20.17:1099/OIService";
IScheduler sched = new StdSchedulerFactory(properties).GetScheduler();
The error that I get is:
"Unable to connect to the remote server" and "No connection could be made because the target machine actively refused it 172.22.22.17:1099"
What did I miss? I am using Quartz.net version 1.1 with RamJobStore and my firewall is off.
I think that's the port which I am using the wrong one. Any help is appreciated
disclaimer: I don't know anything about Quartz Scheduler! :)
However, if you suspect you might have the wrong port number, if you can log into the remote server and run from a command line
netstat -b
This will print out the ports that are in use by process. Hopefully that will allow you to verify the port number.
If the port number is correct, but you still can't communicate with the remote server you can try a couple of things:
run ping.exe [xxx.xxx.xxx.xxx] from the command line to see if remote server responds(*)
verify the firewall on remote server has an exception for the Quartz Scheduler process
try running tracert [xxx.xxx.xxx.xxx] to see if you have a route between the two servers
(*) this is not always conclusive as the remote server, or networking hardware in the middle, might be configured not to respond.
You have more than one problem.
Instance name must be same on client and server side.
properties["quartz.scheduler.instanceName"] = "RemoteClient";
Double properties.Add("quartz.scheduler.rmi.registryPort", "1099");
(it's not problem, but can be removed.)