I running spring-xd-1.3.1.RELEASE run-time container, when I tried to module file with the source from ftp to hdfs, I get an exception in the shell command which is given below.
xd:>module info --name source:ftphdfs
Command failed org.springframework.xd.rest.client.impl.SpringXDException: Could
not find module with name 'ftphdfs' and type 'source'
Also when I tried to use source as http endpoint, I get an exception like this in shell command which is given below.
xd:>module info --name source:http
Information about source module 'http':
Injects data from http endpoint.
Option Name Description
Default
Type
--------------------- -------------------------------------------------------
--------------------------------------------------------------------------------
--------- -------------------------------------------------------------------
---------------------------------
https true for https://
false
boolean
maxContentLength the maximum allowed content length
1048576
int
messageConverterClass the name of a custom MessageConverter class, to convert
HttpRequest to Message; must have a constructor with a 'MessageBuilderFactory'
parameter org.springframework.integration.x.http.NettyInboundMessageConverter
java.lang.String
port the port to listen to
9000
int
sslPropertiesLocation location (resource) of properties containing the locati
on of the pkcs12 keyStore and pass phrase
classpath:httpSSL.properties
java.lang.String
outputType how this module should emit messages it produces
<none>
org.springframework.util.MimeType
Tech stack which I'm currently using is given below.
1) Hadoop 2.7.2
2) Spring-XD-1.3.1.RELEASE
3) Redis 2.6 (Windows Version) - I use this as a transport
4) Zoo-Keeper 3.8
Any help would be appreciated.
It's a job not a stream source...
xd:>module info job:ftphdfs
Information about job module 'ftphdfs':
...
I don't see an exception for source:http above - just a description of the source.
Related
I am using Ansible to create a server in the Hetzner Cloud, the playbook reads:
- name: create the server at Hetzner
hetzner.hcloud.hcloud_server:
name: "{{server_hostname}}"
enable_ipv4: false
enable_ipv6: false
server_type: cx11
location: "{{server_location}}"
image: ubuntu-22.04
ssh_keys:
- "mykey"
state: present
api_token: "{{hetzner_secret}}"
private_networks: ipfire
register: server
My aim is to integrate the new server into the private network named 'ipfire' that I have previously created. The server should not be accessible via the internet, so I have disabled ipv4 and ipv6. Rather, I'd like to access the server by connecting via OpenVPN to the private network 'ipfire' and connect by use of ssh from there.
Unfortunately, I get an error message as follows:
PLAY [Order servers] ********************************************************************************************************
TASK [hetznerserver : create the server at Hetzner] *************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Unsupported parameters for (hetzner.hcloud.hcloud_server) module: private_networks. Supported parameters include: rebuild_protection, api_token, location, enable_ipv6, upgrade_disk, ipv4, endpoint, ipv6, firewalls, server_type, state, force, labels, ssh_keys, delete_protection, image, id, name, enable_ipv4, placement_group, force_upgrade, user_data, datacenter, rescue_mode, allow_deprecated_image, volumes, backups."}
PLAY RECAP ******************************************************************************************************************
localhost : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
The module private_networks does not seem to work like this?
Error messages like Unsupported parameters for (<moduleName>) module: <givenParameter>. Supported parameters include: <supportedParametersList> are usually syntax errors of the module used.
Therefore one may need to look up the respective documentation, in the example case hcloud_server module – Create and manage cloud servers on the Hetzner Cloud.
If the documentation shows the Parameters in question are available, it indicates
either a version mismatch of module used, means the used version is too old and an update is necessary
or an bug within the module code and further debugging and investigation within the module code is necessary
Code and Documentation Links
Community Authors> hetzner> hcloud
ansible-collections / hetzner.hcloud
After further investigation it might turn out that the parameter in question was introduced recently, in example
Github hetzner.hcloud Issue #150 "Unable to create cloud server without public ipv4 and ipv6"
Github hetzner.hcloud Pull #160 "Add possibility to specify private network when creating or updating servers"
which indicates in your example case that you'll need to update the Ansible Collection module in question since the parameter wasn't introduced in your used version of the module but as of v1.9.0.
As the title says. I cannot run h20.init.
I have already downloaded the 64 bit version of the Java SE Development Kit 8u291. I also downloaded the xgboost library in R (install.packages("xgboost") ). Finally, I have updated all my NVIDIA drivers and downloaded the latest CUDA (although, tbh I don't even know what that does). I followed the steps described in the NVIDIA forums to avoid the crash I had when installing (i.e. remove integration with visual studio). FWIW I'm using a DELL Inspiron 15 Gaming and it has a NVIDIA GTX 1050 with 4GB.
Here's the full code I'm using (straight from the h2o download instructions except for the first line):
library(xgboost)
library(h2o)
localH2O = h2o.init()
demo(h2o.kmeans)
Any help would be much appreciated.
The full message I get when running the above code chunk:
H2O is not running yet, starting it now...
Note: In case of errors look at the following log files:
C:\Users\<my username>\AppData\Local\Temp\RtmpcdvCce\file1a106074110b/h2o_<my username>_started_from_r.out
C:\Users\<my username>\AppData\Local\Temp\RtmpcdvCce\file1a10253139db/h2o_<my username>_started_from_r.err
java version "15.0.2" 2021-01-19
Java(TM) SE Runtime Environment (build 15.0.2+7-27)
Java HotSpot(TM) 64-Bit Server VM (build 15.0.2+7-27, mixed mode, sharing)
Starting H2O JVM and connecting: ............................................................Diagnostic HTTP Request:
HTTP Status Code: -1
HTTP Error Message: Failed to connect to localhost port 54321: Connection refused
Cannot load library from path lib/windows_64/xgboost4j_gpu.dll
Cannot load library from path lib/xgboost4j_gpu.dll
Failed to load library from both native path and jar!
Cannot load library from path lib/windows_64/xgboost4j_omp.dll
Cannot load library from path lib/xgboost4j_omp.dll
Failed to load library from both native path and jar!
Cannot load library from path lib/windows_64/xgboost4j_minimal.dll
Cannot load library from path lib/xgboost4j_minimal.dll
Failed to load library from both native path and jar!
Failed to add native path to the classpath at runtime
java.io.IOException: Failed to get field handle to set library path
at ai.h2o.xgboost4j.java.NativeLibLoader.addNativeDir(NativeLibLoader.java:229)
at ai.h2o.xgboost4j.java.NativeLibLoader.initXGBoost(NativeLibLoader.java:43)
at ai.h2o.xgboost4j.java.NativeLibLoader.getLoader(NativeLibLoader.java:66)
at hex.tree.xgboost.XGBoostExtension.initXgboost(XGBoostExtension.java:70)
at hex.tree.xgboost.XGBoostExtension.isEnabled(XGBoostExtension.java:51)
at water.ExtensionManager.isEnabled(ExtensionManager.java:189)
at water.ExtensionManager.registerCoreExtensions(ExtensionManager.java:103)
at water.H2O.main(H2O.java:2203)
at water.H2OStarter.start(H2OStarter.java:22)
at water.H2OStarter.start(H2OStarter.java:48)
at water.H2OApp.main(H2OApp.java:12)
Cannot initialize XGBoost backend! Xgboost (enabled GPUs) needs:
- CUDA 8.0
XGboost (minimal version) needs:
- GCC 4.7+
For more details, run in debug mode: `java -Dlog4j.configuration=file:///tmp/log4j.properties -jar h2o.jar`
ERROR: Unknown argument (<my username>/AppData/Local/Temp/RtmpcdvCce)
Usage: java [-Xmx<size>] -jar h2o.jar [options]
(Note that every option has a default and is optional.)
-h | -help
Print this help.
-version
Print version info and exit.
-name <h2oCloudName>
Cloud name used for discovery of other nodes.
Nodes with the same cloud name will form an H2O cloud
(also known as an H2O cluster).
-flatfile <flatFileName>
Configuration file explicitly listing H2O cloud node members.
-ip <ipAddressOfNode>
IP address of this node.
-port <port>
Port number for this node (note: port+1 is also used by default).
(The default port is 54321.)
-network <IPv4network1Specification>[,<IPv4network2Specification> ...]
The IP address discovery code will bind to the first interface
that matches one of the networks in the comma-separated list.
Use instead of -ip when a broad range of addresses is legal.
(Example network specification: '10.1.2.0/24' allows 256 legal
possibilities.)
-ice_root <fileSystemPath>
The directory where H2O spills temporary data to disk.
-log_dir <fileSystemPath>
The directory where H2O writes logs to disk.
(This usually has a good default that you need not change.)
-log_level <TRACE,DEBUG,INFO,WARN,ERRR,FATAL>
Write messages at this logging level, or above. Default is INFO.
-max_log_file_size
Maximum size of INFO and DEBUG log files. The file is rolled over after a specified size has been reached.
(The default is 3MB. Minimum is 1MB and maximum is 99999MB)
-flow_dir <server side directory or HDFS directory>
The directory where H2O stores saved flows.
(The default is 'C:\Users\<my username>\h2oflows'.)
-nthreads <#threads>
Maximum number of threads in the low priority batch-work queue.
(The default is.)
-client
Launch H2O node in client mode.
-notify_local <fileSystemPath>
Specifies a file to write when the node is up. The file contains one line with the IP and
port of the embedded web server. e.g. 192.168.1.100:54321
-context_path <context_path>
The context path for jetty.
Authentication options:
-jks <filename>
Java keystore file
-jks_pass <password>
(Default is 'h2oh2o')
-jks_alias <alias>
(Optional, use if the keystore has multiple certificates and you want to use a specific one.)
-hostname_as_jks_alias
(Optional, use if you want to use the machine hostname as your certificate alias.)
-hash_login
Use Jetty HashLoginService
-ldap_login
Use Jetty Ldap login module
-kerberos_login
Use Jetty Kerberos login module
-spnego_login
Use Jetty SPNEGO login service
-pam_login
Use Jetty PAM login module
-login_conf <filename>
LoginService configuration file
-spnego_properties <filename>
SPNEGO login module configuration file
-form_auth
Enables Form-based authentication for Flow (default is Basic authentication)
-session_timeout <minutes>
Specifies the number of minutes that a session can remain idle before the server invalidates
the session and requests a new login. Requires '-form_auth'. Default is no timeout
-internal_security_conf <filename>
Path (absolute or relative) to a file containing all internal security related configurations
Cloud formation behavior:
New H2O nodes join together to form a cloud at startup time.
Once a cloud is given work to perform, it locks out new members
from joining.
Examples:
Start an H2O node with 4GB of memory and a default cloud name:
$ java -Xmx4g -jar h2o.jar
Start an H2O node with 6GB of memory and a specify the cloud name:
$ java -Xmx6g -jar h2o.jar -name MyCloud
Start an H2O cloud with three 2GB nodes and a default cloud name:
$ java -Xmx2g -jar h2o.jar &
$ java -Xmx2g -jar h2o.jar &
$ java -Xmx2g -jar h2o.jar &
So... after a lot of poking around I found the answer. Windows Defender ughhh was blocking access to the h2o.jar. The solution was to open PowerShell on the h2o java folder and run the h2o.jar using java -jar h2o.jar. Then you'll get the security prompt asking you to authorize the program (I've had to do it every time, so you might want to check your settings). Once you do that h2o.init() runs very smoothly in R.
I would like to setup ansible on my Mac. I've done something similar in GNS3 and it worked but here there are more factors I need to take into account. so I have the Ansible installed. I added hostnames in /etc/hosts and I can ping using the hostnames I provided there.
I have created ansible folder which I am going to use and put ansible.cfg inside:
[defaults]
hostfile = ./hosts
host_key_checking = false
timeout = 5
inventory = ./hosts
In the same folder I have hosts file:
[tp-lab]
lab-acc0
When I try to run the following command: ansible tx-edge-acc0 -m ping
I am getting the following errors:
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details
[WARNING]: Unhandled error in Python interpreter discovery for host tx-edge-acc0: unexpected output from Python interpreter discovery
[WARNING]: sftp transfer mechanism failed on [tx-edge-acc0]. Use ANSIBLE_DEBUG=1 to see detailed information
[WARNING]: scp transfer mechanism failed on [tx-edge-acc0]. Use ANSIBLE_DEBUG=1 to see detailed information
[WARNING]: Platform unknown on host tx-edge-acc0 is using the discovered Python interpreter at /usr/bin/python, but future installation of another Python interpreter could change the meaning of that path. See
https://docs.ansible.com/ansible/2.10/reference_appendices/interpreter_discovery.html for more information.
tx-edge-acc0 | FAILED! => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"module_stderr": "Shared connection to tx-edge-acc0 closed.\r\n",
"module_stdout": "\r\nerror: unknown command: /bin/sh\r\n",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 0
Any idea what might the problem here? much appreciated
At first glance it seems that you ansible controller does not load configuration files (especially ansible.cfg) when playbook is fired.
(From documentation) Ansible searches for configuration files in the following order, processing the first file it finds and ignoring the rest:
$ANSIBLE_CONFIG if the environment variable is set.
ansible.cfg if it’s in the current directory.
~/.ansible.cfg if it’s in the user’s home directory.
/etc/ansible/ansible.cfg, the default config file.
Edit: For peace of mind it is good to use full paths
EDIT Based on comments
$ cat /home/ansible/ansible.cfg
[defaults]
host_key_checking = False
inventory = /home/ansible/hosts # <-- use full path to inventory file
$ cat /home/ansible/hosts
[servers]
server-a
server-b
Command & output:
# Supplying inventory host group!
$ ansible servers -m ping
server-a | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
server-b | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
I am trying to set up a mock server using wireMock as a standalone process. I downloaded the jar file and executed the following command:
java -jar wiremock-standalone-2.23.2.jar --port 0
I had to dynamically determine a port because I am already using the default 8080 port for another program running on my machine. It gave me the port number 55142, but when I tried accessing that on the web, it gave me the following error:
HTTP ERROR 403
Problem accessing /__files/. Reason:
Forbidden
Powered by Jetty://
It's probably due to the fact that you just entered http://localhost:55142
and as there are no mappings in ./mappings directory and files in ./files directory (the same where you have your wiremock.jar file is located)
2019-06-04 00:10:58.890 Request was not matched as there were no stubs registered:
{
"url" : "/"
...
}
please try call with __admin endpoint to see if WireMock is working
http://localhost:55142/__admin
please see also docs here for more nice admin commands.
I try to install https://github.com/roots/bedrock-ansible to get a bedrock deployment (http://roots.io/wordpress-stack/) running.
When I run "vagrant up", after some time I get the error:
TASK: [capistrano-setup | Setup deploy group] *********************************
skipping: [default]
TASK: [capistrano-setup | Setup deploy user] **********************************
skipping: [default]
TASK: [capistrano-setup | Adding public key to server] ************************
fatal: [default] => could not locate file in lookup: ~/.ssh/id_rsa.pub
FATAL: all hosts have already failed -- aborting
PLAY RECAP ********************************************************************
to retry, use: --limit #/Users/johannes/site.retry
default : ok=46 changed=16 unreachable=1 failed=0
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.
I do not have a clou how i can fix this. Do you have an idea?
It seems the role is trying to find your local public key. It should be in the location in the error message '~/.ssh/id_rsa.pub', but it's not. So either you don't have one, or you keep it in another location.
If you're not familiar with generating SSH keys you probably don't have one. I personally like the GitHub help page for this: https://help.github.com/articles/generating-ssh-keys/
(you only have to perform steps 1 and 2).
If you do have SSH keys, but in a different location, the capistrano-install role in bedrock uses some variables:
deploy_user: deploy
deploy_keys:
- "~/.ssh/id_rsa.pub"
So you can set (multiple) public key files in the deploy_keys list and they will be added to the deploy_user's authorized keys.
All this is needed because Capistrano will use the deploy user to connect to the remote server later. http://blakesmith.me/2010/02/08/understanding-public-key-private-key-concepts.html