I am trying to connect two Crate.io nodes which are in different data centers. Node 1 is the master and is already running. When I try to get Node 2 to join the cluster I get the following error:-
[2016-06-23 09:29:30,475][INFO ][discovery.zen] [Node 2] failed to
send join request to master [[Node 1][flPMSe_vQw-2GEb5dkwqaw][ns1.example.com]
[inet[/11.22.33.44:4300]]{http_address=http://11.22.33.44:4200, master=true}],
reason [RemoteTransportException[[Node 1][inet[/11.22.33.44:4300]][internal:discovery
/zen/join]]; nested: ConnectTransportException[[Node 2][inet[/10.19.0.5:4300]]
connect_timeout[30s]]; nested: ConnectTimeoutException
[connection timed out: /10.19.0.5:4300]; ]
Any ideas?
yes, it seems like you have NAT between those data centers, which is why you need to set the publish host parameter to your public IP (where you have to have port mapping to this/these machine/s).
To fix this, set the publish_host setting in crate.yml and maybe (if required - depends on your port mapping settings) publish_port.
See here: https://crate.io/docs/reference/configuration.html#hosts
In crate.yml
network.publish_host: <ip address of Node 2>
service crate restart
Related
I'm trying to send logs into datadog using rsyslog. Ideally, I'm trying to do this without having the logs stored on the server hosting rsyslog. I've run into an error in my config that I haven't been able to find out much about. The error occurs on startup of rsyslog.
omfwd: could not get addrinfo for hostname '(null)':'(null)': Name or service not known [v8.2001.0 try https://www.rsyslog.com/e/2007 ]
Here's the portion I've added into the default rsyslog.config
module(load="imudp")
input(type="imudp" port="514" ruleset="datadog")
ruleset(name="datadog"){
action(
type="omfwd"
action.resumeRetryCount="-1"
queue.type="linkedList"
queue.saveOnShutdown="on"
queue.maxDiskSpace="1g"
queue.fileName="fwdRule1"
)
$template DatadogFormat,"00000000000000000 <%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% - - - %msg%\n "
$DefaultNetstreamDriverCAFile /etc/ssl/certs/ca-certificates.crt
$ActionSendStreamDriver gtls
$ActionSendStreamDriverMode 1
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer *.logs.datadoghq.com
*.* ##intake.logs.datadoghq.com:10516;DatadogFormat
}
First things first.
The module imudp enables log reception over udp.
The module omfwd enables log forwarding over (tcp, udp, ...)
So most probably - or atleast as far as i can tell - with rsyslog you just want to log messages locally and then send them to datadog.
I don't know anything about the $ActionSendStreamDriver tags, so I can't help you there. But what is jumping out is, that in your action you haven't defined where the logs should be sent to.
ruleset(name="datadog"){
action(
type="omfwd"
target="10.100.1.1"
port="514"
protocol="udp"
...
)
...
}
Creation instance failed with error:
Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance
Controller node has not any errors.
There is exception on compute node:
nova.exception.PortBindingFailed: Binding failed for port b70c2f30-f83c-4cae-abf8-98be39a382d5, please check neutron logs for more information.
Neutron's log no errors.
neutron config
[linux_bridge]
physical_interface_mappings = provider:ens3
[vxlan]
enable_vxlan = true
local_ip = 10.101.1.46
l2_population = true
[securitygroup]
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
How can I fix port binding?
1, check the port create successful by openstack port show b70c2f30-f83c-4cae-abf8-98be39a382d5, I guess it's good because of the nova execute PortBinding.
2, check whether the all network component works by openstack network agent list, and check the not works component's log.
3, make sure the all hypervisor are at consistent time.
did this port created first then try to attach as interface ? if yes pleas check port owner ( project) .
Port type you created is normal or direct ?
I have achieved a small test cloud on 3 pieces of hardware. It works fine when in EDGE mode but when I try to configure it for VPCMIDO, new instances begin to launch but then timeout and move to a terminated state. I can also see the instances' initial volume and config data appear in the NC and CC data directories. Below is my system layout and network.json.
HOST 1 : CLC/UFS/WALRUS/MIDO CLUSTER/MIDO GATEWAY/MIDOLMAN AGENT:
em1 (All Services including Mido Cluster): 10.0.0.21
em3 (Target VPCMIDO Adapter): 10.0.0.22
HOST 2 : CC/SC
em1 : 10.0.0.23
HOST 3 : NC/MIDOLMAN AGENT
em1 : 10.0.0.24
{
"Mido": {
"Gateways": [
{
"Ip": "10.0.0.22",
"ExternalDevice": "em3",
"ExternalCidr": "192.168.0.0/16",
"ExternalIp": "192.168.0.2",
"ExternalRouterIp": "192.168.0.1"
}
]
},
"Mode": "VPCMIDO",
"PublicIps": [
"10.0.100.1-10.0.100.254"
]
}
I may be misunderstanding the intent of reserving an interface just for the mido gateway. All of my eucalyptus/zookeeper/cassandra/midonet configs use the 10.0.0.21 interface and seem to communicate fine. The midonet tunnel zone reports my CLC host and NC host successfully in the tunnel zone. The only part of my config that references the interface I intend to use for the midonet gateway is the network.json. No errors were returned at any time during my config so I think I may be missing something conceptual.
You may need to start eucanetd as described here:
https://docs.eucalyptus.cloud/eucalyptus/4.4.5/index.html#install-guide/starting_euca_clc.html
The eucanetd component in vpcmido mode runs on the cloud controller and is responsible for controlling midonet.
When eucanetd is not running instances will fail to start as the required network resources will not be created.
I configured a bridge on the NC and instances were able to launch and I no longer got an error in my nc.log. Docs and the eucalyptus.conf file comments tell me I shouldn't need to do this in VPCMIDO netowrking mode: https://docs.eucalyptus.cloud/eucalyptus/4.4.5/index.html#install-guide/configuring_bridge.html
Despite all that adding the bridge fixed this issue.
I have tried several combinations of proposed configurations I found either in the official documentation or the internet. However, I'm keep getting the same error shown below regardless of the configuration.
[2016-12-14T13:47:37,706][INFO ][o.e.p.PluginsService ] [vc6lzXh] no plugins loaded
[2016-12-14T13:47:43,048][INFO ][o.e.n.Node ] initialized
[2016-12-14T13:47:43,054][INFO ][o.e.n.Node ] [vc6lzXh] starting ...
[2016-12-14T13:47:43,495][INFO ][o.e.t.TransportService ] [vc6lzXh] publish_address {192.168.34.84:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2016-12-14T13:47:43,514][INFO ][o.e.b.BootstrapCheck ] [vc6lzXh] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
ERROR: bootstrap checks failed
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2016-12-14T13:47:43,538][INFO ][o.e.n.Node ] [vc6lzXh] stopping ...
[2016-12-14T13:47:43,680][INFO ][o.e.n.Node ] [vc6lzXh] stopped
[2016-12-14T13:47:43,681][INFO ][o.e.n.Node ] [vc6lzXh] closing ...
[2016-12-14T13:47:43,700][INFO ][o.e.n.Node ] [vc6lzXh] closed
Elasticsearch v5.1.1 is installed in a virtual machine which has a bridged ip address, 192.168.34.84
The configurations I've tried are as follows:
1.
network.host, 192.168.34.84
2.
network.host: 192.168.34.84
network.public_host: 192.168.34.84
3.
network.bind_host: 192.168.34.84
4.
network.bind_host: 0.0.0.0
network.publish_host: 192.168.34.84
None of these did work for me. I guess there are significant changes in v5.11. any help?
According to the the Bootstrap Checks, make sure you've got the important settings set in your machine.
Setting your network.host to something like this, where network.bind_host and network.public_host would pick up the value of network.host by default:
network.host: 192.168.34.84
Try having only the above property set , without the others. Maybe you could have a look at these for more:
Blog Post
Network Settings
EDIT:
So according to your logs, you have to increase vm.max_map_count to 262144 where you set this, as a root user in your machine:
sysctl -w vm.max_map_count=262144
Have a look at the details here about the VM.
Hope it helps!
After doing some research and trying few solution I found this solution working. In our case we wanted to run our elastic search on one of our dev server instead of local machine.
network.host
The node will bind to this hostname or IP address and publish (advertise) this host to other nodes in the cluster. Accepts an IP address, hostname, a special value, or an array of any combination of these. Note that any values containing a : (e.g., an IPv6 address or containing one of the special values) must be quoted because : is a special character in YAML. 0.0.0.0 is an acceptable IP address and will bind to all network interfaces. The value 0 has the same effect as the value 0.0.0.0.
Open this configuration page elasticsearch.yml
network.host: "192.168.1.1" // Your desired TP address
After changing the values restart your elasticsearch serer and it will resolve the issue.
Hope it helps.
I have 3 network nodes running neutron-server ..
Only one of these nodes is attached to the external network
I use ml2 with openvswitch
in the bridge mapping of the node connected to the external network - VIA FLOATING IPS - , i have external_net mapped to the correct bridge ..
On the other nodes i do not have this mapping defined and i do not have interfaces
The issue i have is the following
When i try to start a virtual machine that is connected to the external network , i have this error in the logs :
neutron-server: 2016-09-07 12:33:00.975 57352 ERROR neutron.plugins.ml2.managers [req-def18170-5e45-4fef-9653-e008faa39913 -
- - - -] Failed to bind port 035a58e1-f18f-428b-b78e-e8c0aaba7d14 on host node002 for vnic_type normal using segments [{'segmentation_id': None, 'phy
sical_network': u'external_net', 'id': u'0d4590e5-0c48-4316-8b78-1636d3f44d43', 'network_type': u'flat'}]
neutron-server: 2016-09-07 12:33:00.975 57352 ERROR neutron.plugins.ml2.managers [req-def18170-5e45-4fef-9653-e008faa39913 -
- - - -] Failed to bind port 035a58e1-f18f-428b-b78e-e8c0aaba7d14 on host node003 for vnic_type normal using segments [{'segmentation_id': None, 'phy
sical_network': u'external_net', 'id': u'0d4590e5-0c48-4316-8b78-1636d3f44d43', 'network_type': u'flat'}]
on both nodes( node002 and node003 ) , because they DO NOT have this network defined ! so is this a bug or such a setup is not valid ?
Thank you
In a typical OpenStack deployment you do not bind Nova instances directly to the external network. As you have already surmised, this won't work because that network isn't provisioned on the compute hosts.
Instead, you attach your instances to an internal network, and then you assign floating ip addresses from the external network using,e.g., nova floating-ip-create and nova floating-ip-associate.
An alternative solution is to use "provider external networks", an arrangement in which your nova instances are attached directly to L2 networks with external connectivity, rather than relying on the floating-ip NAT solution described in the previous paragraphs.
the reason behind the error was bad configuration on nodes that DOES NOT host a provider network
mainly the ml2 core file ml2_conf.ini
parameter :
flat_network should be set to the appropriate value on each node
like on the node which is connected to all flat networks ( including the internal network ) it should be set to
flat_networks = *
and on the node that does not host all flat networks ( the provider network for instance )
flat_networks = physical_internal
I believe it won't work. You need to have binded ports to all your 3 network nodes.
A quick test would be to stop neutron-server, neutron-dhcp-agent, neutron-l3-agent and neutron-metadata-agent services from the 2 network nodes that are not bined to external ports... and test again.