Creation instance failed with error:
Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance
Controller node has not any errors.
There is exception on compute node:
nova.exception.PortBindingFailed: Binding failed for port b70c2f30-f83c-4cae-abf8-98be39a382d5, please check neutron logs for more information.
Neutron's log no errors.
neutron config
[linux_bridge]
physical_interface_mappings = provider:ens3
[vxlan]
enable_vxlan = true
local_ip = 10.101.1.46
l2_population = true
[securitygroup]
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
How can I fix port binding?
1, check the port create successful by openstack port show b70c2f30-f83c-4cae-abf8-98be39a382d5, I guess it's good because of the nova execute PortBinding.
2, check whether the all network component works by openstack network agent list, and check the not works component's log.
3, make sure the all hypervisor are at consistent time.
did this port created first then try to attach as interface ? if yes pleas check port owner ( project) .
Port type you created is normal or direct ?
Related
I want to manage Ovirt VM's network by openstack neutron(OVN).
My setup:
oVirt v4.4.10 (created one VM names: vm1)
Openstack victoria (5.4.0)
My steps:
created network 'vlan-1' at openstack dashboard network page.
created provider 'neutron' and select openstack networking type
import network 'vlan-1' from 'neutron' provider.
create nic1 in vm1 and selected network 'vlan-1'
run vm1.
But vm1 start failed, The engine.log shows:
2022-11-02 12:05:17,674+08 WARN [org.ovirt.engine.core.bll.provider.network.openstack.BaseNetworkProviderProxy] (EE-ManagedThreadFactory-engine-Thread-11256) [a0d16d0c-f5df-4992-808f-0095daff628a]
Host binding id for external network vlan-1 on host ovn-comp-nodes is null, using host id 192.168.3.215 to allocate vNIC nic1 instead. Please provide an after_get_caps hook for the plugin type OPEN_VSWITCH on host ovn-comp-nodes
2022-11-02 12:05:23,180+08 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-21) [6f97b9be] EVENT_ID: VM_DOWN_ERROR(119), VM ovn-comp-node1 is down with error. Exit message:
Cannot get interface MTU on 'vlan-1': No such device.
I am reading Ovirt's source code but still can't found the reason...
I'm Rookie for openstack, I want to create a network with neutron now. But it always show me it fails.
This is my neutron-linuxbridge-agent.log
i see neturon.server.log a error
INFO neutron.api.v2.resource [req-d2d242c1-7b1d-4ad3-a2e8-33d85dcc3ac3 f77a94713b74422fa45805fc91b6c907 380e5668659b4e8b99c66f6ec331aebf - - -] create failed (client error): Invalid input for operation: physical_network 'default' unknown for VLAN provider network.
i find where is problem,because i'm set a wrong label,there is no label named 'default'
I have 3 network nodes running neutron-server ..
Only one of these nodes is attached to the external network
I use ml2 with openvswitch
in the bridge mapping of the node connected to the external network - VIA FLOATING IPS - , i have external_net mapped to the correct bridge ..
On the other nodes i do not have this mapping defined and i do not have interfaces
The issue i have is the following
When i try to start a virtual machine that is connected to the external network , i have this error in the logs :
neutron-server: 2016-09-07 12:33:00.975 57352 ERROR neutron.plugins.ml2.managers [req-def18170-5e45-4fef-9653-e008faa39913 -
- - - -] Failed to bind port 035a58e1-f18f-428b-b78e-e8c0aaba7d14 on host node002 for vnic_type normal using segments [{'segmentation_id': None, 'phy
sical_network': u'external_net', 'id': u'0d4590e5-0c48-4316-8b78-1636d3f44d43', 'network_type': u'flat'}]
neutron-server: 2016-09-07 12:33:00.975 57352 ERROR neutron.plugins.ml2.managers [req-def18170-5e45-4fef-9653-e008faa39913 -
- - - -] Failed to bind port 035a58e1-f18f-428b-b78e-e8c0aaba7d14 on host node003 for vnic_type normal using segments [{'segmentation_id': None, 'phy
sical_network': u'external_net', 'id': u'0d4590e5-0c48-4316-8b78-1636d3f44d43', 'network_type': u'flat'}]
on both nodes( node002 and node003 ) , because they DO NOT have this network defined ! so is this a bug or such a setup is not valid ?
Thank you
In a typical OpenStack deployment you do not bind Nova instances directly to the external network. As you have already surmised, this won't work because that network isn't provisioned on the compute hosts.
Instead, you attach your instances to an internal network, and then you assign floating ip addresses from the external network using,e.g., nova floating-ip-create and nova floating-ip-associate.
An alternative solution is to use "provider external networks", an arrangement in which your nova instances are attached directly to L2 networks with external connectivity, rather than relying on the floating-ip NAT solution described in the previous paragraphs.
the reason behind the error was bad configuration on nodes that DOES NOT host a provider network
mainly the ml2 core file ml2_conf.ini
parameter :
flat_network should be set to the appropriate value on each node
like on the node which is connected to all flat networks ( including the internal network ) it should be set to
flat_networks = *
and on the node that does not host all flat networks ( the provider network for instance )
flat_networks = physical_internal
I believe it won't work. You need to have binded ports to all your 3 network nodes.
A quick test would be to stop neutron-server, neutron-dhcp-agent, neutron-l3-agent and neutron-metadata-agent services from the 2 network nodes that are not bined to external ports... and test again.
Background: I've got two machines with identical hostnames, I need to set up a local spark cluster for testing, setting up a master and a worker works fine, but trying to run an application with the driver causes problems, netty doesn't seem to be picking the correct host (regardless of what I put in there, it just picks the first host).
Identical hostname:
$ dig +short corehost
192.168.0.100
192.168.0.101
Spark config (used by master and the local worker):
export SPARK_LOCAL_DIRS=/some/dir
export SPARK_LOCAL_IP=corehost // i tried various like 192.168.0.x for
export SPARK_MASTER_IP=corehost // local, master and the driver
export SPARK_MASTER_PORT=7077
export SPARK_WORKER_CORES=2
export SPARK_WORKER_MEMORY=2g
export SPARK_WORKER_INSTANCES=2
export SPARK_WORKER_DIR=/some/dir
Spark starts up and I can see the worker in the web-ui.
When I run the spark "job" below:
val conf = new SparkConf().setAppName("AaA")
// tried 192.168.0.x and localhost
.setMaster("spark://corehost:7077")
val sc = new SparkContext(conf)
I get this exception:
15/04/02 12:34:04 INFO SparkContext: Running Spark version 1.3.0
15/04/02 12:34:04 WARN Utils: Your hostname, corehost resolves to a loopback address: 127.0.0.1; using 192.168.0.100 instead (on interface en1)
15/04/02 12:34:04 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
15/04/02 12:34:05 ERROR NettyTransport: failed to bind to corehost.home/192.168.0.101:0, shutting down Netty transport
...
Exception in thread "main" java.net.BindException: Failed to bind to: corehost.home/192.168.0.101:0: Service 'sparkDriver' failed after 16 retries!
at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:393)
at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:389)
at scala.util.Success$$anonfun$map$1.apply(Try.scala:206)
at scala.util.Try$.apply(Try.scala:161)
at scala.util.Success.map(Try.scala:206)
at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235)
at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
15/04/02 12:34:05 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
15/04/02 12:34:05 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
15/04/02 12:34:05 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
Process finished with exit code 1
Not sure how to proceed... its a whole jungle of ip addresses.
Not sure if this is a netty issue either.
My experience with the identical problem is that it revolves around setting things up locally. Try being more verbose in your spark driver code, add the SPARK_LOCAL_IP and driver host ip to the config:
val conf = new SparkConf().setAppName("AaA")
.setMaster("spark://localhost:7077")
.set("spark.local.ip","192.168.1.100")
.set("spark.driver.host","192.168.1.100")
This should tell netty which of the two identical hosts to use.
I'm trying to deploy Openstack Icehouse on Ubuntu Server 14.04 by following the official document. But after Keystone\Nova\Neutron\Glance were deployed, when I tried to launch a CirrOS instance by
nova boot -nic ... -image ... -flavor ...
, it failed.
The log in Nova client shows that:
The Neutron client(Yes, it's neutron. I guess there are interactions between them in booting) tried to connect with Neutron server to create a port on tenant's network.
But Neutron client set up the token-getting request using {username:neutron, password:REDACTED} to Keystone server and used that token to request for creating port to Neutron server.
Finally, the Neutron Server decided that that's an authentication problem.
I'm sure that I requested to create instance using tenant 'demo''s info($OS_TENANT_NAME, $OS_USERNAME, $OS_PASSWORD, $OS_AUTH_URL were properly set with 'demo''s value) by
source demoopenrc.sh
with demo's credential in that file.
Is that something wrong in the Neutron client's configuration or booting process? I paste a part of the neutron.conf here:
the Keystone setting
[keystone_authtoken]
auth_uri = http://controller:5000
auth_host = controller
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = neutron
admin_password = neutronpass
signing_dir = $state_path/keystone-signing
Since the Neutron client used 'neutron' user's credential for token getting, is there something wrong in this part?
The problem has been solved after nearly a month. For anyone still interested in this problem, please visit here