OpenStack error when launching an instance - openstack

I am having a consistent
Error: Failed to launch instance-id": Please try again later [Error: Timeout while waiting on RPC response -topic: "network", RPC method: "get_instance_nw_info" info: ""]
every time I am launching an instance in Openstack. I've tried it both using the OpenStack dashboard and via terminal (nova). Using the terminal, here's the command I ran:
nova boot --flavor "2" --image "b26c9acf-06c0-4ff8-b1c7-aca3052485c8" --num-instances "2" --security-groups "default" --meta description="test" test
When I check the list of instances, here's the output:
+--------------------------------------+-------------------------------------------+---
-----+------------+-------------+----------+
| ID | Name |
Status | Task State | Power State | Networks |
+--------------------------------------+-------------------------------------------+---
-----+------------+-------------+----------+
| a0477666-b810-4c73-94e6-2a66576bccac | test-a0477666-b810-4c73-94e6-2a66576bccac |
ERROR | None | NOSTATE | |
| c5822a6f-4270-4718-95c4-9f28fea8de82 | test-c5822a6f-4270-4718-95c4-9f28fea8de82 |
ERROR | None | NOSTATE | |
Here's a snapshot of the error I am encountering:
Am I missing a configuration entry (i.e. using dashboard) or sub-command (i.e. in using nova via terminal) during launching?
Any feedback is greatly appreciated. Thanks in advance!

Related

Maxscale is writing on slave with router_options=master (slave/master replication) and listeners stopped

I've configured on 2 servers(srv50/51),
one of them is Master and the second one is slave,
Here the configuration of my configuration file /etc/maxscale.cnf :
[Read-Only Service]
type=service
router=readconnroute
servers=server50, server51
user=YYYYYYYYYYYYY
passwd=XXXXXXXXXXXXXX
router_options=slave
[Write-Only Service]
type=service
router=readconnroute
servers=server50, server51
user=YYYYYYYYYYYYY
passwd=XXXXXXXXXXXXXX
router_options=master
[Read-Only Listener]
type=listener
service=Read-Only Service
protocol=MySQLClient
port=4008
[Write-Only Listener]
type=listener
service=Write-Only Service
protocol=MySQLClient
port=4009
As i understool the router_options look who is the master and send the writing query to the master
Maxscale (via maxadmin) seems to discover the 2 serveur and understand witch one is the Master :
MaxScale> list servers
Servers.
-------------------+-----------------+-------+-------------+--------------------
Server | Address | Port | Connections | Status
-------------------+-----------------+-------+-------------+--------------------
server51 | 192.168.0.51 | 3306 | 0 | Slave, Running
server50 | 192.168.0.50 | 3306 | 0 | Master, Running
-------------------+-----------------+-------+-------------+--------------------
But even if I connect in Mysql in local on my Maxscale Write-Only Listener port (4009), Listener are in Stopped mode, is it normal ?
MaxScale> list listeners
Listeners.
---------------------+--------------------+-----------------+-------+--------
Service Name | Protocol Module | Address | Port | State
---------------------+--------------------+-----------------+-------+--------
Read-Only Service | MySQLClient | * | 4008 | Stopped
Write-Only Service | MySQLClient | * | 4009 | Stopped
MaxAdmin Service | maxscaled | * | 6603 | Running
---------------------+--------------------+-----------------+-------+--------
I've try to create a database in srv51 (slave), and it was created only on srv51, not in srv50.
Is something wrong in my configuration ? It's strange because it's not my first cluster, and on other cluster all write go to the master (but listeners are Running). Do i don't understand well the meaning of "router_options=master" ? How to start listeners ? I prefere to keep the 51 in Write list to detect topology change
===== UPDATE =====
After watching Log file /var/log/maxscale/maxscale1.log
I found that my monitor user didn't have the correct password :
[MySQL Monitor]
type=monitor
module=mysqlmon
servers=server50, server51
user=MONITOR
passwd=MONITOR_PASS
monitor_interval=10000
I corrected password for user and restarted maxscale, Now everything is running :
MaxScale> list listeners
Listeners.
---------------------+--------------------+-----------------+-------+--------
Service Name | Protocol Module | Address | Port | State
---------------------+--------------------+-----------------+-------+--------
Read-Only Service | MySQLClient | * | 4008 | Running
Write-Only Service | MySQLClient | * | 4009 | Running
MaxAdmin Service | maxscaled | * | 6603 | Running
---------------------+--------------------+-----------------+-------+--------
But write query are still done on Slave and not on Master
Thanks to MariaDb support, I was trying to connect like this :
mysql -h localhost --port=4009 -u USER -p
But Maxscale & Mysql were installed in the same server, even if Mysql bind port 3306, when you specify 'localhost', the connection is done on Mysql port 3306 and not in Maxscale port 4009, the port is ignore !!
The solution is to connect like this :
mysql -h 127.0.0.1 --port=4009 -u USER -p
or like this :
mysql -h localhost --protocol=tcp --port=4009 -u USER -p
I've try both solution and they works.
The solution about the listener not Running is on update of the question.
If writes are done on the slaves, the simplest explanation would be that you're executing writes on the wrong port or your configuration is wrong. To diagnose these problems, enable the info log level by adding log_info=true under the [maxscale] section.
If enabling the info log and inspecting the log files does not provide any clues, I'd suggest opening a bug report on the Maxscale Jira.

Openstack swift with ceph backend (radosgw)

I am trying to use Openstack (liberty) swift with Ceph (Jewel) using radosgw. The aim is that the objects should be stored under ceph osds. I have a working Openstack and Ceph cluster.
To use Ceph as object storage backend I installed and configured radosgw in ceph cluster . In openstack node I installed "python-swiftclient", created an object-store service and added an endpoint for that service with the URL of radosgw.
I followed the instructions given in the link below.
http://docs.ceph.com/docs/jewel/radosgw/keystone/
ceph.conf
[client.rgw.rgw]
rgw_frontends = "civetweb port=7480"
rgw enable ops log = true
rgw ops log rados = true
rgw thread pool size = 2000
rgw override bucket index max shards = 23
ms dispatch throttle bytes = 209715200
[client.radosgw.gateway]
rgw keystone url = http://controller:35357
rgw keystone admin token = ADMIN
rgw keystone accepted roles = _member_,admin
rgw keystone token cache size = 200
rgw keystone revocation interval = 60
rgw s3 auth use keystone = true
nss db path = /var/ceph/nss
Openstack endpoints
# openstack endpoint list |grep -i object
| 8efd00b48db249e69244a5f3e35356b1 | RegionOne | swift | object-store | True | internal | http://rgw:7480/swift/v1 |
| b7d1c7ccc84640138116d8e6676b28a3 | RegionOne | swift | object-store | True | admin | http://rgw:7480/swift/v1 |
| c7844842b53647a4b623905c54cc6c75 | RegionOne | swift | object-store | True | public | http://rgw:7480/swift/v1 |
Output of swift list from command line
# swift list -v
test_CONTAINER
Output of swift stat from command line
# swift stat -v
StorageURL: http://rgw:7480/swift/v1
Auth Token: AUTH_rgwtk0e00000074657374757365723a737769667431dd200c6d2136112ee6d657300feb16d05ffa8f80a2e53ce6c257b32ec5505ff396e5e8
Account: v1
Containers: 7
Objects: 12
Bytes: 168
Meta Temp-Url-Key: healthseq
X-Account-Bytes-Used-Actual: 40960
X-Timestamp: 1473615022.41820
X-Trans-Id: tx0000000000000000006b3-0057d594ae-1f5cb-default
Content-Type: text/plain; charset=utf-8
Accept-Ranges: bytes
When I try to access the object store - container in openstack dashboard, I get the following error.
http://pastebin.com/ALEvYCX8
Please see the image below for the error that I get while trying to access the object store from dashboard.
just remove this line in your code
[client.radosgw.gateway] and merge the setting

nova boot baremetal, select specific machine in pool to 'boot'

I am using Ironic to help me deploy bare metal in a data center environment using 1U Dell servers. It works very well, I can use Ironic to marshall dozens of servers in the rack, then when I need a bare metal instance (via nova) I just use the flavor associated with those servers and I get one of them. Is there a way I can get a specific one? For example, my servers are numbered from the top, starting with control0, control1 all the way down to control39. So, first I create all of the baremetal servers, introspect them. Then I create a flavor (like below, please forgive the pseudo code) and associate each baremetal server with that profile.
openstack flavor create --id auto --ram 6144 --disk 40 --vcpus 4 control
openstack flavor set --property "cpu_arch"="x86_64" --property "capabilities:boot_option"="local" --property "capabilities:profile"="control" control
i = 0
for each baremetal server's uuid:
ironic node-update server-uuid add name=control$i
i = i + 1
ironic node-update server-uuid add properties/capabilities="profile:control,boot_option:local"
When I loop through the list I know that the servers are in top down physical order. What I would like to be able to do is get nova to create a boot instance on a specific ironic bare metal (like control3). I could create separate flavors for each one but I think there must be a way to select a specific piece of hardware? Or a strategy that would pick them in the order I specify.
I am pretty new to Ironic. I have done quite a bit of googling on the topic but haven't found anything. Here is how I start nova:
nova boot --flavor control --image rhel-server-7.1-x86_64-dvd.iso --nic 'net-id=723e7b11-3e61-481a-827e-e58b369dd28f' mybootinstance1
Which works fine. What I would like to do is have a nova boot line which uses the flavor control, and also the name (control0) or any other property that I can assign to make that machine unique. Something like:
nova boot --flavor control --ironic-instance-name control0 --image rhel-server-7.1-x86_64-dvd.iso --nic 'net-id=723e7b11-3e61-481a-827e-e58b369dd28f' mybootinstance1
This is actually a simplification of the nova pool selection process. I don't want to use a pool, but rather, a specific piece of hardware.
If that isn't possible, is there a big drawback to using 40 flavors to create individual 'pools'?
I think you can use --hint in nova boot to select specific machine in pool.
Preconditions: edit /etc/nova/nova.conf, add 'JsonFilter' in scheduler_default_filters and restart nova-scheduler.Then use nova boot command like this:
nova boot --flavor <flavor> --image <image_id> --nic net-id=<net_id> --hint reservation=<reservation_id> --hint query='["=","$hypervisor_hostname", "<node_uuid>"]' <instance_name>
I'm not quite familiar to this topic, but I'd like to share how to boot an instance to specific host via availability zone.
In my devstack (master) development environment, the procedure is:
$ nova availability-zone-list
+---------------------+----------------------------------------+
| Name | Status |
+---------------------+----------------------------------------+
| internal | available |
| |- fcwszq | |
| | |- nova-conductor | enabled :-) 2015-11-23T06:31:46.000000 |
| | |- nova-cert | enabled :-) 2015-11-23T06:31:41.000000 |
| | |- nova-scheduler | enabled :-) 2015-11-23T06:31:43.000000 |
| | |- nova-network | enabled :-) 2015-11-23T06:31:44.000000 |
| nova | available |
| |- fcwszq | |
| | |- nova-compute | enabled :-) 2015-11-23T06:31:41.000000 |
+---------------------+----------------------------------------+
Note that my environment only gets one compute node whose hostname is fcwszq, but still can be specified as:
nova boot --availability-zone nova:fcwszq --flavor 1 --image c38f0c7e-8ee0-4b0f-8a56-022040b4696f test02
If I specify a non-existent node, for example, nova:non-existent, the instance cannot be created correctly (state is ERROR).
Hope this can help you.
Another way is using host aggregate and flavor metadata to boot instance on a random server in a group, reference: http://docs.openstack.org/liberty/config-reference/content/section_compute-scheduler.html#d6e21786

Cinder volume cant attach or detach

After a power failure of the host machine, the openstack cinder volumes have entered a state in which they can not be attached nor detached.
~$ nova volume-attach ### ###
ERROR: Invalid volume: already attached (HTTP 400) (Request-ID: req-###)
~$ nova volume-detach ### ###
ERROR: Invalid volume: already detached (HTTP 400) (Request-ID: req-###)
The status of the volume its self is listed as attached
cinder list
+-----+-----------+---------------------+------+-------------+----------+-------------+
| ID | Status | Display Name | Size | Volume Type | Bootable | Attached to |
+-----+-----------+---------------------+------+-------------+----------+-------------+
| ### | available | volume-data | 690 | Storage | false | ### |
+-----+-----------+---------------------+------+-------------+----------+-------------+
The volume is not found on the instance even though its listed as attached in the cli and in horizon (the device isnt found in /dev/ nor the mount point in /mnt/)

WebLogic OBIEE Scheduler Component Down

I have an OBIEE 11g installation in a Red Hat machine, but I'm finding problems to make it running. I can start WebLogic and its services, so I’m able to enter the WebLogic console and Enterprise Manager, but problems come when I try to start OBIEE components with opmnctl command.
The steps I’m performing are the following:
1) Start WebLogic
cd /home/Oracle/Middleware/user_projects/domains/bifoundation_domain/bin/
./startWebLogic.sh
2) Start NodeManager
cd /home/Oracle/Middleware/wlserver_10.3/server/bin/
./startNodeManager.sh
3) Start Managed WebLogic
cd /home/Oracle/Middleware/user_projects/domains/bifoundation_domain/bin/
./startManagedWebLogic.sh bi_server1
4) Set up OBIEE Components
cd /home/Oracle/Middleware/instances/instance1/bin/
./opmnctl startall
The result is:
opmnctl startall: starting opmn and all managed processes...
================================================================================
opmn id=JustiziaInf.mmmmm.mmmmm.9999
Response: 4 of 5 processes started.
ias-instance id=instance1
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
ias-component/process-type/process-set:
coreapplication_obisch1/OracleBISchedulerComponent/coreapplication_obisch1/
Error
--> Process (index=1,uid=1064189424,pid=4396)
failed to start a managed process after the maximum retry limit
Log:
/home/Oracle/Middleware/instances/instance1/diagnostics/logs/OracleBISchedulerComponent/
coreapplication_obisch1/console~coreapplication_obisch1~1.log
5) Check the status of components
cd /home/Oracle/Middleware/instances/instance1/bin/
./opmnctl status
Processes in Instance: instance1
---------------------------------+--------------------+---------+---------
ias-component | process-type | pid | status
---------------------------------+--------------------+---------+---------
coreapplication_obiccs1 | OracleBIClusterCo~ | 8221 | Alive
coreapplication_obisch1 | OracleBIScheduler~ | N/A | Down
coreapplication_obijh1 | OracleBIJavaHostC~ | 8726 | Alive
coreapplication_obips1 | OracleBIPresentat~ | 6921 | Alive
coreapplication_obis1 | OracleBIServerCom~ | 7348 | Alive
Read the log files from /home/Oracle/Middleware/instances/instance1/diagnostics/logs/OracleBISchedulerComponent/
coreapplication_obisch1/console~coreapplication_obisch1~1.log.
I would recommend trying the the steps in the below link as this is a common issue when upgrading OBIEE.
http://www.askjohnobiee.com/2012/11/fyi-opmnctl-failed-to-start-managed.html
Not sure, what your log says, but try these below steps and check if it works or not
Login as superuser
cd $ORACLE_HOME/Apache/Apache/bin
chmod 6750 .apachectl
logout and login as ORACLE user
opmnctl startproc process-type=OracleBIScheduler

Resources