Cannot delete Cinder volume with error message "image still has watchers" - openstack

I run Openstack cinder with ceph as its storage backend. when I occasionally tried to delete one of cinder-volume, it failed.
So I turned to use rbd commands to troubleshoot this issue, below is the error message printed by the command: rbd rm ${pool}/${volume-id}
rbd: error: image still has watchers
This means the image is still
open or the client using it crashed. Try again after closing/unmapping
it or waiting 30s for the crashed client to timeout.
Then rbd status ${pool}/${volume-id} shows
Watchers:
watcher=172.18.0.1:0/523356342 client.230016780
cookie=94001004445696
I am confused why the watcher stick on the volume and cause the volume unable to delete, is there any reason or something I did wrong?
And how to delete the volume in this case?

I found a solution to fix this issue, the concept is adding the watcher to the blacklist by using ceph osd blacklist, then the volume will become removable, after deleting, remove the watcher from the blacklist.
add the watcher to the blacklist
$ ceph osd blacklist add 172.18.0.1:0/523356342
blacklisting 172.18.0.1:0/523356342
check status and delete the volume
$ rbd status ${pool}/${volume-id}
Watchers: none
$ rbd rm ${pool}/${volume-id}
Removing image: 100% complete...done.
remove the watcher from the blacklist
$ ceph osd blacklist rm 172.18.0.1:0/523356342
un-blacklisting 172.18.0.1:0/523356342
That's all, but still finding the root cause.

Related

SELinux and cryptsetup: chown failed and can't access temporary keystore

I am trying to set up SELinux and an encrypted additional partition that I mount at startup using a systemd service.
If I run SELinux in permissive mode, everything runs ok (partition is correctly mounted, data can be accessed and service runs properly).
If I run SELinux in enforcing mode (enforcing=1), I am not able to mount such partition with the error:
/dev/mapper/temporary-cryptsetup-1808: chown failed: Permission denied
sh[1777]: Failed to open temporary keystore device.
sh[1777]: Command failed with code 5: Input/output error
Any ideas to fix that?
Audit2allow does not return any additional rules to be added
Solved assigning to cryptsetup the lvm_exec_t context.
In the lvm.fc file cryptsetup was defined as /bin/cryptsetup but I had to change it to /usr/sbin/cryptsetup where it actually was.

Error: Port 5000 is not open, could not start functions emulator

✔ Deploy complete!
Project Console: https://console.firebase.google.com/project/socialape-6b2f7/overview
Ayhan-MacBookPro:socialape-functions macbook$ firebase serve
=== Serving from '/Users/macbook/Desktop/socialape-functions'...
Error: Port 5000 is not open, could not start functions emulator.
Run lsof -t -i tcp:5000 | xargs kill from your Terminal.
A common cause for this error occurs when the Firebase emulator is not cleanly shut down (e.g., closing your IDE that's running the emulator in an embedded Terminal session) This will leave the process running in the background and occupies the emulator's default port.
To resolve the conflict, find the process ID running on the port (here 5000) from your Terminal command line and then kill it.
The above one-liner finds the process ID and pipes it directly to kill (h/t #manav).
For additional info, check out: Find (and kill) process locking port 3000 on Mac
The bug seems to be on not your end
It is caused by a bug in a dependency (node portfinder).
A quick fix to edit it might be to use the old version of node portfinder (v 1.0.21). Alternatively, you can do it by editing node_modules/firebase-tools/lib/emulator/controller.js and changing yield pf.getPortPromise({ port, stopPort: port }) to yield pf.getPortPromise({ port, stopPort: port + 1 }).
You can see the answer to your question completely here in this SO link.
If you are facing this issue in macOS Pro then this solution is for you.
In MacOS , Port 5000 may be claimed by a new "AirPlay Receiver".
This can be disabled in Settings -> Sharing:
I'm also adding the Screenshot of settings panel for disabling AirPlay Receiver.
Disabling the AirPlay Receiver (if you do not need it) frees up port 5000.

502 Bad Gateway Nginx & Chef Server

I've upgraded my chef server. Then I ran chef-server-ctl reconfigure successfully.
However, when I ran chef-server-ctl test, I got error:
Encountered an error attempting to create client pedant_admin_client
Response Code was: 502
502 Bad Gateway
nginx/1.4.4
Can anyone help me?
tl;dr
sudo chef-server-ctl upgrade
WARNING: my org was preserved, but your results may vary
I ran into this too. I suspect an unexpected. "apt-get upgrade". For me all checks pass in status.
chef-server-ctl status
but this failed
sudo chef-server-ctl test
I ran this for more details (beware thousands of lines of output ;- )
sudo chef-server-ctl tail
and found this gem: (note the ".." in the path indicating a path misconfig)
2015-08-24_23:00 mkdir: cannot create directory '/opt/opscode/embedded/service/rabbitmq/sbin/../var': Permission denied
I then ran this and it worked
sudo chef-server-ctl upgrade
Ran into the same thing: Here is what I did to solve the problem.
Chef has logs but a lot of services. Check which one is failing
chef-server-ctl status will indicate what is down.
Go look at that log under
/var/log/chef-server/<problem-service>/current
My particular problem was
2014-09-27_17:33:32.41439 FATAL: could not create shared memory segment: Invalid argument
2014-09-27_17:33:32.41441 DETAIL: Failed system call was shmget(key=5432001, size=4050755584, 03600).
2014-09-27_17:33:32.41442 HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMMAX parameter. You can either reduce the request size or reconfigure the kernel with larger SHMMAX. To reduce the request size (currently 4050755584 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections.
Thus all I needed to do was chef-server-ctl reconfigure
problem solved

How to reset password in MariaDB on Windows?

How to reset password in MariaDB? I use Windows and NOT Linux. Anyone who knows how to reset my MySQL MariaDB password? I tried to search on Google but did not help.
I bumped into the same problem. I lost the root password for a test server on a windows development machine.
Following the Linux step:
After
net stop mysql
Try invoking mysqld with
mysqld --skip-grant-tables
mysqld will only exit with a short message
[Note] mysqld.exe <...5.5.48.MariaDB> starting as process <pid> ...
then it quits. However I tried to launch mysqld directly, there is no mysqld.exe process. The service start command might have some argument combination that enabled mysqld to run. Tried to pass the settings through a configuration file and it works.
put
skip-grant-tables=TRUE
into MariaDB 5.5\data\my.ini
Restart mysqld, by
net stop mysql && net start mysql
then being able to login as root.
Don't forget to remove the inserted line and restart mysqld again.
As this is top result in Google here's a quick way to change the password:
Stop the DB server
create a text file containing your new password:
ALTER USER 'root'#'localhost' IDENTIFIED BY 'DontForgetMeAgain';
Run this command (don't forget to replace the path to file):
mysqld --init-file=C:\\path\\to\\file.txt
And we're done
Using skip-grant-tables led Maria Db to complain when changing a password even if logged in as root.
I had to do a little modification on Tom's answer in the content of the file to make it work for me:
After stopping the DBServer i created a file with this content:
SET PASSWORD FOR 'root'#'localhost' = PASSWORD('mynewpassword');
FLUSH PRIVILEGES;
then I run:
C:\Program Files\MariaDB 10.1\bin>mysqld --init-file=C:\\path\\to\\file.txt
I then managed to log in

Boot script execution order (rc.local)?

With some great help from another user on here I've managed to create a script which writes the necessary network configurations required to /etc/network/interfaces and allow public access to a DomU server.
I’ve placed this script in the /etc/rc.local file, and executed chmod u+x /etc/rc.local to enable it.
The server is a DomU Ubuntu server on the a host (Dom0). And rc.local doesn't seem to be executing before the network is brought up at boot/creation time.
So the configuration changes are being made to the /etc/network/interfaces file, but are not active once the boot process completes. I have to reboot once more before the changes take effect.
I've tried adding /etc/init.d/networking restart to the the end of the rc.local script (before exit 0), but with no joy.
I also tried adding the script to the S35networking file, but again without success.
Any advice or suggestions on getting this script to execute before the network device is brought up would be greatly appreciated.?

Resources