I am learning Chef + Test Kitchen on a CentOS VM at the moment and it seems that every time I run kitchen converge, some packages fail and throw the same error:
Chef::Exceptions::Exec
----------------------
returned 1, expected 0
And ALL of the errors are located in the package resource. For example:
Compiled Resource:
------------------
# Declared in /tmp/kitchen/cookbooks/nginx/recipes/package.rb:39:in `from_file'
package("nginx") do
action :install
retries 0
retry_delay 2
guard_interpreter :default
package_name "nginx"
version "1.0.15-5.el6"
cookbook_name :nginx
recipe_name "package"
end
However, when I login to the VM using kitchen login and manually run
yum install nginx
It runs just OK. Also, sometimes it just installs fine when I run kitchen converge for the second time.
My recipe file is:
# create vtapp user
user node.default['railsapp']['user'] do
supports :manage_home => true
system true
home "/home/#{node.default['railsapp']['user']}"
shell '/bin/bash'
end
# install git
package 'git'
# install mysql and run the service
mysql_service 'default'
# install redis and run the service
include_recipe 'redis::server'
# install rbenv to vtapp user, and install ruby 2.1.0 along with bundler
include_recipe "ruby_build"
node.default['rbenv']['user_installs'] = [
{
'user' => node.default['railsapp']['user'],
'rubies' => ['2.1.0'],
'gems' => {
'2.1.0' => [
{ 'name' => 'bundler' }
]
}
}
]
include_recipe "rbenv::user"
# install monit
include_recipe "monit"
# install nginx
include_recipe "nginx"
Did I miss something?
Well, as crazy as it seems, after I increased the memory allocation for Vagrant to 1024 MB as is described in the link below:
https://github.com/test-kitchen/kitchen-vagrant/issues/22
The intermittent issue above suddenly gone...
update:
I have repeatedly run the full kitchen test command with success after I increased the memory allocation :-)
update (2):
I have delved more about Chef and another possible cause is the timeout set within Chef to execute an action, 15 mins if I recall correctly. Possible solutions I have used are 1) Installing proxy server to accelerate download times, 2) Increasing internet bandwidth, 3) Allow vagrant to allocate more CPU cores to the VM.
You must also pay attention about the minimum memory required for the application. For example, I had installed ZenOSS with Chef, which require 3 GB memory at minimum, and kept failing with the error code above if I allocated memory below that.
Related
I would like to setup ansible on my Mac. I've done something similar in GNS3 and it worked but here there are more factors I need to take into account. so I have the Ansible installed. I added hostnames in /etc/hosts and I can ping using the hostnames I provided there.
I have created ansible folder which I am going to use and put ansible.cfg inside:
[defaults]
hostfile = ./hosts
host_key_checking = false
timeout = 5
inventory = ./hosts
In the same folder I have hosts file:
[tp-lab]
lab-acc0
When I try to run the following command: ansible tx-edge-acc0 -m ping
I am getting the following errors:
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details
[WARNING]: Unhandled error in Python interpreter discovery for host tx-edge-acc0: unexpected output from Python interpreter discovery
[WARNING]: sftp transfer mechanism failed on [tx-edge-acc0]. Use ANSIBLE_DEBUG=1 to see detailed information
[WARNING]: scp transfer mechanism failed on [tx-edge-acc0]. Use ANSIBLE_DEBUG=1 to see detailed information
[WARNING]: Platform unknown on host tx-edge-acc0 is using the discovered Python interpreter at /usr/bin/python, but future installation of another Python interpreter could change the meaning of that path. See
https://docs.ansible.com/ansible/2.10/reference_appendices/interpreter_discovery.html for more information.
tx-edge-acc0 | FAILED! => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"module_stderr": "Shared connection to tx-edge-acc0 closed.\r\n",
"module_stdout": "\r\nerror: unknown command: /bin/sh\r\n",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 0
Any idea what might the problem here? much appreciated
At first glance it seems that you ansible controller does not load configuration files (especially ansible.cfg) when playbook is fired.
(From documentation) Ansible searches for configuration files in the following order, processing the first file it finds and ignoring the rest:
$ANSIBLE_CONFIG if the environment variable is set.
ansible.cfg if it’s in the current directory.
~/.ansible.cfg if it’s in the user’s home directory.
/etc/ansible/ansible.cfg, the default config file.
Edit: For peace of mind it is good to use full paths
EDIT Based on comments
$ cat /home/ansible/ansible.cfg
[defaults]
host_key_checking = False
inventory = /home/ansible/hosts # <-- use full path to inventory file
$ cat /home/ansible/hosts
[servers]
server-a
server-b
Command & output:
# Supplying inventory host group!
$ ansible servers -m ping
server-a | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
server-b | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
After understanding how to add an ospd scanner, verify it etc ...
I though I could finally use it but got an error through UI to add it to a task.
In my case, I run OpenVAS 9 on a debian 9 and I'm trying to include a w3af scanner but I got the same issue with every OSP scanner I add.
my pip freeze :
ospd==1.2.0
ospd-debsecan==1.2b1
ospd-nmap==1.0b1
ospd-w3af==1.0.0
Note that here is an example of w3af but the issue is the same for debsecan scanner and nmap scanner.
my openvas-check-setup :
Step 1: Checking OpenVAS Scanner ...
OK: OpenVAS Scanner is present in version 5.1.1.
OK: redis-server is present in version v=3.2.6.
OK: scanner (kb_location setting) is configured properly using the redis-server socket: /tmp/redis.sock
OK: redis-server is running and listening on socket: /tmp/redis.sock.
OK: redis-server configuration is OK and redis-server is running.
OK: NVT collection in /usr/local/var/lib/openvas/plugins contains 47727 NVTs.
WARNING: Signature checking of NVTs is not enabled in OpenVAS Scanner.
SUGGEST: Enable signature checking (see http://www.openvas.org/trusted-nvts.html).
OK: The NVT cache in /usr/local/var/cache/openvas contains 47727 files for 47727 NVTs.
Step 2: Checking OpenVAS Manager ...
OK: OpenVAS Manager is present in version 7.0.2.
OK: OpenVAS Manager database found in /usr/local/var/lib/openvas/mgr/tasks.db.
OK: Access rights for the OpenVAS Manager database are correct.
OK: sqlite3 found, extended checks of the OpenVAS Manager installation enabled.
OK: OpenVAS Manager database is at revision 184.
OK: OpenVAS Manager expects database at revision 184.
OK: Database schema is up to date.
OK: OpenVAS Manager database contains information about 47727 NVTs.
OK: At least one user exists.
OK: OpenVAS SCAP database found in /usr/local/var/lib/openvas/scap-data/scap.db.
OK: OpenVAS CERT database found in /usr/local/var/lib/openvas/cert-data/cert.db.
OK: xsltproc found.
Step 3: Checking user configuration ...
WARNING: Your password policy is empty.
SUGGEST: Edit the /usr/local/etc/openvas/pwpolicy.conf file to set a password policy.
Step 4: Checking Greenbone Security Assistant (GSA) ...
OK: Greenbone Security Assistant is present in version 7.0.2.
OK: Your OpenVAS certificate infrastructure passed validation.
Step 5: Checking OpenVAS CLI ...
OK: OpenVAS CLI version 1.4.5.
Step 6: Checking Greenbone Security Desktop (GSD) ...
SKIP: Skipping check for Greenbone Security Desktop.
Step 7: Checking if OpenVAS services are up and running ...
OK: netstat found, extended checks of the OpenVAS services enabled.
OK: OpenVAS Scanner is running and listening on a Unix domain socket.
OK: OpenVAS Manager is running and listening on a Unix domain socket.
OK: Greenbone Security Assistant is listening on port 443, which is the default port.
Step 8: Checking nmap installation ...
WARNING: Your version of nmap is not fully supported: 7.40
SUGGEST: You should install nmap 5.51 if you plan to use the nmap NSE NVTs.
Step 10: Checking presence of optional tools ...
OK: pdflatex found.
WARNING: PDF generation failed, most likely due to missing LaTeX packages. The PDF report format will not work.
SUGGEST: Install required LaTeX packages.
OK: ssh-keygen found, LSC credential generation for GNU/Linux targets is likely to work.
OK: rpm found, LSC credential package generation for RPM based targets is likely to work.
OK: alien found, LSC credential package generation for DEB based targets is likely to work.
OK: nsis found, LSC credential package generation for Microsoft Windows targets is likely to work.
To create the scanner in openvas, I use:
openvasmd --create-scanner="w3af" --scanner-host=127.0.0.1 --scanner-port=1235 --scanner-type="OSP" \
--scanner-ca-pub=/usr/local/var/lib/openvas/CA/cacert.pem \
--scanner-key-pub=/usr/local/var/lib/openvas/CA/clientcert.pem \
--scanner-key-priv=/usr/local/var/lib/openvas/private/CA/clientkey.pem
To run ospd-w3af scanner, I use:
~# ospd-w3af -b 127.0.0.1 -p 1235 -k \
/usr/local/var/lib/openvas/private/CA/clientkey.pem -c \
/usr/local/var/lib/openvas/CA/clientcert.pem --ca-file \
/usr/local/var/lib/openvas/CA/cacert.pem -L DEBUG
When I verify the scanner with openvasmd --verify-scanner xxxxx I got
Scanner version: 2018.8.22.
note: in the logs of the scanner I got this for every verify I do, I don't know if it's related or no and I didn't find a way to fix this:
2018-10-15 14:27:47,413 ospd.ospd: DEBUG: New connection from 127.0.0.1:60078
2018-10-15 14:27:49,430 ospd.ospd: DEBUG: Error: ('The read operation timed out',)
2018-10-15 14:27:49,433 ospd.ospd: DEBUG: 127.0.0.1:60078: Connection closed
So, my verification made, I want to create a task that uses this scanner but I can't save it due to error "Given scanner_type was invalid" :
https://i.stack.imgur.com/fvIJd.png
I got 0 connection to the chosen scanner at this moment and I can't find anything in the logs (maybe I can't search). I suspect the gsad UI being responsible for this but I can't find it.
I don't know what to do and if someone more expert than me (not very hard) could help that'd be great :)
Thanks in advance.
I solved this issue by creating a scan configuration for the ospd scanner (I though it didn't need one since it import them)
I faced another issue concerning ospd-w3af configuration, I couldn't create one because it needs ospd 1.0.0 installed, I modified the dependencies few days ago and it doesn't work with ospd 1.2.0
Now I'm facing the issue where the scans doesn't start properly. It stops at 1%
Getting openvas 9 running on new install of Ubuntu 18 was a pain. once i got past all my errors by creating files and ln -s for redis-server socks connections my tasks crapped out at 1%. My fix was install sudo apt install libopenvas-dev after that scans work and check-setup worked. Check-setup report no scanner but openvassd was running and openvasmd --verify-scanner (uuid) showed the scanner.
Just for a trial I got my hands dirty installing Meteor 1.6 using Chocolatey.
When I ran the command 'choco install meteor', after little time, I receive error below
Downloading meteor 64 bit
from 'https://packages.meteor.com/bootstrap-link?arch=os.windows.x86_64'
Progress: 48% - Saving 83.27 MB of 171.74 MB C
hocolatey timed out waiting for the command to finish. The timeout
specified (or the default value) was '2700' seconds. Perhaps try a
higher `--execution-timeout`? See `choco -h` for details.
The install of meteor was NOT successful.
Error while running 'C:\ProgramData\chocolatey\lib\meteor\tools\chocolateyinstal
l.ps1'.
See log for details.
Chocolatey installed 1/2 packages. 1 packages failed.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
Failures
- meteor (exited -1) - Error while running 'C:\ProgramData\chocolatey\lib\meteo
r\tools\chocolateyinstall.ps1'.
See log for details.
By default, Chocolatey has a command timeout of 45 minutes (2700 seconds) as documented here:
https://chocolatey.org/docs/chocolatey-configuration#general
Is the download of the meteor installer taking longer than this?
If so, you can change the value using a command similar to this:
choco config set --name <nameFromBelow> --value <value>
i.e. in your case (again, as an example):
choco config set --name commandExecutionTimeoutSeconds --value 5400
Or, if you only want this timeout to apply to a single command, you can try:
choco install meteor --execution-timeout 5400
As mentioned in the error output.
I am trying use the package manager in chef and installing nginx server but every time i ran the cook book on my client it is just saying that
Recipe: nginx::default
* yum_package[nginx] action install[2017-03-11T06:16:01-05:00] INFO: Processing yum_package[nginx] action install (nginx::default line 11)
* No candidate version available for nginx
================================================================================
Error executing action `install` on resource 'yum_package[nginx]'
================================================================================
Chef::Exceptions::Package
-------------------------
No candidate version available for nginx
Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/nginx/recipes/default.rb
11: package "nginx" do
12: action :install
13: end
14:
Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/nginx/recipes/default.rb:11:in `from_file'
yum_package("nginx") do
package_name "nginx"
action [:install]
retries 0
retry_delay 2
default_guard_interpreter :default
declared_type :package
cookbook_name "nginx"
recipe_name "default"
flush_cache {:before=>false, :after=>false}
end
Platform:
---------
x86_64-linux
[2017-03-11T06:16:29-05:00] INFO: Running queued delayed notifications before re-raising exception
Running handlers:
[2017-03-11T06:16:29-05:00] ERROR: Running exception handlers
Running handlers complete
[2017-03-11T06:16:29-05:00] ERROR: Exception handlers complete
Chef Client failed. 1 resources updated in 05 minutes 44 seconds
And even i tried to install epel-release packages but denied with similar errors.
Any idea how we could install nginx with CHEF recipe.
I tried also using yum_package but had no luck in installing
yum_package "nginx" do
action :install
end
Thanks
This means there is no package named nginx in your repositories. If you log into the machine you want to provision (With kitchen login, for example), you can try to search package nginx.
The best way to install it if it is not in your repositories is either adding nginx´s official repo with a chef repository resource (Like yum_repository for Centos) or downloading the tarball with Chef resource remote_file.
If you choose the last option, be sure to generate a sha256 of the tarball you download and add it to the remote_file resource, so among other things you prevent Chef from downloading every run the file.
-Edit-
As Szymon says, you can also use the Nginx cookbook for this and don't write any special recipe.
As discussed here, you can use official NGINX chef cookbook or just install epel-release before installing NGINX:
if platform_family?('rhel')
package 'epel-release'
end
if platform_family?('debian')
apt_update 'update'
end
package 'nginx'
On my Macbook I'm using VirtualBox+Vagrant. For Vagrant I downloaded VagrantPress (from vagrantpress.org), which is a Vagrantfile with Wordpress and some Puppet scripts. It looks like this:
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vm.box = "base"
config.vm.box_url = "http://cloud-images.ubuntu.com/vagrant/precise/current/precise-server-cloudimg-amd64-vagrant-disk1.box"
config.vm.network :forwarded_port, guest: 80, host: 8080
config.vm.provision :puppet do |puppet|
puppet.manifests_path = "puppet/manifests"
puppet.module_path = "puppet/modules"
puppet.manifest_file = "init.pp"
puppet.options="--verbose --debug"
end
end
At first, this was running as intended (viewing the Wordpress installation on localhost), but I have it installed on a Dropbox sub-directory, and while trying to get it to work on another computer which runs Ubuntu, I may have changed something that broke the whole setup.
Now, when running 'Vagrant up' on my macbook (which used to run well), I see this:
Bringing machine 'default' up with 'virtualbox' provider...
[default] Clearing any previously set forwarded ports...
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] -- 80 => 8080 (adapter 1)
[default] Booting VM...
GuestAdditions versions on your host (4.2.18) and guest (4.1.12) do not match.
Reading package lists...
Building dependency tree...
Reading state information...
dkms is already the newest version.
dkms set to manually installed.
linux-headers-3.2.0-54-generic is already the newest version.
linux-headers-3.2.0-54-generic set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Copy iso file /Applications/VirtualBox.app/Contents/MacOS/VBoxGuestAdditions.iso into the box /tmp/VBoxGuestAdditions.iso
mount: warning: /mnt seems to be mounted read-only.
Installing Virtualbox Guest Additions 4.2.18 - guest version is 4.1.12
Verifying archive integrity... All good.
Uncompressing VirtualBox 4.2.18 Guest Additions for Linux............
VirtualBox Guest Additions installer
You appear to have a version of the VBoxGuestAdditions software
on your system which was installed from a different source or using a
different type of installer. If you installed it from a package from your
Linux distribution or if it is a default part of the system then we strongly
recommend that you cancel this installation and remove it properly before
installing this version. If this is simply an older or a damaged
installation you may safely proceed.
Do you wish to continue anyway? [yes or no]
Cancelling installation.
An error occurred during installation of VirtualBox Guest Additions 4.2.18. Some functionality may not work as intended.
[default] Waiting for machine to boot. This may take a few minutes...
[default] Machine booted and ready!
[default] The guest additions on this VM do not match the installed version of
VirtualBox! In most cases this is fine, but in rare cases it can
cause things such as shared folders to not work properly. If you see
shared folder errors, please update the guest additions within the
virtual machine and reload your VM.
Guest Additions Version: 4.1.12
VirtualBox Version: 4.2
[default] Mounting shared folders...
[default] -- /vagrant
[default] -- /tmp/vagrant-puppet/manifests
[default] -- /tmp/vagrant-puppet/modules-0
and localhost on Chrome says 'No data received'.
After that, I 'Vagrant ssh' and ran:
curl 'http://localhost/'
and get:
cURL Error (7): couldn't connect to host ...
I have been trying to fix this for days now. I even tried reinstalling Vagrant and VirtualBox but it didn't work.
Also, I tried to manually update the Guest Additions (although this was not necessary before), and after getting 'GuestAdditions 4.2.18 running --- OK.' after booting the VM, I still get the same errors for localhost.
What am I doing wrong?
If I understand your setup correctly, you would want to try http://localhost:8080/ on the host system (in order to connect with port 80 on the guest).
Inside the guest OS, are you sure the webserver is running? It might not be starting automatically.
Also, I think you can ignore the Guest Additions version mismatch. It's likely not related to the problem you're seeing.