I'm trying to build up some EC2 resources using saltstack on Ubuntu 14.04. Problem is that the python-boto package that comes with the 14.04 is too old, and I need to use pip to install something more up to date. I have the following SLS to express this state:
python-boto:
pkg.purged
python-pip:
pkg.installed
boto:
pip.installed:
- name: boto >= 2.6
- require:
- pkg: python-pip
default_lc:
boto_lc.present:
- require:
- pip: boto
- image_id: ami-3d2cce5d
- key_name: uberhaus
- security_groups:
- default
- instance_type: m3.medium
- volume_type: gp2
The first time I run a highstate, the boto_lc resource fails. On subsequent highstate runs, it succeeds.
How do I deal with this? Do I need to use Reactor or Orchestrate to somehow ensure the boto resources are run only after a previous run to install boto?
I think that you need a 'reload_modules: True' in the pip.installed state, which will then make the boto module available for salt to use in the subsequent state(s).
Related
I am trying to configure an external pillar in github, but no matter what I cannot get the minions to successfully read top.sls. Below is my ext_pillar and pillar_roots config:
pillar_roots:
base:
- /srv/pillar
fileserver_backend:
- gitfs
- roots
gitfs_update_interval: 60
gitfs_base: main
gitfs_remotes:
- https://gituser:gittoken#github.com/gitaccount/saltstack.git:
- mountpoint: salt://
ext_pillar:
- git:
- main https://gituser:gittoken#github.com/gitaccount/saltpillar.git
I have the following in the root of my saltpillar repo:
top.sls:
base:
'*':
- data
data.sls:
info: some test data from remote pillar
Repos are accessible with the URIs provided. When I run salt '*' saltutil.refresh_pillar and then salt '*' pillar.items I get no results. However, I can put top.sls and data.sls directly into /srv/pillar and it works. I put the master in debug mode and don't see any errors running the commands. Any help is appreciated.
Does the following ext_pillar configuration fix your issue? I'm assuming your top.sls you posted is still in the main branch of your git repo.
ext_pillar:
- git:
- main https://gituser:gittoken#github.com/gitaccount/saltpillar.git
- env: base
Your top.sls must reference your actual branch name or you can add the env option to specify a different name.
https://docs.saltproject.io/en/latest/ref/pillar/all/salt.pillar.git_pillar.html
As TravisCI.org is no longer free for small open source projects, I am trying to setup CircleCI and CodeCov.
Creating the Coverage report in CircleCI seems to work:
But uploading to CodeCov fails, claming report cannot be found:
I followed the instructions at https://circleci.com/docs/2.0/code-coverage/#codecov
Used orb codecov/codecov#1.0.2
Allowed unprivate orbs
Using CircleCI 2.1
Generating phpdbg
I tried with store_artificats and without, unclear to me if this shall be used with codecov, but both fail
Thats my config.yml:
# PHP CircleCI 2.0 configuration file
# See: https://circleci.com/docs/2.0/language-php/
version: 2.1
orbs:
codecov: codecov/codecov#1.0.2
# Define a job to be invoked later in a workflow.
# See: https://circleci.com/docs/2.0/configuration-reference/#jobs
jobs:
build:
# Specify the execution environment. You can specify an image from Dockerhub or use one of our Convenience Images from CircleCI's Developer Hub.
# See: https://circleci.com/docs/2.0/configuration-reference/#docker-machine-macos-windows-executor
docker:
# Specify the version you desire here
- image: circleci/php:7.2-node-browsers
# Specify service dependencies here if necessary
# CircleCI maintains a library of pre-built images
# documented at https://circleci.com/docs/2.0/circleci-images/
# Using the RAM variation mitigates I/O contention
# for database intensive operations.
# - image: circleci/mysql:5.7-ram
#
# - image: redis:2.8.19
# Add steps to the job
# See: https://circleci.com/docs/2.0/configuration-reference/#steps
steps:
- checkout
- run: sudo apt update # PHP CircleCI 2.0 Configuration File# PHP CircleCI 2.0 Configuration File sudo apt install zlib1g-dev libsqlite3-dev
- run: sudo docker-php-ext-install zip
# Download and cache dependencies
- restore_cache:
keys:
# "composer.lock" can be used if it is committed to the repo
- v1-dependencies-{{ checksum "composer.json" }}
# fallback to using the latest cache if no exact match is found
- v1-dependencies-
- run: composer install -n --prefer-dist
- save_cache:
key: v1-dependencies-{{ checksum "composer.json" }}
paths:
- ./vendor
# run tests with phpunit or codecept
#- run: ./vendor/bin/phpunit
- run:
name: "Run tests"
command: phpdbg -qrr vendor/bin/phpunit --coverage-html build/coverage-report
- codecov/upload:
file: build/coverage-report
Here is the failing build:
https://app.circleci.com/pipelines/github/iwasherefirst2/laravel-multimail/25/workflows/57e6a71c-7614-4a4e-a7cc-53f015b3d437/jobs/35
Codecov is not able to process HTML coverage reports. You should ask phpunit to output XML as well by either changing or appending your command to read --coverage-clover coverage.xml
You can view a list of the supported and unsupported coverage formats at https://docs.codecov.com/docs/supported-report-formats
[1] Saved https://web.archive.org/web/20220113142241/https://docs.codecov.com/docs/supported-report-formats
I have a repository with salt states for provisioning my cluster of servers in the cloud. Over time, I kept on adding more states - the .sls files - into this repo. Now im starting to struggle what is what and what is where.
I am wondering if there is a there is some software utility/package that will generate documentation off my states repository, preferably as html pages, so that I can browse them and see their interdependencies.
UPDATE:
The state sls files look like this:
include:
- states.core.pip
virtualenv:
pip.installed:
- require:
- sls: states.core.pip
virtualenvwrapper:
pip.installed:
- require:
- sls: states.core.pip
And another sls example:
{% set user_home = '/home/username' %}
my_executable_virtualenv:
virtualenv.managed:
- name: {{ user_home }}/.virtualenvs/my_executable_virtualenv
- user: username
- system_site_packages: False
- pip_pkgs:
- requests
- numpy
- pip_upgrade: True
- require:
- sls: states.core
my_executable_supervisor_entry:
file.managed:
- name: /etc/supervisor/conf.d/my_executable.conf
- source: salt://files/supervisor_config/my_executable.conf
- user: username
- group: username
- mode: 644
- makedirs: False
- require:
- sls: states.core
I did some research and found that salt stack has created one. It does work as HTML pages too. According to the documentation. If you have python installed installing Sphinx is as easy as doing
C:\> pip install sphinx
Salt-stacks docs on this can be found here. According to the docs making the HTML documentation is as easy as doing:
cd /path/to/salt/doc
make HTML
I hope this answer is what you were looking for!
This needs a custom plugin which needs to be written.
There is no plugins directly available to render sls files.
There are some plugins available for rendering YAML files, may be you can modify the same to suite your requirement.
You can use some of the functions in the state module to list all the everything in the highstate for a minion:
# salt-call state.show_states --out=yaml
local:
- ufw.package.install
- ufw.config.file
- ufw.service.enable
- ufw.service.reload
- ufw.config.services
- ufw.config.applications
- ufw.service.running
- apt.apt_conf
- apt.unattended
- cacerts
- kerberos
- network
- editor
- mounts
- openssh
- openssh.config_ini
- openssh.known_hosts
...
And then view the compiled data for each one (also works with states not in the highstate):
# salt-call state.show_sls editor --out=yaml
local:
vim-tiny:
pkg:
- installed
- order: 10000
__sls__: csrf.editor
__env__: base
editor:
alternatives:
- path: /usr/bin/vim.tiny
- set
- order: 10001
__sls__: csrf.editor
__env__: base
Or to get the entire highstate at once with state.show_highstate.
I'm not aware of any tools to build HTML documentation from that. You'd have to do that yourself.
To access all states (not just a particular highstate), you can use salt-run fileserver.file_list | grep '.sls$' to find every state, and salt-run state.orchestrate_show_sls to get the rendered data for each (though you may need to supply pillar data).
Let's assume I have the next simple Ansible playbook:
---
tasks:
- name: Upgrade installed packages
become: true
apt:
upgrade: safe
- name: Install NGINX web server
become: true
apt:
name: nginx
state: latest
notify:
- Restart NGINX
handlers:
- name: Restart NGINX
become: true
service:
name: nginx
state: restarted
As you see, I upgrade installed APT packages first and only then make sure I have the latest Nginx version. The problem is that if there's an update for Nginx, it will be installed in the first task and if so, the second task won't be marked as changed and the handler won't be fired. Is it true? Or Ansible is clever enough to somehow fire this handle only when Nginx was upgraded in the first task?
I wonder about the best practice for this case. Is there a better way than move all the separate installation tasks (which should fire handlers on their change) before the task which upgrades all the installed packages?
Thanks!
This is not "The Ansible way", but it is an option.
one why you can do it is by using lsof to find all the pid's which need restart and pass this information to systemd to get the service name for each pid. And then go over the list of services and restart each one of them.
some one all ready wrote a perl-script like that
- see example here: https://rwmj.wordpress.com/2014/07/10/which-services-need-restarting-after-an-upgrade/
another option is but the same is the restart-services script from the debian-goodies repo/package.
for my tests I need a dummy server which outputs sample/test code. I use a node http server for that and start it before my scripts with node ./test/server.js.
I can start it, but the issue is that it's taking up the instance and therefore can't run the tests now.
So the question is, how can I run the server in the background/new instance so it doesn't conflict with that? I do stop the server with the end_script so I don't have to terminate it.
This is my travis-config so far:
language: node_js
node_js:
- "6.1"
cache:
directories:
— node_modules
install:
- npm install
before_script:
- sh -e node ./test/server.js
script:
- ./node_modules/mocha-phantomjs/bin/mocha-phantomjs ./test/browser/jelly.html
- ./node_modules/mocha-phantomjs/bin/mocha-phantomjs ./test/browser/form.html
- ./node_modules/mocha/bin/mocha ./test/node/jelly.js
after_script:
- curl http://localhost:5555/close
You can background the process by appending a &:
before_script:
- node ./test/server.js &