CodeDeploy fails at DownloadBundle saying Broken pipe # io_write - <STDERR> - aws-code-deploy

I am using CloudFormation, CodePipeline, CodeDeploy, & CodeCommit to deploy a Rails app to an EC2 instance.
Once the stack is built, CodePipeline kicks off and the first deploy succeeds without issue, but each deploy after that fails at the DownloadBundle step. If I restart the codedeploy-agent on the EC2 instance then the next deploy will succeed, but the following deploys will fail as before.
From the AWS Dashboard, CodeDeploy complains that "too few healthy instances are available" (Error code: HEALTH_CONSTRAINTS)
Under the "View Events" link of CodeDeploy it shows:
Error Code: UnknownError
Script Name
Message: Broken pipe # io_write - <STDERR>
Log Tail
Here is the output from the codedeploy-agent.log:
2017-02-23 21:18:33 INFO [codedeploy-agent(28647)]: [Aws::CodeDeployCommand::Client 200 0.041592 0 retries] put_host_command_complete(command_status:"Failed",diagnostics:{format:"JSON",payload:"{\"error_code\":5,\"script_name\":\"\",\"message\":\"Broken pipe # io_write - <STDERR>\",\"log\":\"\"}"},host_command_identifier:" . . . . ")
2017-02-23 21:18:33 ERROR [codedeploy-agent(28647)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Error during perform: Errno::EPIPE - Broken pipe # io_write - <STDERR> - /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/entry.rb:573:in `write' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/entry.rb:573:in `warn' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/entry.rb:573:in `rescue in set_time' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/entry.rb:571:in `set_time' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/entry.rb:365:in `read_c_dir_entry' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/entry.rb:191:in `read_c_dir_entry' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/central_directory.rb:127:in `block in read_central_directory_entries' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/central_directory.rb:126:in `times' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/central_directory.rb:126:in `read_central_directory_entries' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/central_directory.rb:138:in `read_from_stream' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/file.rb:77:in `block in initialize' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/file.rb:76:in `open' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/file.rb:76:in `initialize' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/file.rb:98:in `new' /usr/local/lib/ruby/gems/2.2.0/gems/rubyzip-1.2.1/lib/zip/file.rb:98:in `open' /opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:317:in `unpack_bundle' /opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:98:in `block in <class:CommandExecutor>' /opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:62:in `execute_command' /opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_poller.rb:132:in `process_command' /opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_poller.rb:65:in `perform' /opt/codedeploy-agent/lib/instance_agent/agent/base.rb:28:in `run' /opt/codedeploy-agent/lib/instance_agent/runner/child.rb:38:in `block in run' /opt/codedeploy-agent/lib/instance_agent/runner/child.rb:55:in `with_error_handling' /opt/codedeploy-agent/lib/instance_agent/runner/child.rb:37:in `run' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:70:in `block in run_with_error_handling' /opt/codedeploy-agent/lib/instance_agent/runner/child.rb:55:in `with_error_handling' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:69:in `run_with_error_handling' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:33:in `block in start' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:22:in `loop' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:22:in `start' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:206:in `block in spawn_child' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:204:in `fork' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:204:in `spawn_child' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:196:in `block in spawn_children' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:195:in `times' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:195:in `spawn_children' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:134:in `start' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:37:in `block in start' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:36:in `fork' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:36:in `start' /opt/codedeploy-agent/bin/../lib/codedeploy-agent.rb:41:in `block (2 levels) in <main>' /opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/command_support.rb:130:in `call' /opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/command_support.rb:130:in `execute' /opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/app_support.rb:262:in `block in call_command' /opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/app_support.rb:275:in `call' /opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/app_support.rb:275:in `call_command' /opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/app_support.rb:69:in `run' /opt/codedeploy-agent/bin/../lib/codedeploy-agent.rb:88:in `<main>'
I have seen others having similar problems, but not specifically what I'm seeing here.
Some solutions I've come across:
Configure a cron to periodically restart the codedeploy agent.
# Restarting the agent is not a practical solution.
Other issues seem to be related to how the code is pulled from S3 or GitHub.
# This deploy is using CodeCommit instead of S3 or GitHub.
I can provide more detail if needed.

Solved
It turns out that one of my deploy scripts was generating a ruby error with the CodeDeploy agent (see the codedeploy-agent output in the original question). Once I removed that script from my appspec.yml, the deploy began working again.
This was not very intuitive since the CodeDeploy error was showing up in the DownloadBundle step. Apparently the deploy was progressing past that step and failing on the second deploy script in the AfterInstall step.
I wasted a lot of time chasing what I thought was a permissions issue.

Related

"Environment data not found in the schema." with database-cleaner when calling cucumber repeatedly on Rails 6

Note: This is NOT a dupe of Rails 5, Rspec: Environment data not found in the schema, though it is similar. The affected versions are different, and the solve doesn't work for me.
My Rails setup is as follows:
Rails 6.0.1
cucumber-rails 2.0.0
database_cleaner 1.7.0
Whenever I run cucumber a second or subsequent time, I get this error:
3924 ~/fork/my-proj$ bundle exec cucumber
rails aborted!
ActiveRecord::NoEnvironmentInSchemaError:
Environment data not found in the schema. To resolve this issue, run:
rails db:environment:set RAILS_ENV=test
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/activerecord-6.0.1/lib/active_record/migration.rb:1151:in `last_stored_environment'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/activerecord-6.0.1/lib/active_record/tasks/database_tasks.rb:60:in `check_protected_environments!'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/activerecord-6.0.1/lib/active_record/railties/databases.rake:15:in `block (2 levels) in <main>'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/activerecord-6.0.1/lib/active_record/railties/databases.rake:491:in `block (3 levels) in <main>'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/railties-6.0.1/lib/rails/commands/rake/rake_command.rb:23:in `block in perform'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/railties-6.0.1/lib/rails/commands/rake/rake_command.rb:20:in `perform'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/railties-6.0.1/lib/rails/command.rb:48:in `invoke'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/railties-6.0.1/lib/rails/commands.rb:18:in `<main>'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/bootsnap-1.4.5/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/bootsnap-1.4.5/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `block in require_with_bootsnap_lfi'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/bootsnap-1.4.5/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/bootsnap-1.4.5/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `require_with_bootsnap_lfi'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/bootsnap-1.4.5/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/activesupport-6.0.1/lib/active_support/dependencies.rb:325:in `block in require'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/activesupport-6.0.1/lib/active_support/dependencies.rb:291:in `load_dependency'
/Users/grantb/.rvm/gems/ruby-2.6.5/gems/activesupport-6.0.1/lib/active_support/dependencies.rb:325:in `require'
bin/rails:4:in `<main>'
Tasks: TOP => db:test:load => db:test:purge => db:check_protected_environments
(See full trace by running task with --trace)
Using the default profile...
[...cucumber continues as normal...]
How the heck can I get rid of that?
If I run rails db:environment:set RAILS_ENV=test, it'll get rid of it for the next run of cucumber, but it will always come back on the run after that.
I found one other answer that suggested adding this to my config, but it did exactly nothing:
# This did NOT fix my problem:
DatabaseCleaner.clean_with(
:truncation,
except: %w(ar_internal_metadata)
)
Clearly database_cleaner is deleting environment data, but I don't know how to make it stop.
Can anyone help me get rid of this annoying warning?
Turns out this problem was specific to database_cleaner 1.7.
A recent upgrade to 1.8.3 made this error stop happening.
(associated Github issue)

Ruby App works with webrick but not with passenger

I'm trying to deploy a ruby app to a server running Ubuntu -> Nginx -> Passenger.
I can access the site fine using 'rails s' however passenger keeps returning the bellow error into the nginx log. I've checked that rake 12.0.0 is installed and also tried setting up a new rails project to check if passenger would provide the "your on rails" page. But that doesn't work either.
Any help would be greatly appreciated, I've tried googleing and performing various bundle updates and gem installs.
Error ID: a06f05a9
Error details saved to: /tmp/passenger-error-EVNeAF.html
Message from application: <p>It looks like Bundler could not find a gem. Maybe you didn't install all the gems that this application needs. To install your gems, please run:</p>
<pre class="commands">bundle install</pre>
<p>If that didn't work, then the problem is probably caused by your application being run under a different environment than it's supposed to. Please check the following:</p>
<ol>
<li>Is this app supposed to be run as the <code>dmn</code> user?</li>
<li>Is this app being run on the correct Ruby interpreter? Below you will
see which Ruby interpreter Phusion Passenger attempted to use.</li>
<li>Please check whether the correct RVM gemset is being used.</li>
<li>Sometimes, RVM gemsets may be broken.
Try resetting them.</li>
</ol>
<p>-------- The exception is as follows: -------</p>
Could not find rake-12.0.0 in any of the sources (Bundler::GemNotFound)
<pre> /home/dmn/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.4/lib/bundler/spec_set.rb:87:in `block in materialize'
/home/dmn/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.4/lib/bundler/spec_set.rb:81:in `map!'
/home/dmn/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.4/lib/bundler/spec_set.rb:81:in `materialize'
/home/dmn/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.4/lib/bundler/definition.rb:159:in `specs'
/home/dmn/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.4/lib/bundler/definition.rb:218:in `specs_for'
/home/dmn/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.4/lib/bundler/definition.rb:207:in `requested_specs'
/home/dmn/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.4/lib/bundler/runtime.rb:109:in `block in definition_method'
/home/dmn/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.4/lib/bundler/runtime.rb:21:in `setup'
/home/dmn/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.4/lib/bundler.rb:101:in `setup'
/home/dmn/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.4/lib/bundler/setup.rb:19:in `<top (required)>'
/home/dmn/.rvm/rubies/ruby-2.3.3/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/home/dmn/.rvm/rubies/ruby-2.3.3/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/usr/lib/ruby/vendor_ruby/phusion_passenger/loader_shared_helpers.rb:430:in `activate_gem'
/usr/lib/ruby/vendor_ruby/phusion_passenger/loader_shared_helpers.rb:297:in `block in run_load_path_setup_code'
/usr/lib/ruby/vendor_ruby/phusion_passenger/loader_shared_helpers.rb:435:in `running_bundler'
/usr/lib/ruby/vendor_ruby/phusion_passenger/loader_shared_helpers.rb:296:in `run_load_path_setup_code'
/usr/share/passenger/helper-scripts/rack-preloader.rb:100:in `preload_app'
/usr/share/passenger/helper-scripts/rack-preloader.rb:156:in `<module:App>'
/usr/share/passenger/helper-scripts/rack-preloader.rb:30:in `<module:PhusionPassenger>'
/usr/share/passenger/helper-scripts/rack-preloader.rb:29:in `<main>'</pre>"

Restore fails for a drupal app on openshift

I consistently get failures when trying to restore my drupal app to an openshift gear. The only way I have been able to restore my app is to delete it then recreate the app then add the restore data to the newly created app. It looks like a permissions error.
Anyone else having this issue or have ideas for a solution?
Here is what the process looks like:
andy#hermes:~/Siesta Solutions Projects and Files/Marias/Backups$ rhc snapshot restore mariadaycare -f mariasdaycare.tar.gz
Restoring from snapshot mariasdaycare.tar.gz to application 'mariadaycare' ...
Error in trying to restore snapshot. You can try to restore manually by running:
cat 'mariasdaycare.tar.gz' | ssh
54034e07e0b8cd2608000659#mariadaycare-siestasolutions.rhcloud.com 'restore
INCLUDE_GIT'
andy#hermes:~/Siesta Solutions Projects and Files/Marias/Backups$ cat 'mariasdaycare.tar.gz' | ssh 54034e07e0b8cd2608000659#mariadaycare-siestasolutions.rhcloud.com 'restore INCLUDE_GIT'
Removing old git repo: ~/git/mariadaycare.git/
Removing old data dir: ~/app-root/data/*
Restoring ~/git/mariadaycare.git and ~/app-root/data
/bin/tar: app-root/data/sites/default/settings.php: Cannot open: Permission denied
/bin/tar: Exiting with failure status due to previous errors
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.28.6/lib/openshift-origin-node/utils/shell_exec.rb:133:in `block (2 levels) in oo_spawn': Shell command '/bin/tar --strip=2 --overwrite -xmz ./*/app-root/data ./*/app-deployments ./*/git --transform="s|${OPENSHIFT_GEAR_NAME}/data|app-root/data|" --transform="s|git/.*\.git|git/${OPENSHIFT_GEAR_NAME}.git|" --exclude="./*/app-root/runtime/data" --exclude="./*/mysql/data" --exclude="./*/php/data" --exclude="./*/phpmyadmin/data" --exclude="./*/cron/data" --exclude="./*/git/*.git/hooks" 1>&2' returned an error. rc=2 (OpenShift::Runtime::Utils::ShellExecutionException)
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.28.6/lib/openshift-origin-node/utils/shell_exec.rb:96:in `pipe'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.28.6/lib/openshift-origin-node/utils/shell_exec.rb:96:in `block in oo_spawn'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.28.6/lib/openshift-origin-node/utils/shell_exec.rb:95:in `pipe'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.28.6/lib/openshift-origin-node/utils/shell_exec.rb:95:in `oo_spawn'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-container-selinux-0.10.1/lib/openshift/runtime/containerization/selinux_container.rb:368:in `run_in_container_context'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.28.6/lib/openshift-origin-node/model/application_container.rb:795:in `run_in_container_context'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.28.6/lib/openshift-origin-node/model/application_container_ext/snapshots.rb:259:in `extract_restore_archive'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.28.6/lib/openshift-origin-node/model/application_container_ext/snapshots.rb:162:in `restore'
from /usr/bin/gear:585:in `block (2 levels) in <main>'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/command.rb:180:in `call'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/command.rb:180:in `call'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/command.rb:155:in `run'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/runner.rb:385:in `run_active_command'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/runner.rb:62:in `run!'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/delegates.rb:11:in `run!'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/import.rb:10:in `block in <top (required)>'
andy#hermes:~/Siesta Solutions Projects and Files/Marias/Backups$
I had the same issue many times. One time, it failed and then I login in, change the permission of the settings.php and default/ in the gear (chmod +w, not the snapshot), uploaded the snapshot to the gear and manually restore the snapshot(cat '/path/to/snapshot_file.tar.gz' | oo-restore INCLUDE_GIT). And it worked. I know it seems not logical but maybe you can try.

Rubymine RSpec kind of error after successful test

I am a newbie just learnt how to use RSpec. My RSpec works fine in windows terminal. However, in Rubymine build-in RSpec. It errors after giving success message. Which is really annoying.
Git: https://github.com/ivawzh/sample_app
Please help.
Thanks and best regards.
RSpec works fine in native cmd
Not sure if this matters, this is the Spork drb server
Rubymine build-in RSpec, it says the tests are passed. But errors come after success message.
Error messages:
Running specs...
Command line:
["C:\\Ruby200\\bin\\ruby.exe", "-e", "$stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift)", "C:\\Ruby200\\bin/rspec", "--require", "teamcity/spec/runner/formatter/teamcity/formatter", "--format", "Spec::Runner::Formatter::TeamcityFormatter", "E:/RoR docs/sample_app/spec/controllers/pages_controller_spec.rb", "--drb", "--backtrace"]
2 examples, 0 failures, 2 passed
Finished in 0.17701 seconds
<-- Slave(1) run done!
C:/Ruby200/lib/ruby/gems/2.0.0/gems/rake-10.1.0/lib/rake/file_utils.rb:54:in `block in create_shell_runner': Command failed with status (1): [C:\Ruby200\bin\ruby.exe -e $stdout.sync=tr...] (RuntimeError)
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rake-10.1.0/lib/rake/file_utils.rb:45:in `call'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rake-10.1.0/lib/rake/file_utils.rb:45:in `sh'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rake-10.1.0/lib/rake/file_utils_ext.rb:37:in `sh'
from C:/Program Files (x86)/JetBrains/RubyMine 5.4.2/rb/testing/runner/specs_in_folder_runner.rb:139:in `<top (required)>'
from -e:1:in `load'
from -e:1:in `<main>'
Process finished with exit code 1
Environment:
Ruby version 2.0.0 (i386-mingw32)
RubyGems version 2.0.2
Rack version 1.5
Rails version 4.0.0
Environment development
Database adapter postgresql

Chef Cookbook Compilation Error While Bootstrapping The Node

I am using chef to bootstrap a node. I have created encrypted data bags and copied the secret key on the node as /etc/chef/encrypted_data_bag_secret.
When I try to bootstrap the node, I am getting the following error:
================================================================================
Recipe Compile Error in /var/chef/cache/cookbooks/webapps-configs/recipes/httpd.rb
================================================================================
Net::HTTPServerException
------------------------
404 "Not Found"
Cookbook Trace:
---------------
/var/chef/cache/cookbooks/webapps-configs/recipes/httpd.rb:12:in `from_file'
Relevant File Content:
----------------------
/var/chef/cache/cookbooks/webapps-configs/recipes/httpd.rb:
10:
11: CHEF_ENV = node.chef_environment
12>> http_ssl_creds = Chef::EncryptedDataBagItem.load("#{CHEF_ENV}", "http-ssl")
13: PRIVATE_KEY = http_ssl_creds["private_key"]
14: CHAIN_FILE= http_ssl_creds["chain_file"]
15: CERT= http_ssl_creds["cert"]
16:
ERROR: Running exception handlers
FATAL: Saving node information to /var/chef/cache/failed-run-data.json
ERROR: Exception handlers complete
Chef Client failed. 0 resources updated
FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
FATAL: Net::HTTPServerException: 404 "Not Found"
Here is the stack trace:
Net::HTTPServerException: 404 "Not Found"
/opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2632:in `error!'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/rest.rb:199:in `block in raw_http_request'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/rest.rb:280:in `retriable_rest_request'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/rest.rb:164:in `raw_http_request'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/rest.rb:158:in `api_request'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/rest.rb:99:in `get'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/data_bag_item.rb:149:in `load'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/encrypted_data_bag_item.rb:294:in `load'
/var/chef/cache/cookbooks/webapps-configs/recipes/httpd.rb:12:in `from_file'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/mixin/from_file.rb:30:in `instance_eval'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/mixin/from_file.rb:30:in `from_file'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/cookbook_version.rb:346:in `load_recipe'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/run_context.rb:151:in `load_recipe'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/run_context/cookbook_compiler.rb:139:in `block in compile_recipes'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/run_context/cookbook_compiler.rb:137:in `each'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/run_context/cookbook_compiler.rb:137:in `compile_recipes'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/run_context/cookbook_compiler.rb:74:in `compile'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/run_context.rb:86:in `load'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:224:in `setup_run_context'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:467:in `do_run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:200:in `run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application.rb:190:in `run_chef_client'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:297:in `block in run_application'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:290:in `loop'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:290:in `run_application'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application.rb:73:in `run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/bin/chef-client:26:in `<top (required)>'
/usr/bin/chef-client:23:in `load'
/usr/bin/chef-client:23:in `<main>'
Any help or pointers?
I have resolved this issue. It appears that the environment was not created properly. So when I was bootstrapping the node, its environment was being set to "_default", hence the node was unable to locate the items.
I recreated the environment, and bootstrapped node by passing --environment parameter, and I was able to bootstrap without any issues.
Once again, thanks to all the contributors for their inputs. This community is awesome!
Try adding the following line to your recipe:
11. CHEF_ENV = node.chef_environment
+ 12. Chef::Config[:encrypted_data_bag_secret]
13. http_ssl_creds = Chef::EncryptedDataBagItem.load("#{CHEF_ENV}", "http-ssl")
14. PRIVATE_KEY = http_ssl_creds['private_key']
15. CHAIN_FILE= http_ssl_creds['chain_file']
16. CERT= http_ssl_creds['cert']
Another thing to check is that the node environment is the name of your encrypted data bag.
since you 've copied the /etc/chef/encrypted_data_bag_secret to the node, that should do the trick.

Resources