Im trying to integrate Sagemaker pipeline with Jenkins. Im using aws-cli ( version - 2.1.24 ).
Since this version doesnt support --pipeline-definition-s3-location, Im trying to do something like below -
aws s3 cp s3://some_bucket/folder1/pipeine_definition.json - | \ jq -c . | \ tee /dev/stderr | \ xargs -0 -I{} aws sagemaker update-pipeline --pipeline-name "pipelinename" --role-arn "arn:aws:iam::<account_id>:role/sagemaker-role" --pipeline-definition '{}'
And I found this error -
An error occurred (ValidationException) when calling the UpdatePipeline operation: Pipeline definition: At least 1 step must be provided
When I recheck the definition.json, Im able to see the steps defined inside json.
Can someone help me?
I tried adding the quotes for --pipeline-definition, which isnt working.
Since jenkins has aws-cli 2.1.24 version, I want to someone copy the contents of json file in s3 and pass it to --pipeline-definition argument using aws sagemaker --update-pipeline command.
Related
My deployments fail on last step Validate Service with error message:
The overall deployment failed because too many individual instances failed deployment, too few healthy instances are available for deployment, or some instances in your deployment group are experiencing problems.
Events log
No lines are selected.
My validate_service.sh contain
#!/bin/bash
# verify we can access our webpage successfully
curl -v --silent localhost:80 2>&1 | grep Welcome
Can someone advice what should I change ?
Script return value matters. Yours looks good to me. I just added couple of seconds to wait until application starts up.
In case you use bash -x together with pipeline of commands, you better add shopt -s pipefail so all pipeline fails when one of the commands fails.
Checkout my script:
#!/bin/bash
sleep 5
curl http://localhost:3009 | grep Welcome
I am running this command:
{{ aws ec2 describe-availability-zones --region ca-central-1 | jq '.AvailabilityZones[]|(.ZoneName)}}'
on 2 identical MacOs and one Amazon Linux.
The MacOs subject to this question is showing this error :
parse error: Invalid numeric literal at line 1, column 18
However, the Amazon and the Other MacOS are showing the correct output
Please help me! This is driving me crazy
This error message indicates that the input piped to jq wasn't valid JSON. Since this input comes directly from the output of the aws ec2 describe-instances command it looks like it isn't emitting JSON, or it's emitting other text as well as JSON.
The fastest way to diagnose this is going to be for you to find out what text the aws command is emitting.
One possible cause could be that you have it configured via an environment variable (AWS_DEFAULT_OUTPUT) or configuration file (e.g. ~/.aws/config) to output YAML or text or tables. (In fact, I consider this probable. I can reproduce the error message exactly down to the column number if I set it to output YAML.) You could rule this out by explicitly specifying --output json.
Beyond that, I suggest you compare these machines to each other. For example, try this on each machine and see what's different on the odd machine:
echo Versions:
aws --version
jq --version
echo Environment:
env | grep '^AWS_'
echo AWS configuration:
aws configure list
echo AWS config file:
cat ~/.aws/config
Originally posted in https://github.com/flyway/flyway/issues/2429
I have an issue (probably a wrong configuration) using flyway placeholders; I can use placeholders for my own variables; but it fails cos one value in an sql query has a similar syntax as flyway placeholder syntax.
Which version and edition of Flyway are you using?
5.2.4 using official docker image
If this is not the latest version, can you reproduce the issue with the latest one as well? (Many bugs are fixed in newer releases and upgrading will often resolve the issue)
5.2.4 tag is the latest version in docker hub (https://hub.docker.com/r/boxfuse/flyway/)
Which client are you using? (Command-line, Java API, Maven plugin, Gradle plugin)
Command line thru the docker image
Which database are you using (type & version)?
MySQL Server version: 5.7.26 - MySQL Community Server (GPL) - This is a legacy project
Which operating system are you using?
Linux CentOS 7 x64 (uname -r = 3.10.0-957.5.1.el7.x86_64)
What did you do?
(Please include the content causing the issue, any relevant configuration settings, the SQL statement that failed (if relevant) and the command you ran.)
I apply flyway to initialize/update a MySQL database; here are a couple of SQL commands.
Here I use placeholders with xxx prefixes:
CREATE USER IF NOT EXISTS '${xxxdbuser}'#'${xxxdbclip}' IDENTIFIED WITH mysql_native_password BY '${xxxdbpass}';
GRANT ALL PRIVILEGES ON ${xxxdbbase}.* TO '${xxxdbuser}'#'${xxxdbclip}';
FLUSH PRIVILEGES;
... then in another SQL script, from a thirdparty app, I insert a content with ${row}. I don't want Flyway to interpret ${row} as a placeholder, only my own vars starting by ${xxx such as ${xxxdbuser}
INSERT INTO `xxx_xxx` (`name`, `template`, `lang`, `group`, `version`, `data`, `size`, `style`, `modified`) VALUES
... ('addressbook.email.rows', '', '', 0, '1.3.001', 'a:1:{i:0;a:6:{ ... \"label\";s:21:\"$row_cont[type_label]\";s:4:\"name\";s:12:\"${row}[type]\";s:5:\"align\";... :{i:0;s:4:\"100%\";}}}', '100%', '', 1150326789), ...
I guess the placeholderPrefix parameter described into https://flywaydb.org/documentation/commandline/info or FLYWAY_PLACEHOLDER_PREFIX env var described into https://flywaydb.org/documentation/envvars#FLYWAY_PLACEHOLDER_PREFIX is for that purpose; but I didn't succeed in using them!
Here is my command using docker:
docker run --rm --network="$(docker network ls --filter name=app_mysql_dev --filter "label=type=app" --format '{{.ID}}')" \
-v `pwd`/code/Admin/install:/flyway/sql \
-e FLYWAY_URL=jdbc:mysql://${host}:${port}?useSSL=false \
-e FLYWAY_SCHEMAS=${base} \
-e FLYWAY_USER=root \
-e FLYWAY_PASSWORD=${root_pwd} \
-e FLYWAY_PLACEHOLDERS_PREFIX="\${xxx" \
-e FLYWAY_PLACEHOLDERS_XXXDBBASE=${base} \
-e FLYWAY_PLACEHOLDERS_XXXDBUSER=${user} \
-e FLYWAY_PLACEHOLDERS_XXXDBPASS=${pass} \
-e FLYWAY_PLACEHOLDERS_XXXDBCLIP=${clip} \
-e FLYWAY_PLACEHOLDERS_XXXVHOST=${vhost} \
-e FLYWAY_PLACEHOLDERS_XXXSCHEME=${scheme} \
-e FLYWAY_CONNECT_RETRIES=5 \
boxfuse/flyway:5.2.4 -locations=filesystem:/flyway/sql/custom/ \
migrate
What did you expect to see?
All ${xxx placeholders should be replaced by their corresponding ENV values; and the ${row} chain in SQL code stay unchanged.
What did you see instead?
Flyway error:
Flyway Community Edition 5.2.4 by Boxfuse
Database: jdbc:mysql://tasks.atlas-mysql:3306 (MySQL 5.7)
ERROR: No value provided for placeholder expressions: ${row}. Check your configuration!
I guess I did not configure my command correctly... any help, advise and/or command line example would help.
Regrads,
Chris
I think there are a couple of problems in your command:
-e FLYWAY_PLACEHOLDERS_PREFIX="\${xxx"
should be FLYWAY_PLACEHOLDER_PREFIX (no S), and
-e FLYWAY_PLACEHOLDERS_XXXDBBASE=${base}
should be FLYWAY_PLACEHOLDERS_DBBASE (as XXX is part of the prefix, it's not included in the placeholder name; and analogously for following lines).
In my Dockerfile I'm trying to download the latest WordPress version without any content inside it, but I'm having trouble automating the latest version number so that I don't have to manually change it when the new version of WordPress comes out.
In my Dockerfile I have
ARG LATESTWPVER="$(curl -s https://api.wordpress.org/core/version-check/1.5/ | head -n 4 | tail -n 1)"
ADD $(https://downloads.wordpress.org/release/wordpress-$LATESTWPVER-no-content.zip) /var/www/latest.zip
But the problem is that my LATESTWPVER is not 4.9.8, and I get the error
ADD failed: stat /var/lib/docker/tmp/docker-builder962069305/$(https:/downloads.wordpress.org/release/wordpress-$(curl -s https:/api.wordpress.org/core/version-check/1.5/ | head -n 4 | tail -n 1)-no-content.zip): no such file or directory
It passes the entire command and I'd like to have the output of that command.
In my shell file the
#!/bin/bash
WP_LATEST="$(curl -s https://api.wordpress.org/core/version-check/1.5/ | head -n 4 | tail -n 1)"
echo $WP_LATEST
will return the number 4.9.8.
From the error, I'm guessing that you can only assign something to the variable, but not execute it. Is there a way to execute a command and assign it to a variable and pass it as an argument?
A Dockerfile is not a shell or a build script, so it will not execute what you pass in ARG. There is a workaround - define the version as an ARG and pass that during build.
Dockerfile:
--
FROM ubuntu:latest
ARG LATESTWPVER
RUN echo $LATESTWPVER
ADD https://downloads.wordpress.org/release/wordpress-$LATESTWPVER-no-content.zip /var/www/latest.zip
docker build --build-arg LATESTWPVER=`curl -s https://api.wordpress.org/core/version-check/1.5/ | head -n 4 | tail -n 1` .
Sending build context to Docker daemon 6.656kB
Step 1/4 : FROM ubuntu:latest
---> 113a43faa138
Step 2/4 : ARG LATESTWPVER
---> Using cache
---> 64f47dcfe7fa
Step 3/4 : RUN echo $LATESTWPVER
---> Running in eb5fdd005d77
4.9.8
Removing intermediate container eb5fdd005d77
---> 1015629b927e
Step 4/4 : ADD https://downloads.wordpress.org/release/wordpress-$LATESTWPVER-no-content.zip /var/www/latest.zip
Downloading [==================================================>] 7.118MB/7.118MB
---> 72f0d3790e51
Successfully built 72f0d3790e51
I have followed below document to deploy openstack using kolla and I have built all the docker images successfully. I am following this guide for all-in-one installation.
http://docs.openstack.org/developer/kolla/quickstart.html
I have cloned stable/liberty branch.
But while issuing kolla-ansible deploy getting below error.
:# kolla-ansible deploy
Deploying Playbook : ansible-playbook -i /usr/local/share/kolla/ansible/inventory/all-in-one -e #/etc/kolla/globals.yml -e #/etc/kolla/passwords.yml /usr/local/share/kolla/ansible/site.yml
ERROR: merge_configs is not a legal parameter in an Ansible task or handler
Command failed ansible-playbook -i /usr/local/share/kolla/ansible/inventory/all-in-one -e #/etc/kolla/globals.yml -e #/etc/kolla/passwords.yml /usr/local/share/kolla/ansible/site.yml
I have searched a lot about this error but could not find anything, any idea about this error?
Please make sure you have right version of ansible on your deployment node. It should expect version >1.9.4 but < 2.0