I am curious if you can control the output "src" folder in AWS CodeBuild.
Specifically, I see this when debugging the build in CodeBuild.
/codebuild/output/src473482839/src/github.....
I would love to be able to set/change/remove the src473482839 part of that path, because I have a feeling it is causing my sbt to recompile my scala source files, although I am using CodeBuilds new localcache to cache my target folders between builds, the compiled class's canonical path change between builds, which is what I suspect is causing the problem
After some more debugging I have managed to get my 6 minute builds down to 1:30s.
Although you are not able to set or override the CODEBUILD_SRC_DIR I have found a work around in my buildspec.
This is what my buildspec looks like now, with local caching enabled in codebuild.
version: 0.2
phases:
pre_build:
commands:
- mkdir -p /my/build/folder/
- cp -a ${CODEBUILD_SRC_DIR}/. /my/build/folder
build:
commands:
- cd /my/build/folder
- sbt compile test
cache:
paths:
- '/root/.ivy2/cache/**/*'
- '/root/.cache/**/*'
- 'target/**/*'
- 'any other target folders you may need'
The key change I had to make was copy over the source(cached target directories) in the pre_build phase, and change directory and compile from the new, static directory
I hope this helps someone else down the road until CodeBuild allows a person to set/override the CODEBUILD_SRC_DIR folder
Related
I'm currently using semantic-release for versioning of my react library.
https://github.com/semantic-release/semantic-release
Question:
Is it possible to generate the artifacts without publishing it?
For example, in my use case I would like to generate:
- Version Release Number (#semantic-release/commit-analyzer)
- tar file that will be publish to npm (#semantic-release/npm)
- change log (#semantic-release/release-notes-generator)
If you run the dry run option, it will print the version release number and the change log to console, but I want to store it to a file. One workaround is I could pipe the results and then parse, but it'll be nice if it can pass the plugin could put the data in a file during the dry run.
The dry run won't run the publish stage which is where the files get tar'ed up.
Any Advice appreciated,
Thanks,
Derek
You can use the npmPublish option of the #semantic-release/npm plugin. This will generate the tar file for the npm package but it won't publish it to the npm registry.
// In your package.json file add the following property which ensures that npm will not publish
"private": "true"
// In your github action's workflow/release.yml file use the following to store your tar file
- run: |
mkdir -p ~/new/artifact
echo ${{contents_of_your_file}} > ~/new/artifact/yourtarfile
- uses: actions/upload-artifact#v2
with:
name: artifactname
path: '~/new/**/*'
- name: download
uses: actions/download-artifact#v1
with:
name: artifactname
path: '~/new/**/*'
So I have a react with SSR project that I am trying to create a CI/CD pipeline. My project needs to deploy the following artifacts:
dist folder created by Webpack
appspec.yml file needed by Code Deploy
script folder for scripts that are references in appspec.yml
When I tried to get the files one by one using this buildspec.yml:
version: 0.2
phases:
install:
runtime-versions:
nodejs: 10
commands:
- npm install
build:
commands:
- npm run build
artifacts:
files:
- 'dist/*'
- 'appspec.yml'
- 'deploy-scripts'
I got dist with only part of it's content and the appspec.yml and the deploy scripts folder.
Then I tried a different approach:
version: 0.2
phases:
install:
runtime-versions:
nodejs: 10
commands:
- npm install
build:
commands:
- npm run build
artifacts:
files:
- '**/*'
base-directory: "dist"
discard-paths: yes
The dist folder has the scripts and the appspec file inside it. Now it does deploy, but I lose the folder structure of dist which is necessary for my deploy scripts.
I need to get all the contents of dist in their folder structure. As well as the scripts and the appspec.yml file. The scripts and the appspec.yml can't be inside dist, but dist needs to have all of it's content.
Can anyone help with this?
The solution was to use the first buildspec file and adding "**/*" to the dist directory.
So in the dist line it ends up being this: "dist/**/*".
So if we apply this to the general context, anytime you want to get a directory to be sent along with single files in the build phase, you can add it like this:
"[directory_name]/**/*"
And that will get you both the directory and everything inside it in a recursive way.
The dotnet publish command published into the projects bin/netcoreapp2.2/Debug/publish folder. Where netcoreapp2.2 presumably changes with the dotnet version and Debug changes with whatever configuration the -c parameter specifies.
For CI/CD purposes this is clearly undesirable. Alternatively one can pass -o to pass an explicit output path, but again, in a CI/CI environment this path should be inside the project folder structure, e.g. something like:
dotnet publish -o publish
But, because the publish command globs up all files, it picks up previous publish attempts and recursively stores them. This can be mitigated by either cleaning the publish folder explicitly, and/or adding a to the csproj for the project but now there is a dependency between the build script and the csproj: if the publish path is changed in the build scripts for any reason without a corresponding csproj update things break.
So, the least fragile option seems to be to use the default output path as thats automatically excluded from globbing, but how to remove the version & configuration sensitivity? Is there a particularly safe way to get dotnet to tell my CI/CD environment what its output path for build / publish is?
IMP: I do not have enough so reputation to add comment
refer : dotnet publish
you could use relative path with -o option and you may end up avoiding folder name with run-time, platform identification explicitly.
or why do not you consider using build command with publish profile where you can specify explicit path. but generally relative path is less error prone.
Hope this may help you !
I'm working on an SBT project that has to be built with the options like:
-Xmx2G -Xss256M -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
This means that every new developer has to read the readme and assign the options to SBT_OPTS in bash profile or put them in the sbtopts file. Similarly, this has to be configured on Jenkins and this applies to all the projects (so if someone wants to use -XX:+UseG1GC with other projects it becomes an issue). Is it possible to specify the required options in the build file itself? This seems logical to me, as the options are project-specific and without them, you cannot build the project.
Create a .sbtopts file at the root of the build with contents:
-J-Xmx2G
-J-Xss256M
-J-XX:+UseConcMarkSweepGC
-J-XX:+CMSClassUnloadingEnabled
I've the following environment: One Apache2 Webserver on an Ubuntu machine with three vhosts (one vhost per project). Those three project run all on Symfony2 (but differnt versions, from 2.2 to 2.4). Each of the projects (and its paths) have an own user. I'm deploying the projects with capifony on this server, each has an own receipt.
Sass version
Sass 3.2.14 (Media Mark)
Imagine I'm deploying application 1 to the Webserver. When the deploymentprocess comes to dumping all production assets, its writing the temporary generated sass files into the following folder:
/tmp/600d657f6ac2358f30ba6bc0ab4cd7ffb6194ced
as user1.
If I'm deploying now application 2 to the Webserver, dumping the assets would like to write into the exactly same folder, this time as user2 and the following error occurs:
An error occurred while running:
* [err :: 10.0.106.103] '/usr/bin/ruby' '/usr/local/bin/sass' '--load-path' '/srv/vhosts/myproject.com/releases/20140619124055/app/../web/sass' '--scss' '--cache-location' '/tmp' '/tmp/assetic_sassbsrcle'
* [err :: 10.0.106.103]
* [err :: 10.0.106.103] Error Output:
Errno::ENOENT: No such file or directory - /tmp/600d657f6ac2358f30ba6bc0ab4cd7ffb6194ced/assetic_sassbsrclec20140619-27927-aw8xrk.lock
My workaround currently is, to remove this /tmp/600d657f6ac2358f30ba6bc0ab4cd7ffb6194ced folder every time before deployment.
I didn't find any pathconfiguration in the capifony receipts nor in the Symfony2 config files in all projects.
Any help is appreciated.
Best,
Ramo
This has been referenced on the official assetic repository as an issue, as well. Since you can set sys_get_temp_dir() via the environment variable TMPDIR (among others), I would recommend doing that for your dump. You could base it off the current unix time, or the commit you're deploying, or a combination of the application, time, and intent. Really, anything could work. The line responsible for setting the cache location is here, in case you wanted to fork Assetic and change that, which is quite doable as well. I would suggest the TMPDIR route at first to confirm a potential fix.
Small edit, there is also this pull to semi-address this issue.