I'm trying to copy the latest artifact from one path to another using Artifactory API.
POST /api/copy/{srcRepoKey}/{srcFilePath}?to=/{targetRepoKey}/{targetFilePath}[&dry=1][&suppressLayouts=0/1(default)][&failFast=0/1]
Let's say I have a few RPMs named: artifact-1.0-1.rpm, artifact-1.0-2.rpm and artifact-1.0-3.rpm.
How to automatically copy the third artifact ?
With the next release of Jfrog's CLI, planned in a couple of weeks, you'll be able to use SORT and LIMIT in the COPY command.
This will allow you to fetch only the latest item\artifact by SORTing by date and LIMITing to the result set to 1.
For now, you can use 2 sequential CURL commands to try and accomplish what you're after:
First use an AQL SEARCH with you're SORT and LIMIT to retrieve the relevant item's path, and then use your COPY command with that path.
Note: the CLI's SORT and LIMIT feature has already been checked in to the CLI's dev branch, so if you wish to use a snapshot you can "download and build" the dev branch from github, and then test if the solution suites you.
I doubt that you can automatically copy all these artifacts in one statement. You can copy the folder but no regex or pattern can defined in copy command.
Related
I have a product for which I store software builds in Artifactory.
I name the software artifacts like this, so it is possible to se what a downloaded file contains: system-pcm33-base-v0.0.0.0_65_ga03970a.raucb
Thus it is also possible to download directly via an URL, not using jfrog
https://artifactory.deif.com/ui/native/amc-sw/pcm33/master/system-pcm33-base-v0.0.0.0_65_ga03970a.raucb
Now I would like to make a quick way to download the latest master build. To do this I have in my build made a symlink
system-pcm33-base.raucb -> system-pcm33-base-v0.0.0.0_65_ga03970a.raucb
I can also push this symlink to artifactory, but it only works from the GUI and via jfrog. I do not get this symlink as I had hoped:
https://artifactory.deif.com/ui/native/amc-sw/pcm33/master/system-pcm33-base.raucb
Is there a way to do this?
It is of course possible to upload the file twice under two different names, and thus update system-pcm33-base.raucb on every build. But that is a bit more heavy.
Artifactory doesn't handle symbolic links as in the Linux file system.
Based on the described use case, you can upload the file twice (as suggested) - first with the actual version, second as the latest. The important part is - when you upload for the 2nd time, as the latest, use Checksum Deploy.
Artifactory has a checksum based storage, which means that every file is actually stored once, even if it is uploaded to different target paths. In order to tell Artifactory to create/update a path without actually sending the binary, you can send the checksum of the binary, and Artifactory will link the path to the binary with that checksum. This operation is quite cheap.
Another possible approach is to define and use a custom Repository Layout. This way, in order to download the latest version of the file, you can use the [RELEASE] placeholder. The actual latest version ill be automatically resolved by the extracted version value based on the layout.
See also:
How to create simple versioning custom layout in Artifactory
How to find the latest artifact version based on layout?
Thanks to yinon explaining that the checksum is used, I found this simple solution
jf rt copy --flat amc-sw/pcm33/master/system-pcm33-base-v0.0.0.0_65_ga03970a.raucb amc-sw/pcm33/master/system-pcm33-base.raucb
This copies ALL the properties, but then a download query will return two files, so a property has to be changed
jf rt sp amc-sw/pcm33/master/system-pcm33-base.raucb artifact=last_bsp
In JFrog Artifactory, I need to delete the artifacts which are not downloaded for a period of 6 months. I have gone through the JFrog repository. It is mentioned to run via REST API, but I am not pretty sure how to use it.
How can we implement this? or if there is some other way?
I would recommend to refer to this user plugin, artifactCleanUp plugin can be used to delete artifacts which are not downloaded for certain days/months.
The easiest way would be to use the delete command of the JFrog CLI.
The command accepts a file spec.
Write an AQL query with Relative Time Operators, and pass it in the aql field of the file spec.
I have 3 stages (dev / staging / production). I've successfully set up publishing for each, so that the code will be deployed, using msbuild, to the correct location, with the correct web configs transformed - all within Jenkins.
The problem I'm having is that I don't know to deploy the code to staging from what was built on dev (and staging to production). I'm currently using SVN as the source control, so I think I would need to somehow save the latest revision number dev has built and somehow tell Jenkins to build/deploy staging based on that number?
Is there a way to do this, or a better alternative?
Any help would be appreciated.
Edit: Decided to use the save the revision number method, which parses a file containing the revision number to the next job -- to do this, I followed this answer:
How to promote a specific build number from another job in Jenkins?
It explains how to copy an artifact from one job to another using the promotion plugin. For the artifact itself, I added a "Execute Windows batch command" build step after the main build with:
echo DEV_ENVIRONMENT_CORE_REVISION:%SVN_REVISION%>env.properties
Then in the staging job, using that above guide, copied that file, and then using a plugin EnvInject, to read from that file and set an environment variable, which can then be used as a parameter to the SVN Repository URL.
You should be able to identify the changeset number that was built in DEV and manually pass that changeset to the Jenkins build to pull that same changeset from SVN. Obviously that makes your deployment more manual. Maybe you can setup Jenkins to publish the changeset number to a file and then have the later env build to read that file for the changeset number.
We used to use this model as well and it was always complex. Eventually we moved to a build once and deploy many times model using WebDeploy. This has made the process much more simple. Check it out - http://www.dotnetcatch.com/2016/04/16/msbuild-once-msdeploy-many-times/
I have followed every advice on http://r-pkgs.had.co.nz/git.html and on the subsection http://r-pkgs.had.co.nz/git.html#git-branch and I am still getting error.
The steps I need/did (different from what Hadley's page dictates).
grab URL of GitHub repo (e.g, https://github.com/OHDSI/Achilles.git )
create versioned project in RStudio with this URL
set up my global user names for git
select a dev branch here (for example devXYZ)
At this point I got "detached at origin/devXYZ) message.
Per instructions in Hadley book - I tried to do fix this using this command
git push --set-upstream origin devXYZ
but it fails. The error is: origin does not appear to be a git repository or src refspec devXYZ does not match any
I tried fixing it with doing this command (may be wrong)
git remote add origin https://github.com/OHDSI/Achilles.git
I am using windows, latest R, latest RStudio, latest git from https://git-scm.com/download/win
EDIT: I also tried making a new branch using the recommended mechanism but it also fails. The goal is to get instructions where there is not git init and the whole process starts with an URL and new project in RStudio.
The desired future steps to work would be 5. modify and commit into the devXYZ branch.
THIS ONLY APPLIES TO NON-MASTER BRANCHES:
If you are newbie to git - simply don't try to do the git part in R at all.
Instead, use GitHub Desktop or SourceTree.
Point that tool to the desired repo, switch to desired branch
Start RStudio and do any development
Close RStudio and use that external tool to perform any git steps.
FOR MASTER BRANCHES:
integrated RStudio git implementation works great.
I think I might know what the problem is. You're trying to push directly to the main repo. I'm guessing you're not one of the main contributors for that repo so it won't allow you to create a branch there directly. I'm guessing in that book he's probably using his own repository as an example rather than using an existing one
The reason you're getting that error is because that branch doesn't exist on the remote repo so it can't get the reference to it which is inferred from this src refspec devXYZ does not match any
The preferred workflow is to work on a fork of the main repo (basically its your own personal copy of the main repo that is stored on the server). Even if you end up as a contributor at some point I think this is a good workflow to follow
Here's a good explanation on how use the fork workflow. There's other information on stackoverflow as well
Once you've made updates you'd create what's called a pull request to the original repo (commonly referred to as upstream). This basically is a request to merge your changes from the fork into the main repo. This allows the repo owner to review the changes and decide whether to accept them or make changes
Since you're just going over a tutorial I'd say use your fork as the origin wherever its used in the book for now
I have a nexus server version 2.11.1-01 that has a scheduled task setup to run over our snapshots repo to remove snapshots with the minimum snapshot count set to 3 and a retention of 5 days. The scheduled task is showing that it runs, but when I look through our repository there are as many as 26 snapshots for one artifact gav spanning 6 months.
Is there something not configured correctly or a way to find out why it isn't running correctly?
As reported in this nexus Jira:
The snapshot remover removes timestamped snapshots within one version
folder. It does not remove previous versions of snapshots. Note that
it is very common for there to be multiple active versions of
snapshots for a given artifact, some being produced by branch builds
and others by trunk builds.
The "remove if released" option can be used to clean up all snapshots
from a particular version.
Also, as reported in this other Jira, you have to check file naming layout:
Nexus supports Maven2/3 layout only
If you want to delete multiple versions of an artifact, as reported here, you have to use REST apis:
curl -X DELETE -u admin:admin123
http://localhost:8081/nexus/service/local/repositories/releases/content/com/test/project/1.0/
Or, alternatively you can delete them directly from the repository's
local storage on disk. If you delete directly from local storage then
use the REST command to rebuild metadata for the affected path:
curl -X DELETE -u admin:admin123
http://localhost:8081/nexus/service/local/metadata/repositories/snapshots/content/com/test/project/
You'll also need to update the search indexes. If you schedule a
nightly "update indexes" task it will pick up any changes made
directly to the storage area.