I'm struggling to find a solution to clean up all RPM repositories from Artifactory using AQL and CLI. I would like for example to delete everything which was not downloaded in the last 12 months (also if the creation date is older than 1 year and it was never downloaded). The hardest part in my opinion is that I also need to keep the last x number of RPM files in each repository (for example the last 3 rpm files which were last downloaded). Any idea how to perform this task or if it is possible at all, especially the part with keeping the last downloads. Any suggestion is appreciated.
Thank you,
Dany
Clean up old artifacts using jfrog CLI and AQL
JFrog CLI is a compact and smart client that provides a simple interface to automate access to Artifactory. As a wrapper to the REST API.
Installing JFrog CLI
On MAC
brew install jfrog-cli-go
With Curl
curl -fL https://getcli.jfrog.io | sh
JFrog CLI syntax
jfrog target command-name global-options command-options arguments
Target - product on which you wish to execute the command:
rt: JFrog Artifactory
bt: JFrog Bintray
mc: JFrog Mission Control
xr: JFrog Xray
Configure JFrog cli
jfrog rt c Artifactory --url=https://artifactory.eqs.intra/artifactory --apikey=<add api key> #can generate api key from user profile
Generate API Key
Login to artifactory -> Click on username and edit profile -> under Authentication Settings
JFrog CLI configuration Official Documentation
Search for Artifacts
display artifacts from
- registry - docker
- path - *
- download status = null # Not downloaded
- created before 1 month
{
"files": [
{
"aql": {
"items.find": {
"repo": {"$eq":"docker"},
"path": {"$match":"*"},
"name": {"$match":"*"},
"stat.downloads":{"$eq":null},
"$or": [
{
"$and": [
{
"created": { "$before":"1mo" }
}
]
}
]
}
}
}
]
}
AQL Official Documentation.
Run a search and verify the packages you are about to delete
jfrog rt s --spec artifactory.spec
Delete artifacts as per the spec file
jfrog rt del --spec artifactory.spec
Insight Page
Related
I am trying to download the packages of grafana from github releases using our Jfrog Artifactory instance.
The github url is :- https://github.com/grafana/k6/releases/download/v0.42.0/k6-v0.42.0-linux-amd64.tar.gz
Inorder to achieve this , I create a generic typr of remote repo (named as grafana-generic
) in jfrog Artifactory by pointing the remote url to https://github.com.
I referred this solution forn stackoverflow, But it didnt help.
The url which I tried to download the package is as below
https://myrepo/artifactory/grafana-generic/grafana/k6/releases/download/v0.42.0/k6-v0.42.0-linux-amd64.tar.gz
Error is getting as below
{
"errors": [
{
"status": 404,
"message": "Item grafana-generic-cache:grafana/k6/releases/download/v0.42.0/k6-v0.42.0-linux-amd64.tar.gz does not exist"
}
]
}
If you want to fetch a VCS Release you can use the Download a VCS Release REST API command
This is what my setup looks like:
JFrog CLI 1.26.2
I have a local directory with the following items:
spec (JFrog Upload Spec)
myartifact/1.0.0/myartifact-1.0.0-1.txt
My spec looks like so this:
{
"files": [
{
"pattern": "myartifact/*",
"regexp": "false",
"target": "testrepo-release/testbuilds/",
"recursive": "true",
"flat": "false",
"explode": "false"
}
]
}
Then I run build-add-dependencies:
shell>jfrog rt build-add-dependencies --spec=spec myartifact 1
[Info] Running Build Add Dependencies command...
[Info] Adding dependency: myartifact/1.0.0/myartifact-1.0.0-1.txt
{
"status": "success",
"totals": {
"success": 1,
"failure": 0
}
}
Finally I run build publish:
shell>jfrog rt build-publish --url=https://server.com/artifactory/ --user=user --password=password --build-url=https://fake myartifact 1
[Info] Deploying build info...
[Info] Build info successfully deployed. Browse it in Artifactory under https://server.com/artifactory/webapp/builds/myartifact/1
I can't figure out why there is nothing (builds or artifacts) at the target testrepo-release/testbuilds/
Thanks for your help!
The build-add-dependencies command collects files located on the local file-system and adds them as dependencies to the build-info. It does not upload the files to Artifactory.
In order to upload files, you first need to upload the files to Artifactory using the upload command. You can use the same file soec.
Omce the files are uploaded, you can then use the build-add-depedencies command to add the files as dependencies to the build.
Notice that the build-add-depedencies collects the files from the local file-system and not from Artifactory. Future releases of JFrog CLI may add the functionality of also collecting files from Artifactory and adding them to the build as dependencies (the latest release of JFrog when adding this answer is 1.27.0).
Something else to notice:
Both the upload and download commands accept two optional flags: --build-name and --build-number. These flags make the command register the uploaded files as build artifacts and the downloaded files as build dependencies.
We somehow managed to create a 'build' in Artifactory that does not have any artifacts (files) associated with it.
Through the web interface I only seem to be able to delete all builds for that particular release - which I obviously don't want.
Assuming we have 3 good 'builds' (has files etc.):
Moon -- 12.3 Build 23
Moon -- 12.4 Build 34
Moon -- 12.5 Build 89
and the aforementioned bad 'build'
Moon -- 12.0 Build 100
In the WebUI, 'Moon #12.0 Build 100' is listed in 'Last Deployed Builds' and also shows up as '12.0 Build 100' under 'Builds' for the 'Moon' release.
How do I delete the '12.0 Build 100' from Artifactory?
The WebUI seems to only allow deletion of all 'Moon' builds.
If the WebUI is a no go, how would I do so using the JFrog CLI or curl?
I am already having trouble coming up with the correct file spec to even find the 'build' - and obviously don't want to accidentally wipe out artifacts we want to keep.
{
"files": [
{
"pattern": "/api/build/Moon"
}
]
}
... doesn't return anything
{
"files": [
{
"pattern": "my-repo/Moon/*"
}
]
}
... returns all artifacts - but not the builds.
Thank you for any advice.
You can delete a specific build is using the delete builds REST API, for example
curl -X DELETE http://myserver:8081/artifactory/api/build/my-build?buildNumbers=100
In the upcoming version of Artifactory - 6.6, there will be a new option to delete a specific build from the UI (assuming you have the right permissions).
Is there a way to get the binary HASH code(SHA1, SHA256) from artifactory through jfrog cli?
Or at least to download only binaries with specific HASH.
I tried to use the props like below, but it does not work.
jfrog rt download --props "sha1=506438fbfc4a029ffee8b001fdce3c2cbd5541ec --server-id af-server.com afrepo/test_repo/test.txt
"
an Artifact sha-1 is not a property but but an item.field.
You can therefore query an artifact by sha1 using the field name (it is actually "actual_sha1") using AQL.
You can use AQL with curl (see some examples here) or with the CLI using spec files (examples here)
(Note that the AQL structure is slightly different between spec files and curl)
With all that said, your sha1 search example would look like this.
CLI command:
jfrog rt s --spec="/MyPath/MySpecFile"
Spec file
{ "files": [
{
"aql": {
"items.find":
{
"repo": "my-local-repo",
"actual_sha1": {"$eq": "6aebc7357ba46916aab5e9d29b3f8e7180cd7089"}
}
}
}]}
I have some artifacts in Artifactory (OpenSource):
I can download an artifact from using jfrog CLI:
jfrog rt config --user=admin --password=**** --url=http://foo:8081/artifactory
jfrog rt download testproject/01_Develop/01_CI/HPCC-Package-70.zip --flat=true
How can I download the LATEST(highest number) artifact?
JFrog CLI recently started supporting 3 new options from many of the Artifactory commands: --sort-by --sort-order and --limit.
You can use these options to search, download, copy, move or delete the latest artifact created in Artifactory. For example, to download the latest file uploaded to the all-my-frogs folder in the my-local-repo repository, use the following command:
jfrog rt dl "my-local-repo/all-my-frogs/" --sort-by=created
--sort-order=desc --limit=1
You can use the JFrog Cli search command.
jfrog rt s "testproject/01_Develop/01_CI/HPCC-Package-*.zip"
The search command will return a list of paths which you can then sort using external tools such as jq.
If you wish to do it from a Jenkins groovy file, you could use:
def downloadSpec = """{
"files": [
{
"pattern": pattern,
"target": "",
"flat" : "true",
"sortBy": ["name"],
"sortOrder": "desc",
"limit": 1
}
]
}"""