How would you delete all artifacts that match a pattern (e.g older than 6 months old) from artifactory?
Using either curl, or the go library
The jfrog cli takes a 'spec file' to search for artifacts. See here for information on jfrog spec files
The jfrog cli documentation is available here:
Create an aql search query to find just the artifacts you want:
If your aql search syntax were like:
/tmp/foo.query
items.find(
{
"repo":"foobar",
"modified" : { "$lt" : "2016-10-18T21:26:52.000Z" }
}
)
And you could find the artifacts like so:
curl -X POST -u admin:<api_key> https://artifactory.example.com/artifactory/api/search/aql -T foo.query
Then the spec file would be
/tmp/foo.spec
{
"files": [
{
"aql": {
"items.find": {
"repo": "foobar",
"$or": [
{
"$and": [
{
"modified": { "$lt": "2016-10-18T21:26:52.000Z"}
}
]
}
]
}
}
}
]
}
And you would use the golang library like so:
jfrog rt del --spec /tmp/foo.spec --dry-run
Instead of modified, you can also do a relative date
"modified": { "$before":"6mo" }
If you get error 405 Method not allowed, verify you have an api or password correct, and try using PUT instead of POST
Install jforg cli
configure jfrog cli to make API calls
jfrog rt c Artifactory --url=https://<>url/artifactory --apikey=<add api key> #can generate api key from user profile
Create a spec file called artifactory.spec You can change the find query as required. Below query will
display artifacts from
registry - docker-staging
path - *
download status = null # Not downloaded
created before 6 month
{
"files": [
{
"aql": {
"items.find": {
"repo": {"$eq":"docker-staging"},
"path": {"$match":"*"},
"name": {"$match":"*"},
"stat.downloads":{"$eq":null},
"$or": [
{
"$and": [
{
"created": { "$before":"6mo" }
}
]
}
]
}
}
}
]
You can search for the packages avalable to delete by
jfrog rt s --spec artifactory.spec
Run the delete command
jfrog rt del --spec artifactory.spec
We've built a tool for Artifactory artifacts management (cleanup) https://github.com/devopshq/artifactory-cleanup
it'd look like
# artifactory-cleanup.yaml
artifactory-cleanup:
server: https://repo.example.com/artifactory
# $VAR is auto populated from environment variables
user: $ARTIFACTORY_USERNAME
password: $ARTIFACTORY_PASSWORD
policies:
- name: Remove all files from repo-name-here older then 7 days
rules:
- rule: Repo
name: "reponame"
- rule: DeleteOlderThan
days: 7
then you should run it to see what it'll remove and when you are ready - add --destroy flag to it!
artifactory-cleanup
Related
We are trying to automate the repos, group and permission creation in Jfrog as part of the azuredevops pipeline. So here I decided to use JfrogCLI and API calls as the azuredevops tasks. But facing some difficulties in Jfrog cli or API calls related as the idempotent behavior is not enabled for most of the API calls operations and CLI commands.
Scenario:- For Application app-A1,
2 repos(local-A1 and virtual-A1) ,
2 groups(appA1-developers, appA1-contributors)
1 permission target (appA1-permission) where we will include the created 2 repos and groups with the permissions
Below is what I tried to achieve so far.
For creating the groups
jf rt group-create appA1-developers --url https://myrepo/artifactory --user jfuser --password jfpass
jf rt group-create appA1-contributors --url https://myrepo/artifactory --user jfuser --password jfpass
Create Repos using the command as below
Repo creation and Update template
jfrog rt rc local-repo-template
{
"description": "$variable",
"excludesPattern":"$variable",
"includesPattern":"$variable",
"key":"$variable",
"notes":"$variable",
"packageType":"$variable",
"rclass":"$variable"
}
Repo Update Command
jfrog rt ru local-repo-template
{
"description": "$variable",
"excludesPattern":"$variable",
"includesPattern":"$variable",
"key":"$variable",
"notes":"$variable",
"packageType":"$variable",
"rclass":"$variable"
}
For creating permission
curl -u 'jfuser' -X PUT "https://myrepo/artifactory/api/v2/security/permissions/java-developers" -H "Content-type: application/json" -T permision.json
{
"name": "appA1-developers",
"repo": {
"include-patterns": ["**"],
"exclude-patterns": [""],
"repositories": ["appA1-local"],
"actions": {
"groups" : {
"appA1-developers" : ["manage","read","annotate"]
}
}
},
"build": {
"include-patterns": ["testmaven/**"],
"exclude-patterns": [""],
"repositories": ["artifactory-build-info"],
"actions": {
"groups" : {
"appA1-developers" : ["manage","read","write","annotate","delete"]
}
}
}
}
But when I trying in Azuredevops all the above tasks, Not able to identify and condition if any of the resources above are already existing.
Looking for a way to Identify first,
If the specified groupname is already existing or not, if existing skip group creation, else create the groups
check if the repos already existing,
if not existing create as per the template
But if it exists and all the properties are same- Skip it
But if it exists and there are property changes- use the update command
Similarly, the permission target also need to be updated only if there are changes , But existing properties or settings shouldnt be altered.
I tried making a GET request to this url:
https://artifactory.xxxxxx.com/artifactory/api/search/dates?dateFields=created&from=1675050197406&name=yyyyy-*.tgz/
but I ran into this error:
{
"errors": [
{
"status": 500,
"message": "Item not found for id '2649778966'"
}
]
}
Thanks in advance.
You can use the AQL (Artifactory Query Language) to search for artifacts in Artifactory by name and last modified date. Here is an example AQL query to search for artifacts with a name starting with "my-artifact" modified after Jan 1, 2022:
items.find(
{
"name": {"$match": "my-artifact*"},
"$and": [
{"modified": {"$gt": "2022-01-01T00:00:00.000Z"}}
]
}
)
I'm interested in the value of one field (Module ID) but there seems to be no way of obtaining this specifically. A complete dump of all field values would also suffice but I haven't succeeded in finding a way to do that either. I've looked at and tried the searches available within the documentation here: https://www.jfrog.com/confluence/display/JFROG/Artifactory+REST+API#ArtifactoryRESTAPI-SEARCHES
If it helps, I'm trying to query an on-premise installation of Artifactory.
More fields can be added to the AQL using the "include" element.
For example - to list all artifacts under "libs-release-local" repository, including their module names, run the following query:
items.find(
{
"repo":{"$eq":"libs-release-local"}
}
).include("artifact.module")
Response example:
{
"results": [
{
"repo": "libs-release-local",
"path": "org/jfrog/test/multi2/2.17.0",
"name": "multi2-2.17.0.jar",
"type": "file",
"size": 1022,
"created": "2021-09-11T13:51:33.878Z",
"created_by": "deployer",
"modified": "2021-09-11T13:51:33.631Z",
"modified_by": "deployer",
"updated": "2021-09-11T13:51:33.881Z",
"artifacts": [
{
"modules": [
{
"module.name": "org.jfrog.test:multi2:2.17.0"
}
]
}
]
}
]
}
You can find all of the required information under AQL documentation.
I can't update IAM policy in my AI Platform Notebook.
I created a new AI Platform Notebooks instance:
gcloud beta notebooks instances create nb1 \
--vm-image-project=deeplearning-platform-release \
--vm-image-family=tf-latest-cpu \
--machine-type=n1-standard-4 \
--location=us-west1-b
When I try to apply a new IAM policy I get an Error:
gcloud beta notebooks instances set-iam-policy nb1 --location=us-west1-b notebooks.policy
ERROR: (gcloud.beta.notebooks.instances.set-iam-policy) INTERNAL: An
internal error has occurred (506011f7-b62e-4308-9bde-10b97dd7b99c)
My policy looks like this:
{
"bindings": [
{
"members": [
"user:myuser#gmail.com",
],
"role": "roles/notebooks.admin"
}
],
"etag": "BwWlgdvxWT0=",
"version": 1
}
when I do a
gcloud beta notebooks instances get-iam-policy nb1 --location=us-west1-b --format=json
I get:
ACAB
As there is no policy set.
Please take a look at etag field:
An etag is returned in the response to getIamPolicy, and systems are expected to put that etag in the request to setIamPolicy to ensure that their change will be applied to the same version of the policy.
From documentation here
string (bytes format)
etag is used for optimistic concurrency control as a way to help prevent simultaneous updates of a policy from overwriting each other. It is strongly suggested that systems make use of the etag in the read-modify-write cycle to perform policy updates in order to avoid race conditions: An etag is returned in the response to getIamPolicy, and systems are expected to put that etag in the request to setIamPolicy to ensure that their change will be applied to the same version of the policy.
Important: If you use IAM Conditions, you must include the etag field whenever you call setIamPolicy. If you omit this field, then IAM allows you to overwrite a version 3 policy with a version 1 policy, and all of the conditions in the version 3 policy are lost.
A base64-encoded string.
You can just easily change your policy etag to ACAB, which is the default one.
{
"bindings": [
{
"members": [
"user:myuser#gmail.com",
],
"role": "roles/notebooks.admin"
}
],
"etag": "ACAB",
"version": 1
}
or you can use add-iam-policy-binding command, to create a new policy, then you can extract the etag using get-iam-policy and update your JSON file with it, finally run the set-iam-policy
You may also use this format:
{
"policy": {
"bindings": [
{
"members": [
"user:myuser#gmail.com"
],
"role": "roles/notebooks.admin"
}
],
"etag": "ACAB",
"version": 1
}
}
Artifactory 6.9.1
The artifact existing in the Artifactory is shown below:
I am using the following query based on the REST API doc. for artifact version search:
curl --request GET "https://repository.net/artifactory/api/search/versions?g=com.name&a=core-api-error&repos=core-services&v=0.4.0-56204b7*"
{
"results" : [ {
"version" : "0.4.0-56204b7",
"integration" : false
} ]
}
but if I try to use the exact version by removing '*' from the 'v' argument,
curl --request GET "https://repository.name.net/artifactory/api/search/versions?g=com.name&a=core-api-error&repos=core-services&v=0.4.0-56204b7"
{
"errors" : [ {
"status" : 404,
"message" : "Unable to find artifact versions"
} ]
}
The usage of '*' is dangerous as it might return some other versions. I just want to check if the artifact with that version exists.