I'm working with JFROG Cli and need to cleanup artifacts from folder under repository and keep only 5 latest artifacts (latest by created date).
I have already created some code which removes artifacts which were created 7 and more days. But I need to keep 5 latest artifacts. Anyone has any ideas?
{
"files": [
{
"aql": {
"items.find": {
"repo": "maven-repo",
"path": {"$match":"com/mqjbnd64/7.1"},
"name": {"$match":"*"},
"$or": [
{
"$and": [
{
"created": { "$before":"7d" }
}
]
}
]
}
}
}
]
}
You can create an initial query sorting by create date and limiting the number of records returned to 5.
Than you can execute another query, to get all artifacts in this path, and deleted the ones not returned by the previous query.
Related
Question
How to handle complex structures, such as arrays, when using the ConfigurationBuilder's AddEnvironmentVariables approach for setting config.
i.e. What would the environment variables be called to represent the below structure?
{
"MyApp": {
"SendAlertsTo": [
{"Name": "Joe Blogger", "Email": "JoeBlogger#example.com"},
{"Name": "Jane Doer", "Email": "JaneDoer#example.com"}
]
}
}
Detailed version with context
We're writing apps that will be hosted in containers. Initially we'd continued to use the AppSettings.json file to hold settings, populating this file's contents from environment variables at startup via an entrypoint.sh shell script.
We later realised that we could use the ConfigurationBuilder's AddEnvironmentVariables method to pull the values directly from the environment variables, and thus avoid the overhead of maintaining the entrypoint.sh script as being another place to update each time a new setting was added.
This works well for an app settings file such as:
{
"MyApp": {
"SmtpServer": "smtp.example.com",
"FromAddress": "JoeyBlogger#example.com"
},
"Logging": {
"LogLevel": "Default"
}
}
.. as this translates simply to:
export MyApp:SmtpServer=smtp.example.com
export MyApp:FromAddress=JoeyBlogger#example.com
export Logging:LogLevel=Default
However, I recently saw a developer's app settings file for some new monitoring, which looks like the below (note; this won't be the actual settings used in our hosted environments; it's just an example which triggered this question). In this case, using the above approach doesn't work so well, since there would be two different settings with the name Serilog:WriteTo:Name, and ambiguity over which entry anything starting Serilog:WriteTo:Args:* relates to.
{
"Serilog"
"WriteTo": [
{
"Name": "Async",
"Args": {
"configure": [
{
"Name": "File",
"Args": {
"path": ".\\log.txt",
"rollingInterval": "Day",
"retainedFileCountLimit": 7,
"buffered": true
}
}
]
}
},
{
"Name": "Async",
"Args": {
"configure": [
{
"Name": "Console"
}
]
}
}
]
}
}
My guess is we'd handle this via some sort of index; but I've not been able to find anything in the docs about this.
export Serilog:WriteTo[0]:Name=Async
export Serilog:WriteTo[0]:Args:configure[0]:Name=File
export Serilog:WriteTo[0]:Args:configure[0]:Args:path=.\\log.txt
export Serilog:WriteTo[0]:Args:configure[0]:Args:rollingInterval=Day
export Serilog:WriteTo[0]:Args:configure[0]:Args:retainedFileCountLimit=7
export Serilog:WriteTo[0]:Args:configure[0]:Args:buffered=true
export Serilog:WriteTo[1]:Name=Async
export Serilog:WriteTo[1]:Args:configure[0]:Name=Console
However, running a quick PoC shows that the above doesn't work. How can this be done?
I found the solution here: MS Extensions Configuration Deep Dive.
The answer is that indexes are used, but the index number is treated like a named element; i.e.
{
"MyApp": {
"SendAlertsTo": [
{"Name": "Joe Blogger", "Email": "JoeBlogger#example.com"},
{"Name": "Jane Doer", "Email": "JaneDoer#example.com"}
]
}
}
Becomes:
export MyApp:SendAlertsTo:0:Name=Joe Blogger
export MyApp:SendAlertsTo:0:Email=JoeBlogger#example.com
export MyApp:SendAlertsTo:1:Name=Jane Doer
export MyApp:SendAlertsTo:1:Email=JaneDoer#example.com
Additionally, for Linux hosting there's another point I'd not been aware of / unrelated to the array question.. the : separator should be __, so the above becomes:
export MyApp__SendAlertsTo__0__Name=Joe Blogger
export MyApp__SendAlertsTo__0__Email=JoeBlogger#example.com
export MyApp__SendAlertsTo__1__Name=Jane Doer
export MyApp__SendAlertsTo__1__Email=JaneDoer#example.com
We are using the ELK v6.6 version.
We have requirement in Kibana that we have list of records which are displaying in the Kibana data table. We need to display last updated record on top of the table. For which we are using below DSL query but which is not working.Can you please help us to resolve this issue.
{
"query": {
"match_all": { }
},
"aggs": {
"actual_trans_type.keyword": {
"terms": {
"field": "#timestamp"
},
"aggs": {
"top_hit": {
"sort": [
{
"fieldName": {
"order": "desc"
}
}
],
"size": 2
}
}
}
}
}
https://i.stack.imgur.com/eCpde.png
https://i.stack.imgur.com/9XBHC.png
Edit the visualization, click on the timestamp column head to sort it in the descending order and then save the visualization. Now you will have the table in the descending order of timestamp field by default.
I am trying to update dozens of services to au.com.dius:pact-jvm-consumer-junit_2.12:3.5.12 from au.com.dius:pact-jvm-consumer-junit_2.11:3.2.13 but it seems that the new consumer version is generating pacts that the old provider version (au.com.dius:pact-jvm-provider-junit_2.11:3.2.13) cannot handle.
The old pact had a hashmap with matching rules being added at the root as show below
{
"consumer": {
"name": "consumer-amqp"
},
"provider": {
"name": "prodvider-amqp"
},
"messages": [
{
"description": "amqp contract",
"contents": {
"body": {
"guidProperty": "795ecfd5-a3a5-430f-a0cd-1569df61bff6"
}
},
"matchingRules": {
"$.body.body.guidProperty": {
"regex": "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"
}
}
}
],
"metadata": {
"pact-specification": {
"version": "3.0.0"
},
"pact-jvm": {
"version": "3.2.13"
}
}
}
The new consumer adds and wrapper body around the matchers. Below is an example of the same pact generated with the new consumer version
{
"consumer": {
"name": "consumer-amqp"
},
"provider": {
"name": "prodiver-amqp"
},
"messages": [
{
"description": "contract",
"contents": {
"body": {
"guidProperty": "e2490de5-5bd3-43d5-b7c4-526e33f71304"
}
},
"matchingRules": {
"body": {
"$.guidProperty": {
"matchers": [
{
"match": "regex",
"regex": "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"
}
],
"combine": "AND"
}
}
}
}
],
"metadata": {
"pact-specification": {
"version": "3.0.0"
},
"pact-jvm": {
"version": "3.5.12"
}
}
}
Because of that change the provider fails to parse the matching rules with the following error:
body
^
10:21:24.526 [main] DEBUG au.com.dius.pact.matchers.JsonBodyMatcher - compareValues: No matcher defined for path List($, body, body, guidProperty), using equality
10:21:24.527 [main] WARN au.com.dius.pact.matchers.Matchers$ - Path expression body is invalid, ignoring: [1.1] failure: `$' expected but `b' found
java.lang.AssertionError:
comparison
{$.body.body.guidProperty=Expected 'e2490de5-5bd3-43d5-b7c4-526e33f71304' but received 'aff876f5-5014-937c-6855-c099f9857437'
Looking at the v3 spec the new message seems to be valid, does the old provider library (v3.2.13) not support it? I looked through the code and found this commit which seems to me where the change was introduced.
From my testing the new provider library (3.5.12) can handle both old and new formats, but if both the new provider and old consumer libraries are present in the classpath http contract tests fail with run-time errors.
Questions:
1) Is there a way to force the new consumer to create the pacts the old way, and is that form spec compliant?
2) Is there a way to update the provider to the new version and still have the old consumer library in the path and not get failures?
As J_A_X pointed out, it looks like the commit you referenced fixed a bug where pacts that were supposed to be version 3 were not fully version 3 compliant (they used the old matchers format)
1) Is there a way to force the new consumer to create the pacts the old way, and is that form spec compliant?
Yes and yes. You should be able to fix this by setting the system property pact.provider.version to 2 - then both the old and the new versions will be able to read the generated pact.
2) Is there a way to update the provider to the new version and still have the old consumer library in the path and not get failures?
Yes, as long as you ask the old consumer version to generate version 2 pacts (but ideally, why not update both to the same version?)
Is there a way to download an artifact using AQL?
I have a query sent with:
curl -u user:pass \
-X POST https://artifactoryURL/artifactory/api/search/aql \
-H 'content-type: text/plain' \
-T query.aql
And my query.aql:
items.find(
{
"repo":{"$eq":"repo"},
"$and": [
{
"path": { "$match": "path/*"},
"name": { "$match": "*.rpm"}
}
]
}
)
.sort({ "$desc": ["modified"] })
.limit(1)
Now that I know it returns what I want, Is there a way to change the request from api/search/aql to something like api/download/aql and get those items?
EDIT:
I had also tried doing this with the jfrog cli but they don't fully support AQL there (sort and limit didn't work).
This is the command I tried:
jfrog rt s --spec=query-aql.json
And the spec that failed to sort and limit results:
{
"files": [
{
"aql": {
"items.find": {
"repo": "repo",
"$and": [
{
"path": { "$match": "path/*"},
"name": { "$match": "*.rpm"}
}
]
}
},
"sort": {
"$asc": ["modified"]
},
"limit": 1
}
]
}
EDIT 2:
Added a jfrog-cli-go issue: https://github.com/JFrogDev/jfrog-cli-go/issues/56
An easy way to use your aql script to download files from artifactory is to use the JFrog cli as mention here : https://www.jfrog.com/confluence/display/CLI/CLI+for+JFrog+Artifactory#CLIforJFrogArtifactory-Download,CopyandMoveCommandsSpecSchema
The cli can be downloaded as an executable for linux, mac or windows and should fit your needs
With the curl command the only thing you can do is then to parse the result from your aql query and perform download request for each file.
I was just looking for a very similar thing -- use a download spec file to download the latest artifact from a given repo and path. I don't care if I use AQL or not, I just want it in the download-spec.json file. If you go to the link above look at Example 5.
Modified for your example:
{
"files": [
{
"pattern": "path/*.rpm",
"target": "my/download/path/",
"sortBy": "created",
"sortOrder": "desc",
"limit": 1,
}
]
}
jfrog cli supports --limit, --sort-order, and --sort-by arguments.
The following works for me:
jfrog rt search --spec=/tmp/jfrogfilespec.json --sort-by created --sort-order=desc --limit 1
The contents of the json spec file are:
{ "files": [ { "aql": { "items.find": { "repo":{"$eq":"my-release-repo"}, "name":{"$match":"my-artifact-prefix*"} } } } ] }
This generates the following query (according to debug mode):
items.find( { "repo":{"$eq":"my-release-repo"}, "name":{"$match":"my-artifact-prefix*"} } ).include("name","repo","path","actual_md5","actual_sha1","size","type","created").sort({"$desc":["created"]}).limit(1)
What is frustrating is that I cannot seem to find a way to use "jfrog rt search" with a filespec that allows me to influence the "include" modifier portion of the search.
I'm trying to query all artifacts that are older than 6 months old. I'm able to delete them if I hard code a date into my query.
{
"files": [
{
"aql": {
"items.find": {
"repo": "foobar",
"$or": [
{
"$and": [
{
"modified": { "$lt": "2016-10-18T21:26:52.000Z"}
}
]
}
]
}
}
}
]
}
jfrog rt del --spec /tmp/foo.spec --dry-run
How can I do a query with a relative date? (e.g today - 6 months)
I'm going to put this into a cron job, and I'd rather not munge a spec file every time the cron job runs.
AQL queries support relative time operators.
In this case, modify the query:
"modified": { "$lt": "2016-10-18T21:26:52.000Z"}
To:
"modified": { "$before": "6mo"}
See full documantation at: AQL Relative Time Operators.