I realize that the Artifactory support team reads these SO posts, so my question is either a regular question or a feature request.
I want to programmatically update various artifacts and when my operations complete (copies, writes, moves, deletes), create an alias/symlink to the new/updated artifact URLs.
For example, I would like to have a "latest" link which always points to the latest build for a number of different artifacts: Java, yum, Python and generic binaries, that is, native executables that do not use pypy/Maven/yum toolsets.
I don't see a way to do this and if that is the case, I'd like to request this feature in a new version of Artifactory.
Related
I have a product for which I store software builds in Artifactory.
I name the software artifacts like this, so it is possible to se what a downloaded file contains: system-pcm33-base-v0.0.0.0_65_ga03970a.raucb
Thus it is also possible to download directly via an URL, not using jfrog
https://artifactory.deif.com/ui/native/amc-sw/pcm33/master/system-pcm33-base-v0.0.0.0_65_ga03970a.raucb
Now I would like to make a quick way to download the latest master build. To do this I have in my build made a symlink
system-pcm33-base.raucb -> system-pcm33-base-v0.0.0.0_65_ga03970a.raucb
I can also push this symlink to artifactory, but it only works from the GUI and via jfrog. I do not get this symlink as I had hoped:
https://artifactory.deif.com/ui/native/amc-sw/pcm33/master/system-pcm33-base.raucb
Is there a way to do this?
It is of course possible to upload the file twice under two different names, and thus update system-pcm33-base.raucb on every build. But that is a bit more heavy.
Artifactory doesn't handle symbolic links as in the Linux file system.
Based on the described use case, you can upload the file twice (as suggested) - first with the actual version, second as the latest. The important part is - when you upload for the 2nd time, as the latest, use Checksum Deploy.
Artifactory has a checksum based storage, which means that every file is actually stored once, even if it is uploaded to different target paths. In order to tell Artifactory to create/update a path without actually sending the binary, you can send the checksum of the binary, and Artifactory will link the path to the binary with that checksum. This operation is quite cheap.
Another possible approach is to define and use a custom Repository Layout. This way, in order to download the latest version of the file, you can use the [RELEASE] placeholder. The actual latest version ill be automatically resolved by the extracted version value based on the layout.
See also:
How to create simple versioning custom layout in Artifactory
How to find the latest artifact version based on layout?
Thanks to yinon explaining that the checksum is used, I found this simple solution
jf rt copy --flat amc-sw/pcm33/master/system-pcm33-base-v0.0.0.0_65_ga03970a.raucb amc-sw/pcm33/master/system-pcm33-base.raucb
This copies ALL the properties, but then a download query will return two files, so a property has to be changed
jf rt sp amc-sw/pcm33/master/system-pcm33-base.raucb artifact=last_bsp
I'm a new Artifactory user. My company just setup Artifactory v6.5.2 and I'm looking to use is for managing software deployed for our production team. What I need is a download link that will get documented in our product management system that directly points to the exact file that Software deployed for Production to use. I was anticipating this would look like this:
https://artifactory.mycompany.com/artifactory/myrepo/mymodule/mypkgfile_v1_b30b890becfb4a02510ed12a7283c676.tgz
I'm not seeing that Artifactory can do this for me. What I see is I can do this:
http://artifactory.mycompany.com/artifactory/myrepo/mymodule/mypkgfile_v1.tgz
However if another artifact is deployed with the same name, it's not reflected in the download link. This means that the link could return different results.
Am I missing something or am I asking Artifactory do something it's not intended to do?
Artifactory returns the URL based on on the filename and the path (as any web server would do). Here are two options to achieve what you need:
Name the artifacts uniquely (timestamps are the simplest). Instead of naming the artifact mypkgfile_v1.tgz, name it mypkgfile_v1-1553038888.tgz (I used the Unix Epoch time, but everything unique enough will do).
This one is more evolved but doesn't require you to change the naming scheme.
First, configure a custom repository layout to match your versioning.
Once you've done that, every time you deploy an artifact, attach a unique identifier to the artifact as property during deployment (using matrix params, for example), deploying your artifact as mypkgfile_v1;timestamp=1553038888.
On the revrieval, use the token for the latest release together with the timestamp you need as a matrix param:mypkgfile_v[RELEASE];timestamp=1553038888
We're making use of a remote repository and are storing artifacts locally. However, we are running into a problem because of the fact the remote repository regularly rebuilds all artifacts that it hosts. In our current state, we update metadata (e.x. repodata/repomd.xml), but artifacts are not updated.
We have to continually clear our local remote-repository-cache out in order to allow it to download the rebuilt artifacts.
Is there any way we can configure artifactory to allow it to recache new artifacts as well as the new artifact metadata?
In our current state, the error we regularly run into is
https://artifactory/artifactory/remote-repo/some/path/package.rpm:
[Errno -1] Package does not match intended download.
Suggestion: run yum --enablerepo=artifactory-newrelic_infra-agent clean metadata
Unfortunately, there is no good answer to that. Artifacts under a version should be immutable; it's dependency management 101.
I'd put as much effort as possible to convince the team producing the artifacts to stop overriding versions. It's true that it might be sometimes cumbersome to change versions of dependencies in metadata, but there are ways around it (like resolving the latest patch during development, as supported in the semver spec), and in any way, that's not a good excuse.
If that's not possible, I'd look into enabling direct repository-to-client streaming (i.e. disabling artifact caching) to prevent the problem of stale artifacts.
Another solution might be cleaning up the cache using a user plugin or a script using JFrog CLI once you learn about newer artifacts being published in the remote repository.
I'm trying to develop a script to query Artifactory for artifact details. The AQL api does the trick for providing me SHA1, SIZE and location. However I need to restrict this information to a specific release. Does Artifactory provide an API endpoint that will return the latest release version for a specific repo? I've already looked over the Artifact Latest Version Search Based on Layout API and while this will work I would need to provide a specific artifact to key off of.
Basically I'm trying to gather a complete list of artifact details from a given repo based on latest release. I'm after the sha1, size, and name of a given artifact for a specific release.
We have a series of scripts that are used when building Apache. Specifically these scripts are used to:
patch the vanilla source code,
patch in customer specific functionality,
build and install the Apache server,
build customer specific modules, and
create a custom install tarball ready for eventual deployment.
N.B. The deployment of the tarballs and configs is handled by other independent mechanisms. This system creates the install tarball for deliver to Ops who then handle the deployment to the web farm.
We have versions of these patches and modules for:
different OS's and OS versions, and
different Apache releases.
N.B. The different Apache releases tend to exist only during testing and migration to a new release of Apache across the web farm.
I have now brought the patches and custom modules under CVS and currently have them tagged according to Apache release version.
Should I keep the tagging simple and use subdirectories to distinguish between Linux specific and Solaris specific elements? Or should I keep one directory structure and use more complex tagging to distinguish elements for the different OS's?
We are currently using CVS and are about to migrate to SVN.
For platform-specific versions, it usually helps to:
have template scripts (i.e. scripts with variables in them)
a deployment script which will copy the template and replace the variables with platform specific data (hostname, port number, paths, ...)
That is not always possible for all type of scripts, but if the differences are limited, that approach allows you to go with:
one simple tagging
one simple tag
one central place to manage platform-specific differences (the "deployment" script)