I have an Artifactory pro (without support) server installed in my local network.
One major use case for this artifactory was to use it as local cache for remote artifacts from e.g. repo1 maven repository or lightbend ivy2 repository. The hope was that I could speedup resolution of dependencies hosted on repo1 when caching them on my local artifactory.
I am pretty sure my development machine is configured correctly to exclusively resolve artifacts against my local artifactory.
However, every once in a while (suspiciously close to the interval configured as Metadata Retrieval Cache Period (Sec) in the Advanced Tab of the remote repository settings), the resolution of dependencies originally hosted on maven repo 1 takes far longer then usual.
I suspect that at these times artifactory refreshes artifact meta data (pom, ivy.xml) of remote artifacts. But this takes far longer than I would expect, assuming that a simple pom or ivy download should not take several seconds but rather a few milli seconds.
I am currently requesting root access to the server for attempting a tcpdump from OPs which may take time...
So my question is
Has anyone an idea what actually might happen that takes several seconds per dependency of a remote repository to refresh meta data files or am I looking in the wrong direction?
Update
My Artifactory version is
Artifactory Professional 5.1.3 rev 50019
We had a similar issue but with npm repo's where the meta data re-calculation was taking quite sometime and eventually we came to know that it was a bug in artifactory and was resolved in version 6.1.0. Worth checking the artifactory jira's for any such bugs. Hope so this helps!
Artifactory Jira Link
Related
Installing PyPI or NPM packages from our company's Artifactory instance is about 5-10 times slower then installing from PyPI server or simple NFS share. The network infrastructure is the same in all cases and seems fine..
Does Artifactory suppose to be slower (Because extra security checks or something)?
How can I debug and fix the slowness?
You should not be seeing such a difference between resolving from the public pypi/npm repositories and Artifactory.
There are a couple of things which may impact performance:
Location of your Artifactory server - if your Artifactory server is located outside your network (for example if your client is on-prem and Artifactory is hosted in the cloud), make sure you have good network connection to Artifactory
In case you are self hosting Artifactory - make sure that the resources provisioned to Artifactory are meeting the minimum system requirements. An overloaded database or a slow storage can affect download speed. In case your Artifactory is under a heavy load, you can take a look at some tuning best practices
In case Artifactory is using LDAP/SAML for authentication, any latency in the communication with those will affect the download time. More information about debugging LDAP issues can be found in the knowledge base
The type of repository you are resolving from can have an affect: if you are resolving from a remote repository, the connection to the remote URL can affect the download speed.You can get useful debug information by using the trace capability. Try downloading an artifact directly (using a browser or curl) and add the trace parameter, for example: http://localhost:8081/artifactory/npm-local/drorb/craftyjs-npm-example/-/drorb/craftyjs-npm-example-1.0.0.tgz?trace
I'm setting up a new atrifactory installation for the first time in my life. Downloaded the tar and extraceted it ok. Got some firewall rules in place to allow https to jcenter.bintray.com. After an initial refresh I see loads of artifacts in the com tree that must come from jcenter, so all seems fine, but when I preform simple maven tasks like mvn help:active-profiles I only get warnings and errors that indicate that none of the relevant stuff is available from my artifactory.
I have accessed the firewall logs and I found no outgoing traffic from my artifactory server to anything that's not permitted. What have I missed? My artifactory is OSS version 7.5.7 rev 705070900.
Artifactory remote repositories are not working as a mirror or the external repository they are pointing at.
Remote Artifactory repositories are proxying the external repository, which means that you have to actively request for artifacts. When requesting for an artifact, Artifactory will request it from the external repository and cache it inside Artifactory. Farther requests for a cached artifact will be served from Artifactory without the need to go out to the external repository.
The list of artifacts we are seeing, are ones which are available in the external repository. This is a feature is called remote browsing and available for some of the package types supported by Artifactory.
I found the issue, sort of. For reasons I now understand I have plugin repositories. I added the true source for the plugins to my list of plugin repositories, and that solved the issue for me.
We're making use of a remote repository and are storing artifacts locally. However, we are running into a problem because of the fact the remote repository regularly rebuilds all artifacts that it hosts. In our current state, we update metadata (e.x. repodata/repomd.xml), but artifacts are not updated.
We have to continually clear our local remote-repository-cache out in order to allow it to download the rebuilt artifacts.
Is there any way we can configure artifactory to allow it to recache new artifacts as well as the new artifact metadata?
In our current state, the error we regularly run into is
https://artifactory/artifactory/remote-repo/some/path/package.rpm:
[Errno -1] Package does not match intended download.
Suggestion: run yum --enablerepo=artifactory-newrelic_infra-agent clean metadata
Unfortunately, there is no good answer to that. Artifacts under a version should be immutable; it's dependency management 101.
I'd put as much effort as possible to convince the team producing the artifacts to stop overriding versions. It's true that it might be sometimes cumbersome to change versions of dependencies in metadata, but there are ways around it (like resolving the latest patch during development, as supported in the semver spec), and in any way, that's not a good excuse.
If that's not possible, I'd look into enabling direct repository-to-client streaming (i.e. disabling artifact caching) to prevent the problem of stale artifacts.
Another solution might be cleaning up the cache using a user plugin or a script using JFrog CLI once you learn about newer artifacts being published in the remote repository.
We are running into a 404 error when pulling a specific package from the npm remote repository. It seems to only happen with the #ngrx/effects#2.0.2. We are able to install the 2.0.0 version and other scoped packages correctly.
tested it with scoped and unscoped packages that we have never installed before and it works successfully. Just this package seems to have a problem.
We are on version 5.1.0
The issue is the metadata retrieval cache periods. In order to avoid the latency associated with upstream connections, Artifactory will cache certain metadata from the remote site (NPMJS in this case). This can mean that the period has to pass before you can see anything new.
You can read more about the settings on Artifactory Wiki entry for Advanced Settings. In your case, the relevant settings are Metadata Retrieval Cache Period and Missed Retrieval Cache Period. If you want to always receive the most up-to-date information, simply set those to zero (or a couple of minutes). This may slow down your builds a tad but it's a compromise between speed and completeness.
As administering my Artifactory install was not an option, I found an easy fix:
Remove the line containing the token to your artifactory server in ~/.npmrc.
This may be done with npm logout, however I didn't try that. In any case, the token being present resulted in 404 responses from the server.
I am running Nexus 2.3.1-01. I define a proxy repository that proxies snapshots from an upstream nexus instance. When I browse the remote repository associated with this proxy, I can see the snapshot artifact of interest. However, when I search for all versions of this artifact in the Nexus admin web ui, older versions of the snapshot artifact appear, but not the more recent versions of interest. Yet those more recent versions are clearly visible when I browse the remote.
I've struggled with this for a few hours, and have tried expiring the proxy cache, rebuilding the index, and repairing the index. This is a fresh installation of Nexus, so a damaged index seems unlikely.
Might someone provide some guidance on what I can try next? I should add that my mvn clients cannot resolve the snapshot dependency of interest either.
I figured it out. My mistake, of course.
The POM my test project was using did not have a clause in it pointing to an appropriate repository. The only hint of a repository was in my settings.xml file, and that repository was in a clause, which I want, but which is not sufficient.
What was the final hint? When I dumped the effective POM (mvn help:effective-pom), I saw the only repository configured was Maven Central. And snapshots were disabled. I (actually a coworker) realized that this single repository could not bootstrap the ability to resolve snapshots.
So I added a repository clause to my POM, enabled snapshots on it, and now everything, releases and snapshots resolve fine. Of course, the repository has to be setup to hand back releases and snapshots, but I already that part of my Nexus config right.