So, I am trying to set up a CI/CD pipeline with the s4sdk. I successfully completed all the steps descriped in this blog. Everything seems to be running smoothly, however my build is failing with the following error message:
The following artifacts could not be resolved: com.sap.xs2.security:security-commons:jar:0.28.6, com.sap.xs2.security:java-container-security:jar:0.28.6, com.sap.xs2.security:java-container-security-api:jar:0.28.6, com.sap.security.nw.sso.linuxx86_64.opt:sapjwt.linuxx86_64:jar:1.1.19: Could not find artifact com.sap.xs2.security:security-commons:jar:0.28.6 in s4sdk-mirror (http://s4sdk-nexus:8081/repository/mvn-proxy/)
Now, this error messages makes sense to me, since I remember downloading these artifacts from the SAP download center and therefore those artifacts are not available on maven central.
I think this error can be resolved by manually uploading those artifacts to the nexus server, but I don't know how. According to the nexus documentation, there is a web ui reachable under http://< cx-server-ip>:8081, but it is somehow not responding.
I can confirm with docker ps that both the jenkins and nexus container are running and that the nexus container is listening on TCP 8081. I am also able to reach the jenkin's frontend to configure and run my pipeline.
What am I missing? Is uploading the missing artifacts to the nexus the right approach? Any help is appreciated.
The nexus container you see acts as a download cache and is by design not accessible from outside to prevent accidental changes to it. Also, its life-cycle is controlled by the cx-server script, so even if you installed packages there manually, they would be gone once you upgrade the Jenkins.
I think the best way to handle this would be to set up another Nexus instance where you install the required packages and configure the pipeline to use that as described here (mvn_repository_url). This nexus needs to be configured as a mirror for Maven central. We don't have specific docs on how to do that, but this post describes a similar setup.
In this set up, you might want to disable the download cache as it is redundant (cache_enabled to false).
I hope this helps.
Kind regards
Florian
The sidecar nexus acts as a read-only cache for maven and npm artifacts on the host (and agents) where cx server is running. By default it looks up artifacts from maven central and the default npm registry. In the current implementation, the cache will be completely deleted after stopping cx server, leading to a loss of all internal state.
If you want to use custom sources, you can set them in server.cfg via mvn_repository_url and npm_registry_url. This is documented in the operations guide, which you can find here: https://github.com/SAP/cloud-s4-sdk-pipeline/blob/master/doc/operations/operations-guide.md
In your case, you have to specify a maven repository which includes the dependencies in question.
Related
I have an instance of Artifactory OSS (latest version) running in a docker container locally.
We have a remote instance of Artifactory (non-OSS) running as well.
In my local instance, I set up a remote repository of package type Ivy pointing to each of the repositories we have set up in our remote non-OSS instance.
Once I have created a remote repository configuration, I can view each remote repository I created and its artifacts being served under the Application --> Artifactory --> Artifacts page HOWEVER under the Administration --> Repositories --> Repositories page where I am expecting to be able to make changes to the configurations (I have logged in as Admin with administrator privileges btw), none of the repositories I set up are actually visible here! I only see a 0 Repositories count and a No remote repositories message where I am expecting to see a list.
When I do Ivy resolves against my local (remote repository) it works as expected, so they're definitely working... just not showing up for administration.
I have tried rolling back to earlier versions of Artifactory OSS but that hasn't changed anything.
I can tediously work around this with a combination of the REST API and the UI but I REALLY JUST want to be able to administer the configurations within the web app... Am I just being dumb about something or is there a known issue regarding this? I have searched around and I haven't found an answer to this issue, so any help or direction would be greatly appreciated! Thanks!
I was able to reproduce it on my end on as well on version 7.19.1, seems this issue happens when working with Ivy on Artifactory OSS.
I have opened a bug for this on JFrog Jira.
I'm setting up a new atrifactory installation for the first time in my life. Downloaded the tar and extraceted it ok. Got some firewall rules in place to allow https to jcenter.bintray.com. After an initial refresh I see loads of artifacts in the com tree that must come from jcenter, so all seems fine, but when I preform simple maven tasks like mvn help:active-profiles I only get warnings and errors that indicate that none of the relevant stuff is available from my artifactory.
I have accessed the firewall logs and I found no outgoing traffic from my artifactory server to anything that's not permitted. What have I missed? My artifactory is OSS version 7.5.7 rev 705070900.
Artifactory remote repositories are not working as a mirror or the external repository they are pointing at.
Remote Artifactory repositories are proxying the external repository, which means that you have to actively request for artifacts. When requesting for an artifact, Artifactory will request it from the external repository and cache it inside Artifactory. Farther requests for a cached artifact will be served from Artifactory without the need to go out to the external repository.
The list of artifacts we are seeing, are ones which are available in the external repository. This is a feature is called remote browsing and available for some of the package types supported by Artifactory.
I found the issue, sort of. For reasons I now understand I have plugin repositories. I added the true source for the plugins to my list of plugin repositories, and that solved the issue for me.
I am using nexus on our companies build server as proxy. Sometimes developers add new dependencies to their projects without telling me. Hence, the list of proxy repositories is sometimes not in sync what is really required. As a result, the jobs in our jenkins build server fail because of missing artificats. The jenkins is configured to use the nexus proxy repositories.
Is it possible to tell nexus to download the artifacts from the original repository if it is not found in the proxied ones?
I assume you mean that developers add repository entries into their Maven pom file to get further dependencies and/or modify their settings.xml.
On the other hand the CI server is configured to get everything from Nexus with mirrorOf *.
There is no automatic addition of repositories based on this setup. You can do two things imho
create scripts that do that for you using the Nexus REST API
or educate your developers to tell you to add the proxy repos to Nexus
Potentially you can even use the Maven enforcer rule to disallow repositories in the POM and set up an explicit message and allow them to create proxy repositories in Nexus. Just dont forget to have them added to the group you are using on the CI server.
I am running Nexus 2.3.1-01. I define a proxy repository that proxies snapshots from an upstream nexus instance. When I browse the remote repository associated with this proxy, I can see the snapshot artifact of interest. However, when I search for all versions of this artifact in the Nexus admin web ui, older versions of the snapshot artifact appear, but not the more recent versions of interest. Yet those more recent versions are clearly visible when I browse the remote.
I've struggled with this for a few hours, and have tried expiring the proxy cache, rebuilding the index, and repairing the index. This is a fresh installation of Nexus, so a damaged index seems unlikely.
Might someone provide some guidance on what I can try next? I should add that my mvn clients cannot resolve the snapshot dependency of interest either.
I figured it out. My mistake, of course.
The POM my test project was using did not have a clause in it pointing to an appropriate repository. The only hint of a repository was in my settings.xml file, and that repository was in a clause, which I want, but which is not sufficient.
What was the final hint? When I dumped the effective POM (mvn help:effective-pom), I saw the only repository configured was Maven Central. And snapshots were disabled. I (actually a coworker) realized that this single repository could not bootstrap the ability to resolve snapshots.
So I added a repository clause to my POM, enabled snapshots on it, and now everything, releases and snapshots resolve fine. Of course, the repository has to be setup to hand back releases and snapshots, but I already that part of my Nexus config right.
We upload artifacts to Nexus through the file protocol with Maven deploy plugin. Sometimes, those artifacts do not appear directly in Nexus Web interface. I have to do 'expire cache' and refresh the page. Moreover, this causes builds dependant of this artifact to fail.
I guess this is because, we deploy though file protocol. Is there a way to prevent this ? I saw the 'Not Found Cache TTL' in Nexus interface. Not sure to understand the doc. If I set this to zero, will this work ?
Thanks
PW
Deploying directly to the file system should only be used in extreme cases such as bulk manipulations or imports. In order to make Nexus fully recognize the changes on disk, you would need to expire the cache and then you may have to rebuild the metadata. Both of these can be triggered from the repository screen. If you want the artifacts to be searchable, you would also have to fire off the indexer task as well.
All of those things happen automatically when you deploy via http/https directly to Nexus which is the way it is intended to be used