is it possible to proxy a private docker registry that runs on docker distribution using nexus oss 3?
i am able to successfully proxy the docker hub, however when i try to proxy my own internal registry, i just end up with image not found errors.
2016-08-31 15:58:21,457+0000 WARN [qtp331814152-140] admin org.sonatype.nexus.repository.docker.internal.V1Handlers - Error: GET /v1/repositories/company-npm/images: 404 - org.sonatype.nexus.repository.docker.internal.V1Exception$ImagesNotFound: images not found
2016-08-31 15:58:30,764+0000 WARN [qtp331814152-140] admin org.sonatype.nexus.repository.docker.internal.V2Handlers - Error: GET /v2/library/company-java/manifests/latest: 404 - org.sonatype.nexus.repository.docker.internal.V2Exception: manifest unknown
2016-08-31 15:58:30,811+0000 WARN [qtp331814152-51] admin org.sonatype.nexus.repository.docker.internal.V1Handlers - Error: GET /v1/repositories/company-java/images: 404 - org.sonatype.nexus.repository.docker.internal.V1Exception$ImagesNotFound: images not found
2016-08-31 15:58:46,379+0000 WARN [qtp331814152-164] admin org.sonatype.nexus.repository.docker.internal.V2Handlers - Error: GET /v2/library/company-java/manifests/6.0.0: 404 - org.sonatype.nexus.repository.docker.internal.V2Exception: manifest unknown
the documentation for the feature does not seem to indicate if this is supported.
I had this same issue with Nexus 3.0.1-01. For me the problem came down to namespacing. Nexus inserts the /library namespace for all repository access commands when a namespace is left blank. See https://books.sonatype.com/nexus-book/3.0/reference/docker.html section 9.8.
So for example if I push an image to a hosted repository:
docker push my-registry.com:5000/myimage:latest
The proxy registry looks for the image as:
docker pull my-registry.com:5000/library/myimage:latest
Which of course doesn't exist. (It would be really great if Nexus would add the /library namespace automatically on image push, or at least make this a configurable option at the repo level).
If you were to do the following:
docker push my-registry:5000/library/myimage:latest
or even:
docker push my-registry:5000/mynamespace/myimage:latest
The your proxy will be able to find the image.
docker pull my-proxy-registry:5000/mynamespace/myimage:latest
Related
I'm having a dickens of a time publishing to our Artifactory instance using poetry (1.1.12). I've tried a couple different URL's to publish to, and I either get a 405 (Method not Allowed), or a 415 (Unsupported Media Type)
https://my.domain/artifactory/my-cool-reg/ --> 405
https://my-cool-reg.my.domain/artifactory/api/pypi/pypi/simple/ -> 415
Is there some magic that I'm missing to get publishing going? I have tried passing my credentials both with -u and -p on the CLI, as well as with the credential config for the repository via -r my_repo. Below is an example error message (from the pypi url test)
UploadError
HTTP Error 415: Unsupported Media Type
at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/publishing/uploader.py:216 in _upload
212│ self._register(session, url)
213│ except HTTPError as e:
214│ raise UploadError(e)
215│
→ 216│ raise UploadError(e)
217│
218│ def _do_upload(
219│ self, session, url, dry_run=False
220│ ): # type: (requests.Session, str, Optional[bool]) -> None
geudrik - For the PyPI deployment, local configuration (.pypirc) should be defined with the API endpoint as http://pythonpublishtest.com/artifactory/api/pypi/test-pypi-local
/api/pypi is the required endpoint and the test-pypi-local I have mentioned in the URL here is the local repository. The reason for the HTTP 405 and HTTP 415 errors could be due to missing/incorrect API endpoint or the missing deployment repository.
I see that you are attempting to publish with the following URL (https://my-cool-reg.my.domain/artifactory/api/pypi/pypi/simple/). Could you please remove "simple" from the URI and include your PyPI local repository's name & verify if that works?
[Additional note]
Artifactory also supports the deployment of artifacts through the virtual repository. In order to enable it, we need to define the local repositories to be aggregated by the virtual repository as a target for deployment. It can be achieved by setting the Default Deployment Repository under the Virutal repository settings.
By gum, I've figured it out
$ poetry config repositories.poetry-repo-name https://artifactory.domain.lan/artifactory/api/pypi/your-cool-repository
We moved to a new authenticated Nexus to act as a proxy to get dependencies.
I've tried to to give SBT (1.1.1) the credentials it needs, in multiple ways, but I always endup getting :
[error] Unable to find credentials for [Sonatype Nexus Repository Manager # nexus3.company.com]
[debug] CLIENT ERROR: Unauthorized url=https://nexus3.company.com/repository/maven2-proxy-all/org/scala-sbt/actions_2.12/1.1.1/actions_2.12-1.1.1.pom
It's repeated for a lot of dependencies.
I've created a .credentials file in my project as follow:
realm=Sonatype Nexus Repository Manager
host=nexus3.company.com
user=xxxxx
password=xxxxx
Here's what I've tried, based on inputs I got from other threads on the internet:
Adding the path to this credentials file in the command : -Dsbt.boot.credentials=.credentials
Adding the path to this credentials file to an environment variable : $SBT_CREDENTIALS = PATH
Adding the following line in the build.sbt : credentials += Credentials(new File(".credentials"))
Adding the following line in the build.sbt : credentials += Credentials("Sonatype Nexus Repository Manager", "nexus3.company.com", "xxxxx", "xxxxxx")
Checking what's going on with a proxy : my requests don't seem to have any authorization header and all come back as HTTP 401
And yet, when I access the URL mentioned from the same machine, with the credentials in the file, there is no issue at all.
I'm running out of ideas here :(
After more attempts, adding :
~/.sbt/1.0/credentials.sbt
credentials += Credentials("Sonatype Nexus Repository Manager", "nexus3.company.com", USER, PWD)
AND
The SBT_CREDENTIALS variable mentioned above,
Seems to do the job.
I also updated the image we use for our pipelines, not sure if it helped.
I'm trying to use Concourse to grab a dockerfile defintion from a git repository, do some work, build the docker image, and push the new image to Artifactory. See below for the pipeline definition. At this time I have all stages up to the artifactory stage (the one that pushes to Artifactory) working. The artifactory stage exits with error with the following output:
waiting for docker to come up...
sha256:c6039bfb6ac572503c8d97f42b6a419b94139f37876ad331d03cb7c3e8811ff2
The push refers to repository [artifactory.server.com:2077/base/golang/alpine]
a4ab5bf94afd: Preparing
unauthorized: The client does not have permission to push to the repository.
This would seem straight-forward as an Artifactory permissions issue, except that I've tested locally with the docker cli and am able to push using the same user/pass as specified within destination_username and destination_password. I double checked the credentials to make sure I'm using the same ones and find that I am.
Question #1: is there any other known cause for getting this error? I've scoured the resource github page without finding anything. Any ideas why I may be getting the permissions error?
Without having an answer to the above question, I'd really like to dig deeper into troubleshooting the problem. To do so I use fly hijack to get a shell in the corresponding container. I notice that docker is installed on the container, so next step I think would be to do a docker import on the tarball for the image I'm trying to push and then perform a docker push to push it to the repo. When attempting to run the import I get the error:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is
the docker daemon running?
Question #2: Why can't I use docker commands from within the container? Perhaps this has something to do with the issue I'm seeing with pushing to repo when running the pipeline (I don't think so)? Is it because the container isn't running with privilege? I thought that the privileged argument would be supplied in the resource type definition, but if not, how can I run with privilege?
resources:
- name: image-repo
type: git
source:
branch: master
private_key: ((private_key))
uri: ssh://git#git-server/repo.git
- name: artifactory
type: docker-image
source:
repository: artifactory.server.com:2077/((repo))
tag: latest
username: ((destination_username))
password: ((destination_password))
jobs:
- name: update-image
plan:
- get: image-repo
- task: do-stuff
file: image-repo/scripts/do-stuff.yml
vars:
repository-directory: ((repo))
- task: build-image
privileged: true
file: image-repo/scripts/build-image.yml
- put: artifactory
params:
import_file: image/image.tar
Arghhhh. Found after much troubleshooting that the destination_password wasn't being picked up properly due to special characters and a lack of quotes. Fixed the issue by properly setting the password within yaml file being included with the --load-vars flag.
I am setting up Nexus3 repository as a remote repository in Artifactory.
But when I update the Nexus3 repo URL(https://domainname/repository/reponame/) & necessary credentials for authentication in the admin section, during testing i am getting,
Connection failed: Error 404.
I have tried providing the rest URL(http://domainname/service/rest/repository/browse/reponame).
In this case, connection to the nexus server establishes successfully & able to see the directory structure for the remote repo in the Artifacts section , but could not find the artifacts inside & seeing below output/error,
{ "errors" : [ {
"status" : 404,
"message" : "Couldn't find item: XXXX:XXXXXXX" } ]
You have to ignore the error while saving. This is because of a header mismatch.
Repo path should be like:
https://<host>:<port>/repository/reponame
Once you save the repository and try to download, it will work.
The test is failing when creating an Artifactory remote repository which point to a hosted Nexus repository, since Artifactory is using a HEAD request for checking the remote repository and for some reason Nexus will return a 404 (while returning a 200 when the same request is sent using the GET method). This behavior does not happen with Nexus group repositories.
Creating a Proxy repository on Nexus that proxies Oracle Maven Repo (http://download.oracle.com/maven/) marks the Oracle Repo as "Attempting to Proxy and Remote Unavailable"
The problem might be that Oracle disabled directory listing and every attempt to get the content without the full GAV returns a 404 code.
How to workaround this on Nexus?
Using Nexus OS Edition 1.9.2.2
Configuration:
Remote Storage Location = http://download.oracle.com/maven/
Download Remote Indexes = True
Auto Blocking Active = False
File Content Validation = True
CheckSum Policy = Warn
There should be no need to proxy the oracle repos, we've merged all of that content into Central now, so you can safely remove these from your Nexus.
The url you are using is wrong. Did you mean the java.net repo at http://download.java.net/maven/2/