Artifactory Migration - URL to files not working - artifactory

I'm in the process of upgrading and migrating Artifactory version 6.11 (zip install, housed on RH7) to the 7.35 version (housed on a new server and hostname, rpm install). I'm doing this on a cloned VM as a test, so the only thing that is different from our original system is the hostname. As the documentation recommends, I first upgraded 6.11 to 7.35 and everything seemed to go well. I followed the upgrade steps and the migration.sh script completed successfully.
The major issue I'm having is that when I go into Artifacts, the 'url to file' is bringing up a 502 Bad Gateway nginx error. It seems to me that a pointer is incorrect somewhere and I'm confused as to where it could be. The upgrade was successful, so I know the data is there, but Artifactory is not able to link to it properly.
Update/clarification: To improve my description: When I head into Application bar / Artifactory / Artifacts and select a repo from the left-hand column, the 'url to file' fails to load. I'm assuming this is the tree view?
On the server that is currently working, a url such as https://acme/artifactory/repo leads to a directory listing. However, on the new server, a url such as https://new-acme-server/artifactory/repo would lead to a 502 Bad Gateway or an nginx error if I use http (no cert is installed on the test VM, but is installed on the orignal server).
In v7.35, I went into the 'http settings' and switched the server provider as both nginx and apache (Tomcat was set as default) and while the site operated fine under both, the url to the repo files still fails with an nginx error, regardless of the server provider.
When I did a full system export of the original server, the documentation had me uncheck "Exclude data". I also exported the repos out as well and imported those in via a path. Everything seems to show up correctly just like on the original server, but I'm still unable to view a directory listing when I click on the url.
Could it be the location of the filestore being different? If so, how would I go about pointing it to the right location?
V7.35: /opt/jfrog/artifactory/var/data/artifactory/filestore
V6.11: /opt/artifactory/artifactory-pro-6.11.3/data/filestore
The base URL is the same as the original installation http(s)://domain/artifactory
Output from artifactory-service.log
2022-03-25T16:58:40.429Z [jfrt ] [INFO ] [3bb67ba1f30d560e] [ifactoryApplicationContext:564] [ttp-nio-8081-exec-10] - Artifactory application context set to READY by reload
2022-03-25T16:58:40.430Z [jfrt ] [INFO ] [3bb67ba1f30d560e] [c.CentralConfigServiceImpl:933] [ttp-nio-8081-exec-10] - Configuration reloaded.
2022-03-25T17:09:04.013Z [jfrt ] [INFO ] [708a8ae7c307ec92] [c.CentralConfigServiceImpl:914] [http-nio-8081-exec-5] - Reloading configuration... old revision 212, new revision 213
2022-03-25T17:09:04.121Z [jfrt ] [INFO ] [708a8ae7c307ec92] [c.CentralConfigServiceImpl:542] [http-nio-8081-exec-5] - New configuration with revision 213 saved.
2022-03-25T17:09:04.121Z [jfrt ] [INFO ] [708a8ae7c307ec92] [ifactoryApplicationContext:564] [http-nio-8081-exec-5] - Artifactory application context set to NOT READY by reload
2022-03-25T17:09:04.181Z [jfrt ] [INFO ] [708a8ae7c307ec92] [ifactoryApplicationContext:564] [http-nio-8081-exec-5] - Artifactory application context set to READY by reload
2022-03-25T17:09:04.181Z [jfrt ] [INFO ] [708a8ae7c307ec92] [c.CentralConfigServiceImpl:933] [http-nio-8081-exec-5] - Configuration reloaded.
2022-03-25T17:36:47.707Z [jfrt ] [INFO ] [d7bb51eedd93b03c] [aseBundleCleanupServiceImpl:84] [art-exec-20 ] - Starting to cleanup incomplete Release Bundles
2022-03-25T17:36:47.708Z [jfrt ] [INFO ] [d7bb51eedd93b03c] [b.ReleaseBundleServiceImpl:415] [art-exec-20 ] - Finished deleting orphan/unidentified items from _intransit repository
2022-03-25T17:36:47.709Z [jfrt ] [INFO ] [d7bb51eedd93b03c] [aseBundleCleanupServiceImpl:90] [art-exec-20 ] - Finished incomplete Release Bundles cleanup

Your filestore location for both Artifactory 6 and Artifactory7 is correct.
This indicates to me that the issue is with your reverse proxy.
In order to confirm, can you check the below two things.
Open your Artifactory on its IP and Port.
http://localhost:8082/ (The default port will be 8082 if you have not modified). Now go to tree view in Application tab of Artifactory and try to download a specific file. If you are able to download, then the issue is might not be with filestore or upgrade. Mostly it should be reverse proxy.
In that case, navigate to Artifactory > Administration > Artifactory > HTTP Settings > Genereate new settings > Place in reverse proxy and restart.
In the above test, if you are still not able to download, then check in the logs ($JFROG_HOME/artifactory/var/log/artifactory-service.log) if you are observing a message something similar to the below.
2022-03-24T20:15:35.072Z [jfrt ] [WARN ] [2a73d62655afd1ad] [.r.ArtifactoryResponseBase:136] [http-nio-8081-exec-9] - Sending HTTP error code 500: Could not process download request: Binary provider has no content for '165c79f8dff2f9e7d3ccadcbc295f7ef8e6e95f0'
If yes, then it indicates that, Artifactory is not able to find the binary. If none of the above helped, post the log snippet from the above log file here while you are trying to download a file along with the file name.

For your update/clarification questions, please allow me to clarify.
In Artifactory 6.x Artifactory was acting as both the server and UI, therefore you could use the "http://acme/artifactory URL, however in Artifactory 7.x, Artifactory changed to work with multiple microservices, and the UI has moved to its own microservice (now it is named "Frontend"). You can try access the "Native Browser" by using this URL http://acme/ui/native/REPOSITORY/.
To add to the above and to Ganapathi's reply, the URL for your Artifactory has changed from http://acme:8081/artifactory to http://acme:8082 since Artifactory is now utilizing the "router" (external port is 8082 and internal 8046) microservice to redirect all the requests to the respective microservices. You can check the full list here.
I hope this clarifies more.

Related

Artifactory doesn't work after restoring nuget packages

When a team member restores nuget packages using donet, artifactory enters a loop with the errors below until the memory overflows.
The problem only happens when he adds in his Nuget.Config the tag protocolVersion="3"
<add key="Company" value="https://repo.company.com/artifactory/api/nuget/v3/rdi-nuget-virtual" protocolVersion="3" />
In the tests we noticed that the use of the protocolversion=3 tag causes this.
This tag is necessary because the download of the file is much faster
Anyone have an idea what it could be?
Could you help me please?
logs below
Version jfrog: 7.29.8 rev 72908900
2022-01-11T14:00:57.263Z [jfrt ] [WARN ] [6cdecc92eaf486c7] [.r.ArtifactoryResponseBase:136] [ttp-nio-8081-exec-43] - Sending HTTP error code 403: Download request for repo:path 'nuget-remote-cache:.nuGetV3/feed.json' is forbidden for user: 'lcunha'.
2022-01-11T14:00:57.264Z [jfrt ] [ERROR] [6cdecc92eaf486c7] [etV3VirtualAndRemoteCommon:274] [ttp-nio-8081-exec-43] - Failed to download resource in repo: nuget-remote, at url: https://api.nuget.org/v3/index.json. HTTP STATUS CODE: 403
2022-01-11T14:00:57.264Z [jfrt ] [ERROR] [6cdecc92eaf486c7] [etV3VirtualAndRemoteCommon:133] [ttp-nio-8081-exec-43] - Failed to convert artifactory url (https://repostaging.companysoftware.com:443/artifactory/api/nuget/v3/company-nuget-virtual/registration-semver2) to original remote url for repo: nuget-remote, package: xunit.core
java.lang.NullPointerException: null
at java.base/java.util.Objects.requireNonNull(Objects.java:221)
2022-01-11T14:01:56.786Z [jfrou] [ERROR] [2016c910242342cc] [external_topology.go:82 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": net/http: request canceled (Client.Timeout exceeded while awaiting headers)
2022-01-11T14:02:04.871Z [jfrou] [WARN ] [7f3fb3a30ade9665] [local_topology.go:268 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy"
2022-01-11T14:02:09.877Z [jfrou] [ERROR] [7f3fb3a30ade9665] [local_topology.go:128 ] [main ] - periodic send heartbeat failed for 4 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out
java.lang.OutOfMemoryError: Java heap space
-XX:OnOutOfMemoryError="kill -9 %p"
Executing /bin/sh -c "kill -9 3974"...
2022-01-11T14:02:11.185Z [jfrou] [WARN ] [5a89519a8048b91d] [local_topology.go:268 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy"
2022-01-11T14:02:11.196Z [jfrou] [ERROR] [79bb63bc55c1ed15] [external_topology.go:82 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": read tcp 127.0.0.1:55970-127.0.0.1:8040: read: connection reset by peer
2022/01/11 14:02:11 httputil: ReverseProxy read error during body copy: read tcp 127.0.0.1:56788->127.0.0.1:8045: read: connection reset by peer
2022/01/11 14:02:11 httputil: ReverseProxy read error during body copy: read tcp 127.0.0.1:56788->127.0.0.1:8045: read: connection reset by peer
2022/01/11 14:02:11 httputil: ReverseProxy read error during body copy: read tcp 127.0.0.1:56788->127.0.0.1:8045: read: connection reset by peer
2022-01-11T14:02:11.208Z 35[jfob ] [WARN ] [1ed879c85a5af005] [access_join.go:70 ] [main ] - Refreshing platform config change events gRPC stream - target server is unavailable - if issue persists check communication with access [access_client]
/opt/jfrog/artifactory/app/bin/artifactory.sh: line 359: 3974 Killed $TOMCAT_HOME/bin/catalina.sh run
Can you share your setup?
-The application server is the CentOS Linux release 7.9.2009 (Core)
-MySQL as backend Database
-Also, we are using the Apache as reverse proxy to apply the SSL certificate.
How are you running Artifactory?
-The Artifactory is container based: releases-docker.jfrog.io/jfrog/artifactory-pro:7.29.8
How much resources to you give it? Memory and CPU
The server has allocated:
2 vCPU
16 GB RAM
SSD
What are the java memory settings? Mostly interested in heap settings (Xms and Xmx)?
We are using the standard configuration. No changes were made in the Xms and Xms parameters.
As you are using Artifactory v7.x? You may refer to our System Requirements wiki page for the recommended hardware based on your environment.
Also, from on the error message shared, it looks like the user does not have proper permissions hence we see 403 errors. Please do validate and assign the required permissions to the user and let us know the results.
Sending HTTP error code 403:
Download request for repo:
path 'nuget-remote-cache:.nuGetV3/feed.json' is forbidden for user: 'lcunha'.

Pushing an image to a Docker repo crashes artifactory

What reasons could there be that a push of a docker image to an Artifactory repo could crash the entire app?
The symptoms I see are that a layer hangs during the push:
ยป docker push my-repo/my-app:20211207-afdf6438-test
The push refers to repository [my-repo/my-app]
d4cde41bcf33: Layer already exists
ac14050b2264: Layer already exists
27e5cc646fd0: Pushing [==================================================>] 562.7MB
797b1ec2507f: Layer already exists
34de87854e34: Layer already exists
cde140fcdbee: Layer already exists
be3883e87d34: Layer already exists
9ec86c039eae: Layer already exists
371ce8b24b31: Layer already exists
7e718b9c0c8c: Layer already exists
It hangs on pushing that layer, then Artifactory becomes unresponsive until it finally restarts itself, which takes a while.
When this happens, the logs don't give me much to go on.
2021-12-09T20:06:01.066Z [jfrt ] [INFO ] [44ac995c9c11dc2e] [o.a.e.UploadServiceImpl:465 ] [http-nio-8081-exec-2] - Deploy to 'docker-local:my-app/_uploads/60766841-fac7-4781-b325-da9ef66ba2cf' Content-Length: 1 (estimation) artificial: false
2021-12-09T20:06:27.249Z [jfrt ] [INFO ] [ ] [ffectedConfigStreamObserver:32] [Stream_1639080147248] - publishing full invalidation and attempting to resubscribe to affected configuration changes
2021-12-09T20:06:28.145Z [jfrt ] [INFO ] [ ] [ffectedConfigStreamObserver:32] [Stream_1639080153144] - publishing full invalidation and attempting to resubscribe to affected configuration changes
2021-12-09T20:06:29.646Z [jfrt ] [INFO ] [ ] [ectedEntitiesStreamObserver:35] [Stream_1639080154645] - publishing full invalidation and attempting to resubscribe to affected permissions changes
2021-12-09T20:07:26.615Z [jfrt ] [ERROR] [59398681a5a1be14] [o.j.a.c.h.AccessHttpClient:136] [http-nio-8081-exec-6] - Error while executing /api/v1/users/ on access. Exception message: Read timed out
2021-12-09T20:07:51.686Z [jfrt ] [ERROR] [6a934e4bf3e126b4] [o.j.a.c.h.AccessHttpClient:136] [http-nio-8081-exec-4] - Error while executing /api/v1/users/ on access. Exception message: Read timed out
2021-12-09T20:07:56.622Z [jfrt ] [ERROR] [2fe4c5688866ed57] [o.j.a.c.h.AccessHttpClient:136] [http-nio-8081-exec-8] - Error while executing /api/v1/users/ on access. Exception message: Read timed out
2021-12-09T20:07:56.622Z [jfrt ] [ERROR] [2b94004eb23bb041] [o.j.a.c.h.AccessHttpClient:136] [ttp-nio-8081-exec-12] - Error while executing /api/v1/users/ on access. Exception message: Read timed out
Terminating Artifactory
This is self-hosted Artifactory 7.27.10 running in Kubernetes
This was a disk performance problem. I use EFS for my Artifactory data volume. For some reason still unknown to me, Artifactory started requiring higher IOPS than the EFS volume could provide. I was able to solve the problem by changing the EFS volume to provisioned IOPS.
Pulls worked fine, because my backing datastore is s3. So the problem only manifested on pushes, after all the docker layers were uploaded, and when log files were written to the EFS volume. It was on log writes that the system choked and restarted.

Installing Xray and configure with Jfrog Artifactory using Docker compose

We were trying to Integrate Xray with our Jfrog Artifactory. In Amazon Linux 2 we are trying to install with docker compose,while we run the config.sh
After running the bellow docker compose commands
start rabbitmq: docker-compose -p xray-rabbitmq -f docker-compose-rabbitmq.yaml up -d
start postgresql: docker-compose -p xray-postgres -f docker-compose-postgres.yaml up -d
start: docker-compose -p xray up -d
xray router is getting restarting after 20sec with following error:
We have checked whether any selinux, firewalld, or iptables are blocking,but all are in disable state.
Can someone help us to resolve the issue?
Now Private IP is able to reach Artifactory server,we have created Xray in same VPC of Artifactory.
Now all containers of Xray are running in Xray server,but now we have a different issue.
In xray server container we are getting the below logs
2021-08-12T13:41:17.601Z [jfxr ] [INFO ] [469946e5f04dd2c6] [updates_service:486 ] [main ] Initializing JFrog vendor
2021-08-12T13:41:17.700Z [jfxr ] [ERROR] [ ] [bin_mgr_cache:50 ] [main ] Failed to get binary managerid:failed on GetAllBinaryManagerIds query
--- at /go/src/jfrog.com/xray/internal/dbaccess/dao/binary_managers_dao.go:367 (binMgrDao.GetBinaryManagerId) ---
Caused by: not found
2021-08-12T13:41:17.701Z [jfxr ] [ERROR] [ ] [bin_mgr_cache:59 ] [main ] Failed to get binary manager'' version, err :failed to fetch binary manager
--- at /go/src/jfrog.com/xray/internal/dbaccess/dao/binary_managers_dao.go:290 (binMgrDao.GetBinMgrByID) ---
Caused by: not found
2021-08-12T13:41:17.701Z [jfxr ] [WARN ] [ ] [indexed_resources_cache:36 ] [main ] Failed to get binary managerfor cache:failed to fetch binary manager
--- at /go/src/jfrog.com/xray/internal/dbaccess/dao/binary_managers_dao.go:290 (binMgrDao.GetBinMgrByID) ---
Caused by: not found
Any idea on this?
#praseeb It appears you are giving JFrogURL as the node IP of xray. It should be the reachable URL of artifactory from the xray machine, Please pick it from Admin > Security > Settings as indicated.
I had similar issue with some custom Docker Compose files.
It was a network issue, the containers (server, indexer, analysis, persist) did not start in the same network as the router. This occurs because I use docker-compose [...] --no-start.
With the --no-start option, the network_mode: service:router was ignored and the containers goes to the default bridge network. So they cannot communicate with the router on local ports (8046, etc).

404 after upgrading artifactory from 6.20 to 7.6.2

I am getting 404 accesing to https://my-dmain/ui/. If I try to access to https://my-dmain/artifactory it redirects to https://my-dmain/ui/ with 404. No log errors, only one warning:
2020-07-10T08:06:04.535L [35m[tomct][0m [WARNING] [ ]
[org.apache.catalina.startup.HostConfig]
[org.apache.catalina.startup.HostConfig deployDescriptor] - A docBase
[/opt/jfrog/artifactory/app/artifactory/tomcat/webapps/artifactory.war]
inside the host appBase has been specified, and will be ignored
2020-07-10T08:06:04.540L [35m[tomct][0m [WARNING] [ ]
[org.apache.catalina.startup.HostConfig]
[org.apache.catalina.startup.HostConfig deployDescriptor] - A docBase
[/opt/jfrog/artifactory/app/artifactory/tomcat/webapps/access.war]
inside the host appBase has been specified, and will be ignored
Just to confirm it, can you try to access the Artifactory using the server IP and port, like HTTP://1.2.3.4:8082? If you are able to access the Artifactory UI using the server IP and Port, I believe you need to tweak the reverse proxy being used.
Your problem is that with Artifactory 7.x the reverse proxy configuration is different. In this KB article you can find a working NGINX configuration.
One easy way to generate such configuration is to bypass your reverse proxy and go to Artifactory directly, there in the UI you will be able to log in, head to HTTP settings, and generate a new Apache or NGINX config.

I am having an issue recursively pulling a folder from Artifactory using JFrog CLI

I am using an Artifactory server. I am trying to store a folder of third party files in my repo.
I was able to upload a zipfile and explode it so I have my files in MY_REPO/MY_FOLDER/
I am trying to recursively download the contents of the folder using:
jfrog rt dl --recursive MY_REPO/MY_FOLDER
I have enabled debug and I get:
[Info] Searching items to download...
[Debug] Searching Artifactory using AQL query:
items.find({"repo": "MY_REPO","path": {"$ne": "."},"$or": [{"$and":[{"path": {"$match": "MY_FOLDER"},"name": {"$match": "*"}}]},{"$and":[{"path": {"$match": "MY_FOLDER/*"},"name": {"$match": "*"}}]}]}).include("name","repo","path","actual_md5","actual_sha1","size","type","property")
[Error] Artifactory response: 405 Method Not Allowed
{
"status": "failure",
"totals": {
"success": 0,
"failure": 0
}
}
[Error] Download finished with errors. Please review the logs
I have tried variations of MY_FOLDER, MY_FOLDER/, MY_FOLDER/. and with/without recursive. I get the same error.
I know I have permissions because I can upload a file:
jfrog rt u TEST_FILE MY_FOLDER
succeeds just fine. But I can't turn around and pull it down:
jfrog rt dl MY_FOLDER/TEST_FILE
results in:
[Info] Searching items to download...
[Debug] Searching Artifactory using AQL query:
items.find({"repo": "MY_REPO","$or": [{"$and":[{"path": {"$match": "."},"name": {"$match": "MY_FILE"}}]}]}).include("name","repo","path","actual_md5","actual_sha1","size","type","property")
[Error] Artifactory response: 405 Method Not Allowed
{
"status": "failure",
"totals": {
"success": 0,
"failure": 0
}
}
[Error] Download finished with errors. Please review the logs
I'm not certain what isn't working, there's nothing on StackOverflow, JFrog dox or elsewhere with issues using jfrog cli. :(
My jfrog-cli.conf looks like this:
{
"artifactory": [
{
"url": "https://SITE_NAME:PORT_NUM/artifactory/list/MY_REPO/",
"user": "USERNAME",
"serverId": "MY_REPO",
"isDefault": true,
"apiKey": "API_KEY_FROM_ARTIFACTORY"
}
],
"Version": "1"
}
UPDATED
I was running version 1.22.1 for Windows 64-bit. I am now using jfrog cli version 1.34.1 for Windows 64-bit. Per first comment I updated to the latest binary and tried running a search, both with and without MY_FOLDER:
jfrog rt s TEST_FILE and
jfrog rt s MY_FOLDER/TEST_FILE
Both return the same results. I enabled DEBUG and with the newer binary I actually seem to get two errors. The older binary just reported the 405 error. The newer one reports a 404 (Not Found) then reports the error as a 405 Method Not Allowed.
[Debug] Sending usage info...
[Info] Searching artifacts...
[Debug] Searching Artifactory using AQL query:
items.find({"$or":[{"$and":[{"repo":"TEST_FILE","path":{"$match":"*"},"name":{"$match":"*"}}]}]}).include("name","repo","path","actual_md5","actual_sha1","size","type","modified","created","property")
[Debug] Sending HTTP POST request to: https://SITE_NAME:8443/artifactory/list/MY_REPO/api/search/aql
[Debug] Sending HTTP GET request to: https://SITE_NAME:8443/artifactory/list/MY_REPO/api/system/version
[Debug] Artifactory response: 404 Not Found
{
"errors": [
{
"status": 404,
"message": "File not found."
}
]
}
[Error] Artifactory response: 405 Method Not Allowed
Which is still odd because I'm looking for a file I just uploaded. I even deleted the uploaded file and re-uploaded to verify the newer cli can still upload files. I tried a wildcard search to find all files but it returned the same error.
I can also see the file where it is supposed to be using the Artifactory web interface, and download the file and delete it. So Artifactory knows it's there and I have access to it. :-/
The url in your configuration should only contain the base url of Artifactory, i.e: "SITE_NAME:PORT_NUM/artifactory".
You should provide the directory path in the upload and download commands.
Use the ping command to verify your setup works, then progress from there.
BTW: The reason the upload works but download fails is by coincidence. The upload command just uses the upload REST API which expects a path (which you provided in your config), but the download command first sends a search request that does not expect a path, so it fails.

Resources