How to migrate from Nexus Repository to JFrog Artifactory? - artifactory

I have two servers, one with Nexus Repository and one with JFrog Artifactory OSS.
I would like to migrate to JFrog Artifactory but because it is open source there is no tool to migrate. If it were the pro version I could migrate easily.
I need some idea how to migrate from Nexus Repository to JFrog Artifactory OSS.

The way that I accomplished this task was to:
manually download the repos from Nexus. For nexus2 you can directly copy the repos from the filesystem. For nexus3 you will need to use the API. For nexus3 I used a script that I will put below.
Zip the repos into a folder
Upload the zipped file to artifactory with their import tool. Go to admin panel, artifactory, import, repositories.
You should be good to go from there more or less. I had to manually set some permissions and snapshot settings.
WARNING! This script doesn't work 100% you may/will have to modify it. WARNING!
sourceServer=
sourceRepo=
sourceUser=
sourcePassword=
logfile=$sourceRepo-backup.log
outputFile=$sourceRepo-artifacts.txt
# ======== GET DOWNLOAD URLs =========
url=$sourceServer"/service/rest/v1/assets?repository="$sourceRepo
contToken="initial"
while [ ! -z "$contToken" ]; do
if [ "$contToken" != "initial" ]; then
url=$sourceServer"/service/rest/v1/assets?continuationToken="$contToken"&repository="$sourceRepo
fi
echo Processing repository token: $contToken | tee -a $logfile
response=`curl -ksSL -u "$sourceUser:$sourcePassword" -X GET --header 'Accept: application/json' "$url"`
readarray -t artifacts < <( jq '[.items[].downloadUrl]' <<< "$response" )
printf "%s\n" "${artifacts[#]}" > artifacts.temp
sed 's/\"//g' artifacts.temp > artifacts1.temp
sed 's/,//g' artifacts1.temp > artifacts.temp
sed 's/[][]//g' artifacts.temp > artifacts1.temp
cat artifacts1.temp >> $outputFile
#for filter in "${filters[#]}"; do
# cat artifacts.temp | grep "$filter" >> $outputFile
#done
#cat maven-public-artifacts.txt
contToken=( $(echo $response | sed -n 's|.*"continuationToken" : "\([^"]*\)".*|\1|p') )
done
# ======== DOWNLOAD EVERYTHING =========
echo Downloading artifacts...
urls=($(cat $outputFile)) > /dev/null 2>&1
for url in "${urls[#]}"; do
path=${url#http://*:*/*/*/}
dir=$sourceRepo"/"${path%/*}
mkdir -p $dir
cd $dir
pwd
curl -vks -u "$sourceUser:$sourcePassword" -D response.header -X GET "$url" -O >> /dev/null 2>&1
responseCode=`cat response.header | sed -n '1p' | cut -d' ' -f2`
if [ "$responseCode" == "200" ]; then
echo Successfully downloaded artifact: $url
else
echo ERROR: Failed to download artifact: $url with error code: $responseCode
fi
rm response.header > /dev/null 2>&1
cd $curFolder
done

Related

RPM .spec with bash must changed but not know what wrong

Pls answer me and hope my question not so stupid but i first time put inside rpm bash skrip. May someone can sent link or tip what i must coorrect in my .spec
This rpm pakage must make just two thing - copy in linux folder /bin/ one file(inet.dbg) and next when cp make must start bash skrip for check library for this file.
This my bash skript
#!/bin/bash
ldd inet.dbg > t.t
ERR=`grep -i "not" t.t | wc -l`
if [[ $ERR -gt 0 ]]; then
grep -i "not" t.t > erg.e
echo "Die folgenden Bibliotheken wurden nicht gefunden:"
cat erg.e
rm t.t erg.e
else
ls -a ss.tar &> t.t
if [[ $? -eq 0 ]]; then
echo alles ok
else
echo no
fi
fi '
This .spec
# %_topdir and %_tmppath are defined in ~/.rpmmacros
%define name inetdbg
%define version 0.1
%define release 1
%define buildroot %{_tmppath}/%{name}-%{version}-%{release}
%define tarfile %{tarversion}.tar
%define installscript /home/adis/rpmbuild/SOURCES/skript.sh
Name: %{name}
Version: %{version}
Release: %{release}
BuildArch: noarch
Summary: adisc
License: -
Source1: %{installscript}
Source2: /home/adis/rpmbuild/SOURCES/inetdbg.tar.xz
BuildRoot: %{_builddir}/%{name}-root
%description
Tested RPM
%prep
%build
%install
mkdir -p %{buildroot}%{_bindir}
install -D -pm 755 %{SOURCE2} %{buildroot}/bin/inetdbg.tar.xz
cp -a %{installscript} %{buildroot}%{_bindir}/
chmod a+x %{buildroot}%{_bindir}/%{installscript}
%files
%{_bindir}/%{installscript}
%{SOURCE2}
%post
%{_bindir}%{installscript}
%clean
[ ${RPM_BUILD_ROOT} != "/" ] && rm -rf ${RPM_BUILD_ROOT}
%changelog
* 25 Jan 2023 <user>
- Add script.sh
- Add definit

how to install Nginx on CentOs7 without internet connection with root permission?

I need to install Nginx on my target which there is no internet connection, how can I install Nginx with all dependencies in an offline mode?? thanks in advance for your answers.
I have recently gone through this procedure and this is what worked for me on centos7:
You need an online Linux server to download dependencies. You can use virtual machines or anything else.
On your online server create a .sh file and copy script below in it. (I named it download_dependencies)
#!/bin/bash
# This script is used to fetch external packages that are not available in standard Linux distribution
# Example: ./fetch-external-dependencies ubuntu18.04
# Script will create nms-dependencies-ubuntu18.04.tar.gz in local directory which can be copied
# into target machine and packages inside can be installed manually
set -eo pipefail
# current dir
PACKAGE_PATH="."
mkdir -p $PACKAGE_PATH
declare -A CLICKHOUSE_REPO
CLICKHOUSE_REPO['ubuntu18.04']="https://repo.clickhouse.tech/deb/lts/main"
CLICKHOUSE_REPO['ubuntu20.04']="https://repo.clickhouse.tech/deb/lts/main"
CLICKHOUSE_REPO['centos7']="https://repo.clickhouse.tech/rpm/lts/x86_64"
CLICKHOUSE_REPO['centos8']="https://repo.clickhouse.tech/rpm/lts/x86_64"
CLICKHOUSE_REPO['rhel7']="https://repo.clickhouse.tech/rpm/lts/x86_64"
CLICKHOUSE_REPO['rhel8']="https://repo.clickhouse.tech/rpm/lts/x86_64"
declare -A NGINX_REPO
NGINX_REPO['ubuntu18.04']="https://nginx.org/packages/mainline/ubuntu/pool/nginx/n/nginx/"
NGINX_REPO['ubuntu20.04']="https://nginx.org/packages/mainline/ubuntu/pool/nginx/n/nginx/"
NGINX_REPO['centos7']="https://nginx.org/packages/mainline/centos/7/x86_64/RPMS/"
NGINX_REPO['centos8']="https://nginx.org/packages/mainline/centos/8/x86_64/RPMS/"
NGINX_REPO['rhel7']="https://nginx.org/packages/mainline/rhel/7/x86_64/RPMS/"
NGINX_REPO['rhel8']="https://nginx.org/packages/mainline/rhel/8/x86_64/RPMS/"
CLICKHOUSE_KEY="https://repo.clickhouse.com/CLICKHOUSE-KEY.GPG"
NGINX_KEY="https://nginx.org/keys/nginx_signing.key"
declare -A CLICKHOUSE_PACKAGES
# for Clickhouse package names are static between distributions
# we use ubuntu/centos entries as placeholders
CLICKHOUSE_PACKAGES['ubuntu']="
clickhouse-server_21.3.10.1_all.deb
clickhouse-common-static_21.3.10.1_amd64.deb"
CLICKHOUSE_PACKAGES['centos']="
clickhouse-server-21.3.10.1-2.noarch.rpm
clickhouse-common-static-21.3.10.1-2.x86_64.rpm"
CLICKHOUSE_PACKAGES['ubuntu18.04']=${CLICKHOUSE_PACKAGES['ubuntu']}
CLICKHOUSE_PACKAGES['ubuntu20.04']=${CLICKHOUSE_PACKAGES['ubuntu']}
CLICKHOUSE_PACKAGES['centos7']=${CLICKHOUSE_PACKAGES['centos']}
CLICKHOUSE_PACKAGES['centos8']=${CLICKHOUSE_PACKAGES['centos']}
CLICKHOUSE_PACKAGES['rhel7']=${CLICKHOUSE_PACKAGES['centos']}
CLICKHOUSE_PACKAGES['rhel8']=${CLICKHOUSE_PACKAGES['centos']}
declare -A NGINX_PACKAGES
NGINX_PACKAGES['ubuntu18.04']="nginx_1.21.3-1~bionic_amd64.deb"
NGINX_PACKAGES['ubuntu20.04']="nginx_1.21.2-1~focal_amd64.deb"
NGINX_PACKAGES['centos7']="nginx-1.21.4-1.el7.ngx.x86_64.rpm"
NGINX_PACKAGES['centos8']="nginx-1.21.4-1.el8.ngx.x86_64.rpm"
NGINX_PACKAGES['rhel7']="nginx-1.21.4-1.el7.ngx.x86_64.rpm"
NGINX_PACKAGES['rhel8']="nginx-1.21.4-1.el8.ngx.x86_64.rpm"
download_packages() {
local target_distribution=$1
if [ -z $target_distribution ]; then
echo "$0 - no target distribution specified"
exit 1
fi
mkdir -p "${PACKAGE_PATH}/${target_distribution}"
# just in case delete all files in target dir
rm -f "${PACKAGE_PATH}/${target_distribution}/*"
readarray -t clickhouse_files <<<"${CLICKHOUSE_PACKAGES[${target_distribution}]}"
readarray -t nginx_files <<<"${NGINX_PACKAGES[${target_distribution}]}"
echo "Downloading Clickhouse signing keys"
curl -fs ${CLICKHOUSE_KEY} --output "${PACKAGE_PATH}/${target_distribution}/clickhouse-key.gpg"
echo "Downloading Nginx signing keys"
curl -fs ${NGINX_KEY} --output "${PACKAGE_PATH}/${target_distribution}/nginx-key.gpg"
for package_file in "${clickhouse_files[#]}"; do
if [ -z $package_file ]; then
continue
fi
file_url="${CLICKHOUSE_REPO[$target_distribution]}/$package_file"
save_file="${PACKAGE_PATH}/${target_distribution}/$package_file"
echo "Fetching $file_url"
curl -fs $file_url --output $save_file
done
for package_file in "${nginx_files[#]}"; do
if [ -z $package_file ]; then
continue
fi
file_url="${NGINX_REPO[$target_distribution]}/$package_file"
save_file="${PACKAGE_PATH}/${target_distribution}/$package_file"
echo "Fetching $file_url"
curl -fs $file_url --output $save_file
done
bundle_file="${PACKAGE_PATH}/nms-dependencies-${target_distribution}.tar.gz"
tar -zcf $bundle_file -C "${PACKAGE_PATH}/${target_distribution}" .
echo "Bundle file saved as $bundle_file"
}
target_distribution=$1
if [ -z $target_distribution ]; then
echo "Usage: $0 target_distribution"
echo "Supported target distributions: ${!CLICKHOUSE_REPO[#]}"
exit 1
fi
# check if target distribution is supported
if [ -z ${CLICKHOUSE_REPO[$target_distribution]} ]; then
echo "Target distribution is not supported."
echo "Supported distributions: ${!CLICKHOUSE_REPO[#]}"
exit 1
fi
download_packages "${target_distribution}"
Then on the same directory that contains download_dependencies.sh run command below:
download_dependencies.sh <your linux version>
In my case, I ran code below (leave it blank to see options):
download_dependencies.sh centos7
It should start to download and when it finished you should see nms-dependencies-rhel7.tar.gz in your directory.
Copy that file(.tar.gz) to your offline target.
Now on your target machine, go to directory which you copied your file and run the code below:
tar -zxvf nms-dependencies-rhel7.tar.gz
sudo yum install *.rpm
After installation you can start nginx using systemctl:
sudo systemctl start clickhouse-server
sudo systemctl start nginx
Your nginx service must be running now!
you can download tar file in another system and copy
did you try this link?
https://gist.github.com/taufiqibrahim/d7f697de6bb8b93ca348a5b94d6adbfc

RPM-SPEC - Bash Script not installing packages specified in %post SPEC file

I wrote a bash script to install multiple packages and calling the bash script in %post. But its not executing the script. I am new to this area and I am not sure what I am missing.
If i manually execute the script its working fine but not through the rpm package.
InstallRPM.spec file:
Name: InstallRPM
Version: 1
Release: 1%{?dist}
Summary: Install RPM Packages
License: Script
URL: NA
Source0: InstallRPM-1.tar.gz
BuildArch: noarch
BuildRoot: %{_tmppath}/%{name}-buildroot
Requires: /bin/sh
%description
Install RPM files from the /tmp/ folder
%prep
%setup -q
%install
mkdir -p "$RPM_BUILD_ROOT"
cp -R * "$RPM_BUILD_ROOT"
%post
echo "Executing the script /tmp/InstallRPM.sh"
/tmp/InstallRPM.sh
%clean
rm -rf $RPM_BUILD_ROOT
%files
%defattr(-,root,root,-)
/tmp/InstallRPM.sh
Shell Script:
#!/bin/bash
echo "Installing package using YUM"
package="vim zip"
echo "Install $package ..."
if [ "`cat /etc/centos-release | awk '{print $1}'`" == "CentOS" ]
then
{
for i in $package
do
{
#if [ "`/bin/rpm -qa $i | cut -d. -f1`" == "" ]
if [ "`/bin/rpm -qa $i`" == "" ]
then
{
echo "Clean Install"
/usr/bin/sudo /bin/yum install -y $i
}
else
{
echo "Upgrade"
/usr/bin/sudo /bin/yum update -y $i
}
fi
echo "****************"
}
done
}
fi
I ran the rpmbuild -ba InstallRPM.spec and it created a rpm file which I executed but nothing happened. Its getting hung in the below mentioned spot.
Downloading packages:
vim-enhanced-7.4.629-6.el7.x86_64.rpm | 1.1 MB 00:00:02
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
^C
You are using rpm the wrong way. Don't install dependencies in a %post script. I suggest you to read the rpm packaging guide first.
You should use Requires in your spec file, like this:
Requires: vim
Requires: zip

Trial AppDynamics package fails to install

Getting below error during platform installation:
"Required libaio package is not found. ..."
However, above package is already installed:
rpm -q libaio
libaio-0.3.107-10.el6.x86_64
Here is output from the installation script:
./platform-setup-x64-linux-4.4.3.10393.sh
Unpacking JRE ...
Preparing JRE ...
Starting Installer ...
May 30, 2018 6:51:23 PM java.util.prefs.FileSystemPreferences$2 run
INFO: Created system preferences directory in java.home.
Verifying if the libaio package is installed. /opt/appdynamics/platform/installer/checkLibaio.sh
I got this too... I was running from command-line as a non-root user:
./platform-setup-x64-linux-4.4.3.10393.sh -q -varfile /appd/home/Install/response.varfile
I added the shell expand(-x) switch and log to the command(s) like so:
bash -x ./platform-setup-x64-linux-4.4.3.10393.sh -q -varfile /appd/home/Install/response.varfile > install.log 2>&1
If we tail the last bit of that log you get, this response in debug mode:
Verifying if the libaio package is installed. /opt/appdynamics/platform/installer/checkLibaio.sh
Required libaio package is not found. For instructions on installing
the missing package, refer to https://docs.appdynamics.com/display/PRO44/Enterprise+Console+Requirements
and the script checkLibaio.sh isn't left there... so you cannot figure it out easily. I also have a RedHat variant with the packages installed:
rpm -qa | grep libaio
libaio-0.3.109-13.el7.x86_64
Strangely enough I have one VM from the same image that will install the distribution just fine, and one that will not, so on the broken install (where I really want to install this). I ran another command from the expanded view of the install.log, which was a really long JVM command line. Anyways I got it to work and then made a looping script to retrieve the file (Because AppD for some reason removes the check script before you can look at it). The script is as follows:
#!/bin/sh
# Script used to check if the machine has libaio on it or not.
cat /dev/null > /opt/appdynamics/platform/installer/.libaio_status
chmod 777 /opt/appdynamics/platform/installer/.libaio_status
# Check if the dpkg or rpm command exists before running it.
command -v dpkg >/dev/null 2>&1
OUT=$?
if [ $OUT -eq 0 ];
then
if [ `dpkg -l | grep -i libaio* | wc -l` -gt 0 ];
then
echo SUCCESS >> /opt/appdynamics/platform/installer/.libaio_status
exit 0
fi
else
command -v rpm >/dev/null 2>&1
OUT=$?
if [ $OUT -eq 0 ];
then
if [ `rpm -qa | grep -i libaio* | wc -l` -gt 0 ];
then
echo SUCCESS >> /opt/appdynamics/platform/installer/.libaio_status
exit 0
fi
fi
fi
echo FAILURE >> /opt/appdynamics/platform/installer/.libaio_status
exit 1
I you run this script like me on the faulty platform what you will discover is that your version of Linux has both:
dpkg
and
rpm
installed. To work around this you should temporarily make a name change to one of these two package manager executables so it cannot be found (by your shell environment).
Most common here will be that you are running a RedHat variant where someone chose to install dpkg (For who knows what reason). If so desired remove that package and the install should be successful.

How to remove the previous version app folder when update(-Uvh) the RPM package(version number modified)?

#define program installation destination
%define app_destination /opt
%define app_name MY_APP_NAME
%define app_version 2.1
%define app_release 7%{?dist}
%define app_dir %{app_name}-%{app_version}
%define compress_file %{app_dir}.tar.gz
%define app_service_softlink /etc/init.d/%{app_name}
%define app_dir_softlink %{app_destination}/%{app_name}
Name: %{app_name}
Version: %{app_version}
Release: %{app_release}
Summary: MY APP ONE-SENTENCE SUMMARY %{app_version}
# An open source software license
License: GPLv3+
URL: http://www.starscriber.com/
Source0: http://ftp.gnu.org/gnu/%{compress_file}
%description
MY APP DESCRIPTION
%pre
#each time before install/upgrade RPM, check and remove the softlinks provided below
echo "pre..."
if [ -L %{app_service_softlink} ];then
rm %{app_service_softlink}
elif [ -f %{app_service_softlink} ];then
rm %{app_service_softlink}
fi
if [ -L %{app_dir_softlink} ]; then
rm %{app_dir_softlink}
elif [ -d %{app_dir_softlink} ]; then
rmdir %{app_dir_softlink}
fi
%prep
%setup -q
echo "prep..."
# Script commands to "build" the program (e.g. to compile it) and
# get it ready for installing. The program should come with
# instructions on how to do this.
%build
%install
echo "install..."
# uses relative paths
# creates buildroot/destination directory
mkdir -p %{buildroot}%{app_destination}
# copies tar.gz file from source directory to buildroot/destination directory
cp %{_sourcedir}/%{compress_file} %{buildroot}%{app_destination}
# changes directory to buildroot/destination
cd %{buildroot}%{app_destination}
# extracts compression file
tar xf %{compress_file}
# removes the compression file
rm -rf %{compress_file}
cd %{buildroot}%{app_destination}
#invoked after %post when RPM pkg is removal or upgrade
%preun
echo "preun..."
#leftover cleanup
#invoked after %preun when RPM pkg is removal or upgrade
%postun
echo "postun..."
if [ "$1" == "0" ]; then
rm -rf %{app_destination}/%{app_dir}
fi
if [ ! -d %{app_destination}/%{app_dir} ]; then
if [ -L %{app_service_softlink} ]; then
rm %{app_service_softlink}
elif [ -f %{app_service_softlink} ]; then
rm %{app_service_softlink}
fi
if [ -L %{app_dir_softlink} ]; then
rm %{app_dir_softlink}
elif [ -d %{app_dir_softlink} ]; then
rmdir %{app_dir_softlink}
fi
fi
%files
#all files under the provided folder will be gathered up to create RPM pkg
%{app_destination}/%{app_dir}/bin
%{app_destination}/%{app_dir}/conf
%{app_destination}/%{app_dir}/misc
%post
echo "post"
#symbolic link to the new appdir with version
echo "builds new symbolic link for the app folder"
ln -sf %{app_destination}/%{app_dir} %{app_dir_softlink}
echo "builds new symbolic link for the app service"
# make a symbolic for the service file using the new created softlink
ln -sf %{app_destination}/%{app_name}/misc/%{app_name} %{app_service_softlink}
I am trying to create my own RPM package, and here's the SPEC file, it works properly when install(rpm -ivh app-2.1-6.el6.x86_64.rpm), or upgrade(rpm -Uvh app-2.1-7.el6.x86_64.rpm) or remove (rpm -e app-2.1-7.el6.x86_64.rpm)
For RPM package app-2.1-7.el6.x86_64.rpm, the version is 2.1 and release number is 7.
My question is, no matter how I modify the release number, install/upgrade/remove are working properly, but if I modify the version number to 2.2 or 3.2, the previous version folder(/opt/app-2.1) will not be deleted, can anyone help me, how should I delete the previous version folder(/opt/app-2.1) when I update(-Uvh) the RPM package?
The problem is that your package doesn't "own"
the directory /opt/2.1 directory.
Just like tar, rpm will create all "missing" directories
in order to install content on a path.
But on erase, rpm will only remove directories that are
mentioned explicitly in the %files manifest.
Short answer:
If you want rpm --erase to remove a directory path,
the mention in %files.
Shorter anser:
Add
%dir /opt/app-%{version}
to %files. If the directory is empty (i.e. all other files
in /opt/app-%{version} are "owned" and can be removed), the
the "owned /opt/app=%{version} will be removed as well.

Resources