I have two servers, one with Nexus Repository and one with JFrog Artifactory OSS.
I would like to migrate to JFrog Artifactory but because it is open source there is no tool to migrate. If it were the pro version I could migrate easily.
I need some idea how to migrate from Nexus Repository to JFrog Artifactory OSS.
The way that I accomplished this task was to:
manually download the repos from Nexus. For nexus2 you can directly copy the repos from the filesystem. For nexus3 you will need to use the API. For nexus3 I used a script that I will put below.
Zip the repos into a folder
Upload the zipped file to artifactory with their import tool. Go to admin panel, artifactory, import, repositories.
You should be good to go from there more or less. I had to manually set some permissions and snapshot settings.
WARNING! This script doesn't work 100% you may/will have to modify it. WARNING!
sourceServer=
sourceRepo=
sourceUser=
sourcePassword=
logfile=$sourceRepo-backup.log
outputFile=$sourceRepo-artifacts.txt
# ======== GET DOWNLOAD URLs =========
url=$sourceServer"/service/rest/v1/assets?repository="$sourceRepo
contToken="initial"
while [ ! -z "$contToken" ]; do
if [ "$contToken" != "initial" ]; then
url=$sourceServer"/service/rest/v1/assets?continuationToken="$contToken"&repository="$sourceRepo
fi
echo Processing repository token: $contToken | tee -a $logfile
response=`curl -ksSL -u "$sourceUser:$sourcePassword" -X GET --header 'Accept: application/json' "$url"`
readarray -t artifacts < <( jq '[.items[].downloadUrl]' <<< "$response" )
printf "%s\n" "${artifacts[#]}" > artifacts.temp
sed 's/\"//g' artifacts.temp > artifacts1.temp
sed 's/,//g' artifacts1.temp > artifacts.temp
sed 's/[][]//g' artifacts.temp > artifacts1.temp
cat artifacts1.temp >> $outputFile
#for filter in "${filters[#]}"; do
# cat artifacts.temp | grep "$filter" >> $outputFile
#done
#cat maven-public-artifacts.txt
contToken=( $(echo $response | sed -n 's|.*"continuationToken" : "\([^"]*\)".*|\1|p') )
done
# ======== DOWNLOAD EVERYTHING =========
echo Downloading artifacts...
urls=($(cat $outputFile)) > /dev/null 2>&1
for url in "${urls[#]}"; do
path=${url#http://*:*/*/*/}
dir=$sourceRepo"/"${path%/*}
mkdir -p $dir
cd $dir
pwd
curl -vks -u "$sourceUser:$sourcePassword" -D response.header -X GET "$url" -O >> /dev/null 2>&1
responseCode=`cat response.header | sed -n '1p' | cut -d' ' -f2`
if [ "$responseCode" == "200" ]; then
echo Successfully downloaded artifact: $url
else
echo ERROR: Failed to download artifact: $url with error code: $responseCode
fi
rm response.header > /dev/null 2>&1
cd $curFolder
done
Related
Pls answer me and hope my question not so stupid but i first time put inside rpm bash skrip. May someone can sent link or tip what i must coorrect in my .spec
This rpm pakage must make just two thing - copy in linux folder /bin/ one file(inet.dbg) and next when cp make must start bash skrip for check library for this file.
This my bash skript
#!/bin/bash
ldd inet.dbg > t.t
ERR=`grep -i "not" t.t | wc -l`
if [[ $ERR -gt 0 ]]; then
grep -i "not" t.t > erg.e
echo "Die folgenden Bibliotheken wurden nicht gefunden:"
cat erg.e
rm t.t erg.e
else
ls -a ss.tar &> t.t
if [[ $? -eq 0 ]]; then
echo alles ok
else
echo no
fi
fi '
This .spec
# %_topdir and %_tmppath are defined in ~/.rpmmacros
%define name inetdbg
%define version 0.1
%define release 1
%define buildroot %{_tmppath}/%{name}-%{version}-%{release}
%define tarfile %{tarversion}.tar
%define installscript /home/adis/rpmbuild/SOURCES/skript.sh
Name: %{name}
Version: %{version}
Release: %{release}
BuildArch: noarch
Summary: adisc
License: -
Source1: %{installscript}
Source2: /home/adis/rpmbuild/SOURCES/inetdbg.tar.xz
BuildRoot: %{_builddir}/%{name}-root
%description
Tested RPM
%prep
%build
%install
mkdir -p %{buildroot}%{_bindir}
install -D -pm 755 %{SOURCE2} %{buildroot}/bin/inetdbg.tar.xz
cp -a %{installscript} %{buildroot}%{_bindir}/
chmod a+x %{buildroot}%{_bindir}/%{installscript}
%files
%{_bindir}/%{installscript}
%{SOURCE2}
%post
%{_bindir}%{installscript}
%clean
[ ${RPM_BUILD_ROOT} != "/" ] && rm -rf ${RPM_BUILD_ROOT}
%changelog
* 25 Jan 2023 <user>
- Add script.sh
- Add definit
I need to install Nginx on my target which there is no internet connection, how can I install Nginx with all dependencies in an offline mode?? thanks in advance for your answers.
I have recently gone through this procedure and this is what worked for me on centos7:
You need an online Linux server to download dependencies. You can use virtual machines or anything else.
On your online server create a .sh file and copy script below in it. (I named it download_dependencies)
#!/bin/bash
# This script is used to fetch external packages that are not available in standard Linux distribution
# Example: ./fetch-external-dependencies ubuntu18.04
# Script will create nms-dependencies-ubuntu18.04.tar.gz in local directory which can be copied
# into target machine and packages inside can be installed manually
set -eo pipefail
# current dir
PACKAGE_PATH="."
mkdir -p $PACKAGE_PATH
declare -A CLICKHOUSE_REPO
CLICKHOUSE_REPO['ubuntu18.04']="https://repo.clickhouse.tech/deb/lts/main"
CLICKHOUSE_REPO['ubuntu20.04']="https://repo.clickhouse.tech/deb/lts/main"
CLICKHOUSE_REPO['centos7']="https://repo.clickhouse.tech/rpm/lts/x86_64"
CLICKHOUSE_REPO['centos8']="https://repo.clickhouse.tech/rpm/lts/x86_64"
CLICKHOUSE_REPO['rhel7']="https://repo.clickhouse.tech/rpm/lts/x86_64"
CLICKHOUSE_REPO['rhel8']="https://repo.clickhouse.tech/rpm/lts/x86_64"
declare -A NGINX_REPO
NGINX_REPO['ubuntu18.04']="https://nginx.org/packages/mainline/ubuntu/pool/nginx/n/nginx/"
NGINX_REPO['ubuntu20.04']="https://nginx.org/packages/mainline/ubuntu/pool/nginx/n/nginx/"
NGINX_REPO['centos7']="https://nginx.org/packages/mainline/centos/7/x86_64/RPMS/"
NGINX_REPO['centos8']="https://nginx.org/packages/mainline/centos/8/x86_64/RPMS/"
NGINX_REPO['rhel7']="https://nginx.org/packages/mainline/rhel/7/x86_64/RPMS/"
NGINX_REPO['rhel8']="https://nginx.org/packages/mainline/rhel/8/x86_64/RPMS/"
CLICKHOUSE_KEY="https://repo.clickhouse.com/CLICKHOUSE-KEY.GPG"
NGINX_KEY="https://nginx.org/keys/nginx_signing.key"
declare -A CLICKHOUSE_PACKAGES
# for Clickhouse package names are static between distributions
# we use ubuntu/centos entries as placeholders
CLICKHOUSE_PACKAGES['ubuntu']="
clickhouse-server_21.3.10.1_all.deb
clickhouse-common-static_21.3.10.1_amd64.deb"
CLICKHOUSE_PACKAGES['centos']="
clickhouse-server-21.3.10.1-2.noarch.rpm
clickhouse-common-static-21.3.10.1-2.x86_64.rpm"
CLICKHOUSE_PACKAGES['ubuntu18.04']=${CLICKHOUSE_PACKAGES['ubuntu']}
CLICKHOUSE_PACKAGES['ubuntu20.04']=${CLICKHOUSE_PACKAGES['ubuntu']}
CLICKHOUSE_PACKAGES['centos7']=${CLICKHOUSE_PACKAGES['centos']}
CLICKHOUSE_PACKAGES['centos8']=${CLICKHOUSE_PACKAGES['centos']}
CLICKHOUSE_PACKAGES['rhel7']=${CLICKHOUSE_PACKAGES['centos']}
CLICKHOUSE_PACKAGES['rhel8']=${CLICKHOUSE_PACKAGES['centos']}
declare -A NGINX_PACKAGES
NGINX_PACKAGES['ubuntu18.04']="nginx_1.21.3-1~bionic_amd64.deb"
NGINX_PACKAGES['ubuntu20.04']="nginx_1.21.2-1~focal_amd64.deb"
NGINX_PACKAGES['centos7']="nginx-1.21.4-1.el7.ngx.x86_64.rpm"
NGINX_PACKAGES['centos8']="nginx-1.21.4-1.el8.ngx.x86_64.rpm"
NGINX_PACKAGES['rhel7']="nginx-1.21.4-1.el7.ngx.x86_64.rpm"
NGINX_PACKAGES['rhel8']="nginx-1.21.4-1.el8.ngx.x86_64.rpm"
download_packages() {
local target_distribution=$1
if [ -z $target_distribution ]; then
echo "$0 - no target distribution specified"
exit 1
fi
mkdir -p "${PACKAGE_PATH}/${target_distribution}"
# just in case delete all files in target dir
rm -f "${PACKAGE_PATH}/${target_distribution}/*"
readarray -t clickhouse_files <<<"${CLICKHOUSE_PACKAGES[${target_distribution}]}"
readarray -t nginx_files <<<"${NGINX_PACKAGES[${target_distribution}]}"
echo "Downloading Clickhouse signing keys"
curl -fs ${CLICKHOUSE_KEY} --output "${PACKAGE_PATH}/${target_distribution}/clickhouse-key.gpg"
echo "Downloading Nginx signing keys"
curl -fs ${NGINX_KEY} --output "${PACKAGE_PATH}/${target_distribution}/nginx-key.gpg"
for package_file in "${clickhouse_files[#]}"; do
if [ -z $package_file ]; then
continue
fi
file_url="${CLICKHOUSE_REPO[$target_distribution]}/$package_file"
save_file="${PACKAGE_PATH}/${target_distribution}/$package_file"
echo "Fetching $file_url"
curl -fs $file_url --output $save_file
done
for package_file in "${nginx_files[#]}"; do
if [ -z $package_file ]; then
continue
fi
file_url="${NGINX_REPO[$target_distribution]}/$package_file"
save_file="${PACKAGE_PATH}/${target_distribution}/$package_file"
echo "Fetching $file_url"
curl -fs $file_url --output $save_file
done
bundle_file="${PACKAGE_PATH}/nms-dependencies-${target_distribution}.tar.gz"
tar -zcf $bundle_file -C "${PACKAGE_PATH}/${target_distribution}" .
echo "Bundle file saved as $bundle_file"
}
target_distribution=$1
if [ -z $target_distribution ]; then
echo "Usage: $0 target_distribution"
echo "Supported target distributions: ${!CLICKHOUSE_REPO[#]}"
exit 1
fi
# check if target distribution is supported
if [ -z ${CLICKHOUSE_REPO[$target_distribution]} ]; then
echo "Target distribution is not supported."
echo "Supported distributions: ${!CLICKHOUSE_REPO[#]}"
exit 1
fi
download_packages "${target_distribution}"
Then on the same directory that contains download_dependencies.sh run command below:
download_dependencies.sh <your linux version>
In my case, I ran code below (leave it blank to see options):
download_dependencies.sh centos7
It should start to download and when it finished you should see nms-dependencies-rhel7.tar.gz in your directory.
Copy that file(.tar.gz) to your offline target.
Now on your target machine, go to directory which you copied your file and run the code below:
tar -zxvf nms-dependencies-rhel7.tar.gz
sudo yum install *.rpm
After installation you can start nginx using systemctl:
sudo systemctl start clickhouse-server
sudo systemctl start nginx
Your nginx service must be running now!
you can download tar file in another system and copy
did you try this link?
https://gist.github.com/taufiqibrahim/d7f697de6bb8b93ca348a5b94d6adbfc
I wrote a bash script to install multiple packages and calling the bash script in %post. But its not executing the script. I am new to this area and I am not sure what I am missing.
If i manually execute the script its working fine but not through the rpm package.
InstallRPM.spec file:
Name: InstallRPM
Version: 1
Release: 1%{?dist}
Summary: Install RPM Packages
License: Script
URL: NA
Source0: InstallRPM-1.tar.gz
BuildArch: noarch
BuildRoot: %{_tmppath}/%{name}-buildroot
Requires: /bin/sh
%description
Install RPM files from the /tmp/ folder
%prep
%setup -q
%install
mkdir -p "$RPM_BUILD_ROOT"
cp -R * "$RPM_BUILD_ROOT"
%post
echo "Executing the script /tmp/InstallRPM.sh"
/tmp/InstallRPM.sh
%clean
rm -rf $RPM_BUILD_ROOT
%files
%defattr(-,root,root,-)
/tmp/InstallRPM.sh
Shell Script:
#!/bin/bash
echo "Installing package using YUM"
package="vim zip"
echo "Install $package ..."
if [ "`cat /etc/centos-release | awk '{print $1}'`" == "CentOS" ]
then
{
for i in $package
do
{
#if [ "`/bin/rpm -qa $i | cut -d. -f1`" == "" ]
if [ "`/bin/rpm -qa $i`" == "" ]
then
{
echo "Clean Install"
/usr/bin/sudo /bin/yum install -y $i
}
else
{
echo "Upgrade"
/usr/bin/sudo /bin/yum update -y $i
}
fi
echo "****************"
}
done
}
fi
I ran the rpmbuild -ba InstallRPM.spec and it created a rpm file which I executed but nothing happened. Its getting hung in the below mentioned spot.
Downloading packages:
vim-enhanced-7.4.629-6.el7.x86_64.rpm | 1.1 MB 00:00:02
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
^C
You are using rpm the wrong way. Don't install dependencies in a %post script. I suggest you to read the rpm packaging guide first.
You should use Requires in your spec file, like this:
Requires: vim
Requires: zip
Getting below error during platform installation:
"Required libaio package is not found. ..."
However, above package is already installed:
rpm -q libaio
libaio-0.3.107-10.el6.x86_64
Here is output from the installation script:
./platform-setup-x64-linux-4.4.3.10393.sh
Unpacking JRE ...
Preparing JRE ...
Starting Installer ...
May 30, 2018 6:51:23 PM java.util.prefs.FileSystemPreferences$2 run
INFO: Created system preferences directory in java.home.
Verifying if the libaio package is installed. /opt/appdynamics/platform/installer/checkLibaio.sh
I got this too... I was running from command-line as a non-root user:
./platform-setup-x64-linux-4.4.3.10393.sh -q -varfile /appd/home/Install/response.varfile
I added the shell expand(-x) switch and log to the command(s) like so:
bash -x ./platform-setup-x64-linux-4.4.3.10393.sh -q -varfile /appd/home/Install/response.varfile > install.log 2>&1
If we tail the last bit of that log you get, this response in debug mode:
Verifying if the libaio package is installed. /opt/appdynamics/platform/installer/checkLibaio.sh
Required libaio package is not found. For instructions on installing
the missing package, refer to https://docs.appdynamics.com/display/PRO44/Enterprise+Console+Requirements
and the script checkLibaio.sh isn't left there... so you cannot figure it out easily. I also have a RedHat variant with the packages installed:
rpm -qa | grep libaio
libaio-0.3.109-13.el7.x86_64
Strangely enough I have one VM from the same image that will install the distribution just fine, and one that will not, so on the broken install (where I really want to install this). I ran another command from the expanded view of the install.log, which was a really long JVM command line. Anyways I got it to work and then made a looping script to retrieve the file (Because AppD for some reason removes the check script before you can look at it). The script is as follows:
#!/bin/sh
# Script used to check if the machine has libaio on it or not.
cat /dev/null > /opt/appdynamics/platform/installer/.libaio_status
chmod 777 /opt/appdynamics/platform/installer/.libaio_status
# Check if the dpkg or rpm command exists before running it.
command -v dpkg >/dev/null 2>&1
OUT=$?
if [ $OUT -eq 0 ];
then
if [ `dpkg -l | grep -i libaio* | wc -l` -gt 0 ];
then
echo SUCCESS >> /opt/appdynamics/platform/installer/.libaio_status
exit 0
fi
else
command -v rpm >/dev/null 2>&1
OUT=$?
if [ $OUT -eq 0 ];
then
if [ `rpm -qa | grep -i libaio* | wc -l` -gt 0 ];
then
echo SUCCESS >> /opt/appdynamics/platform/installer/.libaio_status
exit 0
fi
fi
fi
echo FAILURE >> /opt/appdynamics/platform/installer/.libaio_status
exit 1
I you run this script like me on the faulty platform what you will discover is that your version of Linux has both:
dpkg
and
rpm
installed. To work around this you should temporarily make a name change to one of these two package manager executables so it cannot be found (by your shell environment).
Most common here will be that you are running a RedHat variant where someone chose to install dpkg (For who knows what reason). If so desired remove that package and the install should be successful.
#define program installation destination
%define app_destination /opt
%define app_name MY_APP_NAME
%define app_version 2.1
%define app_release 7%{?dist}
%define app_dir %{app_name}-%{app_version}
%define compress_file %{app_dir}.tar.gz
%define app_service_softlink /etc/init.d/%{app_name}
%define app_dir_softlink %{app_destination}/%{app_name}
Name: %{app_name}
Version: %{app_version}
Release: %{app_release}
Summary: MY APP ONE-SENTENCE SUMMARY %{app_version}
# An open source software license
License: GPLv3+
URL: http://www.starscriber.com/
Source0: http://ftp.gnu.org/gnu/%{compress_file}
%description
MY APP DESCRIPTION
%pre
#each time before install/upgrade RPM, check and remove the softlinks provided below
echo "pre..."
if [ -L %{app_service_softlink} ];then
rm %{app_service_softlink}
elif [ -f %{app_service_softlink} ];then
rm %{app_service_softlink}
fi
if [ -L %{app_dir_softlink} ]; then
rm %{app_dir_softlink}
elif [ -d %{app_dir_softlink} ]; then
rmdir %{app_dir_softlink}
fi
%prep
%setup -q
echo "prep..."
# Script commands to "build" the program (e.g. to compile it) and
# get it ready for installing. The program should come with
# instructions on how to do this.
%build
%install
echo "install..."
# uses relative paths
# creates buildroot/destination directory
mkdir -p %{buildroot}%{app_destination}
# copies tar.gz file from source directory to buildroot/destination directory
cp %{_sourcedir}/%{compress_file} %{buildroot}%{app_destination}
# changes directory to buildroot/destination
cd %{buildroot}%{app_destination}
# extracts compression file
tar xf %{compress_file}
# removes the compression file
rm -rf %{compress_file}
cd %{buildroot}%{app_destination}
#invoked after %post when RPM pkg is removal or upgrade
%preun
echo "preun..."
#leftover cleanup
#invoked after %preun when RPM pkg is removal or upgrade
%postun
echo "postun..."
if [ "$1" == "0" ]; then
rm -rf %{app_destination}/%{app_dir}
fi
if [ ! -d %{app_destination}/%{app_dir} ]; then
if [ -L %{app_service_softlink} ]; then
rm %{app_service_softlink}
elif [ -f %{app_service_softlink} ]; then
rm %{app_service_softlink}
fi
if [ -L %{app_dir_softlink} ]; then
rm %{app_dir_softlink}
elif [ -d %{app_dir_softlink} ]; then
rmdir %{app_dir_softlink}
fi
fi
%files
#all files under the provided folder will be gathered up to create RPM pkg
%{app_destination}/%{app_dir}/bin
%{app_destination}/%{app_dir}/conf
%{app_destination}/%{app_dir}/misc
%post
echo "post"
#symbolic link to the new appdir with version
echo "builds new symbolic link for the app folder"
ln -sf %{app_destination}/%{app_dir} %{app_dir_softlink}
echo "builds new symbolic link for the app service"
# make a symbolic for the service file using the new created softlink
ln -sf %{app_destination}/%{app_name}/misc/%{app_name} %{app_service_softlink}
I am trying to create my own RPM package, and here's the SPEC file, it works properly when install(rpm -ivh app-2.1-6.el6.x86_64.rpm), or upgrade(rpm -Uvh app-2.1-7.el6.x86_64.rpm) or remove (rpm -e app-2.1-7.el6.x86_64.rpm)
For RPM package app-2.1-7.el6.x86_64.rpm, the version is 2.1 and release number is 7.
My question is, no matter how I modify the release number, install/upgrade/remove are working properly, but if I modify the version number to 2.2 or 3.2, the previous version folder(/opt/app-2.1) will not be deleted, can anyone help me, how should I delete the previous version folder(/opt/app-2.1) when I update(-Uvh) the RPM package?
The problem is that your package doesn't "own"
the directory /opt/2.1 directory.
Just like tar, rpm will create all "missing" directories
in order to install content on a path.
But on erase, rpm will only remove directories that are
mentioned explicitly in the %files manifest.
Short answer:
If you want rpm --erase to remove a directory path,
the mention in %files.
Shorter anser:
Add
%dir /opt/app-%{version}
to %files. If the directory is empty (i.e. all other files
in /opt/app-%{version} are "owned" and can be removed), the
the "owned /opt/app=%{version} will be removed as well.