I have to differentiate between Cloudera parcel process and writing a cook book or a reciepe of a Chef for installation of packages in a cluster.
So i'm looking for advantages and disadvantage between Parcel over Chef or Vice versa.
If you are using Cloudera Manager, parcels allow you to install/update CDH on your cluster(s) in its entirety via a single package (or, install add-on components such as betas and Cloudera Labs releases). From the docs:
Parcels are self-contained and installed in a versioned directory,
which means that multiple versions of a given parcel can be installed
side-by-side. You can then designate one of these installed versions
as the active one. With packages, only one package can be installed
at a time so there's no distinction between what's installed and
what's active.
Parcel handling automatically downloads, distributes, and activates
the correct parcel for the operating system running on each host in
the cluster.
Parcels can only be installed via CM, so if you're not a CM user, your question is an academic one. If you are a CM user, you can choose to use either parcels (which is certainly easier) or a packages-based approach via Chef or Puppet (not as easy, but some people prefer it nonetheless).
According to Parcels: What and Why? ยท cloudera/cm_ext Wiki, there are a number of benefits using parcels, including:
having a consistent image of the cluster (all components adhere to the same version)
managing rolling upgrades with ease (thanks to having two versions coexist on the same host)
using non privileged folders for the binaries.
It is sometimes observed that one of the most challenging parts of administering a Hadoop cluster are upgrades, so probably parcels will help the most in that sense.
With that respect, note that Cloudera Manager will be able to install the components initially with packages, but it will manage upgrades only if you had chosen the parcels option.
Related
I want to install Openstack on CentOS 8(single node). I am having single machine (physical machine) where I want to install all nodes of Openstack. This setup I required for simulation only not production use.
I have tried to install Openstack using packstac 3 times but couldn't success.
I got different issues during installation:
1.In first attempt After installation, I tried to create instance, but not getting console of instances even after it got created successfully.
2. In second attempt, during deployment of instance, network not getting allocated.
3. In third attempt, it got stuck at packstack, puppet testing only.
I have followed below 2 links:
https://computingforgeeks.com/install-openstack-victoria-on-centos/
https://www.google.com/amp/s/www.linuxtechi.com/install-openstack-centos-8-with-packstack/amp/
I followed each and every steps mention in the likns.
I want to create two Ubuntu VMs on Openstack.
Can someone provide me some links/video, where I can get everything which is required to install Openstack on single node and create two Ubuntu VMs and assign network to them and test the connectivity between these two VMS.
Thanks in advance.
I would use official Packstack documentation. Note that you should start with a totally fresh Centos installation; i.e. don't try to install Packstack on a server where a previous installation failed (or succeeded).
You can also try Devstack. Its default configuration requires a smaller machine than Packstack (in my experience, 8GB RAM should be sufficient). Same remark: Start with a fresh installation of Centos or Ubuntu.
Microstack is another alternative. Its advantage is a very simple and quick installation; its disadvantage is a very strange (in my opinion) configuration and not a lot of documentation. However, it is suitable for your purpose. It claims to work on any Linux, Windows and MacOS; it does require snap.
I suggest directly installation onto Ubuntu Server.
some time ago I wrote a serie of posts in which I explained in detail how to install OpenStack Rocky. The 2 first blog posts ([1] and [2]) contain commands, examples, content of configuration files that cover common scenarios and tips for the successful installation of most OpenStack services (keystone, nova, glance, etc.) in a single node, and the third post [3] describes the installation of a computing node. This 3rd post is installed in a different node for the sake of making it easier to understand how nova works, but the installation can be safely carried out in the same node than the other components.
I find that the posts are short enough and are very easy to follow (I use that blog as my installation tips, and so I have used them for several deployments). The only caveat is that it is based on Ubuntu, but if you know about your installation, it should be easy to translate the installation to CentOS (some colleagues have used these tips for CentOS installations).
I tried to install Openstack several times last week (october 2021): a) with CentOS 8 Stream to metal hardware (real server) with devstack - no one version was installed (neither Master nor Xena & Wallaby, version Viktoria & below are not for Stream OS); b) Virtual machine with CentOS 8 Stream installed with packstack - installation was clearly successful (!), quite easy for install (according to official RDO project and its homepage), however there is the real problem with virtual and actual networking: no external network is accessible, router created was OK with external connection (router IP was detected successfully from outside) but no connection was possible from and to instance. So I conclude the Openstack package is not completely documented to resolve problems, however its installation can be quite easy (when successfully finish ;) )
Addition: Of coarse, there are resources with an information how network can be configured, official Openstack docs describes different network configurations as well (however it is difficult to find it for one click and being newbie), but anyway this system requires a lot of time to study before usage.
Do we have any cli/api to check openstack development version..i mean whether my system installed havana or grizzly.I searched the openstack cli/api docs but i don't find any relevant.
OpenStack has a number of elements if you want to verify state.
Each of the component projects and each of the python-client api bindings have their own versions. Then there are configurable options for addressing API versions in REST queries.
I took a crack at building an API for the very purpose of verifying this data as well as all python dependencies a while back with the aim of cross verifying that against a vulnerability database but I simply haven't had the time to bring it to completion.
This would be a very useful feature I think.
You might look at your pip requires if you installed from source. Alternatively you can follow the debian package version chain from dependencies and that should provide good insight into what is installed on your system though it's not exactly verified.
I currently have a Java application packaged in an RPM that gets built for 32-bit RedHat platforms, and I want to create a 64-bit RPM, which is largely just the same as the 32-bit one, but with a couple different .so files included. All the Java stuff is the same on both platforms, so it's just JNI .so's.
My question is: Is it possible to have rpmbuild on a 32-bit system generate both the 32-bit and 64-bit RPMs (from different .spec files) since it's just repackaging already-built components, or do I need to build the 64-bit RPM on a 64-bit system?
N.B. I'm not actually building anything native on the system. I'm just repackaging stuff that's already built.
... or vice versa, can I build a 32-bit one on a 64-bit system? I really would prefer just to build and package this on one system than have two separate builds run for the separate RPMs.
As Aaron stated you can build an RPM for multiple distros on the same machine (64-bit), but you have to be very careful or you can run into issues. The biggest problem I've run into is you build on RHEL 5, then you try to deploy to RHEL 6, since RHEL 6 has a different version of RPM installed, it can cause conflicts and fail to install. So in this scenario you have a few options:
Build the RPM on two machines, you've stated you don't really want to do this.
If you have the disk space, configure Mock, I've used it a ton before and it's really easy to get going as long as you have the disk space and the package spec was designed to pull in requires properly.
Personally I'd give Mock a shot, it's quite simple to set up, and will allow you to do what you want with minimal effort as long as the proper repos are available. In the event the build fails the log is pretty comprehensive regarding what the RPM build error was.
I have some questions:
Is it possible to install openstack on a Notebook with a 4GB DD3 Ram? Because the website says it needs atleast 8GB of RAM.
They say it requirs a double-QuadCore , I assue that means Octacore. Can we install that on a Quadcore?
They say that there is no possibility to install it on a NAS . Did you find any where if there is a possibility to do?. I dint find any even after asking our friend(google).
All in all, is it at-all possible to install on it a notebook/Desktop?
That advice is for production environments,
so 1)If you just want to play around your notebook will do fine. I had a succesful test-run on a 1.2 Ghz 1GB Netbook. It became incredibly slow when it launched it's first instance...
With a Double Quadcore they actually mean two seperate Quad-cores, as in two quad-core xeon processors on a single motherboard
So 2) yes you can install it on a quad-core.
3) a NAS device running openstack an openstack storage service seems to be unlikely indeed. You will most likely need more computing power.However If your NAS supports NFS or SSH or sth you can probably mount this drive and use it for storage.
4) You can perfectly build a all-in-one openstack test setup on your notebook. Performance will be low, but acceptable for testing.
It depends on what you mean by "install OpenStack". OpenStack itself is an extremely modular framework consisting on many services (Compute, Networking, Image service, Block Storage, Object Storage, Orchestration, Telemetry, ...). On top of that, a typical production deployment of OpenStack also requires several components, like load balancers, caching systems, firewalls, web servers and others. It is definitely possible to install a minimal openstack system, even on an average laptop.
The simplest way to run OpenStack on a laptop/desktop is to use Devstack, a shell script that installs all services from source and run them (by default) on a single machine. It is customizable enough to provide very good testing ground; it's used by OpenStack developers as well as the OpenStack QA team to test latest developments against "real" systems.
To avoid messing up your system, it's generally recommended to install OpenStack in a VM. From devstack doc:
DevStack should run in any virtual machine running a supported Linux release. It will perform best with 2Gb or more of RAM.
As of the time of this writing (Jan 2015), supported distros are:
Ubuntu (latest LTS)
Fedora
CentOS
Regarding NAS: you can of course use it, but "outside" Openstack apis, by providing mount points to your vms. It's even mandatory if you want to support live migration.
On windows applications are typically packaged as MSI, on Redhat Linux as RPM, what would be a best open source packaging method that could be used to deploy applications to all platforms including different flavors of unix and windows?
Contents would include exes, unix binaries, java jar files, user data, even database scripts to be run.
(I recognize contents would vary per destination OS, ie. binaries would be different, win exe vs unix binary etc, but for example config files may be the same or in the case of java even the bytecode jars)
Key feature I'd like the packaging to support is different users and permissions for different directories, however I recognize supporting this feature multiplatform may be very difficult.
Rather than build a package that is supposed to work across all of your platforms, which is likely impossible, you should have your build system build different packages for each target platform.
With CPack (It come with CMake) you can create packages for Windows (with NSIS), Linux (rpm and deb), and OS X with "make package". CMake also simplify cross-platform building.
For a sample you can look at avogadro's CMakeLists.txt and AvoCPack.cmake
I have a client that uses IzPack to create a single installer (it's Java-based) that installs their app on Windows, OS X and Linux.
http://izpack.org/
NSIS is an open-source solution which, as far as I know is able to build installers that run on Windows and UNIX-likes alike. However, for software deployment on Windows (especially in corporate environments) MSI is the way to go and NSIS is more of a headache.
So I wouldn't advise that you try to build a single package/installer for different platforms. But rather, as RibaldEddie indicated, multiple packages: one for each platform. That also allows to restrict the contents of the package to the files relevant to each platform.
If you'd like to support packaging for multiple distributions, I'd suggest helping the packagers for those distributions out; use some sort of well-known build system for your software (GNU's autotools or something like scons or waf), and document the build, optional dependencies, and so forth pretty well.
That way, when a Debian, Ubuntu, Red Hat, SuSE, whatever, packager comes along, they'll be able to create the package for you. You can optionally include packaging templates for one or more distributions in a separate VCS tree that is available, if you'd like.
If you are looking at packaging a closed-source/proprietary application for multiple systems, you'd probably do best to package up a .tar.gz file and document the installation process for it. You'll also want to make sure that the build process used doesn't embed any path information into the application, so that it can be run in /opt, /usr, or /usr/local, which are some popular choices for third-party add-on software.
BitRock InstallBuilder allows you to create installer packages for each one of the platforms you mentioned (as well as creating RPM, DEB, packages etc. from a single project file)