I have a server machine with the following configuration.
1) Dual Quad Core Xeon
2) 24 GB Memory
3) 500 GB Sata
4) 256 * 2 RAID 1
The machine just arrived and we want to install CDH5 in it. We want create a sand box / dev cluster.
I am looking for some expert advice on
A) How many nodes we can create? We are targeting 4-5 nodes. Is that advisable.
B) I read Cloudera manager should be residing in the node with solid configuration. Based on our configuration how much resource should be allocated for the same.
C) We will install Ubuntu 12.4.
We are fairly new to this process. Any help would be really helpful.
Thanks,
Amit
The answer for A and B would be up to the purpose of the cluster. Let me answer in case the cluster is just for test purpose.
A: 4-5 nodes should be reasonable and Cloudera Manager can manage more than hundreds nodes.
B: Cloudera Manager server can be located with the other services (HDFS, MapReduce, or so). Please see the resource requirements here
C: Ubuntu 12.04 is supported.
A) You can start with 2 Master and 4 datanodes
B) Cloudera Manager will reside on one of the master nodes and run the supervising services and will give you a UI for managing services.
C) 16.04 LTS (Xenial), 14.04 LTS (Trusty), 12.04 LTS (Precise) are supported for ubuntu
Related
I want to install Openstack on CentOS 8(single node). I am having single machine (physical machine) where I want to install all nodes of Openstack. This setup I required for simulation only not production use.
I have tried to install Openstack using packstac 3 times but couldn't success.
I got different issues during installation:
1.In first attempt After installation, I tried to create instance, but not getting console of instances even after it got created successfully.
2. In second attempt, during deployment of instance, network not getting allocated.
3. In third attempt, it got stuck at packstack, puppet testing only.
I have followed below 2 links:
https://computingforgeeks.com/install-openstack-victoria-on-centos/
https://www.google.com/amp/s/www.linuxtechi.com/install-openstack-centos-8-with-packstack/amp/
I followed each and every steps mention in the likns.
I want to create two Ubuntu VMs on Openstack.
Can someone provide me some links/video, where I can get everything which is required to install Openstack on single node and create two Ubuntu VMs and assign network to them and test the connectivity between these two VMS.
Thanks in advance.
I would use official Packstack documentation. Note that you should start with a totally fresh Centos installation; i.e. don't try to install Packstack on a server where a previous installation failed (or succeeded).
You can also try Devstack. Its default configuration requires a smaller machine than Packstack (in my experience, 8GB RAM should be sufficient). Same remark: Start with a fresh installation of Centos or Ubuntu.
Microstack is another alternative. Its advantage is a very simple and quick installation; its disadvantage is a very strange (in my opinion) configuration and not a lot of documentation. However, it is suitable for your purpose. It claims to work on any Linux, Windows and MacOS; it does require snap.
I suggest directly installation onto Ubuntu Server.
some time ago I wrote a serie of posts in which I explained in detail how to install OpenStack Rocky. The 2 first blog posts ([1] and [2]) contain commands, examples, content of configuration files that cover common scenarios and tips for the successful installation of most OpenStack services (keystone, nova, glance, etc.) in a single node, and the third post [3] describes the installation of a computing node. This 3rd post is installed in a different node for the sake of making it easier to understand how nova works, but the installation can be safely carried out in the same node than the other components.
I find that the posts are short enough and are very easy to follow (I use that blog as my installation tips, and so I have used them for several deployments). The only caveat is that it is based on Ubuntu, but if you know about your installation, it should be easy to translate the installation to CentOS (some colleagues have used these tips for CentOS installations).
I tried to install Openstack several times last week (october 2021): a) with CentOS 8 Stream to metal hardware (real server) with devstack - no one version was installed (neither Master nor Xena & Wallaby, version Viktoria & below are not for Stream OS); b) Virtual machine with CentOS 8 Stream installed with packstack - installation was clearly successful (!), quite easy for install (according to official RDO project and its homepage), however there is the real problem with virtual and actual networking: no external network is accessible, router created was OK with external connection (router IP was detected successfully from outside) but no connection was possible from and to instance. So I conclude the Openstack package is not completely documented to resolve problems, however its installation can be quite easy (when successfully finish ;) )
Addition: Of coarse, there are resources with an information how network can be configured, official Openstack docs describes different network configurations as well (however it is difficult to find it for one click and being newbie), but anyway this system requires a lot of time to study before usage.
I am not able to locate any docs to upgrade unmanaged cluster (running 5.16) via command line like the one below for 5.15
https://www.cloudera.com/documentation/enterprise/5-15-x/topics/cdh_ig_upgrade_command_line.html
Can someone point me to the corresponding doc for 6.1 ?
This has been answered in cloudera forum. Cross-posting to prevent link rot
You can upgrade the packages based cluster but it's very tedious
process so not recommended.
https://docs.cloudera.com/documentation/enterprise/upgrade/topics/ug_overview.html
The best way is to migrate from packages to parcels and then
upgrade.
https://docs.cloudera.com/documentation/enterprise/6/latest/topics/cm_ig_migrating_packages_to_parce...
Once you migrated you can use this page to get the CM server running
with Web UI.
https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/installation.html
Regarding your ask "Is there any way by which an Unmanaged 5.x
cluster can move to CDH 6.x (may be adding it in Cloudera Manager
and then moving to 6.x)?"
I am thinking that you can build a 1 node CM server and then add these
present cluster by adding node in that CM. (Though I haven't tried
this on packages)
I installed all prerequisites as mentioned for ubuntu machine
(JDK-8, intellij IDEA, git).
I clone the sample code from the git. It's running but system got struct too much.
My system configurations are operating system : ubuntu 16.04, RAM :4Gb , Hard Disk:500 Gb. Is there any alternative solution for this problem any one can help me out.
Thanks in advance.
This question is answered at: Cordapp tutorial crashing in a Fedora VirtualBox Machine
For similar reason: your RAM is too small to handle multiple nodes.
According to our documentation, Corda has a suggested minimal requirement of 1GB of Heap and 2-3GB of Host RAM per node. https://docs.corda.net/docs/corda-enterprise/4.4/node/sizing-and-performance.html#sizing
Can anyone Please clarify me, i have a only 4gb Ram laptop with windows 7 installed ,currently i have working with Apache distributed hadoop1.x in a vmware,i want to practice Cloudera distributed hadoop with cloudera manager ,can you please tell how to install cloudera manager in 2gb ram allocated vmware.is it possible to install cloudera using cloudera manager in 2gb Ram allocated VMWARE with Redhat linux 6 installed?if yes can anyone tell me steps to install it in vmware with only 2gb Ram Size.
Thanks in advance.
In short: While running a CM cluster instance in pseudo-distributed mode on a 2GB VM is theoretically possible, due to the resource constraints it may lead to a sub-par user experience and is therefore not recommended. It's strongly advised to consider either installing CM or using the existing Cloudera Quickstart VM on a machine with no less than 4GB RAM available (after OS overhead).
Reasoning:
The 2GB of RAM will have to be divvied up between all of the selected components chosen at the time of installation (e.g. HDFS, YARN). Given a barebones configuration (HDFS + YARN), this will require the 2GB to be spread among the following services: NameNode, Secondary NameNode, DataNode, ResourceManager, NodeManager,JobHistoryServer, Cloudera Manager Web UI, PostgreSQL or whatever DB backend was chosen at the time of install, and Cloudera Manager Management services (if configured).
This would yield approx. 8-9 applications/services that would be constricted to using anywhere from 128MB - 256MB each, which depending on usage, could lead to utilization challenges such as GC thrashing, OOMs, or even CPU and RAM contention.
I have some questions:
Is it possible to install openstack on a Notebook with a 4GB DD3 Ram? Because the website says it needs atleast 8GB of RAM.
They say it requirs a double-QuadCore , I assue that means Octacore. Can we install that on a Quadcore?
They say that there is no possibility to install it on a NAS . Did you find any where if there is a possibility to do?. I dint find any even after asking our friend(google).
All in all, is it at-all possible to install on it a notebook/Desktop?
That advice is for production environments,
so 1)If you just want to play around your notebook will do fine. I had a succesful test-run on a 1.2 Ghz 1GB Netbook. It became incredibly slow when it launched it's first instance...
With a Double Quadcore they actually mean two seperate Quad-cores, as in two quad-core xeon processors on a single motherboard
So 2) yes you can install it on a quad-core.
3) a NAS device running openstack an openstack storage service seems to be unlikely indeed. You will most likely need more computing power.However If your NAS supports NFS or SSH or sth you can probably mount this drive and use it for storage.
4) You can perfectly build a all-in-one openstack test setup on your notebook. Performance will be low, but acceptable for testing.
It depends on what you mean by "install OpenStack". OpenStack itself is an extremely modular framework consisting on many services (Compute, Networking, Image service, Block Storage, Object Storage, Orchestration, Telemetry, ...). On top of that, a typical production deployment of OpenStack also requires several components, like load balancers, caching systems, firewalls, web servers and others. It is definitely possible to install a minimal openstack system, even on an average laptop.
The simplest way to run OpenStack on a laptop/desktop is to use Devstack, a shell script that installs all services from source and run them (by default) on a single machine. It is customizable enough to provide very good testing ground; it's used by OpenStack developers as well as the OpenStack QA team to test latest developments against "real" systems.
To avoid messing up your system, it's generally recommended to install OpenStack in a VM. From devstack doc:
DevStack should run in any virtual machine running a supported Linux release. It will perform best with 2Gb or more of RAM.
As of the time of this writing (Jan 2015), supported distros are:
Ubuntu (latest LTS)
Fedora
CentOS
Regarding NAS: you can of course use it, but "outside" Openstack apis, by providing mount points to your vms. It's even mandatory if you want to support live migration.