i am really confuse about the main difference between Open Stack and development Stack ?
is it different version or different platform ?
First of all, let's get the terminology correct. You are talking about OpenStack (one word) and DevStack (one word).
OpenStack is a suite of services for managing computing and related resources. You typically use it for managing "cloud computing" resources, consisting of multiple physical compute and storage servers. Setting up a system like this for production use on "real" hardware takes a lot of time and expertise.
DevStack is a essentially a set of scripts that will set up set up a simple OpenStack test environment running as virtual machines on (typically) a Linux PC with a few GB of RAM and a chunk of free disk space. The purpose is to allow a developer to set up a tiny cloud for trying things out, and doing service development work.
OpenStack and DevStack are neither different versions or different platforms. They are different parts of a larger whole. (You could say that DevStack is part of the OpenStack suite of software. Development of OpenStack and DevStack are handled by the same umbrella organization under the same governance, etc, etc.)
Related
I work for a small web startup. They have decided to use OpenStack as IaaS and then on top of it, cloudfoundry as PaaS. I am trying to learn about this technology stack. But I am really confused even after going through documentations and related materials on the web.
What do I want?
I have a web site, that currently runs on a RHEL system (aws instance), with
nginx as web server. I want to shift this to OpenStack-cloudfoundry
stack because the company's management has decided to do so. They also
want me to evaluate if I can put Docker to use anywhere.
From my understanding, OpenStack (Iaas) will provide me with all stuff related to hardware software needs, and cloudfoundry will help me on the development front.
Now, where does nginx (or any web server) come into the picture? Is it part of Openstack or Is it part of cloudfoundry?
On my aws RHEL system, Do I just install Openstack and Cloudfoundry, and then push my app and not at all bother about what happens beneath? I am really confused.. please help out.
And, Is there anywhere I can utilize Docker, in this setup?
You would generally not deploy OpenStack on top of AWS. OpenStack is similar to AWS in that it provides a service for you to create and destroy virtual machine instances, manage networking between and around your VMs, attach and detach block devices to instances, etc. In other words, both are services for managing "infrastructures", where "infrastructure" here means a virtualized datacenter, which at its core means a bunch of hardware running hypervisors that allow you regard the datacenter as a bunch of virtual machines that can be spun up and down on demand, rather than a bunch of "static" physical machines.
AWS is an Infrastructure-as-a-Service provided by Amazon, so you don't have to install AWS yourself, you can just start using it to provision VM instances within Amazon's datacenters. OpenStack is software you install yourself (or pay a vendor to manage for you) on hardware you own or pay for yourself, and once installed OpenStack provides a similar service/interface to AWS.
With a Platform-as-a-Service, you concern yourself more with your application code, and "just pushing it", and don't have to concern yourself as much with what's happening on the underlying machine. You don't have to worry as much about the underlying OS, making sure you have the right runtime and code dependencies of your application, generally don't have to care about the webserver that's serving your code, etc. And you get many more higher level features, e.g. easy ability to scale vertically or horizontally, dynamic routing, automatic log aggregation, automatic health management, etc.
As far as how nginx fits in, it depends how you're using nginx, and what kind of application you have. Cloud Foundry has few couple ways of dealing with applications.
One is the buildpack model, where you simply push your source code to the platform, and it will automatically detect the appropriate runtime and dependencies for your application. For instance, if your application is a Ruby application, it will automatically detect this, and by default automatically run the application using the WEBrick server. However, you can choose other Ruby webservers such as Phusion, Passenger, etc. [1]
If your application is primarily serving static content, it will use nginx as the webserver. [2]
Another is using Docker. You can deploy applications based on Docker images on Cloud Foundry, in which case you could have a container running nginx and your application inside the container, or not, it depends on whether you still need nginx. Pushing a docker application is as simple as:
cf push trainingwebapp --docker-image training/webapp -c 'python app.py'
Here, this uses the sample Hello World web app from the Docker documentation. [3]
[1] https://docs.cloudfoundry.org/buildpacks/ruby/ruby-prod-server.html
[2] https://docs.cloudfoundry.org/buildpacks/staticfile/index.html
[3] https://docs.docker.com/engine/userguide/containers/usingdocker/
With OpenStack's architecture, is it possible to, for instance, have a PowerPC64 (Altivec) machine, a Intel CoreDuo machine, and a ARMv6 all on the same cluster?
Or is this impossible, because of the restrictions in building buildpacks when deploying to multiple architectures?
EDIT: Whoops, I meant OpenStack, not OpenShift ;)
The answer above is correct (answer from developercorey).
Although whether this suits you depends on how its managed and what your trying to achieve. Typically when you add servers with different physical attributes such as CPU, Disk, Network cards etc you group them into different host aggregates.
By default when you launch a VM it will try and find a suitable host, but you can also tag it, so for example if your VM required alot of disk IO, you might want to place it on a host that has SSD drivers. So you can put those hosts into a 'SSD' aggregate, and then when launching your VM you can make sure it goes to a host in that aggregate.
If your just trying to make the most out of the hardware you have, then I don't see any issue by mixing them.
I don't think that they have to be, but I do believe that they only build packages for 1 or 2 architechtures, so I'm not sure how many options you really have there.
I am a newbie with Xen.I want to know how does Xen work.
It's really a puzzle when facing the code and I don't know where to start.
Are there some easy articles for me?
Since you mention looking at the code, I assume you want to understand the technical details of Xen and not just merely how to start a VM.
As with all problems, start with something simple and then work your way up. Some pointers:
Be sure to have the prerequisite experience under your belt. In particular, strong C and Linux affinity, but also x86 paging and virtualized memory workings.
Make sure you have a sound grasp of the general Xen architecture. For instance, paravirtualized versus hardware-supported virtualization, the special role of the management domain (Dom0) compared to unprivileged domains (DomU), etc.
Investigate the the Xen components running in Dom0:
The Xen control library (libxc) which implements much of the logic relating to hypercalls and adds sugar around these (look in tools/libxc).
The swiss army knife for administrating Xen, namely the Xen light library (libxl). This library replaces the deprecated xm tool with the xl tool and takes care of all your maintenance tasks such as starting/stopping a VM, listing all running VMs, etc. For all these operations, it works in tandem with the aforementioned libxc. (Libxl lives in tools/libxl.)
The Xenstore is a tree-like data structure from which all running domains can retrieve and store data. This is necessary since all I/O goes through Dom0 (not the hypervisor!), and domains need to communicate with Dom0 how they are going to pass I/O along. (Look in tools/xenstore.) You can inspect the Xenstore with a tool such as xenstore-ls.
the blkback/netback kernel drivers which pass the data over shared channels to the VMs. (You will find these drivers in a recent Linux kernel (e.g. >= v3.0) that has so-called PVOPS support).
Take a look at the console daemon (tools/console). Note that sometimes the Qemu console is actually used. Qemu also comes in the pictures as a default backend for if you choose a file-backed virtual storage for a VM.
Experiment with the 'Xen-way' of inter-VM communication: Grant tables, event channels and the Xenstore. With these fundamentals you can create your own shared channel between VMs. You can do this, for example, with writing a kernel module that you use in two domains to let them talk to each other.
I can also give some pointers in the source that you can check out:
xen/xen/include/public/xen.h will give you a list of all the hypercalls with comments what they do.
xen/xen/include/xen/mm.h gives you an introduction to the different memory terminology used by Xen (i.e., real versus virtualized addresses and page numbers). If you don't grasp these differences, then reading the hypervisor code will surely be frustrating.
xen/xen/include/asm-x86/config.h gives an overview of the memory layout of Xen.
xen/tools/libxc/xenctrl.h exports a large list of interesting domain control operations, which gives an abstract view of task division between Dom0 and the hypervisor.
Last but not least, the book 'The Definitive Guide to the Xen Hypervisor' by David Chisnall comes highly recommended. It covers all these topics and more in a thorough, technical fashion with plenty of code examples.
The Xen wiki and developer mailing lists are also a great resource for understanding Xen.
If you have a more specific question, then I can give you a more specific answer.
Here are few links which will guide you with ZEN Start up.Hope they will be useful.
http://www.howtoforge.com/howtos/virtualization/xen
http://wiki.xen.org/wiki/Category:HowTo
http://wiki.debian.org/Xen
For me, that is the best and more concrete tutorial with examples and step by step to start. I used it when I started.
Then you can read a lot more on Xen documentation itself or some books but as a starting point that allows you to easily install and test Xen, I choose that tutorial from Debian Wiki.
If you just want an overview, you may read this: http://wiki.xenproject.org/wiki/Xen_Project_Beginners_Guide.
This will introduce you to Xen hypervisor, suggest configuration to set up virtual machines, provide information about the networking and finally have details about tools for the management of virtual machines.
This documentation is to get the Xen specifically on ubuntu (Most importantly, it works!)
https://help.ubuntu.com/community/Xen
===
However, if you want to go to the next level and understand the working of Xen; Xen architecture, memory management, device management, CPU scheduling etc., I would recommend reading the book "The Definitive Guide to the Xen Hypervisor".
I am developing a distributed file system using Java, I cannot give many details at this moment. I need to test some things on Linux, I will use WMWare server an install Linux inside a virtual machine. Is there any difference between the simulated network card and a real ethernet interface?
I am developing a distributed file system ... I will use WMWare server an install Linux inside a virtual machine.
VMware is great for this sort of thing. There should be no difference except, as RichieHindle said, in performance, especially if you're planning to run multiple vms on the same server.
Use real hardware if you want usable performance benchmark results.
Java is it's own 'VM'... on top of a layer of virtualization in the guest OS... on top of VMware... on a virtual execution model CPU. Take a little virtualization here, add a little virtualization there, and pretty soon we're talking about some real abstraction!
From the point of view of application code, no, there's no difference.
The only visible difference might be in performance - the speed of response and exact timings of things might be different, but you're talking microseconds.
There's so much general-purpose software that works flawlessly under VMs that the answer to almost every question of the form "Are VMs different from real machines?" at the application level is "No".
(Things might be different if you were talking about kernel-level driver software.)
Improving Web Development Using Virtualization
https://web.archive.org/web/20090207084158/http://aspnet.4guysfromrolla.com:80/articles/102908-1.aspx
Virtualization is, in essence, creating multiple miniature (virtual) PCs inside of your primary PC. One of the great benefits of this is that it allows you to isolate and test an application or set of applications in an environment that is free of other things to interfere. It used to be that in order to get a new machine with a new development environment on it you had to have another piece of hardware, or you had to rebuild your system to the new environment. With virtualization, you simply install the new environment that you need into one of the virtual machines and you run it as necessary. When you're done you can shut it down.
Virtualization is the ultimate in isolation -- it can allow you to do things on one piece of hardware that are simply not possible without it. For instance, you can install software in a test environment on a member server because it won't run on a domain controller. You simply fire up two virtual machines at the same time -- one being the domain controller and the other being the member server. Both virtual machines can run on the same physical hardware at the same time without either being aware that they are sharing a machine. The result is a quick way to implement testing environments.
Virtualization technology allows for the virtual systems to be frozen in place. In other words, the exact spot in the machine that you are at can be frozen for an indefinite period of time. If you work on one project until it's released and stable and need to come back in a year and start working on it again, you can freeze the system when you stop working on the project and then restart it a year -- or more – later. When the system is restarted it will be like time had not passed. The system will be restored exactly as it was left.
This particular feature is great for developers who support multiple systems including consultants who have different clients with different projects that they will have to support over time. You don't have to worry about recreating an environment to test a bug fix; you simply thaw out your virtual machine and go.
Virtualization programs have a feature described as Undo disks. Undo disks allow you to operate on the system and if you decide that you don't want to save your work you simply don’t' accept the changes in the undo disks. Poof. Like magic everything that you did is undone and it's like it never happened.