Beginners guide to OpenStack

For IT, cloud is a magic because cloud computing transforms how IT has been done for decades. If cloud is magic then what is the magic wand which transforms the data center to something magical?

Resources such as compute, storage and network is usually managed separately. When need arises resources are added manually. This is how computing had been done for decades. We humans can ever settle with anything. Computing doesn’t escape from that notion of human beings. We always try to enhance something which is taking a lot of manual tasks or create something new as the need grows. The need that we are talking about here is the growth of not just mobile apps, the reason for the growth of cloud computing is; people think it is efficient and not so time consuming to do stuff. Cloud computing enables rapid development and deployment model. It reduces the time that is required in each stage of development. Most importantly its self service model attracts many IT organizations to transform their IT.

OpenStack is that magic wand.

Over the past few years OpenStack had gained popularity and it is being widely adopted. Companies like Intel, Walmart and many others are transforming their entire IT infrastructure using OpenStack platform.

What is OpenStack?

OpenStack is a set of tools and services that can be used to build a cloud computing platform. A common myth is that, people think of it as a replacement to popular hypervisor’s. This is not true. OpenStack uses the hardware resources found in data centers such as Storage, Compute and Networking to create a service model. Once the hardware resources are abstracted, OpenStack presents these resources to users as services in various forms – like Infrastructure as a Service (Iaas), Platform as a Service (PaaS), Software as a Service (SaaS). OpenStack platform is suitable for any deployment model (Public, Private, Hybrid and Community Cloud). Consider reading Introduction to Cloud Computing if you want to understand different cloud computing service and deployment model.

openstack-software-diagram
High level architecture of OpenStack

Each year, OpenStack foundation – which is the control body for OpenStack development – releases two major versions of OpenStack. Based on these versions organizations like Red Hat, SUSE, Oracle, SwiftStack, Ubuntu, Rackspace, Mirantis, VMware etc. create their own distribution by customizing services and packaging it in different ways. Some of these distributions eliminate the complexity of deployment. Like wise each of them have their own pros and cons.

Building blocks of OpenStack

OpenStack is not packaged as a single software which can be deployed within few clicks. It is a collection of services which are inter connected together. Within OpenStack development community these service are developed as a project. Each service has its own API, using which they communicate with one another. For example, Compute service (Nova) creates instances (VMs) and manage the instance resources that are allocated to it. Storage services such as Cinder (Block) and Swift (Object) provides storage access to instances. Similarly there are different services available. Following are some of important services in OpenStack,

Service Project name
Dashboard Horizon
Compute Nova
Networking Neutron
Object Storage Swift
Block Storage Cinder
Identity Service Keystone
Image Service Glance
Telemetry Ceilometer
Orchestration Heat

Dashboard

The dashboard service provides centralized view of cloud environment to user as well as cloud administrator. Using dashboard a tenant (User) can self-provision resources, create/destroy Instances; modify networking for instances etc. Cloud administrator also interacts with dashboard and has more control over entire cloud environment.

Compute

Compute services manage Instances life cycle meaning, compute services takes care of Instances form the time it is created until it is destroyed. Compute services does not function alone it needs a Hypervisor to run instances therefore Instance related tasks such as CPU, Memory allocation is taken care by Hypervisor in its own way. Compute services just manages these instances and monitors them. When an instance creation command is received it’s passed on to Hypervisor to execute the task.

Networking

As the name denotes networking service provides network connectivity for OpenStack services. It allows uses to define network connectivity for instances that they own via dashboard. It also allows other network plugins such as VMware NSX, Open vSwitch, for better functionality. In any cloud environment Networking is the most complex part.

Object Storage

Object storage is provided by this service. Objects are stored and retrieved via REST API (HTTP based). Because this API access it can be directly accessed by an application.

Block Storage

This service creates block storage devices that can be directly provisioned to an instance. This block volume can be used for database or any high speed data access needs.

Identity Service

Different services of OpenStack make use of Identity service to communicate with each other. Identity service is an authentication and authorization service.

Image Service

Image service functionality is to store and retrieve virtual machine disk images. Snapshot of an instance can be taken and it can be used as a template for new instances. Virtual machine disk image is a file in which the operating system is installed. Popular formats are VMDK, VDI, VHD, OVF, qcow2 etc.

Telemetry

This service monitors OpenStack cloud for usage information for metering and performance information for statistical purpose. However this services is not an out of the box billing solution.

Orchestration

This service provides template based orchestration for a cloud application. This service executes appropriate API calls to create/modify OpenStack resources.

OpenStack architecture

In OpenStack cloud, the physical machines are represented as Controller nodes, Compute nodes, Network nodes, Block storage nodes, and Object storage nodes. There can be one or many physical machines (clustered). Each type of node cluster has its own set of services running on them. Following figure is a simplistic view of OpenStack infrastructure; the arrow represents its scale-out nature.

Types of NodesController Nodes

Controller nodes runs core services such as Dashboard, Image, identity service and also supporting services like SQL Database service, Message queue, Network time protocol, Compute management service, Networking ML2 Plugin, etc.

Compute Nodes

Compute nodes runs Compute service. Another important component of compute node is the Hypervisor. KVM is the default hypervisor support for OpenStack but there are a number of other hypervisors such as ESXi, XenServer, Hyper-V, Docker, etc. Importantly, compute node also runs networking modular layer 2 (ML2) plugin. This is for virtual network support for Instances.

Please note that what we discuss here applies to KVM as hypervisor. Implementation method varies for a few other hypervisors, i.e. ESXi. If OpenStack is implemented on top of ESXi infrastructure we normally interact directly with vCenter.

The compute service (Nova) running in compute nodes interacts with KVM and acts as a control element. KVM takes jobs from Nova service for instance creation/deletion/modification.

Network Nodes

Networking node runs tenant networking services which provides functionalities such as switching, routing, network address translation (NAT), and Dynamic Host Configuaration Protocol (DHCP). OpenStack networking is called “Neutron”.

When compared with its predecessor – Nova networking – Neutron release supports three tier architecture and provides functionalities such as load balancing, VPN, and firewall. These services are provided to tenant and can be individually charged. It also enhances security. Unlike Nova networking, Neutron allows usage of plugins. Open vSwitch, VMware NSX plugin, and many other plugins can be used with Neutron. Internet connectivity for tenant virtual machines is provided by network nodes.

Block and Object storage nodes

These nodes provide block and object storage. They are standard x86 servers with a bunch of drives from which storage space for instances are carved out. The storage space from these nodes is used for various other purposes such as backup, block volumes, etc.

Variations in architecture

When implementing OpenStack cloud, one does not necessarily need to follow the discussed architecture. It is possible to run storage-related services on controller and compute nodes. However, it is not the recommended way. Various vendors have released their own distribution of OpenStack which is production ready. There are also appliances available to kick start cloud as fast as you can.

Conclusion

We merely scratched the surface of OpenStack on what it is. For detailed documentation head over to OpenStack.org documentation center. I hope the information presented here helped you to understand what OpenStack is and how it used. Please feel free to comment if you have any questions.

Neturon communication failure – Unable to establish connection to http://controller-node:9696/v2.0/extensions.json

After configuring neutron service i tried to verify its operation using command #neutron ext-list the following error encountered. This is not just specific to neutron alone. This error can occur while configuring services like cinder. This post discusses what caused this error and how it can be rectified.

Error : Unable to establish connection to http://controller-node:9696/v2.0/extensions.json

In my environment “controlle-node” is the host name of OpenStack controller node. The first place to look at is neutron server log located at /var/log/neutron/server.log It may help you to identify where it is failing. I verified recently configured /etc/neutron/neutron.conf file for any typos or wrong configuration. Nothing seems to be wrong. I also verified nova.conf and ml2_conf.ini files. All the configuration is apt and done as per the installation manual.

The cloud that I am building is a very small setup so i tried to uninstall neutron completely and reconfigure it from scratch to save some time. But it was not really a good choice to make. After the purge operation nova.conf file is lost. I quickly re configured nova (i had a backup of nova.conf) and verified its successful operation. And then I re configured neutron. But this time while verifying neutron service a different error interrupts the operation.

Error : Unauthorized (HTTP 401) (Request-ID: req-b92da632-9e08-4e7a-a8cf-51df8ed9ec28)

Checked /var/log/neutron/server.log following are the events that happened after executing #neutron ext-list

2015-02-15 10:46:32.945 20602 INFO neutron.wsgi [-] (20602) accepted ('x.x.x.x', 54745)
2015-02-15 10:46:32.947 20602 INFO urllib3.connectionpool [-] Starting new HTTP connection (1): controller-node
2015-02-15 10:46:32.973 20602 WARNING keystonemiddleware.auth_token [-] Unexpected response from keystone service: {u'error': {u'message': u'The request you have made requires authentication.', u'code': 401, u'title': u'Unauthorized'}}
2015-02-15 10:46:32.973 20602 WARNING keystonemiddleware.auth_token [-] Authorization failed for token
2015-02-15 10:46:32.973 20602 INFO keystonemiddleware.auth_token [-] Invalid user token - rejecting request
2015-02-15 10:46:32.978 20602 INFO neutron.wsgi [-] 10.30.109.148 - - [15/Feb/2015 10:46:32] "GET /v2.0/extensions.json HTTP/1.1" 401 268 0.032020

Now its very evident that the keystone service is not accepting the credentials. If its an authentication error verify the password that is set for the neutron user account residing in keystone database. The next option is to verify neutron.conf in network node. In my case while re-configuring neutron password for neutron user in /etc/neutron/neutron.conf is incorrect. After changing the password keystone accepts authentication and neutron service is running successfully.

Please feel free to comment if the error still persists.