A lot has already been said about the false cloud use where the IaaS platform utilized as an hosting extension of the IT organization’s data center and not taking advantage of the elasticity benefits to generate a cost effective and scalable IT operation. Using the public IaaS whether it is Amazon, Rackspace or any other vendor means using a highly dynamic environment which presents an increasing complexity hence loss of control. Checking the list below I can say that cloud (including all its layers IaaS, PaaS and SaaS) control basically contains the same aspects as the good old system management.

What is “System management” ?

“refers to enterprise-wide administration of distributed systems including (and commonly in practice) computer systems.”
“System management may involve one or more of the following tasks:

• Hardware inventories
• Server availability monitoring and metrics
• Software inventory and installation
• Anti-virus and anti-malware management
• User’s activities monitoring
• Capacity monitoring
• Security management
• Storage management
• Network capacity and utilization monitoring”

Read More on Wikipedia

In my opinion “cloud management” is a general term that refers to all the cloud (on all layers) environment’s aspects (such as security, availability, compliance, cost etc.) and their tasks (deployment, administration, monitoring and analysis).
All of the leading IaaS vendors provide out-of-the-box basic capabilities to deploy and monitor the computing resources though It is also a fact that the IaaS vendors invest much more in developing robust APIs than in their web UI console. Supporting their core business they also focus on provisioning (of resources) tools such as support of auto scaling, CDN and full stack deployment of applications’ templates. Only on September 2009, released Amazon AWS their web UI management console and although they stated that they are extending their UI development team, we can still agree that it is sluggish and unstable. Simple quick Google search took me to the Amazon AWS community forum where I found the following example –

“There’s weirdness in the UI that shows all AMIs as “undefined” and when I try to launch it, it only show clusters as options ..” Check it on Amazon AWS forum

The traditional system management market is adjusting to the new cloud era. Developers of IT management solutions such Zenoss, Nagios and Splunk seems to be the first ones to recognize the cloud evolution and already developed plug-in extensions to support integration with the public clouds though these seem to  hold a “patchy manner”. Parallel to the traditional vendors adjustment to the IaaS, there is a rapidly evolving market of new vendors (RightScale which is the most veteran started on 2008 and got an additional investment of $25M only last year) that were natively born for the public cloud. Most of these new vendors started with developing tools on AWS APIs in order support Amazon’s cloud consumers. As new ISVs (SaaS) and a “cloud vendors” they understand perfectly the need to support the economy of scale and embraced the SaaS model by supporting features such as self-provisioning, pay-as-you-go, etc.
Naturally, the traditional vendors invest much more in their enterprise customers by leveraging their core capabilities to establish private and hybrid cloud offerings. In comparison to the traditional vendors, the new cloud management leading vendors are not really pushing towards support of an integration with the private IaaS but they do say that they have the option. In the rest of the article I will concentrate on these new cloud management vendors.
What’s Cloud Management ?
Among all I find that the main two motivators for the cloud management market are cloud portability and the fact that the current IaaS management UI consoles still don’t support all cloud management basic issues as well as their poor UIUX.
Let’s deal with the portability issue first. I think that Gartner analysts were the first to claim the term “Cloud Broker” –

“What’s a cloud broker? Gartner defines it as “a type of cloud service provider that plays an intermediary role in cloud computing.” Perhaps better put, they help you locate the best and most cost-effective cloud provider for your needs.”

In his recent short post say David Linthicum that  the future of cloud brokers is still unknown. Also I am not really sure that the current cloud vendors that claim to do cloud management will agree that they only do cloud brokerage –

“At the same time, the value of a cloud broker is undetermined, considering that cloud computing is still emerging and the provider choices could become more obvious. In such a case, brokers won’t have as much value.”

In regards to the underdeveloped UI consoles I can just say that although these are being developed slowly, eventually the huge IaaS vendors will fill the gap maybe even by buying some of today’s monitoring and management cloud providers. So, does this market have a future ? I tend to say that the answer is yes as I see this market’s rapid growth though it still needs to demonstrate agility while embracing basic guidelines of what it is really solves.
There is a chance that some guys of the cloud management market that are now reading these lines don’t agree with me on the immaturity of  this market, though when I asked several leading vendors to define the term “Cloud Management” the answers I got contained different views and even some clear marketing statements. The answers were missing the objectivity that I probably would have found if I were to ask them to define “IaaS” for example. What is “Cloud Management” and what areas does it cover?
Cloud Management Vendors
Let’s define “monitoring” as it can very easily be counted even as cloud control or even cloud management:

“To monitor or monitoring generally means to be aware of the state of a system” Read more on Wikipedia

I expect from a cloud monitoring tool to provide full visibility including graphical tools. It should also contain an event handler system including alerts and notifications. In fact most of the cloud monitoring tools also enable the option to administrate the environment in some level from new instance’s initiation all the way to full application stacks deployment.
Today you can find a considerably large amount of SaaS vendors that specialize in IaaS monitoring and deployment. From simple solutions that aims to replace Amazon AWS UI such as ElasticFox or monitoring tools with nice graphical interfaces such as ylastic.com, all the way to a comprehensive monitoring and deployment tools such as CloudKick. CloudKick considers itself as a cloud management provider, though checking their product’s feature page (on their site) reveal a sub section for their service called management. The page includes features such as view of all instances and instances tagging capabilities and more – Are these the features of cloud management ? obviously the answer is no. Another vendor who claims to be a cloud management service is Kaavo, though if you will check their demo you will find that their tool concentrate on making the deployment easy including monitoring layer for auditing on changes.
Is there a demand for Amazon AWS infrastructure management services ?
Searching Google for “Cloud Management” the first result you will find is RightScale which is maybe the most veteran vendor in the cloud management market today. RightScale provides the comprehensive deployment capabilities as well as the monitoring and control. On their website under “Cloud Management  Environment”, you will find the following statement:

“RightScale is the leading provider of cloud management solutions that enable you to design, deploy, manage, and automate business-critical applications on the cloud.”

Checking their “Cloud Management Environment section” you will find the following features:

1. Management Category – “Using the dashboard, you’ll have transparent access to and control over all aspects of a cloud deployment including the ServerTemplates.. “

2. lifecycle support – “ RightScale Platform is open and transparent, when there are issues to be resolved or upgrades to install, it’s easy to troubleshoot, stage, test, and roll out with complete version control..“

3. Comprehensive Administration – “RightScale also supports customers with multiple users..”

Following past engagement I had with RightScale as a potential customer I asked their help with cost control, I got to understand that RightScale huge benefit is on the deployment of ready made server templates. Also It seemed that their offering delivery is not straight forward and includes medium investment in managed professional services. It is important to note the RightScale increasing eco-system includes system integrators that enable the IT organization to adjust RightScale to its specific needs. The RightScale eco-system includes also developers of tools that extend the basic capabilities of the product. RightScale is maybe the best product to manage large scale infrastructure and complex cloud environments though it can be expensive and over kill for simple cloud deployments.
Intelligent work load by Wikipedia:

“An emerging paradigm for IT systems management arising from the intersection of dynamic infrastructure, virtualization, identity management, and the discipline of software appliance development.” Read More

In comparison to the system management traditional world, cloud computing brings the need to perform an Intelligent workload management due to the “endless amount of computing resources” and its extreme elastic capability. Considering that The cloud management layer must include an intelligent deployment and monitoring of the resources. Following that I should mention enStratus. On their site it is stated that:

 “enStratus provides you with governance, security, automation, and cross-cloud management to reduce risk and cost. “

The last demonstration I got from the company team included options to deploy a cluster according to “your cloud considerations” such as the required up-time SLA,  the maximum latency or compliance type. For an example you can just ask to deploy a cluster with 99.99% up-time commitment and the system automatically creates the system wrapper to support the requirement. The UI is straight and simple though it seems that the product is still in its first phases and managed services might be included, depends on the deployment complexity. Listen to ”Who manages the cloud”  an interview of George Reese, CTO/Founder of enStratus brought to you by CloudCast.com
I presented the future’s question to George Hadjiyanis, VP, Sales and Marketing at Enstratus – a cloud governance vendor and I found his answer interesting as it represents the vendor perspective –

“Cloud Management will be the on-ramp for cloud computing in the future. Companies will be leveraging multiple cloud providers and will want one, unified management platform with cross-cloud automation and a common set security and governance controls.”

Check Those Additional 3 Amazon AWS Application Deployment Tools
Whats is the best Amazon AWS management tools ?
There is an evolution of new cloud start-ups (disclosure: I am working in such one) that are dealing with different aspects of cloud management such as cost, availability, compliance, etc. I can point on companies such as Cloudability that deals with cost control or some others that deal with security control. I asked Cloudability Co-founder, J.R. Storment about his thoughts in regards to the unified cloud management approach and it seems that he believe in distributed system of vendors. Basically it means that the cloud consumer will need to take advantage of multiple vendors where each deal with a single “cloudy” aspect:

“The cloud is fragmenting IT services and cost is the only constant across multiple vendors, anything else is like comparing apples to oranges. Our goal is to take timely and accurate data and turn it into information our customers can act on using the best tools for the job.”

The cloud management market is getting bigger every year as I hear about new born start-up companies that deal with the cloud management issue and its different aspects. As the IaaS platforms mature so their API capabilities and complexity. Taking advantage of those capabilities to perform smart and automatic mechanisms upon the infrastructure  layer creates new business opportunities in  the cloud industry.