Turbo Talk

“It’s midnight: do you know where your applications are?”

It used to be somewhat easy. The world was static with relatively few moving parts. A single application on a single OS on a single server with attached storage. With a few (sometime more than a few) pointed niche management tools we were able to get our hands around and manage our environments. Well, virtualization changes everything.  No more static boundaries and well defined interactions between the IT silos.

Do you know where your applications are?  Do you know where your virtual machines are?  Do you know what resources they are using?  Do you know how they perform?  Do they need more or less resources to deliver on their goals?   Are there bottlenecks in your environment? Where are the bottlenecks?

More important! Do you know what you need to DO now? In the next minute? Hour? Day? Week? Month? Do you need to start a new VM? Stop a VM? Move a VM? Do you know where to start/move the VM? Do you need to reconfigure any of its resources?  Do you need to provide more resources? What do you need to do to address the bottlenecks? How do you prevent them?

Are the management tools up to the tasks at hand? The answer is NO!

Virtualization brings down the walls between the silos, but management is lagging far behind and continuing on the trajectory to nowhere of the last decades. There are five fundamental gaps in today’s management environment that we need to bridge before we can address the challenges and opportunities introduced by virtualization:

• Business Gap – IT management is not aligned with the supported business and is not governed by business driven goals and polices.
• Management Information Gap – IT is focused on collecting too much data about the infrastructure that has to be deciphered, creating a huge gap between the raw data that is collected and the meaningful, actionable, information required for intelligent cost effective operation.
• Technology Gap – Heterogeneous environments made up of a wide variety of technologies spanning from networks, through server, storage, and all the way up to the applications. All are parts of the IT stack participating in delivering business service, yet they are being managed in silos. Furthermore, within each layer of the stack, each technology and each product is managed separately.
• Operational Gap – Operational disciplines, fault, performance, planning, configuration, provisioning, accounting/billing, security are all done separately.
• Automation Gap – To reduce complexity and operational costs, we strive for self- managed environments, i.e., environments that configure themselves, optimize themselves, secure themselves and practically heal themselves. Yet, these self-managed automation functions are tightly interacting with the managed entities, dealing with the detailed mess of the different entities in the environment. There is no decomposition of services and no decoupling of functional layers. We end up with one big monolithic mess of interactions.

A proper foundation for bridging the gaps must have three tenets, Abstraction, Analysis, and Automation:

• Abstraction – a layer of abstraction provides a model of the managed environments. The models hide the details of the environments by providing a common abstraction of the heterogeneous environments, exposing a common, rich, semantic interface for interacting with the environments.
• Analysis – analysis engines driven by the knowledge captured and represented by the abstraction layer to drive intelligent decision automation.
• Automation – expose the foundations as collections of services orchestrated using workflow engines driven by business policies.

Utilizing these three tenets provides the proper foundation for virtualization management solutions that will:

• Provide intelligent actionable information with minimal monitoring information, reversing the current trend of chasing too much information and trying to discover, represent and rely on continuously changing topological relationships.
• Scale to the increasing sizes of future environments, meeting the challenges of dynamically changing heterogeneous environment utilizing common abstractions.
• And above all, tie the viewing with the doing utilizing intelligent analysis to drive automation and reduce operational costs!

In the last two decades IT management is in a race to nowhere. A race to discover more, collect more and present more. More reports, more graphs, more views. If there is an IT asset out there, there is a tool to discover it. If there is a metric that can be collected, there is a tool to monitor and graph it. If there is a knob to turn, there is a tool to control it. How many management tools do you have in your environment? Which one do you use when? How many different reports/views are you looking at each day? How much time do you spend investigating these reports? Are you in control? Do you sleep better?

IT environments are complex. Managing these environments can be challenging. Many different moving parts with very complex interactions make it very difficult to effectively track, monitor and control the environments. For years, trying to address the broad range of pain points, we kept throwing more and more management tools into the environment. Every pain point we answered with a different tool. Introducing a new technology or product to the environment led to the deployment of yet another management tool. Very quickly we ended up utilizing hundreds of different management tools/products. Instead of addressing the management challenges, we increased the TCO and created an operational and administrative nightmare. In fact, many enterprises have embarked on multimillion dollar projects to reduce the the number of management tools in use from several hundred to an end state of (hopefully) a few hundred.

The IT landscape is transforming. Virtualization, SaaS and cloud lead to increasingly more dynamic, complex, heterogeneous and large IT environments. In the same time, the constraints within which IT needs to operate are becoming more and more stringent. On one end, business constraints and regulation require strong alignment of IT with the business to meet aggressive business and service goals while keeping OPEX and CAPEX under control. On the other end, environmental constraints of energy, space, etc. limit the flexibility and the free ride we used to have.

We are no longer looking at static siloed IT environments where the boundaries between the technology layers and products were well defined and understood. Virtualization brings these walls down. No longer are we managing one application on one known dedicated physical machine and attached storage, but rather a complex dynamic boundary-less compute capacity available for our applications on-demand. Can we scale to these types of environments with the existing brittle intelligence tools and approaches?

In spite of the savings and increased business agility available through the use of virtualization, virtualization management is on the same “old” trajectory as the management of the more established layers of the infrastructure stack.  There are more and more point tools:

• Chasing and collecting more and more data
• Chasing too much knowledge about the intricacies and relationships between the components (In dynamic virtualized  environments these relationships are continuously changing. Chasing an accurate up-to-date picture of the environment at all time is hard and close to impossible.)
• Chasing overly detailed information (Lack of abstraction leads to lack of scalable management solutions)

Virtualization management is relatively a young space, but we already see too many different non-integrated tools. In a recent article, “Managing Performance and Capacity in Virtualized Server and Desktop Environments” (registration required), covering only performance monitoring tools, analyst Bernd Herzog mentions twenty-four different monitoring tools in six different categories: resource and availability management, infrastructure performance management, application performance management, transaction performance management, end user experience management and virtual desktop management. The implication is that to gain visibility and try to assure the application service, one needs to buy six separate point tools. But do you really achieve application service assurance by buying those tools? When a problem occurred somewhere in the environment, which tool would you look at to find out where the problem is?

The trajectory leads to management that is too complex, not scalable, and provides limited value towards the end goal of a self-managed environment.  Instead of reducing the management complexity and the operational costs the current trends only contribute to the increasing management nightmare.

We must stop the trend. “Less is More!” We must look for management solutions that utilize novel approaches to get us on the road to self-managed orchestrated environments. Reversing the current trends of chasing too much information and trying to discover, represent and rely on continuously changing topological relationships. This is the only way we will be able to scale and meet the challenges of the new dynamically changing environments.