Looking at a theoretical distribution for scaling clouds reminded me of the Petit Prince’s hat (which he insisted was an elephant eaten by a snake).
McCrory showed that clouds also eat elephants.
10/4 Update:
It turns out that the elephant breaking into our plot was Hadoop…
There’s a surprising about of a hair pulling regarding IaaS vs PaaS. People in the industry get into shouting matches about this topic as if it mattered more than Lindsay Lohan’s journey through rehab.
The cold hard reality is that while pundits are busy writing XaaS white papers, developers are off just writing software. We are writing software that fits within cloud environments (weak SLA, small VMs), saves money (hosted data instead of data in VMs), and changes quickly (interpreted languages). We’re doing using an expanding tool kit of networked components like databases, object stores, shared cache, message queue, etc.
Using network components in an application architecture is about as novel as building houses made of bricks. So, what makes cloud architectures any better or different?
Nothing! There is no difference if you buy VMs, install services, and wire together your application in its own little cloud bubble. If I wanted to bait trolls, I’d call that an IaaS deployment.
However, there’s an emerging economic driver to leverage lower cost and more elastic infrastructure by using services provided by hosts rather than standing them up in a VM. These services replace dedicated infrastructure with managed network attached services and they have become a key differentiator for all the cloud vendors
Using these services allows developers to focus on the business problems we are solving instead of building out infrastructure to run our applications. We also save money because consuming an elastic managed network service is less expensive (and more consumption based) than standing up dedicated VMs to operate the services.
Ultimately, an application can be written as stateless code (really “externalized state” is more a accurate description) that relies on these services for persistence. If a host were to dynamically instantiate instances of that code based on incoming requests then my application resource requirements would become strictly consumption based. I would describe that as true cloud architecture.
On a bold day, I would even consider an environment that enforced offered that architecture to be a platform. Some may even dare to describe that as a PaaS; however, I think it’s a mistake to look to the service offering for the definition when it’s driven by the application designers’ decisions to use network services.
While we argue about PaaS vs IaaS, developers are just doing what they need. Today they may stand-up their own services and tomorrow they incorporate 3rd party managed services. The choice is not a binary switch, a layer cake, or a holy war.
The choice is about choosing the right most cost effective and scalable resource model.
or, why is everyone DaaZed and Confused
?
Dave McCrory, my co-worker at Dell, posted an interesting analysis of how different roles people have in IT jobs dramatically influences their perception of cloud services.
I think that part of the confusion is how difficult it is for each category of cloud user to see their challenges/issues for the other classes of user.
We see this in spades during internal PaaS discussions. People with development backgrounds has a fundamentally different concept of a PaaS benefits. In many cases, those same benefits (delegation to a provider for core services like database) are considered disadvantages for the other class of user (you want someone else to manage what!).
Ultimately, the applications are at the core of any XaaS conversation and define what “type” of cloud need to be consumed.
Lately there’s a flurry of interest (and hiring demand) for DevOps gurus. It’s obvious to me that there’s as much agreement between the ethical use of ground unicorn horn as there is about the job description of a DevOps tech.
I look at the world very simply:
Before I risk my supply of ethically obtained unicorn powder by defining DevOps, I want to explore why DevOps is suddenly hot. With cloud driving horizontal scale applications (see RAIN posts), there’s been a sea change in the type of expertise needed to manage an application.
Stereotypically, Ops teams get code over the transom from Dev teams. They have the job of turning the code into a smoothly running application. That requires rigid controls and safe guards. Traditionally, Ops could manage most of the scale and security aspects of an application with traditional scale-up, reliability, and network security practices. These practices naturally created some IT expense and policy rigidity; however, that’s what it takes to keep the lights on with 5 nines (or 5 nyets if you’re an IT customer).
Stereotypically, Dev teams live a carpe diem struggle to turn their latest code into deployed product with the least delay. They have the job of capturing mercurial customer value by changing applications rapidly. Traditionally, they have assumed that problems like scale, reliability, and security could be added after the fact or fixed as they are discovered. These practices naturally created a need to constantly evolve.
In the go-go cloud world, Dev teams are by-passing Ops by getting infrastructure directly from an IaaS provider. Meanwhile, IaaS does not provide Ops the tools, access, and controls that they have traditionally relied on for control and management. Consequently, Dev teams have found themselves having to stage, manage and deploy applications with little expertise in operations. Further, Ops teams have found themselves handed running cloud applications that they have to secure, scale and maintain applications without the tools they have historically relied on.
DevOps has emerged as the way to fill that gap. The DevOps hero is comfortable flying blind on an outsourced virtualized cloud, dealing with Ops issues to tighten controls and talking shop with Dev to make needed changes to architecture. It’s a very difficult job because of the scope of skills and the utter lack of proven best practices.
So what is a day in the life of a DevOp? Here’s my list:
This job is so difficult that I think the market cannot supply the needed experts. That deficit is becoming a forcing function where the cloud industry is being driven to adopt technologies and architectures that reduce the dependence for DevOps skills. Now, that’s the topic for a future post!
The Magic 8 Cube
This is the first part of 3 posts that look into the real future for “private clouds.”
This concept is something that was initially developed with Greg Althaus, my colleague at Dell and then further refined in discussions with by our broader team. It grew from my frustration with the widely referenced predictions by the Gartner Group of a private cloud explosion. Their prognostication did not ring true to me because the economics of “public cloud” are so compelling that going private seems to be like fighting your way out of a black hole.
We’ll get to the gravity well (post 3 of 3) in due time. For now, we need to look into the all knowing magic 8 cube.
Our breakthrough was seeing cloud hosting as a 3 dimensional problem. We realized that we could cover all the practical cloud scenarios with these 8 cases. Showing in the picture (right).
Here are the axis:
With 3 axis, we have a 8 point cube.
In thinking this over, we realized that cloud customers were not likely to jump randomly around this cube. If they were using MSH then they may want to consider MDH or MSO. It seemed unlikely that they would go directly from MSH to UDO as Mr. Bittman suggests; however, the market is clearly willing to move directly from UDO to MSH.
We had a good old-fashioned mystery on our hands… the answer will have to wait until my next post.
I’m growing more and more concerned about the preponderance of Frankencloud offerings that I see being foisted into the market place (no, my employer, Dell, is not guiltless). Frankenclouds are “cloud solutions” that are created by using duct tape, twine, wishful marketing brochures, and at least 4 marginally cloud enabled products.
The official Frankencloud recipe goes like this:
Combine the ingredients at high voltage in a climate conditioned environment. Stir in a seriously large amounts of consulting services, training, and Red Bull. At the end of this process, you will have your very own Frankencloud!
Frankenclouds are notoriously difficult to maintain because each part has its own version life cycle. More critically, they also lack a brain.
Unfortunately, there are few alternatives to the Frankencloud today. I think that the alternatives will rewrite the rules that Ops uses to create clouds. Here are the rules that I think help drive a wooden stake through the heart of the Frankencloud (yeah, I mixed monsters):
What do you think we can do to defeat Frankenclouds?
Dos Clouduis
This was created by Dave McCrory and was too awesome not to share. Of course, it’s ripped from Dos Equis. Thanks for the inspiration!
Rarely in my life have I seen true juxtaposition as in the last few weeks. Mearly hours after my long time friend and cloud conspirator, Dave McCrory, joined our team at Dell; the company that we founded, Surgient, was aquired by Quest software. Neither of us had been there for years and had been looking for ways to work together again. Apparently the cosmos required that we could not join forces while our first effort together was still standing.
Cloud Walker
Our cloud team at Dell is full of people who like to both dream and do. Now that we added Dave, I am expecting BIGGER things. We’re actively planning coordinated blogging about some of the issues and inspirations that are driving our plans. Those topics include Dev-Ops, PaaSvsIaaS, and the real “private” cloud.
Dave, welcome back to the party!
Here’s what Dave posted:
A lot has occurred since my last blog post. I am continuing the development of my technology and working in the Cloud, however I have chosen to do this with a great team at Dell. I was approached a while back about this opportunity and as I dug deeper and saw the potential I began to buy in. Finally after meeting the great team of experts involved behind the scenes I decided to join them.
I have worked with some of the team members before including Rob Hirschfeld. Rob and I founded both ProTier (note that PODS ran on VMware’s ESX) and co-founded Surgient together (interestingly Surgient announced its acquisition by Quest Software last week). Rob and I have created a great deal of IP (Intellectual Property) in the past together, including the First Patent around Cloud Computing (This was filed as a Provisional Patent in 2001 and a Full Patent in 2002). Our time at Dell should produce some new and great work in the Applied Architectures and Intellectual Property sides.
A while back (2004), Dave McCrory and 
I patented an interesting curosity that we called the Shared Nothing Virtual Cluster. Basically, the idea is to use OS RAID 1 on a VM but to have the VHDs split between physical hosts. If the host died, the VM could be restarted on a the second host using the RAID mirror.
It was an interesting idea, but seemed less than ideal because everyone was running to SAN storage and falling madly (insanely?) in love with vMotion.
Now that we’re looking towards clouds that beyond SAN scale, the idea of mixing DAS and NAS to create instant redundancy for VMs may suddenly have more value.
Of course, Sugient owns the patent now…
Or “Darth Vader vs Godzilla”
Way way back in January, I’d heard loud and clear that companies where not expecting to mix cloud computing loads. I was treated like a three-eyed Japanese tree slug for suggesting that we could mixing HPC and Analytics loads with business applications in the same clouds. The consensus was that companies would stand up independent clouds for each workload. The analysis work was too important to interrupt and the business applications too critical to risk.
It has always rankled me that all those unused compute cycles (“the dark cycles”) could be put to good use. It’s appeals to my eco-geek side to make best possible use of all those idle servers. Dave McCrory and I even wrote some cloud patents around this.
However, I succumbed to the scorn and accepted the separation.
Now all of a sudden, this idea seems to be playing Godzilla to a Tokyo shaped cloud data center. I see several forces merging together to resurrect mixing workloads.
This trend will only continue as Moore’s Law improves the compute density for hardware. Since our designs are leading towards scale out designs that distribute applications over multiple nodes; it is not practical to expect an application to consume all the power of a single computer.
That leaves a lot of lonely dark cycles looking for work.
Now all of a sudden, this idea seems to be playing Godzilla to a Tokyo shaped cloud data center. I see several forces merging together to resurrect mixing workloads.
This trend will only continue as Moore’s Law improves the compute density for hardware. Since our designs are leading towards scale out designs that distribute applications over multiple nodes; it is not practical to expect an application to consume all the power of a single computer.
That leaves a lot of lonely dark cycles looking for work.
Rob Hirschfeld 