Category Archives: Dave McCrory

Don’t complicate my cloud! It’s just infrastructure with an API

Posted on by
19

Getty MazeI’ve been “in cloud” for over 13 years (@dmcrory and I submitted patents using it starting in 2001) and I’m continually amazed at how complicated people want to make it.

For my role at Dell, I’m continually invited to seasons of meetings to define cloud, cloud architecture and cloud strategy. The reason these meetings go on and on is that everyone wants to make cloud complicated when it’s really very simple.

Cloud is infrastructure with an API.

That’s it. Everything else is just a consequence of having infrastructure with an API because API provides the ability to provide remote control.

What else do people try to lump into cloud?  Here are some of my topic cloud obfuscators:

  • (inter)network.  Yes, networks make an API interesting.  They are just an essential component but they are not cloud.  Most technologies are interesting because of networks: can we stop turning everything networked into cloud?  Thanks to nonsensical mega-dollar marketing campaigns, I despair this is a moot point.
  • as-a-service.  That’s another way of saying “accessible via an API.”  We have many flavors of Platform, Data, Application, Love or whatever as a Service.  That means they have a API.  Infrastructure as a Service (IaaS) is a cloud.
  • virtualization.  VMs were the first good example of hardware with an API; however, we had virtual containers (on Mainframes!) long before we had “cloud.”    They make cloud easier but they are not cloud.

  • pay-as-you-go (service pricing).  This is a common cloud requirement but it’s really a business model.  If someone builds a private cloud then it is still a cloud.

  • multi-tenant.  Another common requirement where we expect a cloud to be able to isolate users.  I agree that this is a highly desirable attribute of a good API implementation; however, it’s not essential to a cloud.  For example, most public clouds do not have true network isolation model.
  • elastic demand.  IMHO, another word for API driven provisioning.
  • live migration.  This is a cool feature often implemented on top of virtualization, but it’s not cloud.  We were doing live migrate with shared storage and clusters before anyone heard of cloud.   I don’t think this is cloud at all but someone out there does so I included it in the list.
  • security.  Totally important consideration and required for deployments large and small, but not presence/lack does not make something cloud.

We start talking about these points and then forget the whole API thing.  These items are important, but they do not make it “a cloud.”  When Dave McCrory and I first discussed API Infrastructure as “cloud,” it was driven by the fact that you could hide the actual infrastructure behind the API.  The critical concept was that the API allowed a you to manage a server anywhere from anywhere.

When Amazon offered the first EC2 service, it had to be a cloud because the servers were remote. It was not a cloud because it was on the internet; plenty of other companies were offering hosted servers. It was a cloud because their offering allowed required operators to use and API to interact with the infrastructure.  I remember that EC2’s lack of UI (and SLA) causing many to predict it would be a failure; instead, it sparked a revolution.

I’m excited now because we’re entering a new generation of cloud where Infrastructure APIs include networking and storage in addition to compute.  Mix in some of the interesting data and network services and we’re going to have truly dynamic and powerful clouds.  More importantly, we going to have some truly amazing applications.

What do you think?  Is API a sufficient definition of cloud in your opinion?

PS: Yes, if you have a physical server/network/store that is completely controllable by an API then you’ve got a cloud on your hands.

Seven Cloud Success Criteria to consider before you pick a platform

Posted on by
1

From my desk at Dell, I have a unique perspective.   In addition to a constant stream of deep customer interactions about our many cloud solutions (even going back pre-OpenStack to Joyent & Eucalyptus), I have been an active advocate for OpenStack, involved in many discussions with and about CloudStack and regularly talk shop with Dell’s VIS Creator (our enterprise focused virtualization products) teams.  And, if you go back ten years to 2002, patented the concept of hybrid clouds with Dave McCrory.

Rather than offering opinions in the Cloud v. Cloud fray, I’m suggesting that cloud success means taking a system view.

Platform choice is only part of the decision: operational readiness, application types and organization culture are critical foundations before platform.

Over the last two years at Dell, I found seven points outweigh customers’ choice of platform.

  1. Running clouds requires building operational expertise both at the application and infrastructure layers.  CloudOps is real.
  2. Application architectures matter for cloud deployment because they can redefine the SLA requirements and API expectations
  3. Development community and collaboration is a significant value because sharing around open operations offers significant returns.
  4. We need to build an accelerating pace of innovation into our core operating principles
  5. There are still significant technology gaps to fill (networking & storage) and we will discover new gaps as we go
  6. We can no longer discuss public and private clouds as distinct concepts.   True hybrid clouds are not here yet, but everyone can already see their massive shadow.
  7. There is always more than one right technological answer.  Avoid analysis paralysis by making incrementally correct decisions (committing, moving forward, learning and then re-evaluating).

Austin OpenStack Meetup (January Minutes) + OpenStack Foundation Web Cast!

Posted on by
2

Sorry for the brevity… At the last Austin OpenStack meetup, we had >60 stackers!  Some from as far away as Portland and Boston (as in Oregon and Massachusetts).

Notes:

  • Suse introduced their OpenStack beta and talked about their Suse Studio that can deploy images against the OpenStack APIs
  • I showed off DevStack.org code that can setup the truck of OpenStack (now Essex) in about 10 minutes on a single node.  Great for developers!
  • I showed an OpenStack Diablo Final deployment from Crowbar.  I focused mainly on Dashboard and used our reference architecture (see below) as illustration of the many parts.
  • Matt Ray suggested everyone watch the webcasts about the OpenStack Foundation (Thurs 6pm central  & Friday 9am central)
  • We planned the next few meetups.
    • For February, we’ll talk about Swift and Dashboard.
    • For March, we’ll talk about Essex and DevStack to prep for the next design summit (in SF).
    • For April, we’ll debrief the conference

Thank you Suse and Dell (my employer) for sponsoring!   The next meetup is sponsored by Canonical.

The 451 Group Cloudscape report strikes chord misses harmony (DevOps, Hybrid Cloud, Orchestration)

Posted on by
3

It’s impossible to resist posting about this month’s  451 Group Cloudscape report when it calls me out by name as a leading cloud innovator:

… ProTier founders Dave McCrory and Rob Hirschfeld. ProTier [note: now part of Quest] was, indeed, the first VMware ecosystem vendor to be tracked by The 451 Group. In the face of a skeptical world, these entrepreneurs argued that virtualization needed automation in order to realize its full potential, and that the test lab was the low-hanging fruit. Subsequent events have more than vindicated their view (pg. 33).

It’s even better when the report is worth reading and offers insights into forces shaping the industry.  It’s nice to be “more than vindicated” on an amazing journey we started over 10 years ago!

Rather than recite 451’s points (hybrid cloud = automation + orchestration + devops + pixie dust), I’d rather look at the problem different way as a counterpoint.

The problem is “how do we deal with applications that are scattered over multiple data centers?”

I do not think orchestration is the complete answer.  Current orchestration is too focused on moving around virtual machines (aka workloads).

Ultimately, the solution lies in application architecture; however, I feel that is also a misdirection because cloud is redefining what an “application architecture” means.

Applications are a dynamic mix of compute, storage, and connectivity.

We’re entering an age when all of these ingredients will be delivered as elastic services that will be managed by the applications themselves.  The concept of self management is an extension of DevOps principles that fuse application function and deployment. There are missing pieces, but I’m seeing the innovation moving to fill those gaps.

If you want to see the future of cloud applications then look at the network and storage services that are emerging.  They will tell you more about the future than orchestration.

 

Please support me for the OpenStack Policy Board

Posted on by
4

I’m posting my OpenStack bio here and asking for support putting me on the Policy Board by voting for me.  NOTE: You can only vote if you’re registered and you got the “Poll: OpenStack Governance Elections” email.

Project Policy Board Objective

I am seeking a role on the OpenStack Policy Board to further the adoption of OpenStack within and beyond the community.  As the OpenStack technology lead within Dell, I am the engineer who is most actively engaged with field deployments; consequently, I am uniquely positioned to represent our development community, hosters and enterprise user bases.  I bring substantial process experience (Agile/Lean/CI) into my decision making.  My focus will be on ensuring OpenStack is deployable and ready for use.

Background

I am a Principal Engineer at Dell working as the lead for our OpenStack cloud initiative (http://dell.com/openstack).  My team at Dell is responsible for bringing hyper-scale cloud solutions to market and works closely with our cloud optimized hardware division (DCS).  Before working on the OpenStack project, I was involved in cloud projects for Azure, Eucalyptus, and Joyent at Dell.
My involvement with OpenStack goes back to the very earliest days before the project was launched where I was part of the evaluation team that advocated for Dell to join the project.  Since then, I have been active participant at every design conference.  It was my recommendation that Dell focus on making deployment capabilities for OpenStack and to ensure that those contributions are open sourced (Apache 2).  At this point in the project, I am Dell’s technical authority on OpenStack for community and customer interactions.
My team is responsible for the Crowbar cloud deployer (http://github.com/dellcloudedge/crowbar).  The purpose of this project is to ensure that OpenStack is be quickly and reliably deployed in a wide range of configurations on any hardware platform.  I believe that ease of deployability is essential for the success of OpenStack as a project because it ensure adoption by non-developers.  I also believe strongly in continuous integration and am working to adapt Crowbar as a CI platform.  I have been the primary driver to ensure that the Crowbar project is open sourced and accepting of input from the community.
My team also designs technical reference architectures (RAs) for OpenStack.  These RAs help drive adoption by providing crisp guidance on how deploy OpenStack.  I am a vocal proponent of open operations (keeping best practices public) and following a DevOps approach for ongoing cloud deployment life-cycles.
In addition to my work at Dell, I work to ensure community access and communication.  My independent blog provides technical detail and insights about the OpenStack and other cloud initiatives.  My blog also focuses on Agile and Lean practices that I believe are essential to success in technology innovation.
I have been working with cloud computing since 2001.  The company I founded with Dave McCrory (@mccrory), now owned by Quest, was the first multi-server VMware ESX deployment ouside of VMware.  We pioneered the concept of elastic vm management (look up the patents!) so I have a very deep understanding of the problems and architectures required.

Cloudcast interview with Dell Cloud Solutions Team (quotes with time stamps)

Posted on by
Reply

Lloyds BarbershopTheCloudcast.net – Thank you for such a great series of questions. Wow, nearly 36+ minutes of cloudicious interview about the work my team at Dell is doing!

Thanks to our hosts for putting together a great series! They are:

Highlights from Episode 16: Dell, Dude you’re getting a cloud

  • 3:40 JBG “we are listening to our customers tell us what they want to accomplish”
  • 4:40 RAH “humility is part of [what we’re] doing … cloud is about learning and collaboration”
  • 6:40 RAH “OpenStack filled a niche. It was the first open source community cloud. … Not just open source, its open community.”
  • 7:15 RAH “We’re beyond critical mass. We’re seeing acceleration… we are transitioning into a community development.”
  • 7:30 RAH “It’s accelerating. It happening so fast.”
  • 8:00 RAH “We felt it was really important for people to be able to use it. We felt that it was important to get away from just people developing into people using. “
  • 8:57 – RAH “Cloud is not just one thing. You have to have all the pieces.”
  • 10: 15 – RAH “Cloud is always ready, never finished”
  • 10:50 – RAH “OpenStack is an alternative to public cloud including hosting providers seeking to offer their own cloud”
  • 12:40 – AD “Dell has been in the Big data space for many years now”
  • 20:15 – JBG “There’s a legacy of great partnerships that we leverage”
  • 20:48 – JBG “Conflicts have not come up because we are focused on the customer”
  • 21:30 – RAH “Shout out to Greg Althaus for solving these problems in such an elegant way. And we rewrote it 3 times”
  • 22:02 – RAH “Crowbar started from our frustration of bringing up a cloud quickly … so we took a DevOps approach.”
  • 22:41 – RAH “You had to have a system view AND a boot strapping view simultaneously”
  • 23:50 – JBG “Crowbar was born out of necessity because we were setting up and blowing away our clouds over and over and over again. “
  • 24:40 – JBG “We realized there were not many people thinking about all the pieces before OpenStack was installed”
  • 25:20 – RAH “We don’t think customers have all the answers before we show up. This is not unique to OpenStack.”
  • 28:20 – JBG “We’re seeing the community pick up Crowbar as a way to deploy”

Collaboration between Dell Crowbar & VMware Cloud Foundry – unleashes your inner cloud

Posted on by
12

Sometimes a single sprint can deliver magic: when I signed up to document how to create a Crowbar module (aka a barclamp) two weeks ago, I had no idea that it would add a new flavor to Crowbar .

I’m proud to announce that the first public non-Dell Crowbar module will be supporting the VMware Cloud Foundry Open PaaS project.

Development is still in progress (on the Crowbar “CF” branch) and you’ll be able to watch us (even help!) collaborate on this project.  Initially, the deployment will be to single server but we’re hoping to quickly expand to a distributed install that fully leverages the capabilities of both projects.

By creating a Crowbar module, Cloud Foundry™ is able to leverage the cloud deployment capabilities that allow it to be setup on any physical or virtualized data center.  This is core to the Crowbar message: the value of a cloud solution can best be realized when it’s coupled with open practices for deploying it.

There are many significant aspects of this collaboration:

  1. Cloud Foundry is taking the right approach to PaaS.  Their team’s perspective on PaaS mirrors my own: A PaaS is a collection of application services.  That approach makes it extensible and flexible.  Plus, they are also multi-language and multi-platform.
  2. Crowbar is proving our breadth of support.  Last week we announced coming RHEL support and now adding Cloud Foundry is a natural extension.  We did not design Crowbar to be a one-trick pony.  It’s modular design makes it easy to extend while leveraging the existing body of work.
  3. Big companies are acting like start-ups.  Both Crowbar and Cloud Foundry are projects that focus on putting the core functionality out quickly to prove their value proposition, get feedback, and change the game.  This collaboration is positive proof of these companies being Agile and starting a project Lean.
  4. Big companies are acting in the open.  Both Dell via Crowbar and VMware via Cloud Foundry are contributing their source and working on it in the open.

Stay tuned for that “how to create a barclamp” post (or check out the barclamp rake task).

For more information:

Why cloud compute will be free

Posted on by
2

Today at Dell, I was presenting to our storage teams about cloud storage (aka the “storage banana”) and Dave “Data Gravity” McCrory reminded me that I had not yet posted my epiphany explaining “why cloud compute will be free.”  This realization derives from other topics that he and I have blogged but not stated so simply.

Overlooking that fact that compute is already free at Google and Amazon, you must understand that it’s a cloud eat cloud world out there where losing a customer places your cloud in jeopardy.  Speaking of Jeopardy…

Answer: Something sought by cloud hosts to make profits (and further the agenda of our AI overlords).

Question: What is lock-in?

Hopefully, it’s already obvious to you that clouds are all about data.  Cloud data takes three primary forms:

  1. Data in transformation (compute)
  2. Data in motion (network)
  3. Data at rest (storage)

These three forms combine to create cloud architecture applications (service oriented, externalized state).

The challenge is to find a compelling charge model that both:

  1. Makes it hard to leave your cloud AND
  2. Encourages customers to use your resources effectively (see #1 in Azure Top 20 post)

While compute demands are relatively elastic, storage demand is very consistent, predictable and constantly grows.  Data is easily measured and difficult to move.  In this way, data represents the perfect anchor for cloud customers (model rule #1).  A host with a growing data consumption foot print will have a long-term predictable revenue base.

However, storage consumption along does not encourage model rule #2.  Since storage is the foundation for the cloud, hosts can fairly judge resource use by measuring data egress, ingress and sidegress (attrib @mccrory 2/20/11).  This means tracking not only data in and out of the cloud, but also data transacted between the providers own cloud services.  For example, Azure changes for both data at rest ($0.15/GB/mo) and data in motion ($0.01/10K).

Consequently, the financially healthiest providers are the ones with most customer data.

If hosting success is all about building a larger, persistent storage footprint then service providers will give away services that drive data at rest and/or in motion.  Giving away compute means eliminating the barrier for customers to set up web sites, develop applications, and build their business.  As these accounts grow, they will deposit data in the cloud’s data bank and ultimately deposit dollars in their piggy bank.

However, there is a no-free-lunch caveat:  free compute will not have a meaningful service level agreement (SLA).  The host will continue to charge for customers who need their applications to operate consistently.  I expect that we’ll see free compute (or “spare compute” from the cloud providers perspective) highly used for early life-cycle (development, test, proof-of-concept) and background analytic applications.

The market is starting to wake up to the idea that cloud is not about IaaS – it’s about who has the data and the networks.

Oh, dem golden spindles!  Oh, dem golden spindles!

Cloud Gravity – launching apps into the clouds

Posted on by
Reply

Dave McCrory‘s Cloud Gravity series (Data Gravity & Escape Velocity) brings up some really interesting concepts and has lead to some spirited airplane discussions while Dell shuttled us to an end of year strategy meeting.  Note: whoever was on American 34 seats 22A/C – we apologize if we were too geek-rowdy for you.

Dave’s Cloud Gravity is the latest unfolding of how clouds are evolving as application architectures before more platform capable.  I’ve explored these concepts in previous posts (Storage Banana, PaaS vs IaaS, CAP Chasm) to show how cloud applications are using services differently than traditional applications.

Dave’s Escape Velocity post got me thinking about how cleanly Data Gravity fits with cloud architecture change and CAP theorem.

My first sketch shows how traditional applications are tightly coupled with the data they manipulate.  For example, most apps work directly on files or a database direct connection.  These apps rely on very consistent and available data access.  They are effectively in direct contact with their data much like a building resting on it’s foundation.  That works great until your building is too small (or too large).  In that case, you’re looking a substantial time delay before you can expand your capcity.

Cloud applications have broken into orbit around their data.  They still have close proximity to the data but they do their work via more generic network connections.  These connections add some latency, but allow much more flexible and dynamic applications.  Working within the orbit analogy, it’s much much easier realign assets in orbit (cloud servers) to help do work than to move buildings around on the surface.

In the cloud application orbital analogy, components of applications may be located in close proximity if they need fast access to the data.  Other components may be located farther away depending on resource availability, price or security.  The larger (or more valuable) the data, the more likely it will pull applications into tight orbits.

My second sketch extends to analogy to show that our cloud universe is not simply point apps and data sources.  There truly a universe of data on the internet with hugh sources (Facebook, Twitter, New York Stock Exchange, my blog, etc) creating gravitational pull that brings other data into orbit around them.  Once again, applications can work effectively on data at stellar distances but benefit from proximity (“location does not matter, but proximity does”).

Looking at data gravity in this light leads me to expect a data race where clouds (PaaS and SaaS) seek to capture as much data as possible.

Jevon’s Paradox

Posted on by
2

I’ve been finding it necessary to quote Jevon’s paradox several times lately and realized that I have NOT referenced it here.  Quite simply, understanding Jevon’s paradox is essential to understanding cloud.

The concept of the paradox is that when we make something more efficient (for example gas in cars), the demand for that resource goes up (we move further into the exurbs because driving is cheaper).  Notably, as Moore’s law drives computer efficiency up, we are using more and more computers.  Specifically, I have more computers in my house every year even though the efficiency of just one smart phone far (far) exceeds power of my son’s Sinclair 1000.

In cloud computing, Jevon’s paradox points us to the expectation that the rush of applications and activity in the cloud will continue to accelerate.  Since I expect competition and Moore’s law to drive increasing gains in cloud efficiency (and therefore customer advantageous price signals) the market will happily convert these utilization improvements into more and more interesting capabilities.

The cloud expansion means that we can sustain more providers entering the market.  In fact, Jevon would tell us that more providers will likely INCREASE demand for cloud as competition and capacity put downward pressure on prices.  [Q: where will they make up the margin?   A: Adjacent Services]

The loser in cloud’s exploration of Jevon’s paradox are non-cloud deployments (Dell strategists are you listening?).  These systems suffer because their ability to improve their efficiency is limited.

As I look down the road on cloud, I can see many opportunities for current applications to take advantage of cheaper cloud resources to provide even more value.  For example, adding map-reduce analytics to scan a customer’s data can provide tremendous insights.  Today, it’s a luxury like flying on the Concorde.  Tomorrow, it will be part like hopping the Nerd Bird from San Jose to Austin – just a normal part of our daily lives.

 

Note: A shout out to Dave McCrory who introduced me to Jevon’s Paradox.