Bootstrapping Hyperscale OpenStack Clouds – slides from 2/3 OpenStack SJC Meetup

The OpenStack meeting lightening talk is only 5 minutes, so the deck is mostly pictures that support points around a more detailed followup.

Here’s the deck: bootstrapping clouds preso

 and my Hyperscale white paper (links through Dell.com)

The theme of the talk is that hyperscale systems requires a fundamentally different management paradigm because at hyperscale

hardware faults are common,manual steps are impractical and small costs add up quickly.

Included in the preso are concepts I introduced at Flatness at the Edge.

2/10 Update: Now you can watch it Thanks to “@opnstk_com_mgr Stephen Spector lighting talks video of Rob Hirschfeld, Dell at Santa Clara, CA Meetup Feb 3, 2011 http://ow.ly/3U8OA

Cloud Culture Clash Creates Opportunities

In my opinion, one of the biggest challenges facing companies like Dell, my employer, is how to help package and deliver this thing called cloud into the market.  I recently had the opportunity to watch and listen to customers try to digest the concept of PaaS.

While not surprising, the technology professionals in the room split into across four major cultural camps: enterprise vs. start-up and dev vs. ops.  Because I have a passing infatuation with pastel cloud shaped quadrant graphs, I was able to analyze the camps for some interesting insights.

The camps are:

  1. Imperialists:  These enterprise type developers are responsible for adapting their existing business to meet the market.  They prefer process oriented tools like Microsoft .Net and Java that have proven scale and supportability.
  2. MacGyvers: These startup type developers are under the gun to create marketable solutions before their cash runs out.  They prefer tools that adapt quick, minimize development time and community extensions.
  3. Crown Jewels: These enterprise type IT workers have to keep the email and critical systems humming.  When they screw up everyone notices.  They prefer systems where they can maintain control, visibility, or (better) both.
  4. Legos: These start-up type operations jugglers are required to be nimble and responsive with shoestring budgets.   They prefer systems that they can change and adapt quickly.  They welcome automation as long as they can maintain control, visibility, or (better) both.

This graph is deceiving because it underplays the psychological break caused by willingness to take risks.  This break creates a cloud culture chasm. 

On one side, the reliable Imperialists want will mount a Royal Navy flotilla to protect the Crown Jewels in a massive show of strength.  They are concerned about the security and reliability of cloud technologies.

On the other side, the MacGyvers are working against a ticking time bomb to build a stealth helicopter from Legos they recovered from Happy Meals™.  They are concerned about getting out of their current jam to compile another day.

Normally Imperialists simply ignore the MacGyvers or run down the slow ones like yesterday’s flotsam.  The cloud is changing that dynamic because it’s proving to be a dramatic force multiplier in several ways:

  1. Lower cost of entry – the latest cloud options (e.g. GAE) do not charge anything unless you generate traffic.  The only barrier to entry is an idea and time.
  2. Rapid scale – companies can fund growth incrementally based on success while also being able to grow dramatically with minimal advanced planning.
  3. Faster pace of innovation – new platforms, architectures and community development has accelerated development.  Shared infrastructure means less work on back office and more time on revenue focused innovation.
  4. Easier access to customers – social media and piggy backing on huge SaaS companies like Facebook, Google or SalesForce bring customers to new companies’ front doors.  This means less work on marketing and sales and more time on revenue focused innovation.

The bottom line is that the cloud is allowing the MacGyvers to be faster, stronger, and more innovative than ever before.  And we can expect them to be spending even less time polishing the brass in the back office because current SaaS companies are working hard to help make them faster and more innovative.

For example, Facebook is highly incented for 3rd party applications to be innovative and popular not only because they get a part of the take, but because it increases the market strength of their own SaaS application.

So the opportunity for Imperialists is to find a way for employee and empower the MacGyvers.  This is not just a matter of buying a box of Legos: the strategy requires tolerating enabling embracing a culture of revenue focused innovation that eliminates process drag.  My vision does not suggest a full replacement because the Imperialists are process specialists.  The goal is to incubate and encapsulate cloud technologies and cultures.

So our challenge is more than picking up cloud technologies, it’s understanding the cloud communities and cultures that we are enabling.

Clouds & Water (Blog Action Day)

Today Change.org is coordinating Blog Action Day 2010 to raise awareness about Water.  It is widely reported (and worth repeating) that scarcity of clean water is more likely to impact your daily life than scarcity of energy, food, shelter or other basic human rights.

Water scarcity has little impact in my daily life.  <shameless plug>While The new cloud servers my employer, Dell, sells consume less power and thereby less cooling water; these efficiencies do relatively little to impact people’s access to fresh water.</shameless plug>

However, waste is a huge impact.  Since Americans are water, food and energy hogs, we are also in the position of wasting disproportionate amount of these limited resources.  I believe that we commit this waste unconsciously without any real gauge on its volume or impact.  Imagine the impact to your driving behavior if you had to fill your gas tank up a cup of gas at a time (64), water your lawn from a 5 gallon bucket (30+) or refill your toilet with a table-spoon (409!).

The key to addressing waste in the land of plenty is to measure and show impacts.  I believe that people abhor waste when they see it.  Our challenge is not to change people, but to show them in real terms the consequences of their choices.

For example, just having an MPG calculator on our cars has changed the way that we drive them.  I am personally disappointed with how little useful feedback these gauges provide, but it’s a start.

One of the things I like about Cloud Computing is that we want to measure and reduce waste.  We get mad about waste: wasted computer time, wasted equipment, wasted power, and especially wasted time.

As we make strides to make computing and information more personal and mobile, I believe we need to include ways to show people data about the choices that they are making.  So next time you water your lawn or flush your toilet, this about what it would mean if you had the haul that water in a bucket up from a well.  Sound crazy?  That’s status quo for more people than those of us that enjoy indoor plumbing.

Microsoft & Dell partner for Azure hosting

Vacation rental in Fort Walton Beach, FLA

Interesting juxtaposition, last week I was vacationing at the Azure condos in Florida and this week Dell is making a strategic announcement to develop a Dell-powered Microsoft Windows Azure Platform Appliance, as well as Azure based services delivered by Dell Services.

Press Release: http://content.dell.com/us/en/corp/d/press-releases/2010-07-12-dell-microsoft-cloud-azure-appliance.aspx

This is a project that I’m involved in, so watch for updates as details emerge.

Just Striping in the RAIN

Or “behold the power of the unreliable”

In a previous post, I discussed the concept of a Redundant Array of Inexpensive Nodes (RAIN) as a way to create more reliable and scalable applications.   Deploying a RAIN application is like being the House in Vegas – it’s about having enough size that the odds come out in your favor.  Even if one player is on a roll, you can predict that nearly everyone else is paying your rent.  Imagine what would happen if all the winning gamblers were in your casino!  If you don’t want to go bankrupt when deploying a RAIN app, then ensure that the players spread out all over the Strip.

One of my core assumptions is that you’ll deploy a RAIN application on a cloud.   This is a significant because we’re assuming that your nodes are

  1. idle most of the time because your traffic loads are cyclic
  2. unreliable because the cloud provider does not offer much SLA
  3. divisible because renting ten 1/10ths of a server costs roughly the same as a whole one
  4. burstable because 1/10th servers can sometimes consume that extra 9/10th server

The burstable concept is a dramatic power multiplier that tips the whole RAIN equation heavily towards clouds. 

Bursting means that under load, your 10 1/10th servers (roughly 1 server in cost) could instantly expand to the power of 10 full servers!  That reflects an order of magnitude spike in demand without any change in your application.

In the past, we’ve racked extra servers to handle this demand.  That meant that we had a lot of extra capacity taking up rack space, clubbing innocent migratory electrons for their soft velvety fur, and committing over provisioning atrocities.  

Today, multi-tenant clouds allow us to average out these bursts by playing the odds on when application bursts will occur.  In this scenario, the infrastructure provider benefits from the fact that applications need to be over provisioned.  The application author benefits because they can instantly tap more resources on demand.  Now, that is what I would call a win-win synergy!

All this goodness depends on

  • standard patterns & practices that developers use to scale out RAIN applications
  • platform improvements in cloud infrastructure to enable smooth scale out
  • commercial models that fairly charge for multi-tenant over-subscription
  • workable security that understands the underlying need for co-mingling workloads

The growing dominance of cloud deployments looks different when you understand the multiplying interplay between multi-tenant clouds and cloud-ready RAIN applications.

IEEE “Pragmatic Clouds” Presentation (2/24/10)

Tonight I presentated “Pragmatic Clouds” to the IEEE Central Texas Consultants Network Meeting.

The abstract was, “Cloud computing seems to mean everything! We’ll talk in specifics about how application development and delivery models are changing based on new “cloud” commercial models. We’ll also explore how the plumbing behind the curtains is changing to reflect these new models.”

In the presentation, I covered drivers for cloud business models and how it impacts creating applications for clouds. I also described how to write a future-proof application that can work for IaaS clouds today and PaaS clouds tomorrow.