Tag Archives: Clouds

We need DevOps without Borders! Is that “Hybrid DevOps?”

Posted on by
3

The RackN team has been working on making DevOps more portable for over five years.  Portable between vendors, sites, tools and operating systems means that our automation needs be to hybrid in multiple dimensions by design.

Why drive for hybrid?  It’s about giving users control.

launch!I believe that application should drive the infrastructure, not the reverse.  I’ve heard may times that the “infrastructure should be invisible to the user.”  Unfortunately, lack of abstraction and composibility make it difficult to code across platforms.  I like the term “fidelity gap” to describe the cost of these differences.

What keeps DevOps from going hybrid?  Shortcuts related to platform entangled configuration management.

Everyone wants to get stuff done quickly; however, we make the same hard-coded ops choices over and over again.  Big bang configuration automation that embeds sequence assumptions into the script is not just technical debt, it’s fragile and difficult to upgrade or maintain.  The problem is not configuration management (that’s a critical component!), it’s the lack of system level tooling that forces us to overload the configuration tools.

What is system level tooling?  It’s integrating automation that expands beyond configuration into managing sequence (aka orchestration), service orientation, script modularity (aka composibility) and multi-platform abstraction (aka hybrid).

My ops automation experience says that these four factors must be solved together because they are interconnected.

What would a platform that embraced all these ideas look like?  Here is what we’ve been working towards with Digital Rebar at RackN:

Mono-Infrastructure IT “Hybrid DevOps”
Locked into a single platform Portable between sites and infrastructures with layered ops abstractions.
Limited interop between tools Adaptive to mix and match best-for-job tools.  Use the right scripting for the job at hand and never force migrate working automation.
Ad hoc security based on site specifics Secure using repeatable automated processes.  We fail at security when things get too complex change and adapt.
Difficult to reuse ops tools Composable Modules enable Ops Pipelines.  We have to be able to interchange parts of our deployments for collaboration and upgrades.
Fragile Configuration Management Service Oriented simplifies API integration.  The number of APIs and services is increasing.  Configuration management is not sufficient.
 Big bang: configure then deploy scripting Orchestrated action is critical because sequence matters.  Building a cluster requires sequential (often iterative) operations between nodes in the system.  We cannot build robust deployments without ongoing control over order of operations.

Should we call this “Hybrid Devops?”  That sounds so buzz-wordy!

I’ve come to believe that Hybrid DevOps is the right name.  More technical descriptions like “composable ops” or “service oriented devops” or “cross-platform orchestration” just don’t capture the real value.  All these names fail to capture the portability and multi-system flavor that drives the need for user control of hybrid in multiple dimensions.

Simply put, we need devops without borders!

What do you think?  Do you have a better term?

Are VMs becoming El Caminos? Containers & Metal provide new choices for DevOps

Posted on by
9

I released “VMS ARE DEAD” this post two weeks ago on DevOps.com.  My point here is that Ops Automation (aka DevOps) is FINALLY growing beyond Cloud APIs and VMs.  This creates a much richer ecosystem of deployment targets instead of having to shoehorn every workload into the same platform.

In 2010, it looked as if visualization had won. We expected all servers to virtualize workloads and the primary question was which cloud infrastructure manager would dominate. Now in 2015, the picture is not as clear. I’m seeing a trend that threatens the “virtualize all things” battle cry.

IMG_20150301_170558985Really, it’s two intersecting trends: metal is getting cheaper and easier while container orchestration is advancing on rockets. If metal can truck around the heavy stable workloads while containers zip around like sports cars, that leaves VMs as a strange hybrid in the middle.

What’s the middle? It’s the El Camino, that notorious discontinued half car, half pick-up truck.

The explosion of interest in containerized workloads (I know, they’ve been around for a long time but Docker made them sexy somehow) has been creating secondary wave of container orchestration. Five years ago, I called that Platform as a Service (PaaS) but this new generation looks more like a CI/CD pipeline plus DevOps platform than our original PaaS concepts. These emerging pipelines obfuscate the operational environment differently than virtualized infrastructure (let’s call it IaaS). The platforms do not care about servers or application tiers, their semantic is about connecting services together. It’s a different deployment paradigm that’s more about SOA than resource reservation.

On the other side, we’ve been working hard to make physical ops more automated using the same DevOps tool chains. To complicate matters, the physics of silicon has meant that we’ve gone from scale up to scale out. Modern applications are so massive that they are going to exceed any single system so economics drives us to lots and lots of small, inexpensive servers. If you factor in the operational complexity and cost of hypervisors/clouds, an small actual dedicated server is a cost-effective substitute for a comparable virtual machine.

I’ll repeat that: a small dedicated server is a cost-effective substitute for a comparable virtual machine.

I am not speaking against virtualize servers or clouds. They have a critical role in data center operations; however, I hear from operators who are rethinking the idea that all servers will be virtualized and moving towards a more heterogeneous view of their data center. Once where they have a fleet of trucks, sports cars and El Caminos.

Of course, I’d be disingenuous if I neglected to point out that trucks are used to transport cars too. At some point, everything is metal.

Want more metal friendly reading?  See Packet CEO Zac Smith’s thinking on this topic.

Bootstrapping Hyperscale OpenStack Clouds – slides from 2/3 OpenStack SJC Meetup

Posted on by
1

The OpenStack meeting lightening talk is only 5 minutes, so the deck is mostly pictures that support points around a more detailed followup.

Here’s the deck: bootstrapping clouds preso

 and my Hyperscale white paper (links through Dell.com)

The theme of the talk is that hyperscale systems requires a fundamentally different management paradigm because at hyperscale

hardware faults are common,manual steps are impractical and small costs add up quickly.

Included in the preso are concepts I introduced at Flatness at the Edge.

2/10 Update: Now you can watch it!  Thanks to “@opnstk_com_mgr Stephen Spector lighting talks video of Rob Hirschfeld, Dell at Santa Clara, CA Meetup Feb 3, 2011 http://ow.ly/3U8OA“

Cloud Culture Clash Creates Opportunities

Posted on by
1

In my opinion, one of the biggest challenges facing companies like Dell, my employer, is how to help package and deliver this thing called cloud into the market.  I recently had the opportunity to watch and listen to customers try to digest the concept of PaaS.

While not surprising, the technology professionals in the room split into across four major cultural camps: enterprise vs. start-up and dev vs. ops.  Because I have a passing infatuation with pastel cloud shaped quadrant graphs, I was able to analyze the camps for some interesting insights.

The camps are:

  1. Imperialists:  These enterprise type developers are responsible for adapting their existing business to meet the market.  They prefer process oriented tools like Microsoft .Net and Java that have proven scale and supportability.
  2. MacGyvers: These startup type developers are under the gun to create marketable solutions before their cash runs out.  They prefer tools that adapt quick, minimize development time and community extensions.
  3. Crown Jewels: These enterprise type IT workers have to keep the email and critical systems humming.  When they screw up everyone notices.  They prefer systems where they can maintain control, visibility, or (better) both.
  4. Legos: These start-up type operations jugglers are required to be nimble and responsive with shoestring budgets.   They prefer systems that they can change and adapt quickly.  They welcome automation as long as they can maintain control, visibility, or (better) both.

This graph is deceiving because it underplays the psychological break caused by willingness to take risks.  This break creates a cloud culture chasm. 

On one side, the reliable Imperialists want will mount a Royal Navy flotilla to protect the Crown Jewels in a massive show of strength.  They are concerned about the security and reliability of cloud technologies.

On the other side, the MacGyvers are working against a ticking time bomb to build a stealth helicopter from Legos they recovered from Happy Meals™.  They are concerned about getting out of their current jam to compile another day.

Normally Imperialists simply ignore the MacGyvers or run down the slow ones like yesterday’s flotsam.  The cloud is changing that dynamic because it’s proving to be a dramatic force multiplier in several ways:

  1. Lower cost of entry – the latest cloud options (e.g. GAE) do not charge anything unless you generate traffic.  The only barrier to entry is an idea and time.
  2. Rapid scale – companies can fund growth incrementally based on success while also being able to grow dramatically with minimal advanced planning.
  3. Faster pace of innovation – new platforms, architectures and community development has accelerated development.  Shared infrastructure means less work on back office and more time on revenue focused innovation.
  4. Easier access to customers – social media and piggy backing on huge SaaS companies like Facebook, Google or SalesForce bring customers to new companies’ front doors.  This means less work on marketing and sales and more time on revenue focused innovation.

The bottom line is that the cloud is allowing the MacGyvers to be faster, stronger, and more innovative than ever before.  And we can expect them to be spending even less time polishing the brass in the back office because current SaaS companies are working hard to help make them faster and more innovative.

For example, Facebook is highly incented for 3rd party applications to be innovative and popular not only because they get a part of the take, but because it increases the market strength of their own SaaS application.

So the opportunity for Imperialists is to find a way for employee and empower the MacGyvers.  This is not just a matter of buying a box of Legos: the strategy requires tolerating enabling embracing a culture of revenue focused innovation that eliminates process drag.  My vision does not suggest a full replacement because the Imperialists are process specialists.  The goal is to incubate and encapsulate cloud technologies and cultures.

So our challenge is more than picking up cloud technologies, it’s understanding the cloud communities and cultures that we are enabling.

Clouds & Water (Blog Action Day)

Posted on by
1

Today Change.org is coordinating Blog Action Day 2010 to raise awareness about Water.  It is widely reported (and worth repeating) that scarcity of clean water is more likely to impact your daily life than scarcity of energy, food, shelter or other basic human rights.

Water scarcity has little impact in my daily life.  <shameless plug>While The new cloud servers my employer, Dell, sells consume less power and thereby less cooling water; these efficiencies do relatively little to impact people’s access to fresh water.</shameless plug>

However, waste is a huge impact.  Since Americans are water, food and energy hogs, we are also in the position of wasting disproportionate amount of these limited resources.  I believe that we commit this waste unconsciously without any real gauge on its volume or impact.  Imagine the impact to your driving behavior if you had to fill your gas tank up a cup of gas at a time (64), water your lawn from a 5 gallon bucket (30+) or refill your toilet with a table-spoon (409!).

The key to addressing waste in the land of plenty is to measure and show impacts.  I believe that people abhor waste when they see it.  Our challenge is not to change people, but to show them in real terms the consequences of their choices.

For example, just having an MPG calculator on our cars has changed the way that we drive them.  I am personally disappointed with how little useful feedback these gauges provide, but it’s a start.

One of the things I like about Cloud Computing is that we want to measure and reduce waste.  We get mad about waste: wasted computer time, wasted equipment, wasted power, and especially wasted time.

As we make strides to make computing and information more personal and mobile, I believe we need to include ways to show people data about the choices that they are making.  So next time you water your lawn or flush your toilet, this about what it would mean if you had the haul that water in a bucket up from a well.  Sound crazy?  That’s status quo for more people than those of us that enjoy indoor plumbing.

Microsoft & Dell partner for Azure hosting

Posted on by
Reply

Vacation rental in Fort Walton Beach, FLA

Interesting juxtaposition, last week I was vacationing at the Azure condos in Florida and this week Dell is making a strategic announcement to develop a Dell-powered Microsoft Windows Azure Platform Appliance, as well as Azure based services delivered by Dell Services.

Press Release: http://content.dell.com/us/en/corp/d/press-releases/2010-07-12-dell-microsoft-cloud-azure-appliance.aspx

This is a project that I’m involved in, so watch for updates as details emerge.

Just Striping in the RAIN

Posted on by
1

Or “behold the power of the unreliable”

In a previous post, I discussed the concept of a Redundant Array of Inexpensive Nodes (RAIN) as a way to create more reliable and scalable applications.   Deploying a RAIN application is like being the House in Vegas – it’s about having enough size that the odds come out in your favor.  Even if one player is on a roll, you can predict that nearly everyone else is paying your rent.  Imagine what would happen if all the winning gamblers were in your casino!  If you don’t want to go bankrupt when deploying a RAIN app, then ensure that the players spread out all over the Strip.

One of my core assumptions is that you’ll deploy a RAIN application on a cloud.   This is a significant because we’re assuming that your nodes are

  1. idle most of the time because your traffic loads are cyclic
  2. unreliable because the cloud provider does not offer much SLA
  3. divisible because renting ten 1/10ths of a server costs roughly the same as a whole one
  4. burstable because 1/10th servers can sometimes consume that extra 9/10th server

The burstable concept is a dramatic power multiplier that tips the whole RAIN equation heavily towards clouds. 

Bursting means that under load, your 10 1/10th servers (roughly 1 server in cost) could instantly expand to the power of 10 full servers!  That reflects an order of magnitude spike in demand without any change in your application.

In the past, we’ve racked extra servers to handle this demand.  That meant that we had a lot of extra capacity taking up rack space, clubbing innocent migratory electrons for their soft velvety fur, and committing over provisioning atrocities.  

Today, multi-tenant clouds allow us to average out these bursts by playing the odds on when application bursts will occur.  In this scenario, the infrastructure provider benefits from the fact that applications need to be over provisioned.  The application author benefits because they can instantly tap more resources on demand.  Now, that is what I would call a win-win synergy!

All this goodness depends on

  • standard patterns & practices that developers use to scale out RAIN applications
  • platform improvements in cloud infrastructure to enable smooth scale out
  • commercial models that fairly charge for multi-tenant over-subscription
  • workable security that understands the underlying need for co-mingling workloads

The growing dominance of cloud deployments looks different when you understand the multiplying interplay between multi-tenant clouds and cloud-ready RAIN applications.

IEEE “Pragmatic Clouds” Presentation (2/24/10)

Posted on by
Reply

Tonight I presentated “Pragmatic Clouds” to the IEEE Central Texas Consultants Network Meeting.

The abstract was, “Cloud computing seems to mean everything! We’ll talk in specifics about how application development and delivery models are changing based on new “cloud” commercial models. We’ll also explore how the plumbing behind the curtains is changing to reflect these new models.”

In the presentation, I covered drivers for cloud business models and how it impacts creating applications for clouds. I also described how to write a future-proof application that can work for IaaS clouds today and PaaS clouds tomorrow.