Clouds & Water (Blog Action Day)

Posted on by
1

Today Change.org is coordinating Blog Action Day 2010 to raise awareness about Water.  It is widely reported (and worth repeating) that scarcity of clean water is more likely to impact your daily life than scarcity of energy, food, shelter or other basic human rights.

Water scarcity has little impact in my daily life.  <shameless plug>While The new cloud servers my employer, Dell, sells consume less power and thereby less cooling water; these efficiencies do relatively little to impact people’s access to fresh water.</shameless plug>

However, waste is a huge impact.  Since Americans are water, food and energy hogs, we are also in the position of wasting disproportionate amount of these limited resources.  I believe that we commit this waste unconsciously without any real gauge on its volume or impact.  Imagine the impact to your driving behavior if you had to fill your gas tank up a cup of gas at a time (64), water your lawn from a 5 gallon bucket (30+) or refill your toilet with a table-spoon (409!).

The key to addressing waste in the land of plenty is to measure and show impacts.  I believe that people abhor waste when they see it.  Our challenge is not to change people, but to show them in real terms the consequences of their choices.

For example, just having an MPG calculator on our cars has changed the way that we drive them.  I am personally disappointed with how little useful feedback these gauges provide, but it’s a start.

One of the things I like about Cloud Computing is that we want to measure and reduce waste.  We get mad about waste: wasted computer time, wasted equipment, wasted power, and especially wasted time.

As we make strides to make computing and information more personal and mobile, I believe we need to include ways to show people data about the choices that they are making.  So next time you water your lawn or flush your toilet, this about what it would mean if you had the haul that water in a bucket up from a well.  Sound crazy?  That’s status quo for more people than those of us that enjoy indoor plumbing.

Blog Action Day: 10/15

Posted on by
Reply
In a few days, I’ll participate in Blog Action Day 2010.  I did this before from my RAVolt.com EV blog.
This year’s topic, Water, is not directly relevant to the types of Clouds that I’m working with; however, it would make me very sad to think that we can create hyper-scale social media game platforms for lonely laptop wielding suburbanites while not putting a drop of effort into fundamental issues.  I’m sure I can conjure a tirade about it in the next few days…stay tuned.

Shaken or stirred? Cloud Cocktail leads to insights

Posted on by
1

Part of my perfessional & personal mission is to kick over mental ant hills.  In the cloud space, I believe that people are trying way too hard to define cloud into neat little buckets.  That leads me to try and reorient around new visualizations.  The purpose of doing this is to strip away historical thought patterns that limit our ability to envision future patterns (meaning: attitude adjustment).

The Cloud Cocktail

With that overly erudite preamble, here’s a tasty potion that I mixed up for you to enjoy on your way to real libations at ACL.

The technologies underlying cloud are complex; however, the core components for cloud are simple: applications, networked services and virtualized infrastructure.  These three components in varying proportions garnished with management APIs form the basis for all cloud solutions. 

This cocktail napkin sketch of a cloud may appear sparse, but it provides the key insights that drive a vision for how to adapt and respond to clouds’ rapid metamorphosis.  It would be ideal to point to a single set of technologies and declare that it is a Cloud; unfortunately, cloud is a transformation, not an end-state. 

PaaS, much ado about network services

Posted on by
9

There’s a surprising about of a hair pulling regarding IaaS vs PaaS.  People in the industry get into shouting matches about this topic as if it mattered more than Lindsay Lohan’s journey through rehab.

The cold hard reality is that while pundits are busy writing XaaS white papers, developers are off just writing software.  We are writing software that fits within cloud environments (weak SLA, small VMs), saves money (hosted data instead of data in VMs), and changes quickly (interpreted languages).  We’re doing using an expanding tool kit of networked components like databases, object stores, shared cache, message queue, etc.

Using network components in an application architecture is about as novel as building houses made of bricks.  So, what makes cloud architectures any better or different?

Nothing!  There is no difference if you buy VMs, install services, and wire together your application in its own little cloud bubble.  If I wanted to bait trolls, I’d call that an IaaS deployment.

However, there’s an emerging economic driver to leverage lower cost and more elastic infrastructure by using services provided by hosts rather than standing them up in a VM.  These services replace dedicated infrastructure with managed network attached services and they have become a key differentiator for all the cloud vendors

  • At Google App Engine, they include Big Tables, Queues, MemCache, etc
  • At Microsoft Azure, they include SQL Azure, Azure Storage, AppFabric, etc
  • At Amazon AWS, they include S3, SimpleDB, RDS (MySQL), Queue & Notify, etc

Using these services allows developers to focus on the business problems we are solving instead of building out infrastructure to run our applications.  We also save money because consuming an elastic managed network service is less expensive (and more consumption based) than standing up dedicated VMs to operate the services.

Ultimately, an application can be written as stateless code (really “externalized state” is more a accurate description) that relies on these services for persistence.  If a host were to dynamically instantiate instances of that code based on incoming requests then my application resource requirements would become strictly consumption based.   I would describe that as true cloud architecture. 

On a bold day, I would even consider an environment that enforced offered that architecture to be a platform.  Some may even dare to describe that as a PaaS; however, I think it’s a mistake to look to the service offering for the definition when it’s driven by the application designers’ decisions to use network services.

While we argue about PaaS vs IaaS, developers are just doing what they need.  Today they may stand-up their own services and tomorrow they incorporate 3rd party managed services.  The choice is not a binary switch, a layer cake, or a holy war.

The choice is about choosing the right most cost effective and scalable resource model.

McCrory on “Cloud Confusion”

Posted on by
Reply

or, why is everyone DaaZed and Confused?

Dave McCrory, my co-worker at Dell, posted an interesting analysis of how different roles people have in IT jobs dramatically influences their perception of cloud services.

I think that part of the confusion is how difficult it is for each category of cloud user to see their challenges/issues for the other classes of user.

We see this in spades during internal PaaS discussions.  People with development backgrounds has a fundamentally different concept of a PaaS benefits.  In many cases, those same benefits (delegation to a provider for core services like database) are considered disadvantages for the other class of user (you want someone else to manage what!).

Ultimately, the applications are at the core of any XaaS conversation and define what “type” of cloud need to be consumed.

DevOps: There’s a new sheriff in Cloudville

Posted on by
8

DevOps SherrifLately there’s a flurry of interest (and hiring demand) for DevOps gurus.  It’s obvious to me that there’s as much agreement between the ethical use of ground unicorn horn as there is about the job description of a DevOps tech.

I look at the world very simply:

  • Developers = generate revenue
  • Ops = control expenses
  • DevOps = write code, setup infrastructure, ??? IDK!

Before I risk my supply of ethically obtained unicorn powder by defining DevOps, I want to explore why DevOps is suddenly hot.  With cloud driving horizontal scale applications (see RAIN posts), there’s been a sea change in the type of expertise needed to manage an application.

Stereotypically, Ops teams get code over the transom from Dev teams.  They have the job of turning the code into a smoothly running application.  That requires rigid controls and safe guards.  Traditionally, Ops could manage most of the scale and security aspects of an application with traditional scale-up, reliability, and network security practices.  These practices naturally created some IT expense and policy rigidity; however, that’s what it takes to keep the lights on with 5 nines (or 5 nyets if you’re an IT customer).

Stereotypically, Dev teams live a carpe diem struggle to turn their latest code into deployed product with the least delay.  They have the job of capturing mercurial customer value by changing applications rapidly.  Traditionally, they have assumed that problems like scale, reliability, and security could be added after the fact or fixed as they are discovered.  These practices naturally created a need to constantly evolve.

In the go-go cloud world, Dev teams are by-passing Ops by getting infrastructure directly from an IaaS provider.  Meanwhile, IaaS does not provide Ops the tools, access, and controls that they have traditionally relied on for control and management.  Consequently, Dev teams have found themselves having to stage, manage and deploy applications with little expertise in operations.  Further, Ops teams have found themselves handed running cloud applications that they have to secure, scale and maintain applications without the tools they have historically relied on.

DevOps has emerged as the way to fill that gap.  The DevOps hero is comfortable flying blind on an outsourced virtualized cloud, dealing with Ops issues to tighten controls and talking shop with Dev to make needed changes to architecture.  It’s a very difficult job because of the scope of skills and the utter lack of proven best practices.

So what is a day in the life of a DevOp?   Here’s my list:

  • Design and deploy scale out architecture
  • Identify and solve performance bottlenecks
  • Interact with developers to leverage cloud services
  • Interact with operations to integrate with enterprise services
  • Audit and secure applications
  • Manage application footprint based on scale
  • Automate actions on managed infrastructure

This job is so difficult that I think the market cannot supply the needed experts.  That deficit is becoming a forcing function where the cloud industry is being driven to adopt technologies and architectures that reduce the dependence for DevOps skills.  Now, that’s the topic for a future post!

Introducing BravoDelta: Erlang BDD based on Cucumber

Posted on by
7

I highly recommend Armstrong's Programming Erlang

I ❤ Erlang.  I learned about Erlang while simultaneously playing w/ CouchDB (written in Erlang) and reading Siebel’s excellent Coders At Work interview of Erlang creator Joe Armstrong.  Erlang takes me back to my lisp and prolog days – it’s interesting, powerful and elegant.  Even better, it’s performant, time tested and proven.

To whet my Erlang skills, I decided to port of the most essential development tools I’ve used: Cucumber BDD.  I think that using BDD is one of the most critical success criteria for a project that wants to move quickly and respond to customers.  If you’d like to see Cucumber in action, check out my WhatTheBus project.  A Cucumber test is written in “simple English” and looks like this:

Scenario: Web Page1
    When I go to the home page.
    Then I should see "Districts".

To run Bravo Delta, you’ll need Erlang installed on your system.  You may also want to setup the WhatTheBus project because the initial drop uses that RoR web site as it’s target.  I’ve uploaded the code onto GitHub project BravoDelta (code contributions welcome).

NOTE: This is a functional core – it is not expected to be a complete Cuke replacement at this point!

The code base consists of the following files:

  • bdd.erl (main code file, start using bdd:test(“scenario”).)
  • bdd.config (config file)
  • bdd_webrat.erl (standard steps that are used by many web page tests)
  • bravodelta.erl (same custom steps, must match feature file name)
  • bravodelta.feature (BDD test file)
  • bdd_utils.erl (utilities called by bdd & webrat)
  • bdd_selftest.erl (unit tests for utils – interesting pattern for selftest in this file!)
  • bdd_selftest.data (data for unit tests)

Erlang makes parsing the feature file very easy.  Unlike Cucumber, there is no RegEx craziness because Erlang has groovy pattern matching.  Basically, each step decomposes into a single line starting with Given, When, or Then.  The code is designed so that developers can easily add custom steps and there are pre-built steps for common web tasks in the “webrat” step file.  A step processor looks like this in Erlang:

step(_Config, _Given, {step_when, _N, ["I go to the home page"]}) ->
	bdd_utils:http_get(_Config, []);
step(_Config, _Result, {step_then, _N, ["I should see", Text]}) ->
	bdd_utils:html_search(Text,_Result).

The steps are called by an Erlang recursive routine in BDD for each scenario in the feature file.  Explaining that code will have to wait for a future post.

The objective for Bravo Delta is to demonstrate simple Erlang concepts.  I wanted to make sure that the framework was easy to extend and could grow overtime.  My experience with Erlang is that my code generally gets smaller and more powerful as I iterate.  For example, moving from three types of steps (given_, when_, then_) to a single step type with atoms for type resulted in a 20% code reduction.

I hope to use it for future BDD projects and grow its capability because it is fast and simple to extend – I found Cucumber to be very slow.  It should also be possible to execute features in parallel with minimal changes to this code base.  That makes Bravo Delta very well suited to large projects with lots of tests and automated build systems.

New Media = multiple audiences, simulateously

Posted on by
Reply

Danah Boyd‘s insights about the social impact of social media constantly astonish me.  Here recent social steganography post has interesting implications for all of us operating in the topsy-turvey mixed-up world of professional personal branding.

I was interested to think of how differently we process public information and easily ignore parts that don’t make sense to us.  Perhaps a blended word, “confuscation,”  would be an easier word to grok than steganography?

Factoring multiple reader’s perspectives into writing (or presenting) is a crucial part of my daily job.  As my team works to include cloud strategy within Dell, understanding the listener’s frame of reference is essential to communicating the message.  For me, this means framing cloud services & software into units & hardware concepts.

In many ways, I think we have a greater challenge overcoming unintended steganography then learning how to enhance it.  Perhaps as we get more deliberate at it, we’ll become better at limiting the unintended confuscation.