Addressing OpenStack API equality: rethinking API via implementation over specification

This post is an extension of my post about OpenStack’s top 5 challenges from my perspective working on Dell’s OpenStack team.  This issue has been the subject of much public debate on the OpenStack lists, at the conference and online (Wilamuna & Shuttleworth).  So far, I have held back from the community discussion because I think both sides are being well represented by active contributors to trunk.

I have been, and remain, convinced that a key to OpenStack success as a cloud API is having a proven scale implementation; however, we need to find a compromise between implementation and specification.

The community needs code that specifies an API and offers an implementation backing that API.  OpenStack has been building an API by implementing the backing functionality (instead of vice versa).  This ordering is important because the best APIs are based on doing real work not by trying to anticipate potential needs.  I’m also a fan of implementation driven API because it emerges quickly.  That does not mean it’s all wild west and cowboy coding.  OpenStack has the concept of API extensions; consequently, its possible to offer new services in a safe way allowing new features to literally surface overnight.

Unfortunately, API solely via implementation risks being fragile, unpredictable, and unfair. 

It is fragile because the scope is reflected by what the implementors choose to support.  They often either expose internals or restrict capabilities in ways that a leading with design specification would not.  This same challenge makes them less predictable because the API may change as the implementation progresses or become locked into exposing internals when downstream consumers depend on exposed functions.

I am ready to forgive fragile and unpredictable, but unfair may prove to be expensive for the OpenStack community.

Here’s the problem: there is more than on right way to solve a problem.  The benefit of an API is that it allows multiple implementations to emerge and prove their benefits.  We’ve seen many cases where, even with a weak API, a later implementation is the one that carries the standard (I’m thinking browsers here).  When your API is too bound to implementation it locks out alternatives that expand your market.  It makes it unfair to innovators who come to the party in the second wave.

I think that it is possible to find a COMPROMISE between API implementation and specification; however, it takes a lot of maturity to make that work.  First, it requires the implementors to slow down and write definitions outside of their implementation.  Second, it requires implementors to emotionally decouple from their code and accept other API implementations.  Of course, a dose of test driven design (TDD) and continuous integration (CI) discipline does not hurt either.

I’m interested in hearing more opinions about this… come to the 10/27 OpenStack meetup to discuss and/or please comment!

11 thoughts on “Addressing OpenStack API equality: rethinking API via implementation over specification

  1. Pingback: OpenStack: Five Challenges & Conference Observations « Rob Hirschfeld's Blog

  2. Pingback: Notes from 10/27 OpenStack Austin Meetup (via Stephen Spector) « Rob Hirschfeld's Blog

  3. Pingback: OpenStack Seattle Meetup 11/30 Notes « Rob Hirschfeld's Blog

  4. Pingback: Why Governance Matters in Open Source: Discussing the OpenStack Foundation « Rob Hirschfeld's Blog

  5. Pingback: Quantum: Network Virtualization in the OpenStack Essex Release « Rob Hirschfeld's Blog

  6. Pingback: Austin OpenStack Meetup: Keystone & Knife (2/20 notes via Greg Althaus) « Rob Hirschfeld's Blog

  7. Pingback: My Dilemma with Folsom – why I want to jump to G « Rob Hirschfeld's Blog

  8. Pingback: Introducing the OpenStack Spider Graph: untangling our web of interdependencies | Rob Hirschfeld

  9. Pingback: OpenStack’s Test Driven Core > it’s where I think “what is core” discussions are heading | Rob Hirschfeld

  10. Pingback: Understanding OpenStack Designated Code Sections – Three critical questions | Rob Hirschfeld

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s