Community dialogue around “What is Core” positions

THIS POST IS #5 IN A SERIES ABOUT “WHAT IS CORE.”

by Rob Hirschfeld (cc) w/ attribution

The OpenStack Foundation Board has been having a broadening conservation about this topic.  Feeling left out?  Please don’t be!  Now is the time to start getting involved: we had to start very narrowly focused to avoid having the discussion continue to go in circles.  As we’ve expanding the dialog, we have incorporated significant feedback to drive consensus.

No matter where I go, people are passionate about the subject of OpenStack Core.

Overall, there is confusion of scope covered by “what is core” because people bring in their perspective from public, private solution, ecosystem or internal deployment objectives.  In discussion, everyone sees that we have to deal with issues around the OpenStack mark and projects first, but they are impatient to get into the deep issues.  Personally, we can get consensus on core and will always have a degree of healthy tension between user types.

The following are my notes, not my opinions.  I strive to faithfully represent a wide range of positions here.  Clarifications, comments and feedback are welcome!

Consensus Topics:

  • Reference/Alternate Implementation (not plug-in): Not using “plug-ins” to describe the idea that OpenStack projects should have a shared API with required code and clearly specified areas where code is replaceable.  It is the Technical Committee (TC) that makes these decisions.  The most meaningful language around this point is to say that OpenStack will have an open reference implementation with allowable alternate implementations.
  • Alternate implementations are useful:  We want to ensure upstream contribution and collaboration on the code base.  Reference implementations ensure that there’s a reason to keep open source OpenStack strong.  Alternate Implementations are important to innovation.
  • Small vs. Large Core: This is an ongoing debate about if OpenStack should have a lot of projects as part of core.  We don’t have an answer but people feel like we’re heading in a direction that resolves this question.
  • Everyone likes tests: We’re heading towards a definition of core that relies heavily on tests.  Everyone expresses concerns that this will place a lot of stress on Tempest (or another framework) and that needs to be addressed as we move forward.

Open Topics:

  • Monolithic vs. Granular Trademark:  We did not discuss if vendors will be able to claim OpenStack trademarks on subcomponents of the whole.  This is related to core but wide considered secondary.
  • API vs. implementation tension:  We accept that OpenStack will lead with implementation.   There’s no official policy that “we are not a standards body” but we may also have to state that tests are not a specification.  There’s a danger that tests will be considered more than they are.  What are they?  “They are an implementation and a source of information.  They are not the definition.”   We expect to have a working model that drives the API not vice versa.
  • Brouhaha about EC2 APIs:  It’s not clear if defining core helps address the OpenStack API discussion.  I hope it will but have not tested it.
  • Usability as core: I had many people insist that usability and ease of use should be as requirements for core because it supports adoption.  Our current positions do not have any statements to support this view.
  • Toxic neighbors: We have not discussed if use of the mark and criteria could be limited by what else you put in your product.  Are there implementation options that we’d consider toxic and automatically violate the mark?  Right now, the positions are worded that if you pass then you play even if you otherwise stink.
  • Which tests are required?  It appears that we’re moving towards using must-pass tests to define the core.  Moving towards tests determining core, we want actual field data to drive which tests are required. That will allow actual user experience to shape which tests are important rather than having it be a theoretical decision.  There’s some interest in asking the User Committee (UC) to recommend which tests are required.  This would be an added responsibility for the UC and needs more discussion.
  • Need visualization:  With 12 positions so far, it’s getting hard to keep it all together.  I’ve taken on an action item to create a diagram that shows which statements apply to which projects against the roles of ownership.

I’ve had some great discussions about core and am looking forward to many more.  I hope these notes help bring you up to speed.   As always, comments and discussion are welcome!

READ POST #6: VISUALIZING CORE

OpenStack steps toward Interopability with Temptest, RAs & RefStack.org

Pipes are interoperableI’m a cautious supporter of OpenStack leading with implementation (over API specification); however, it clearly has risks. OpenStack has the benefit of many live sites operating at significant scale. The short term cost is that those sites were not fully interoperable (progress is being made!). Even if they were, we are lack the means to validate that they are.

The interoperability challenge was a major theme of the Havana Summit in Portland last week (panel I moderated) .  Solving it creates significant benefits for the OpenStack community.  These benefits have significant financial opportunities for the OpenStack ecosystem.

This is a journey that we are on together – it’s not a deliverable from a single company or a release that we will complete and move on.

There were several themes that Monty and I presented during Heat for Reference Architectures (slides).  It’s pretty obvious that interop is valuable (I discuss why you should care in this earlier post) and running a cloud means dealing with hardware, software and ops in equal measures.  We also identified lots of important items like Open OperationsUpstreamingReference Architecture/Implementation and Testing.

During the session, I think we did a good job stating how we can use Heat for an RA to make incremental steps.   and I had a session about upgrade (slides).

Even with all this progress, Testing for interoperability was one of the largest gaps.

The challenge is not if we should test, but how to create a set of tests that everyone will accept as adequate.  Approach that goal with standardization or specification objective is likely an impossible challenge.

Joshua McKenty & Monty Taylor found a starting point for interoperability FITS testing: “let’s use the Tempest tests we’ve got.”

We should question the assumption that faithful implementation test specifications (FITS) for interoperability are only useful with a matching specification and significant API coverage.  Any level of coverage provides useful information and, more importantly, visibility accelerates contributions to the test base.

I can speak from experience that this approach has merit.  The Crowbar team at Dell has been including OpenStack Tempest as part of our reference deployment since Essex and it runs as part of our automated test infrastructure against every build.  This process does not catch every issue, but passing Tempest is a very good indication that you’ve got the a workable OpenStack deployment.

“Stack Shop” cover of Macklemore’s Thrift Shop

Sometimes a meme glitters too strongly for me to resist getting pulled in… that happened to great effect that just before the OpenStack Havana summit. When my code-addled mind kept swapping “poppin’ tags” for “OpenStack” on the radio edit, I stopped fighting and rewrote the Thrift Shop lyrics for OpenStack (see below the split).

With a lot of help from summit attendees (many of them are OpenStack celebrities, CEOs, VPs and members OpenStack Foundation board), I was able to create a freaking awesome cover of Macklemore’s second hand confection (NSFW).

Frankly, I don’t know everyone in the video (what, what?)!

But here’s a list of those that I do know.  I’m happy to update so the victims actors get credit.  Singers (in order):

Rob Hirschfeld (me) & Monty Taylor, Peter Poulliot, Judd Maltin, Forrest Norrod, Josh Kleinpeter, Tristan Goode, Dan Bode, Jay Pipes, Prabhakar Gopalan, Peter Chadwick, Simon Andersen, Vish Ishaya, Wayne Walls, Alex Freedland, Niki Acosta, Ops Track Monday Session 1, Ben Cherian, Eric Windisch, Brandon Draeger, Joseph B George,  Mark Collier, Joseph Heck, Tim Bell,  Chris Kemp, Kyle McDonald & Joshua McKenty,

Continue reading

OpenStack’s next hurdle: Interoperability. Why should you care?

SXSW life size Newton's Cradle

SXSW life size Newton’s Cradle

The OpenStack Board spent several hours (yes, hours) discussing interoperability related topics at the last board meeting.  Fundamentally, the community benefits when uses can operate easily across multiple OpenStack deployments (their own and/or public clouds).

Cloud interoperability: the ability to transfer workloads between systems without changes to the deployment operations management infrastructure.

This is NOT hybrid (which I defined as a workload transparently operating in multiple systems); however it is a prereq to achieve scalable hybrid operation.

Interoperability matters because the OpenStack value proposition is all about creating a common platform.  IT World does a good job laying out the problem (note, I work for Dell).  To create sites that can interoperate, we have to some serious lifting:

At the OpenStack Summit, there are multiple chances to engage on this.   I’m moderating a panel about Interop and also sharing a session about the highly related topic of Reference Architectures with Monty Tayor.

The Interop Panel (topic description here) is Tuesday @ 5:20pm.  If you join, you’ll get to see me try to stump our awesome panelists

  • Jonathan LaCour, DreamHost
  • Troy Toman, Rackspace
  • Bernard Golden,  Enstratius
  • Monty Taylor, OpenStack Board (and HP)
  • Peter Pouliot, Microsoft

PS: Oh, and I’m also talking about DevOps Upgrades Patterns during the very first session (see a preview).