Spinning up OpenStack “DefCore” Committee by spotting elephants

ElephantsThis week, Joshua McKenty, me and a handful of interested individuals (board member Eileen Evans included) met to start organizing the DefCore* Committee.  This standing committee was established by an OpenStack Foundation resolution just before the Hong Kong Summit.  Joshua and I were nominated as co-chairs (and about half the board volunteered to be members).    This action was an immediate result of the unanimous passage of the 10 Principles that I was driving in the DefCore “Spider” cycle.

We heard overwhelmingly at the Hong Kong summit that defining core should be a major focus for the Board.

The good news is that we’re doing exactly that in CoreDef.  Our challenge is to go quickly but not get ahead of community consensus.  So far, that means eating the proverbial elephant in small bites and intentionally deferring topics where we cannot find consensus.

This meeting was primarily about Joshua and I figuring out how to drive DefCore quickly (go fast!) without exceeding the communities ability to review and discuss (build consensus!).  While we had future-post-worthy conceptual discussions, we had a substantial agenda of get-it-done in front of us too.

Here’s a summary of key outcomes from the meeting:

1)      We’ve established a tentative schedule for our first two meetings (12/3 and 12/17).

  1. We’ve started building agendas for these two meetings.
  2. We’ve also established rules for governance that include members to do homework!

2)      We’ve agreed it’s important to present a bylaws change to the committee for consideration by the board.

  1. This change is to address confusion around how core is defined and possibly move towards the bylaws defining a core process not a list of core projects.
  2. This is on an accelerated track because we’d like to include it with the Community Board Member elections.

3)      We’ve broken DefCore into clear “cycles” so we can be clearer about concrete objectives and what items are out of scope for a cycle.  We’re using names to designate cycles for clarity.

  1. The first cycle, “Spider,” was about finding the connections between core issues and defining a process to resolve the tension in those connections.
  2. This cycle, “Elephant,” is about breaking the Core definition into
  3. The next cycle(s) will be named when we get there.  For now, they are all “Future”
  4. We agreed there is a lot of benefit from being clear to community about items that we “kick down the road” for future cycles.  And, yes, we will proactively cut off discussion of these items out of respect for time.

4)      We reviewed the timeline proposed at the end of Spider and added it to the agenda.

  1. The timeline assumes a staged introduction starting with Havana and accelerating for each release.
  2. We are working the timeline backwards to ensure time for Board, TC and community input.

5)      We agreed that consensus is going to be a focus for keeping things moving

  1. This will likely drive to a smaller core definition
  2. We will actively defer issues that cannot reach consensus in the Elephant cycle.

6)      We identified some concepts that may help guide the process in this cycle

  1. We likely need to create categories beyond “core” to help bucket tests
  2. Committee discussion is needed but debate will be time limited

7)      We identified the need to start on test criteria immediately

  1. Board member John Zannos (in absentia) offered to help lead this effort
  2. In defining test criteria, we are likely to have lively discussions about “OpenStack’s values”

8)      We identified some out of scope topics that are important but too big to solve.

  1. We are calling these “elephants” (or the elephant in the room).
  2. The list of elephants needs to be agreed by DefCore and clearly communicated
  3. We expect that the Elephant cycle will make discussing these topics more fruitful

9)      We talked about RefStack code features

  1. Allowing users to upload/post test results from their clouds to enable white box test reporting
  2. Allowing users who have uploaded results to +/- vote on tests they think are important
  3. We established a requirement that posting results requires an OpenStack ID
  4. We established a requirement that only a single Corporate designate (provided by the Foundation) can make a result official for their company.
  5. Collecting opt-in data with test results using tags for things like alternate implementations use, host operating system(s), deployment method, size of cloud, and hypervisor.
  6. We discussed (but did not resolve) that it could be possible to have people run RefStack against public cloud end points and post their results
  7. We agreed that RefStack needs to be able to run locally or as a hosted site.

10)   We identified a lot of missing communication channels

  1. We created a DefCore wiki page to be a home for information.
  2. Joshua and I (and others?) will work with the Foundation staff to create “what is core” video to help the community understand the Principles and objectives for the Elephant cycle.
  3. We are in the process of setting up mail lists, IRC, blog tags, etc.

Yikes!  That’s a lot of progress priming the pump for our first DefCore meeting!

* We picked “DefCore” for the core definition committee name.  One overriding reason for the name is that it has very clean search results.  Since the word “core” is so widely used, we wanted to make sure that commentary on this topic is easy to track against the noisy term core.  We also liked 1) the reference to DefCon and 2) that the Urban Dictionary defines it as going deaf from standing too close to the speakers.

Twelve straw man positions to frame OpenStack “what is core” discussion

THIS POST IS #4 IN A SERIES ABOUT “WHAT IS CORE.”

Train WheelsIt’s time to move to the what from the how and why.  In my experience discussing the framework, it’s important to understand the context at this point because our objective is to define the common ground.  If you mistake the intent then it’s much harder to understand the details, but here they are…

I’m hopeful that the framework we’re defining today will ultimately be taken for granted.  Until then, relax and remember that this straw man is a work in progress.

These positions have been crafted over the course of several weeks and are constantly evolving. Next steps are to expand discussions to the OpenStack Community for input and review at the next Board Meeting on August 6th.   Ultimately, we need to close this issue at the next Summit.

I present them here for reference and encourage the community to bring up dialog on the OpenStack list (positions are sourced here).

Update Aug 13, 2013: These original 12 points have been refined down to 10.

You’re welcome to read the original 12 below, but I recommend skipping over to the refined set.

Continue reading

My insights from OpenStack “what is core” Spider > we need pluggable architectures

THIS POST IS #3 IN A SERIES ABOUT “WHAT IS CORE.”

IdeasSo what did we learn from the spider map exercise?  Above all else, the spider confirmed that to me that OpenStack is a world of paradox.  The perfect definition of core may be elusive, but I believe we can find one that is sufficient.

The goal was understanding not philosophical truth.  In a diverse and vibrant community with many objectives, understanding leads to consensus while being “right” can become very lonely.

Since our goal was not to answer the question, what did we want to accomplish?  Spider success was defined as creating a framework, really a list of agreed positions (post #4), that narrows the scope of the “what is core” dialog.

Too vague a framework leads to uncertainty about what’s included, stable and working while too rigid a baseline could drive away innovation and lead to forking.  Being too aggressively open could discourage commercial investment yet too proprietary an approach contradicts our collaboration and community values.

Having a workable framework that accommodates these diverse positions allows us to move forward.

So what did Alan and I learn from the spider to help the discussion?

  • “Plug-ins” are essential to the definition of core because they create safe places for innovation  [note: there has been much refinement of what "plug-in" means here]
  • It is possible to balance between stability and innovation if we have a way to allow implementations to evolve
  • OpenStack has a significant commercial ecosystem that needs to be accommodated in core
  • We need an approach that allows extension and improvement without having to incubate new projects
  • We need to ensure that we use brand and culture to combat forking
  • Interoperability is a worthy goal
  • Everyone thinks testing is good, but it’s still a sidebar
  • There are multiple distinct audiences with conflicting goals: some want stability and durability while others want innovation and flexibility.

Of these insights, the need to discuss how OpenStack promotes a plug-in architecture seemed address the most points of tension.  [update: in the course of discussion, we've defining plug-in more generally to be something like "a designated section of implementation code that can be altered without negatively changing the base function of the project."]

This is not the only item worth discussing, but it was the one that made the most sense to cover first based on the spider map.  Our idea was that having the community find agreement on how we approach plug-ins would lead us closer to a common ground for the “what is core” discussion.

Finding a common thread shrinks the problem space so the Board, TC and Community can advance in discussion.  So far, that assessment has proven accurate enough to move the dialog forward.

READ POST IS #4: TWELVE POSITIONS

Introducing the OpenStack Spider Graph: untangling our web of interdependencies

This post is #2 in a series about “what is core.”

peaceOne of the OpenStack Foundation’s most fundamental responsibilities is to define “what is core” for the OpenStack project.  A simple yet urgent question with profound implications for the project’s success; consequently, it’s no surprise that finding a consensus answer has been a thorny problem.   In fact, the answer (it’s 42, of course) is so convoluted that we had to step back establish common ground before we could proceed. I’m proud to say that we’ve made substantial progress towards establishing a framework for the discussion.   

This post (#2) is about the process we followed.  My next post (#3) is about what we learned.  Next, we want to engage the community in discussing the baseline positions (post #4) that make up the framework.   The challenge in discussion was that the Board (and Community) lacked common baseline.  When one person talked about commercial extensions to core another would insist that core always be open.  There are compatible statements; yet, we’d often talk past each other because the question was simply too big and fuzzy.

spider boardAlan Clark, Foundation Board Chairman, and I took on the task of breaking the logjam by decomposing the problem.  We used a multilayered mind map that I called the “spider” chart (yes, it’s a misnomer).  In constructing the spider, we first flagged out issue areas (the nodes) and added goals (red text) for each issue area.  Next we linked connected nodes together to form a graph.  We added blue text on connections where goals were opposed or in tension and green text where goals were aligned.

Note: Some of our text is controversial – it was our objective to find the tension.  If you are looking at the image, we also created some grid and flow charts to test the map.

We did not time capture the spider so it’s impossible to sweep back and see how the ideas progressed.  In general, we started at the center from “IncUp Discussion” which was the TC and Board’s collaboration, led by Alan, to define the process by which projects are promoted from incubation to integrated projects.

Some topics were easy like “API vs Implementation” and “Trademark” while “Implementation Led” and “Plug-ins” were more difficult.  Some clarification on those two items is worthwhile:

Implementation Led describes the OpenStack community for starting from working implementations rather than designing an API.  In my experience, this pattern serves the community well as long as we accept that APIs and implementations in incubated projects will take time to mature.   Using this approach gets us out of prolonged design by committee spirals.  It has serious pitfalls also – there’s no milk and honey approach. Plug-ins represented a big pot of politics in the community that was part of the challenge in defining what is core.  The obvious issue is that not all core projects use plug-ins and the ones that do take different approaches.  Even so, we felt that the concept of multiple implementations with a single API was a significant part of the opportunity (and challenge) for OpenStack core.

Ultimately, we decomposed the spider graph into the following table that shows some of our thoughts in raw form.  I’ll discuss the actual insights in my next post (#3) Continue reading