Tag Archives: OpenStack

Interview Transcript about OpenStack, Foundation, Dell & Crowbar

Posted on by
6

Rafael Knuth, Dell Rafael Knuth (@RafaelKnuth with Dell as EMEA Social Media Community Manager) has set an ambitious objective to interview all 24 of the OpenStack Foundation Board members.  I have the privilege of being the first interview posted (Japanese version!).   Here’s the whole series.

This is not puff interview – We spent an hour together and Rafael did not shy away from asking hard questions like “Why did Dell jump into OpenStack?” and “is VMware a threat to OpenStack?”  Rather than posting the whole transcript (it’s posted here), I’m including the questions (as a teaser) below.   There is some real meat in these answers about OpenStack, Dell, Crowbar and challenges facing the project.

WARNING: My job is engineering, not marketing.  You may find my answers (which are MY OWN ANSWERS) to be more direct that you are expecting.  If you find yourself needing additional circumlocution then simply close your browser and move on.

Some highlights…

Dell’s interest in OpenStack has been very pragmatic. OpenStack is something we really see a market need for.

Rackspace …  runs on OpenStack pretty much off trunk …  That’s exactly the type of vibrant community we want to see.  At the same time, there is a growing community that wants to use OpenStack distributions with support, certifications and they are fine with being 6 months behind OpenStack off trunk. That’s good, and we want that shadow, we want that combination of pure minded early adopters and less sophisticated OpenStack users both working together.

We are working with different partners to bring OpenStack to different customers in different ways. It is confusing. Your question about Dell Crowbar was right … it is targeted at a certain class of users, and I don’t want enterprise customers who expect a lot of shiny chrome and zero touch. That’s not the target by now for Dell Crowbar. We definitely need that sort of magic decoder page to help customers understand our commercial offering.

Questions from Interview:

  1. Dell is one of the very early contributors to OpenStack. Why is Dell engaging in this project?
  2. How does Dell contribute to OpenStack?
  3. Let’s talk a bit about Dell Crowbar, your team’s deployment mechanism for OpenStack.
  4. Let’s talk a bit about OpenStack raw vs. OpenStack distributions.
  5. What are the biggest barriers to OpenStack adoption as of now?
  6. What does a customer specifically need to do when moving from OpenStack Essex to Folsom for example?
  7. My next question is around proof of concept versus production, Rob. How are customers using OpenStack and can you give examples for both scenarios?
  8. I hear very often two different statements: “Open Stack is an alternative to Amazon.” The other is: “OpenStack is an alternative to VMware … maybe, hopefully in two or three years from now.”  Which of both statements is true?
  9. How do you view VMware joining OpenStack. Is it a threat to OpenStack or does VMware add value to the project?
  10. Let us speak about market adoption. Who are the early adopters of OpenStack? And when do you expect OpenStack to hit the tipping point for mass market adoption?
  11. Rob, for all those interested in Dell’s commercial offering around OpenStack … can you give a brief overview?
  12. Dell TechCenter that provides customers an overview over our OpenStack offering: Dell Crowbar as our DevOps tool in its various shapes and forms, OpenStack distros we support … cloud services we build around OpenStack … hardware capabilities optimized for OpenStack.
  13. What are the challenges for the OpenStack Board of Directors?

Seeking OpenStack Foundation Board Seat for 2013 (please nominate me)

Posted on by
5
I am seeking another term on the OpenStack Foundation Board.  Please consider nominating me for this position.
The following is the profile information I provided as part of the nomination process.  If you are looking for insights into where OpenStack is going then these question (especially the later ones) will be interesting.

Provide Brief Biography of Yourself

I have been involved in Cloud for over 12 years and launch some of the earliest Cloud companies.  My educational background (Duke and LSU) is in computer science and systems engineering (Mechanical/Industrial) with a focus on distributed systems.  I have always found deployment to be vitally important in development – that lead me to found a SaaS start-up in 1999 and had made me a DevOps advocate.  In addition to core cloud technologies, I am an Agile/Lean process evangelist who strongly believes that how you build and deliver is just as important as what you deliver.
Currently, I am a principal engineer at Dell leading our OpenStack Cloud project  and also a founder of the Crowbar project.  In that role, I am in constant contact with OpenStack users, ecosystem developers and vendors world-wide; consequently, I have a very broad perspective on use and technical needs for OpenStack and related Cloud technologies.

What is your relationship to OpenStack, and why is its success important to you? What would you say is your biggest contribution to OpenStack’s success to date?

I have been involved in OpenStack at the earliest stages and was a key influencer in Dell’s decision to be an initial sponsor.  Further, I formulated Dell’s operations/deployment focused Lean strategy that helped create an early focus on OpenStack operations.   This support was a critical catalyst to building market momentum and we continue help drive operations and user focused requirements for OpenStack.  Providing a very open and community driven DevOps focus has been my biggest contribution (see Crowbar).
I also serve the community in many ways.  In addition to being elected to the 2012 Board, I founded the Austin OpenStack User Group (next meeting 12/6!), have been spoken at every Summit, co-Chaired the Operations Track at the Grizzly Summit and create community awareness of OpenStack through my blog, corporate work, and social media activity.
I have invested considerable time in OpenStack and made professional commitments to its success.  It’s not just about software and an awesome community – I am personally invested in OpenStack winning.

Describe your experience with other non profits or serving as a board member. How does your experience prepare you for the role of a board member?

I am currently serving on the OpenStack Foundation Board and have been an active advocate for a collaborative process and open communication (http://wp.me/pF6d2-vc, http://wp.me/pF6d2-w9).  I hope continue on the board so that we can stay focused on the critical issues at hand.

I have over four years of serving as Secretary for my city’s public Texas 4-B Commercial Development Corporation which governed by state open meetings standards.  This work has proven directly relevant to the OpenStack foundation because the open governance requirements map very well to our communities transparency expectations.

What do you see as the Board’s role in OpenStack’s success?

The board’s job is to ensure that community and collaboration remain OpenStack’s core strength as an open source project.  As adoption and footprint increases there is enormous pressure on the community to try and serve both general and very specialized interests.  Most critically, we must find ways to balance competing financial interests within the community.
The board must ensure that OpenStack provides commercial opportunity because I believe that incent continued investment; however, we cannot let profit drive the community away from our open source values.
I believe the Board must monitor the community’s progress to ensure we maintain this balance.  It is our responsibility to make adjustments, influence changes and take responsibility for driving an innovative and collaborative culture.

What do you think the top priority of the Board should be in 2013?

There are two inter-related top priorities for the Board: we must help deliver guidelines for which projects are truly “core” OpenStack and help move towards a certifiable API specification for OpenStack (instead of implementation).   While distinct, I believe both items must be solved together.  These changes are essential to foster innovation and adoption of OpenStack.
While these issues will occupy the Board (and Technical/User committees) for 2013, my personal priority for OpenStack in general remains focused on operators and users.  I believe we have to make substantial progress on upgrades, migration and operational readiness.  These issues continue to create a serious barrier to adoption.

Impressions from OpenStack Foundation board meeting 10/15

Posted on by
1

20121018-115319.jpgI spent the first day of the OpenStack summit doing my service as a Foundation board member. While you need to wait for a summary by Jonathan Bryce, the Foundation Executive Director, and/or the official minutes to get the details of our actions, I think that it’s reasonable to share my impressions.

In some ways, the chemistry of the board is as important as our actions at this point. We are still working thorough start up issues and, more importantly, learning how to work together. The board is not only big, 24 members, it is also has many companies that often compete (power of we).

Here are my top 5 impressions:

    1. Transparency. The bylaws are crafted to ensure a openness and our meetings are broadcast to the maximum extent possible. Even with that background, there is a consistent self-check and discussion about increasing transparency so that the community is included.
    2. Humor. It’s a great sign of progress that we are laughing together because it shows that we trust and respect each other.
    3. Frustration. Recognition that we have important decisions to make and a degree of impatience to make them. Boards are subtle: we spend a lot of time setting up the right structures that allow us to make hard decisions quickly. Also, grumbling that we’d overlapped the board meeting and the summit.
    4. Consensus. This something I committed to help build. I feel that our actions reflect both healthy discussion as individuals and desire to work together as a board.
    5. Leadership. Looking back from our first meeting, it is clear that Jonathan and Alan have fully assumed their roles. I’m also seeing how the Foundation team (Mark, Lauren, Stephano) are smoothly supporting operations even though we’re really just weeks from the Foundation launch.

I hope that gives you some insights into the board meeting and that more of you can join the broadcast for the next ones (2012 schedule TBD) including a joint tech and foundation meeting

Wednesday morning at the summit was “breakfast with the board.” Look for some notes from that too soon.

Crowbar releases for OpenStack Essex and Folsom

Posted on by
6

On the eve of the OpenStack design summit, it’s worth noting that the Crowbar team at Dell cut our final Essex release (aka Betty) last week. We’ve also committed the initial Folsom deployment scripts to the 1x development trunk under “feature/folsom” if you are doing Crowbar builds from DevTool (see bit.ly/crowbardevtool).

If you don’t want to build it, I’ve cut ISOs from the open source bits and posted them to http://crowbar.zehicle.com.

Andi Abes is presenting about the new Pull From Source (pfs) feature at the Summit on Monday. There’s a feature branch for that too and I’m going to check with him and try to post and ISO for that too.

20121014-132246.jpg

OpenStack Summit: Let’s talk DevOps, Fog, Upgrades, Crowbar & Dell

Posted on by
7

If you are coming to the OpenStack summit in San Diego next week then please find me at the show! I want to hear from you about the Foundation, community, OpenStack deployments, Crowbar and anything else.  Oh, and I just ordered a handful of Crowbar stickers if you wanted some CB bling.

Matt Ray (Opscode), Jason Cannavale (Rackspace) and I were Ops track co-chairs. If you have suggestions, we want to hear. We managed to get great speakers and also some interesting sessions like DevOps panel and up streaming deploy working sessions. It’s only on Monday and Tuesday, so don’t snooze or you’ll miss it.

My team from Dell has a lot going on, so there are lots of chances to connect with us:

At the Dell booth, Randy Perryman will be sharing field experience about hardware choices. We’ve got a lot of OpenStack battle experience and we want to compare notes with you.

I’m on the board meeting on Monday so likely occupied until the Mirantis party.

See you in San Diego!

PS: My team is hiring for Dev, QA and Marketing. Let me know if you want details.

Ballistic Release Cycles: Tracking the Trajectory of OpenStack Milestones

Posted on by
9

I’ve been watching a pattern emerge on the semiannual OpenStack release cycles for a while now. There is a hidden but crucial development phase that accelerates projects faster than many observers realize. In fact, I believe that substantial work is happening outside of the “normal” design cycle during what I call “free fall” development.

Understanding when the cool, innovative stuff happens is essential to getting (and giving) the most from OpenStack.

The published release cycle looms like a 6 stage ballistic trajectory. Launching at the design summit, the release features change and progress the most in the first 3 milestones. At the apogee of the release, maximum velocity is reached just as we start having to decide which features are complete enough to include in the release. Since many are not ready, we have to jettison (really, defer) partial work to ensure that we can land the release on schedule.

I think of the period where we lose potential features as free fall because thing can go in any direction. The release literally reverses course: instead of expanding, it is contracting. This process is very healthy for OpenStack. It favors code stability and “long” hardening times. For operators, this means that the code stops changing early enough that we have more time to test and operationalize the release.

But what happens to the jettisoned work? In free fall, objects in motion stay in motion. The code does not just disappear! It continues on its original upward trajectory.

The developers who invested time in the code do not simply take a 3 month sabbatical, nor do they stop their work and start testing the code that was kept. No, after the short in/out sorting pause, the free fall work continues onward with rockets blasting. The challenge is that it is now getting outside of the orbit of the release plan and beyond the radar of many people who are tracking the release.

The consequence of this ongoing development is that developers (and the features they are working on) show up at the summit with 3 extra months of work completed. It also means that OpenStack starts each release cycle with a bucket of operationally ready code. Wow, that’s a huge advantage for the project in terms of delivered work, feature velocity and innovation. Even better, it means that the design summit can focus on practical discussions of real prototypes and functional features.

Unfortunately, this free fall work has hidden costs:

  • It is relatively hidden because it is outside of the normal release cycle.
  • It makes true design discussions less productive because the implemented code is more likely to make the next release cycle
  • Integration for the work is postponed because it continues before branching
  • Teams that are busy hardening a core feature can be left out of work on the next iteration of the same feature
  • Forking can make it hard to capture bugs caught during hardening

I think OpenStack greatly benefits from free fall development; consequently, I think we need to acknowledge and embrace it to reduce its costs. A more explicit mid-release design synchronization when or before we fork may help make this hidden work more transparent.

OSCBM Seeking Community Input for Long Board Meetings and Candlelight Coding Sessions

Posted on by
1

Happy OpenStack Foundation Launch Day! I’m a little breathless at OpenStack’s sponsored sprint to foundationhood but very proud to be part of the process (you can be too!).   Just looking at the numbers it’s clear that we’re building something important.

While it’s important that OpenStack is innovative, stable and useful cloud infrastructure, it’s equally important the project in collaborative developed.

Collaborative development makes it safe for so many diverse commercial interests to participate.  The Foundation, with a gold and platinum war chest,  is a reflection of the need for the project to remain both openly collaborative and commercially successful for our community.  We must ensure a level technology playing field while we work to ensure members of the community can be commercially successful while contributing.  This balance is one of our core challenges.

As an OpenStack Community Board Member (OSCBM), I want to hear what you think the OpenStack Foundation should be doing for OpenStack!

It is vital that I get input from the OpenStack community!  Unlike 2/3 of the Board, my seat is decided by the community and (re)elected by the community on an annual basis; consequently, it is my responsibility to voice Stackers’ interest, not my employer’s (Dell).

Frankly, the project is still hugely dominated by developers with users/operators only just gaining influence. The Foundation’s primary purpose is to help safe guard its independence. As a board member, my job today is to oversee building  up the critical infrastructure (like having a staff) to perform that mission.

Of course, you also need to know my priorities.

  1. Consensus governance helps ensure minority views get heard while we still act as a unified body.  Consensus includes formalized agendas, rules (aka Robert’s rules) , clarifying motions and simple actions to make it easier for the community to follow.
  2. More community integration in the form of work done in subcomittees that can bring in external voices and integration with technical and user committees.
  3. Make activities more consistent, visible and accessible.  While our actions are open, our practices (and audio bridges)  make it difficult for the community to follow along.  That includes faster turn around on minutes so that board actions are not subject to twitter extrapolation.

The board is still very young and I’m impressed with what we’ve accomplished so far.

OpenStack Board needs Consensus Governance

Posted on by
6

I am humbled by the community support for my election (I finished first in the results) and have been surprised to realize that one of my unlisted credentials, 5 years as Secretary of our local public Community Development Corporation, could also be an important asset. Dealing with Texas Open Government laws around parliamentary minutia such as open discussions, voting, minutes and agendas turns out to translate directly to open source governance (which affects everyone!).

I believe that the OpenStack Board should operate by Consensus rules.

Boards can choose to operate either by Consensus or Majority. A Consensus board typically passes all resolutions unanimously while a Majority board does not need agreement on decisions (see table below).

On first blush, majority process seems more efficient; unfortunately, split votes are divisive and polarizing. The consequence of split votes is the minority positions will seek longer debate, resort to back room politics and procedural overhead. This type of behavior would be destructive for our community.

A Consensus board, which only happens by implied agreement of the members and leadership, works to ensure that decisions can be supported by all the members. This does not mean that all the members agree with the board positions, hold-hands during meetings, or participate in Polynesian drum circles! It does mean that the board as a whole considers minority positions and their motivation before calling a vote. If there is too much difference in opinion, then the majority may defer voting or minority members may abstain from voting. One common aspect of Consensus boards is that members may appear to argue against their own positions to ensure that minority views have been represented.

While the consensus model takes discipline for our Directors, it also takes patience and cooperation from the community that we serve. Board actions may take longer or be less direct than members of the community desire.

I believe that committing to a Consensus board is essential for OpenStack because our board is large (24 members!), our community is diverse and the financial impacts to members are high. So far, I’m proud that we’ve been following that model and will try to ensure we maintain that tradition.

Post Script Table: Consensus vs. Majority Governance Snapshot

Consensus Majority
Voting Unanimous Split
Process Flexible Strict
Position in Discussions Ambiguous Polarized
Controversy Avoided / Postponed Forced / Decisive Wins
Community Encouraged Divided
Minority Interests Incorporated Excluded
Board Unity High Low

I am seeking your vote(s) for the OpenStack Board

Posted on by
4

If registered, you have 8 votes to allocate as you wish.  You will get a link via email – you must use that link.

Joseph B George and I are cross-blogging this post because we are jointly seeking your vote(s) for individual member seats on the OpenStack Foundation board.  This is key point in the OpenStack journey and we strongly encourage eligible voters to participate no matter who you vote for!  As we have said before, success of the Foundation governance process matters just as much as the code because it ensures equal access and limits forking.

We think that OpenStack succeeds because it is collaboratively developed.  It is essential that we select board members who have a proven record of community development, a willingness to partner and have demonstrated investment in the project.

Our OpenStack vision favors production operations by being operator, user and ecosystem focused.  If elected, we will represent these interests by helping advance deployability, API specifications, open operations and both large and small scale cloud deployments.

Of the nominees, we best represent OpenStack users and operators (as opposed to developers).  We have the most diverse experience in real-world OpenStack deployments because our solution has been deployed broadly (both as Dell and through Crowbar.  We have a proven record of collaborating broadly with contributors, demonstrated skills at building the OpenStack community and doing real open source work to ensure that OpenStack is the most deployable cloud platform anywhere.

Let’s get specific about our leadership in the OpenStack project and community:

  • We have been active and vocal leaders in the OpenStack community
    • our team has established two very active user groups (Austin & Boston)
    • we have lead multiple world-wide deploy day events (March 2012  &  May 2012).
    • we have substantial experience in the field and know the challenges of running OpenStack for a wide variety of real-world deployments
    • our first solution came out on Cactus!  We’ve been delivering on Essex since OSCON 2012 (http://www.oscon.com/ ).
  • We represent a broad range of deployment scenarios ranging from hosting, government, healthcare, retail, education, media, financial and more!
  • We have broad engagements and partnerships at the infrastructure (SUSE, Canonical, Redhat), consulting (Canonical, Mirantis) and ecosystem layers (enStratus) and beyond!
  • We have a proven track record of collaboration instead of forking/disrupting – a critical skill for this project reflected by our consistent actions to preserve the integrity of the project.
  • We have led the “make OpenStack deployable” campaign with substantial investments (open source Crowbar, white papers, documentation & cookbooks.
  • We have very long and consistent history with the project starting even before the first OpenStack summit in Austin.

Of course, we’re asking for you to consider for both of us; however, if you want to focus on just one then here’s the balance between us.  Rob (bio) is a technologist with deep roots in cloud technology, data center operations and open source.  Joseph is a business professional with experience new product introduction and enterprise delivery.

Not sure if you can vote?  If you registered as an individual member then your name should be on the voting list.  In that case, you can vote between 8/20 and 8/24.

Crowbar 2.0 Design Summit Notes (+ open weekly meetings starting)

Posted on by
1

I could not be happier with the results Crowbar collaborators and my team at Dell achieved around the 1st Crowbar design summit. We had great discussions and even better participation.

The attendees represented major operating system vendors, configuration management companies, OpenStack hosting companies, OpenStack cloud software providers, OpenStack consultants, OpenStack private cloud users, and (of course) a major infrastructure provider. That’s a very complete cross-section of the cloud community.

I knew from the start that we had too little time and, thankfully, people were tolerant of my need to stop the discussions. In the end, we were able to cover all the planned topics. This was important because all these features are interlocked so discussions were iterative. I was impressed with the level of knowledge at the table and it drove deep discussion. Even so, there are still parts of Crowbar that are confusing (networking, late binding, orchestration, chef coupling) even to collaborators.

In typing up these notes, it becomes even more blindingly obvious that the core features for Crowbar 2 are highly interconnected. That’s no surprise technically; however, it will make the notes harder to follow because of knowledge bootstrapping. You need take time and grok the gestalt and surf the zeitgeist.

Collaboration Invitation: I wanted to remind readers that this summit was just the kick-off for a series of open weekly design (Tuesdays 10am CDT) and coordination (Thursdays 8am CDT) meetings. Everyone is welcome to join in those meetings – information is posted, recorded, folded, spindled and mutilated on the Crowbar 2 wiki page.

These notes are my reflection of the online etherpad notes that were made live during the meeting. I’ve grouped them by design topic.

Introduction

  • Contributors need to sign CLAs
  • We are refactoring Crowbar at this time because we have a collection of interconnected features that could not be decoupled
  • Some items (Database use, Rails3, documentation, process) are not for debate. They are core needs but require little design.
  • There are 5 key topics for the refactor: online mode, networking flexibility, OpenStack pull from source, heterogeneous/multi operating systems, being CDMB agnostic
  • Due to time limits, we have to stop discussions and continue them online.
  • We are hoping to align Crowbar 2 beta and OpenStack Folsom release.

Online / Connected Mode

  • Online mode is more than simply internet connectivity. It is the foundation of how Crowbar stages dependencies and components for deploy. It’s required for heterogeneous O/S, pull from source and it has dependencies on how we model networking so nodes can access resources.
  • We are thinking to use caching proxies to stage resources. This would allow isolated production environments and preserves the run everything from ISO without a connection (that is still a key requirement to us).
  • Suse’s Crowbar fork does not build an ISO, instead it relies on RPM packages for barclamps and their dependencies.
  • Pulling packages directly from the Internet has proven to be unreliable, this method cannot rely on that alone.

Install From Source

  • This feature is mainly focused on OpenStack, it could be applied more generally. The principals that we are looking at could be applied to any application were the source code is changing quickly (all of them?!). Hadoop is an obvious second candidate.
  • We spent some time reviewing the use-cases for this feature. While this appears to be very dev and pre-release focused, there are important applications for production. Specifically, we expect that scale customers will need to run ahead of or slightly adjacent to trunk due to patches or proprietary code. In both cases, it is important that users can deploy from their repository.
  • We discussed briefly our objective to pull configuration from upstream (not just OpenStack, but potentially any common cookbooks/modules). This topic is central to the CMDB agnostic discussion below.
  • The overall sentiment is that this could be a very powerful capability if we can manage to make it work. There is a substantial challenge in tracking dependencies – current RPMs and Debs do a good job of this and other configuration steps beyond just the bits. Replicating that functionality is the real obstacle.

CMDB agnostic (decoupling Chef)

  • This feature is confusing because we are not eliminating the need for a configuration management database (CMDB) tool like Chef, instead we are decoupling Crowbar from the a single CMDB to a pluggable model using an abstraction layer.
  • It was stressed that Crowbar does orchestration – we do not rely on convergence over multiple passes to get the configuration correct.
  • We had strong agreement that the modules should not be tightly coupled but did need a consistent way (API? Consistent namespace? Pixie dust?) to share data between each other. Our priority is to maintain loose coupling and follow integration by convention and best practices rather than rigid structures.
  • The abstraction layer needs to have both import and export functions
  • Crowbar will use attribute injection so that Cookbooks can leverage Crowbar but will not require Crowbar to operate. Crowbar’s database will provide the links between the nodes instead of having to wedge it into the CMDB.
  • In 1.x, the networking was the most coupled into Chef. This is a major part of the refactor and modeling for Crowbar’s database.
  • There are a lot of notes captured about this on the etherpad – I recommend reviewing them

Heterogeneous OS (bare metal provisioning and beyond)

  • This topic was the most divergent of all our topics because most of the participants were using some variant of their own bare metal provisioning project (check the etherpad for the list).
  • Since we can’t pack an unlimited set of stuff on the ISO, this feature requires online mode.
  • Most of these projects do nothing beyond OS provisioning; however, their simplicity is beneficial. Crowbar needs to consider users who just want a stream-lined OS provisioning experience.
  • We discussed Crowbar’s late binding capability, but did not resolve how to reconcile that with these other projects.
  • Critical use cases to consider:
    • an API for provisioning (not sure if it needs to be more than the current one)
    • pick which Operating Systems go on which nodes (potentially with a rules engine?)
    • inventory capabilities of available nodes (like ohai and factor) into a database
    • inventory available operating systems