Category Archives: Clouds

Seeking OpenStack Foundation Board Seat for 2013 (please nominate me)

Posted on by
5
I am seeking another term on the OpenStack Foundation Board.  Please consider nominating me for this position.
The following is the profile information I provided as part of the nomination process.  If you are looking for insights into where OpenStack is going then these question (especially the later ones) will be interesting.

Provide Brief Biography of Yourself

I have been involved in Cloud for over 12 years and launch some of the earliest Cloud companies.  My educational background (Duke and LSU) is in computer science and systems engineering (Mechanical/Industrial) with a focus on distributed systems.  I have always found deployment to be vitally important in development – that lead me to found a SaaS start-up in 1999 and had made me a DevOps advocate.  In addition to core cloud technologies, I am an Agile/Lean process evangelist who strongly believes that how you build and deliver is just as important as what you deliver.
Currently, I am a principal engineer at Dell leading our OpenStack Cloud project  and also a founder of the Crowbar project.  In that role, I am in constant contact with OpenStack users, ecosystem developers and vendors world-wide; consequently, I have a very broad perspective on use and technical needs for OpenStack and related Cloud technologies.

What is your relationship to OpenStack, and why is its success important to you? What would you say is your biggest contribution to OpenStack’s success to date?

I have been involved in OpenStack at the earliest stages and was a key influencer in Dell’s decision to be an initial sponsor.  Further, I formulated Dell’s operations/deployment focused Lean strategy that helped create an early focus on OpenStack operations.   This support was a critical catalyst to building market momentum and we continue help drive operations and user focused requirements for OpenStack.  Providing a very open and community driven DevOps focus has been my biggest contribution (see Crowbar).
I also serve the community in many ways.  In addition to being elected to the 2012 Board, I founded the Austin OpenStack User Group (next meeting 12/6!), have been spoken at every Summit, co-Chaired the Operations Track at the Grizzly Summit and create community awareness of OpenStack through my blog, corporate work, and social media activity.
I have invested considerable time in OpenStack and made professional commitments to its success.  It’s not just about software and an awesome community – I am personally invested in OpenStack winning.

Describe your experience with other non profits or serving as a board member. How does your experience prepare you for the role of a board member?

I am currently serving on the OpenStack Foundation Board and have been an active advocate for a collaborative process and open communication (http://wp.me/pF6d2-vc, http://wp.me/pF6d2-w9).  I hope continue on the board so that we can stay focused on the critical issues at hand.

I have over four years of serving as Secretary for my city’s public Texas 4-B Commercial Development Corporation which governed by state open meetings standards.  This work has proven directly relevant to the OpenStack foundation because the open governance requirements map very well to our communities transparency expectations.

What do you see as the Board’s role in OpenStack’s success?

The board’s job is to ensure that community and collaboration remain OpenStack’s core strength as an open source project.  As adoption and footprint increases there is enormous pressure on the community to try and serve both general and very specialized interests.  Most critically, we must find ways to balance competing financial interests within the community.
The board must ensure that OpenStack provides commercial opportunity because I believe that incent continued investment; however, we cannot let profit drive the community away from our open source values.
I believe the Board must monitor the community’s progress to ensure we maintain this balance.  It is our responsibility to make adjustments, influence changes and take responsibility for driving an innovative and collaborative culture.

What do you think the top priority of the Board should be in 2013?

There are two inter-related top priorities for the Board: we must help deliver guidelines for which projects are truly “core” OpenStack and help move towards a certifiable API specification for OpenStack (instead of implementation).   While distinct, I believe both items must be solved together.  These changes are essential to foster innovation and adoption of OpenStack.
While these issues will occupy the Board (and Technical/User committees) for 2013, my personal priority for OpenStack in general remains focused on operators and users.  I believe we have to make substantial progress on upgrades, migration and operational readiness.  These issues continue to create a serious barrier to adoption.

Impressions from OpenStack Foundation board meeting 10/15

Posted on by
1

20121018-115319.jpgI spent the first day of the OpenStack summit doing my service as a Foundation board member. While you need to wait for a summary by Jonathan Bryce, the Foundation Executive Director, and/or the official minutes to get the details of our actions, I think that it’s reasonable to share my impressions.

In some ways, the chemistry of the board is as important as our actions at this point. We are still working thorough start up issues and, more importantly, learning how to work together. The board is not only big, 24 members, it is also has many companies that often compete (power of we).

Here are my top 5 impressions:

    1. Transparency. The bylaws are crafted to ensure a openness and our meetings are broadcast to the maximum extent possible. Even with that background, there is a consistent self-check and discussion about increasing transparency so that the community is included.
    2. Humor. It’s a great sign of progress that we are laughing together because it shows that we trust and respect each other.
    3. Frustration. Recognition that we have important decisions to make and a degree of impatience to make them. Boards are subtle: we spend a lot of time setting up the right structures that allow us to make hard decisions quickly. Also, grumbling that we’d overlapped the board meeting and the summit.
    4. Consensus. This something I committed to help build. I feel that our actions reflect both healthy discussion as individuals and desire to work together as a board.
    5. Leadership. Looking back from our first meeting, it is clear that Jonathan and Alan have fully assumed their roles. I’m also seeing how the Foundation team (Mark, Lauren, Stephano) are smoothly supporting operations even though we’re really just weeks from the Foundation launch.

I hope that gives you some insights into the board meeting and that more of you can join the broadcast for the next ones (2012 schedule TBD) including a joint tech and foundation meeting

Wednesday morning at the summit was “breakfast with the board.” Look for some notes from that too soon.

Open Source is The Power of We (Blog Action Day)

Posted on by
2

This post is part of a world wide “blog action day” where thousands of bloggers post their unique insights about a single theme. For 2012, it’s the “power of we is as a celebration of people working together to make a positive difference in the world, either for their own communities or for people they will never meet half way around the world.”

I’ve choosing open source software because I think that we are establishing models for building ideas collaboratively that can be extended beyond technology into broader use. The way we solve open source challenges translates broadly because we are the tool makers of the global interaction.

I started using open source¹ as a way to solve a problem; I did not understand community or how groups of loosely connected people came together to create something new. Frankly, the whole process of creating free software seemed to be some hybrid combination of ninja coders and hippy hackers. That changed when I got involve on the ground floor of the OpenStack project (of which I am now a Foundation board member).

I was not, could not have been, prepared for the power and reality of community and collaboration that fuels OpenStack and other projects. We have the same problems as any non-profit project except that we are technologists: we can make new tools to solve our teaming and process problems.

It is not just that open source projects solve problems that help people. The idea of OpenStack and Hadoop being used by medical researches to find cures for cancer is important; however, the learning how to build collaboratively is another critical dimension. Our world is getting more connected and interconnected by technology, but the actual tools for social media are only in their earliest stages.

Not only are the tools evolving, the people using the tools are changing too! We are training each other to work together in ways that were beyond our imagine even 10 years ago. It’s the combination of both new technology and new skills that is resetting the rules for collaboration.

Just a few years ago, open source technology was considered low quality, risky and fringe. Today, open source projects like OpenStack and Hadoop are seen as more innovative and equally secure and supportable compared to licensed products. This transformation represents a surprising alignment and collaboration between individuals and entities that would normally be competing. While the motivation for this behavior comes from many sources, we all share the desire to do collaborative effectively.

I don’t think that we have figured out how to really do this the best way yet. We are making progress and getting better and better. We are building tools (like etherpad, wikis, irc, twitter, github, jenkins, etc) that improve collaboration. We are also learning building a culture of collaboration.

Right now, I’m on a train bound for the semi-annual OpenStack summit that brings a world wide audience together for 4½ days of community work. The discussions will require a new degree of openness from people and companies that are normally competitive and secretive about product development. During the summit, we’ll be doing more than designing OpenStack, we will be learning the new skills of working together. Perhaps those are the most important deliverables.

Open source projects combination of both new technologies and new skills creates the Power of We.

——————

PS¹: Open source software is a growing class of applications in which the authors publish the instructions for running the software publicly so that other people can use the software. Sometimes (but not always) this includes a usage license that allows other people to run the software without paying the author royalties. In many cases, the author’s motivation is that other users will help them test, modify and improve the software so that improves more quickly than a single creator could do alone.

Crowbar releases for OpenStack Essex and Folsom

Posted on by
6

On the eve of the OpenStack design summit, it’s worth noting that the Crowbar team at Dell cut our final Essex release (aka Betty) last week. We’ve also committed the initial Folsom deployment scripts to the 1x development trunk under “feature/folsom” if you are doing Crowbar builds from DevTool (see bit.ly/crowbardevtool).

If you don’t want to build it, I’ve cut ISOs from the open source bits and posted them to http://crowbar.zehicle.com.

Andi Abes is presenting about the new Pull From Source (pfs) feature at the Summit on Monday. There’s a feature branch for that too and I’m going to check with him and try to post and ISO for that too.

20121014-132246.jpg

OpenStack Summit: Let’s talk DevOps, Fog, Upgrades, Crowbar & Dell

Posted on by
7

If you are coming to the OpenStack summit in San Diego next week then please find me at the show! I want to hear from you about the Foundation, community, OpenStack deployments, Crowbar and anything else.  Oh, and I just ordered a handful of Crowbar stickers if you wanted some CB bling.

Matt Ray (Opscode), Jason Cannavale (Rackspace) and I were Ops track co-chairs. If you have suggestions, we want to hear. We managed to get great speakers and also some interesting sessions like DevOps panel and up streaming deploy working sessions. It’s only on Monday and Tuesday, so don’t snooze or you’ll miss it.

My team from Dell has a lot going on, so there are lots of chances to connect with us:

At the Dell booth, Randy Perryman will be sharing field experience about hardware choices. We’ve got a lot of OpenStack battle experience and we want to compare notes with you.

I’m on the board meeting on Monday so likely occupied until the Mirantis party.

See you in San Diego!

PS: My team is hiring for Dev, QA and Marketing. Let me know if you want details.

Big Data to tame Big Government? The answer is the Question.

Posted on by
Reply

Today my boss at Dell, John Igoe, is part of announcing of the report from the TechAmerica Federal Big Data Commission (direct pdf), I was fully expecting the report to be a real snoozer brimming with corporate synergies and win-win externalities. Instead, I found myself reading a practical guide to applying Big Data to government. Flipping past the short obligatory “what is…” section, the report drives right into a survey of practical applications for big data spanning nearly every governmental service. Over half of the report is dedicated to case studies with specific recommendations and buying criteria.

Ultimately, the report calls for agencies to treat data as an asset. An asset that can improve how government operates.

There are a few items that stand out in this report:

  1. Need for standards on privacy and governance. The report calls out a review and standardization of cross agency privacy policy (pg 35) and a Chief Data Officer position each agency (pg 37).
  2. Clear tables of case studies on page 16 and characteristics on page 11 that help pin point a path through the options.
  3. Definitive advice to focus on a single data vector (velocity, volume or variety) for initial success on page 28 (and elsewhere)

I strongly agree with one repeated point in the report: although there is more data available, our ability to comprehend this data is reduced. The sheer volume of examples the report cites is proof enough that agencies are, and will be continue to be, inundated with data.

One short coming of this report is that it does not flag the extreme storage of data scientists. Many of the cases discussed assume a ready army of engineers to implement these solutions; however, I’m uncertain how the government will fill positions in a very tight labor market. Ultimately, I think we will have to simply open the data for citizen & non-governmental analysis because, as the report clearly states, data is growing faster than capability to use it.

I commend the TechAmerica commission for their Big Data clarity: success comes from starting with a narrow scope. So the answer, ironically, is in knowing which questions we want to ask.

Continuous Release combats disruptions of “Free Fall” development

Posted on by
Reply

Since I posted the “Free Fall” development post, I’ve been thinking a bit about the pros and cons of this type of off-release development.

The OpenStack Swift project does not do free fall because they are on a constant “ship ready” state for the project and only loosely flow the broader OpenStack release track.  My team at Dell also has minimal free fall development because we have a more frequent release clock and choose to have the team focus together through dev/integrate/harden cycles as much as possible.

From a Lean/Agile/CI perspective, I would work to avoid hidden development where possible.  New features are introduced by split test (they are in the code, but not active for most users) so that the all changes in incremental.  That means that refactoring, rearchitecture and new capabilities appear less disruptively.  While it may this approach appears to take more effort in the short term; my experience is that it accelerates delivery because we are less likely to over develop code.

Unfortunately, free fall development has the opposite effect.  Having code that appears in big blocks is contrary to best practices in my opinion.  Further, it rewards groups that work asynchronously and

While I think that OpenStack benefits from free fall work, I think that it is ultimately counter-productive.

Ballistic Release Cycles: Tracking the Trajectory of OpenStack Milestones

Posted on by
9

I’ve been watching a pattern emerge on the semiannual OpenStack release cycles for a while now. There is a hidden but crucial development phase that accelerates projects faster than many observers realize. In fact, I believe that substantial work is happening outside of the “normal” design cycle during what I call “free fall” development.

Understanding when the cool, innovative stuff happens is essential to getting (and giving) the most from OpenStack.

The published release cycle looms like a 6 stage ballistic trajectory. Launching at the design summit, the release features change and progress the most in the first 3 milestones. At the apogee of the release, maximum velocity is reached just as we start having to decide which features are complete enough to include in the release. Since many are not ready, we have to jettison (really, defer) partial work to ensure that we can land the release on schedule.

I think of the period where we lose potential features as free fall because thing can go in any direction. The release literally reverses course: instead of expanding, it is contracting. This process is very healthy for OpenStack. It favors code stability and “long” hardening times. For operators, this means that the code stops changing early enough that we have more time to test and operationalize the release.

But what happens to the jettisoned work? In free fall, objects in motion stay in motion. The code does not just disappear! It continues on its original upward trajectory.

The developers who invested time in the code do not simply take a 3 month sabbatical, nor do they stop their work and start testing the code that was kept. No, after the short in/out sorting pause, the free fall work continues onward with rockets blasting. The challenge is that it is now getting outside of the orbit of the release plan and beyond the radar of many people who are tracking the release.

The consequence of this ongoing development is that developers (and the features they are working on) show up at the summit with 3 extra months of work completed. It also means that OpenStack starts each release cycle with a bucket of operationally ready code. Wow, that’s a huge advantage for the project in terms of delivered work, feature velocity and innovation. Even better, it means that the design summit can focus on practical discussions of real prototypes and functional features.

Unfortunately, this free fall work has hidden costs:

  • It is relatively hidden because it is outside of the normal release cycle.
  • It makes true design discussions less productive because the implemented code is more likely to make the next release cycle
  • Integration for the work is postponed because it continues before branching
  • Teams that are busy hardening a core feature can be left out of work on the next iteration of the same feature
  • Forking can make it hard to capture bugs caught during hardening

I think OpenStack greatly benefits from free fall development; consequently, I think we need to acknowledge and embrace it to reduce its costs. A more explicit mid-release design synchronization when or before we fork may help make this hidden work more transparent.

OSCBM Seeking Community Input for Long Board Meetings and Candlelight Coding Sessions

Posted on by
1

Happy OpenStack Foundation Launch Day! I’m a little breathless at OpenStack’s sponsored sprint to foundationhood but very proud to be part of the process (you can be too!).   Just looking at the numbers it’s clear that we’re building something important.

While it’s important that OpenStack is innovative, stable and useful cloud infrastructure, it’s equally important the project in collaborative developed.

Collaborative development makes it safe for so many diverse commercial interests to participate.  The Foundation, with a gold and platinum war chest,  is a reflection of the need for the project to remain both openly collaborative and commercially successful for our community.  We must ensure a level technology playing field while we work to ensure members of the community can be commercially successful while contributing.  This balance is one of our core challenges.

As an OpenStack Community Board Member (OSCBM), I want to hear what you think the OpenStack Foundation should be doing for OpenStack!

It is vital that I get input from the OpenStack community!  Unlike 2/3 of the Board, my seat is decided by the community and (re)elected by the community on an annual basis; consequently, it is my responsibility to voice Stackers’ interest, not my employer’s (Dell).

Frankly, the project is still hugely dominated by developers with users/operators only just gaining influence. The Foundation’s primary purpose is to help safe guard its independence. As a board member, my job today is to oversee building  up the critical infrastructure (like having a staff) to perform that mission.

Of course, you also need to know my priorities.

  1. Consensus governance helps ensure minority views get heard while we still act as a unified body.  Consensus includes formalized agendas, rules (aka Robert’s rules) , clarifying motions and simple actions to make it easier for the community to follow.
  2. More community integration in the form of work done in subcomittees that can bring in external voices and integration with technical and user committees.
  3. Make activities more consistent, visible and accessible.  While our actions are open, our practices (and audio bridges)  make it difficult for the community to follow along.  That includes faster turn around on minutes so that board actions are not subject to twitter extrapolation.

The board is still very young and I’m impressed with what we’ve accomplished so far.

Do Be Dense! Dell C8000 unit merges best of bladed and rackable servers

Posted on by
Reply

“Double wide” is not a term I’ve commonly applied to servers, but that’s one of the cool things about this new class of servers that Dell, my employer, started shipping today.

My team has been itching for the chance to start cloud and big data reference architectures using this super dense and flexible chassis. You’ll see it included in our next Apache Hadoop release and we’ve already got customers who are making it the foundation of their deployments (Texas Adv Computing Center case study).

If you’re tracking the latest big data & cloud hardware then the Dell PowerEdge C8000 is worth some investigation.

Basically, the Dell C8000 is a chassis that holds a flexible configuration of compute or storage sleds. It’s not a blade frame because the sleds minimize shared infrastructure. In our experience, cloud customers like the dedicated i/o and independence of sleds (as per the Bootstrapping clouds white paper). Those attributes are especially well suited for Hadoop and OpenStack because they support a “flat edges” and scale out design. While i/o independence is valued, we also want shared power infrastructure and density for efficiency reasons. Using a chassis design seems to capture the best of both worlds.

The novelty for the Dell PowerEdge C8000 is that the chassis are scary flexible. You are not locked into a pre-loaded server mix.

There are a plethora of sled choices so that you can mix choices for power, compute density and spindle counts. That includes double-wide sleds positively brimming with drives and expanded GPU processers. Drive density is important for big data configurations that are disk i/o hungry; however, our experience is the customer deployments are highly varied based on the planned workload. There are also significant big data trends towards compute, network, and balanced hardware configurations. Using the C8000 as a foundation is powerful because it can cater to all of these use-case mixes.

That reminds me! Mike Pittaro (our team’s Hadoop lead architect) did an excellent Deploy Hadoop using Crowbar video.

Interested in more opinions about the C8000? Check out Barton George & David Meyer.