Category Archives: Open source

From orphans to open source, data matters

Posted on by
2

TMF ChildrenMy wife’s day job helps Indian orphans through the Miracle Foundation here in Austin. On the surface, our jobs are very different; however, there is lately more and more intersection in both form and substance. It was not always like that, initially the Miracle Foundation primary engagement had been an emotional appeal: “look, these orphans are sad, they need you. Did we mention that they are orphans?”

Joking aside, there are plenty of kind people who want to help children; however, there are a lots of worthy causes with equally strong appeal. The question is how do you pick which one? Donors/Contributors want one that is both emotionally appealing and effective.

While radically different in human impact, both of raising orphans and building open source rely heavily on personal engagement and passion for success. Just like non-profits, there are many open source projects that want you to invest your time in installing and contributing to their most worthy technology.

About 18 months ago, the Miracle Foundation pivoted their strategy from tending individual children towards cultivating whole orphanages (the “NEST program”, video below). They started tracking things like how much milk and fruit each child ate and if they had been vaccinated. They connected observable data like hemoglobin levels of children to their ability to pay attention in school. They were even aware of additional days girls spent in school just because they got monthly hygiene products.

NEST Spider Graph

Used with Permission, The Miracle Foundation

With this new program, the Miracle Foundation can tell you exactly how much benefit each child will receive from each dollar. These are real results derived from collecting real data, and the results are powerful.

The children the Miracle Foundation nurtures are going from subsistence to flourishing. This is not happening because people care more about these children than before. It is happening because someone is keeping the data and making sure that the support they give gets the results they want. This in turn helps donors (become one) feel confident that their emotional response is delivering tangible improvements. Both are essential to TMF’s mission.

Open source projects have a similar gestalt.

People and companies contributing time and resources to a project want to both believe in the technology and see tangible metrics to validate adoption. Open source transparency makes it easier to find active projects and people are engaged contributors, but it can be harder to determine if the project is having broader impact.

For OpenStack, these tangible metrics began to surface in the last few days. Before the summit, Stephano Muffulli, community manager, launched the OpenStack Activity Board to show commit and quality data for the project. Last Monday, Tim Bell & Ryan Lane presented the results of the first user survey which showed how and what users are adopting for OpenStack.

If you like seeing this type of data driven behavior then vote with your keyboard and become part of an active open source project. For non-profits like the Miracle Foundation, voting is even easier – you just need a credit card to join in their Mothers’ Day campaign. Your mom may not understand anything you add to open source, but she can understand when you help orphans.

Continue reading →

OpenStack’s next hurdle: Interoperability. Why should you care?

Posted on by
6
SXSW life size Newton's Cradle

SXSW life size Newton’s Cradle

The OpenStack Board spent several hours (yes, hours) discussing interoperability related topics at the last board meeting.  Fundamentally, the community benefits when uses can operate easily across multiple OpenStack deployments (their own and/or public clouds).

Cloud interoperability: the ability to transfer workloads between systems without changes to the deployment operations management infrastructure.

This is NOT hybrid (which I defined as a workload transparently operating in multiple systems); however it is a prereq to achieve scalable hybrid operation.

Interoperability matters because the OpenStack value proposition is all about creating a common platform.  IT World does a good job laying out the problem (note, I work for Dell).  To create sites that can interoperate, we have to some serious lifting:

At the OpenStack Summit, there are multiple chances to engage on this.   I’m moderating a panel about Interop and also sharing a session about the highly related topic of Reference Architectures with Monty Tayor.

The Interop Panel (topic description here) is Tuesday @ 5:20pm.  If you join, you’ll get to see me try to stump our awesome panelists

  • Jonathan LaCour, DreamHost
  • Troy Toman, Rackspace
  • Bernard Golden,  Enstratius
  • Monty Taylor, OpenStack Board (and HP)
  • Peter Pouliot, Microsoft

PS: Oh, and I’m also talking about DevOps Upgrades Patterns during the very first session (see a preview).

Creating Communities: the intersection between Twitter celebrities and open source

Posted on by
Reply

calvin_leeOne of the unexpected perks of my Chevy SXSW experience was access to some real social media celebrities such as Josh Estrin, Calvin Lee, Kristin Brandt, Doug Mora, Samantha Needham and Jennie Chen.  They are all amazing, fun, wicked smart and NOT INTO CLOUD COMPUTING.

While I already knew Samantha (via Dell) and Jennie (via TechRanch), all of Chevy’s guests brought totally different perspectives to Chevy’s SXSW team ranging from pop culture  and mommies to hypermilers and gearheads.

The common thread is that we are all looking to engage our communities.

We each wanted to find something that would be interesting for our very different audiences to discuss.  That meant using our experiences at SXSW, Chevy and with each other to start a conversation within our communities.  We need good content as a seed but the goal is to drive the interaction.

Josh was the most articulate about this point saying that he measured his success when his followers talked to each other more than to him.   Being able to create content that engages people to do that is a true talent.

Calvin’s focus was more on helping people connect.  He felt successful when he was able to bring people together through his extended network. In those cases and others, the goals and challenges of a social media celebrity were remarkably similar to those helping lead open source projects.

In building communities, you must measure success in member communication and interaction.

If you are intent on being at the center of the universe then your project cannot grow; however, people also need celebrities to bring them together.  The amazing thing about the the people I met at SXSW through Chevy is that they managed to both attract the attention needed to build critical mass and get out of the way so communities could form around them. That’s a skill that we all should practice and foster.

PS: I also heard clearly that “I ate …” tweets are some of their most popular.  Putting on my collaboration hat: if you’re looking to engage a community then food is the most universal and accessible discussion topic.  Perhaps I’ll have to eat crow on that one.  

5 things keeping DevOps from playing well with others (Chef, Crowbar and Upstream Patterns)

Posted on by
6

Sharing can be hardSince my earliest days on the OpenStack project, I’ve wanted to break the cycle on black box operations with open ops. With the rise of community driven DevOps platforms like Opscode Chef and Puppetlabs, we’ve reached a point where it’s both practical and imperative to share operational practices in the form of code and tooling.

Being open and collaborating are not the same thing.

It’s a huge win that we can compare OpenStack cookbooks. The real victory comes when multiple deployments use the same trunk instead of forking.

This has been an objective I’ve helped drive for OpenStack (with Matt Ray) and it has been the Crowbar objective from the start and is the keystone of our Crowbar 2 work.

This has proven to be a formidable challenge for several reasons:

  1. diverging DevOps patterns that can be used between private, public, large, small, and other deployments -> solution: attribute injection pattern is promising
  2. tooling gaps prevent operators from leveraging shared deployments -> solution: this is part of Crowbar’s mission
  3. under investing in community supporting features because they are seen as taking away from getting into production -> solution: need leadership and others with join
  4. drift between target versions creates the need for forking even if the cookbooks are fundamentally the same -> solution: pull from source approaches help create distro independent baselines
  5. missing reference architectures interfere with having a stable baseline to deploy against -> solution: agree to a standard, machine consumable RA format like OpenStack Heat.

Unfortunately, these five challenges are tightly coupled and we have to progress on them simultaneously. The tooling and community requires patterns and RAs.

The good news is that we are making real progress.

Judd Maltin (@newgoliath), a Crowbar team member, has documented the emerging Attribute Injection practice that Crowbar has been leading. That practice has been refined in the open by ATT and Rackspace. It is forming the foundation of the OpenStack cookbooks.

Understanding, discussing and supporting that pattern is an important step toward accelerating open operations. Please engage with us as we make the investments for open operations and help us implement the pattern.

double Block Head with OpenStack+Equallogic & Crowbar+Ceph

Posted on by
9

Block Head

Whew….Yesterday, Dell announced TWO OpenStack block storage capabilities (Equallogic & Ceph) for our OpenStack Essex Solution (I’m on the Dell OpenStack/Crowbar team) and community edition.  The addition of block storage effectively fills the “persistent storage” gap in the solution.  I’m quadrupally excited because we now have:

  1. both open source (Ceph) and enterprise (Equallogic) choices
  2. both Nova drivers’ code is in the open at part of our open source Crowbar work

Frankly, I’ve been having trouble sitting on the news until Dell World because both features have been available in Github before the announcement (EQLX and Ceph-Barclamp).  Such is the emerging intersection of corporate marketing and open source.

As you may expect, we are delivering them through Crowbar; however, we’ve already had customers pickup the EQLX code and apply it without Crowbar.

The Equallogic+Nova Connector

block-eqlx

If you are using Crowbar 1.5 (Essex 2) then you already have the code!  Of course, you still need to have the admin information for your SAN – we did not automate the configuration of the storage system, but the Nova Volume integration.

We have it under a split test so you need to do the following to enable the configuration options:

  1. Install OpenStack as normal
  2. Create the Nova proposal
  3. Enter “Raw” Attribute Mode
  4. Change the “volume_type” to “eqlx”
  5. Save
  6. The Equallogic options should be available in the custom attribute editor!  (of course, you can edit in raw mode too)

Want Docs?  Got them!  Check out these > EQLX Driver Install Addendum

Usage note: the integration uses SSH sessions.  It has been performance tested but not been tested at scale.

The Ceph+Nova Connector

block-ceph

The Ceph capability includes a Ceph barclamp!  That means that all the work to setup and configure Ceph is done automatically done by Crowbar.  Even better, their Nova barclamp (Ceph provides it from their site) will automatically find the Ceph proposal and link the components together!

Ceph has provided excellent directions and videos to support this install.

My Dilemma with Folsom – why I want to jump to G

Posted on by
13

When your ship sailsThese views are my own.  Based on 1×1 discussions I’ve had in the OpenStack community, I am not alone.

If you’ve read my blog then you know I am a vocal and active supporter of OpenStack who is seeking re-election to the OpenStack Board.  I’m personally and professionally committed to the project’s success. And, I’m confident that OpenStack’s collaborative community approach is out innovating other clouds.

A vibrant project requires that we reflect honestly: we have an equal measure of challenges: shadow free fall Dev, API vs implementation, forking risk and others.  As someone helping users deploy OpenStack today, I find my self straddling between a solid release (Essex) and a innovative one (Grizzly). Frankly, I’m finding it very difficult to focus on Folsom.

Grizzly excites me and clearly I’m not alone.  Based on pace of development, I believe we saw a significant developer migration during feature freeze free fall.

In Grizzly, both Cinder and Quantum will have progressed to a point where they are ready for mainstream consumption. That means that OpenStack will have achieved the cloud API trifecta of compute-store-network.

  • Cinder will get beyond the “replace Nova Volume” feature set and expands the list of connectors.
  • Quantum will get to parity with Nova Network, addresses overlapping VM IPs and goes beyond L2 with L3 feature enablement like  load balancing aaS.
  • We are having a real dialog about upgrades while the code is still in progress
  • And new projects like Celio and Heat are poised to address real use problems in billing and application formation.

Everything I hear about Folsom deployment is positive with stable code and significant improvements; however, we’re too late to really influence operability at the code level because the Folsom release is done.  This is not a new dilemma.  As operators, we seem to be forever chasing the tail of the release.

The perpetual cycle of implementing deployment after release is futile, exhausting and demoralizing because we finish just in time for the spotlight to shift to the next release.

I don’t want to slow the pace of releases.  In Agile/Lean, we believe that if something is hard then we do should it more.  Instead, I am looking at Grizzly and seeing an opportunity to break the cycle.  I am looking at Folsom and thinking that most people will be OK with Essex for a little longer.

Maybe I’m a dreamer, but if we can close the deployment time gap then we accelerate adoption, innovation and happy hour.  If that means jilting Folsom at the release altar to elope with Grizzly then I can live with that.

Seeking OpenStack Foundation Board Seat for 2013 (please nominate me)

Posted on by
5
I am seeking another term on the OpenStack Foundation Board.  Please consider nominating me for this position.
The following is the profile information I provided as part of the nomination process.  If you are looking for insights into where OpenStack is going then these question (especially the later ones) will be interesting.

Provide Brief Biography of Yourself

I have been involved in Cloud for over 12 years and launch some of the earliest Cloud companies.  My educational background (Duke and LSU) is in computer science and systems engineering (Mechanical/Industrial) with a focus on distributed systems.  I have always found deployment to be vitally important in development – that lead me to found a SaaS start-up in 1999 and had made me a DevOps advocate.  In addition to core cloud technologies, I am an Agile/Lean process evangelist who strongly believes that how you build and deliver is just as important as what you deliver.
Currently, I am a principal engineer at Dell leading our OpenStack Cloud project  and also a founder of the Crowbar project.  In that role, I am in constant contact with OpenStack users, ecosystem developers and vendors world-wide; consequently, I have a very broad perspective on use and technical needs for OpenStack and related Cloud technologies.

What is your relationship to OpenStack, and why is its success important to you? What would you say is your biggest contribution to OpenStack’s success to date?

I have been involved in OpenStack at the earliest stages and was a key influencer in Dell’s decision to be an initial sponsor.  Further, I formulated Dell’s operations/deployment focused Lean strategy that helped create an early focus on OpenStack operations.   This support was a critical catalyst to building market momentum and we continue help drive operations and user focused requirements for OpenStack.  Providing a very open and community driven DevOps focus has been my biggest contribution (see Crowbar).
I also serve the community in many ways.  In addition to being elected to the 2012 Board, I founded the Austin OpenStack User Group (next meeting 12/6!), have been spoken at every Summit, co-Chaired the Operations Track at the Grizzly Summit and create community awareness of OpenStack through my blog, corporate work, and social media activity.
I have invested considerable time in OpenStack and made professional commitments to its success.  It’s not just about software and an awesome community – I am personally invested in OpenStack winning.

Describe your experience with other non profits or serving as a board member. How does your experience prepare you for the role of a board member?

I am currently serving on the OpenStack Foundation Board and have been an active advocate for a collaborative process and open communication (http://wp.me/pF6d2-vc, http://wp.me/pF6d2-w9).  I hope continue on the board so that we can stay focused on the critical issues at hand.

I have over four years of serving as Secretary for my city’s public Texas 4-B Commercial Development Corporation which governed by state open meetings standards.  This work has proven directly relevant to the OpenStack foundation because the open governance requirements map very well to our communities transparency expectations.

What do you see as the Board’s role in OpenStack’s success?

The board’s job is to ensure that community and collaboration remain OpenStack’s core strength as an open source project.  As adoption and footprint increases there is enormous pressure on the community to try and serve both general and very specialized interests.  Most critically, we must find ways to balance competing financial interests within the community.
The board must ensure that OpenStack provides commercial opportunity because I believe that incent continued investment; however, we cannot let profit drive the community away from our open source values.
I believe the Board must monitor the community’s progress to ensure we maintain this balance.  It is our responsibility to make adjustments, influence changes and take responsibility for driving an innovative and collaborative culture.

What do you think the top priority of the Board should be in 2013?

There are two inter-related top priorities for the Board: we must help deliver guidelines for which projects are truly “core” OpenStack and help move towards a certifiable API specification for OpenStack (instead of implementation).   While distinct, I believe both items must be solved together.  These changes are essential to foster innovation and adoption of OpenStack.
While these issues will occupy the Board (and Technical/User committees) for 2013, my personal priority for OpenStack in general remains focused on operators and users.  I believe we have to make substantial progress on upgrades, migration and operational readiness.  These issues continue to create a serious barrier to adoption.

Open Source is The Power of We (Blog Action Day)

Posted on by
2

This post is part of a world wide “blog action day” where thousands of bloggers post their unique insights about a single theme. For 2012, it’s the “power of we is as a celebration of people working together to make a positive difference in the world, either for their own communities or for people they will never meet half way around the world.”

I’ve choosing open source software because I think that we are establishing models for building ideas collaboratively that can be extended beyond technology into broader use. The way we solve open source challenges translates broadly because we are the tool makers of the global interaction.

I started using open source¹ as a way to solve a problem; I did not understand community or how groups of loosely connected people came together to create something new. Frankly, the whole process of creating free software seemed to be some hybrid combination of ninja coders and hippy hackers. That changed when I got involve on the ground floor of the OpenStack project (of which I am now a Foundation board member).

I was not, could not have been, prepared for the power and reality of community and collaboration that fuels OpenStack and other projects. We have the same problems as any non-profit project except that we are technologists: we can make new tools to solve our teaming and process problems.

It is not just that open source projects solve problems that help people. The idea of OpenStack and Hadoop being used by medical researches to find cures for cancer is important; however, the learning how to build collaboratively is another critical dimension. Our world is getting more connected and interconnected by technology, but the actual tools for social media are only in their earliest stages.

Not only are the tools evolving, the people using the tools are changing too! We are training each other to work together in ways that were beyond our imagine even 10 years ago. It’s the combination of both new technology and new skills that is resetting the rules for collaboration.

Just a few years ago, open source technology was considered low quality, risky and fringe. Today, open source projects like OpenStack and Hadoop are seen as more innovative and equally secure and supportable compared to licensed products. This transformation represents a surprising alignment and collaboration between individuals and entities that would normally be competing. While the motivation for this behavior comes from many sources, we all share the desire to do collaborative effectively.

I don’t think that we have figured out how to really do this the best way yet. We are making progress and getting better and better. We are building tools (like etherpad, wikis, irc, twitter, github, jenkins, etc) that improve collaboration. We are also learning building a culture of collaboration.

Right now, I’m on a train bound for the semi-annual OpenStack summit that brings a world wide audience together for 4½ days of community work. The discussions will require a new degree of openness from people and companies that are normally competitive and secretive about product development. During the summit, we’ll be doing more than designing OpenStack, we will be learning the new skills of working together. Perhaps those are the most important deliverables.

Open source projects combination of both new technologies and new skills creates the Power of We.

——————

PS¹: Open source software is a growing class of applications in which the authors publish the instructions for running the software publicly so that other people can use the software. Sometimes (but not always) this includes a usage license that allows other people to run the software without paying the author royalties. In many cases, the author’s motivation is that other users will help them test, modify and improve the software so that improves more quickly than a single creator could do alone.

Join in Blog Action Day on 10/15

Posted on by
Reply

You’ll have to wait for the reveal on 10/15 but I wanted to throw out the link for blog action day and encourage fellow bloggers to participate in the event.

I participated in this event while I blogging about electric cars (I converted 96 RAV4 to EV) and energy issues.  It’s an interesting perspective to have a large focus on a single topic.  The breadth of discussion is impressive.

This year, I’m going to be topical to my latest interests and offering insights from the floor of the OpenStack summit.

Big Data to tame Big Government? The answer is the Question.

Posted on by
Reply

Today my boss at Dell, John Igoe, is part of announcing of the report from the TechAmerica Federal Big Data Commission (direct pdf), I was fully expecting the report to be a real snoozer brimming with corporate synergies and win-win externalities. Instead, I found myself reading a practical guide to applying Big Data to government. Flipping past the short obligatory “what is…” section, the report drives right into a survey of practical applications for big data spanning nearly every governmental service. Over half of the report is dedicated to case studies with specific recommendations and buying criteria.

Ultimately, the report calls for agencies to treat data as an asset. An asset that can improve how government operates.

There are a few items that stand out in this report:

  1. Need for standards on privacy and governance. The report calls out a review and standardization of cross agency privacy policy (pg 35) and a Chief Data Officer position each agency (pg 37).
  2. Clear tables of case studies on page 16 and characteristics on page 11 that help pin point a path through the options.
  3. Definitive advice to focus on a single data vector (velocity, volume or variety) for initial success on page 28 (and elsewhere)

I strongly agree with one repeated point in the report: although there is more data available, our ability to comprehend this data is reduced. The sheer volume of examples the report cites is proof enough that agencies are, and will be continue to be, inundated with data.

One short coming of this report is that it does not flag the extreme storage of data scientists. Many of the cases discussed assume a ready army of engineers to implement these solutions; however, I’m uncertain how the government will fill positions in a very tight labor market. Ultimately, I think we will have to simply open the data for citizen & non-governmental analysis because, as the report clearly states, data is growing faster than capability to use it.

I commend the TechAmerica commission for their Big Data clarity: success comes from starting with a narrow scope. So the answer, ironically, is in knowing which questions we want to ask.