Tag Archives: OpenStack

Hungry for Operational Excellence? ChefConf 2012 satisfies!

Posted on by
1

Since my team at Dell sponsored the inaugural ChefConf, we had the good fortune to get a handful of passes and show up at the event in force.  I was also tapped for a presentation (Chef+Crowbar gets Physical+OpenStack Cloud) and Ignite session (Crowbar history).

I live demo’ed using a single command window with knife to manage both physical and cloud infrastructure.    That’s freaking cool!  (and thanks to Matt Ray for helping to get this working)

It’s no surprise that I’m already a DevOps advocate and Opscode enthusiast, there were aspects of the conference that are worth reiterating:

  • Opscode is part of the cadre of leaders redefining how we operate infrastructure.  The energy is amazing.
  • The acknowledgement of the “snowflake” challenge where all Ops environments are alike, but no two are the same.
  • A tight integration between Operations and lean delivery because waterfall deployments are not sustainable
  • Opscode’s vision is rooted in utility.  You can be successful without design and then excel when you add it.  I find that refreshing.
  • There was a fun, friendly (“hug driven development?!”) and laid back vibe.  This group laughed A LOT.
  • For a first conference, Opscode did a good job with logistics and organization.
  • I saw that the back rooms and hallways are buzzing with activity.  This means that people are making money with the technology.

Crowbar + Chef installs & manages OpenStack Essex (Live Demo, 45 minutes):

 

Ignite Talk about Dell Crowbar History (5 minutes)

Four OpenStack Trends from Summit: Practical, Friendly, Effective and Deployable

Posted on by
1

With the next OpenStack Austin meetup on Thursday (sponsored by Puppet), I felt like it was past time for me to post my thoughts and observations about the Spring 2012 OpenStack design conference.  This was my fifth OpenStack conference (my notes about Bexar, Cactus, Diablo & Essex).  Every conference has been unique, exciting, and bigger than the previous.

My interest lies in the trend lines of OpenStack.  For details about sessions, I recommend Stefano Maffulli‘s  excellent link aggregation post for the Summit.

1. Technology Trend: Practical with Potential.

OpenStack started with a BIG vision to become the common platform for cloud API and operations.  That vision is very much alive and on-track; however, our enthusiasm for what could be is tempered by the need to build a rock solid foundation.  The drive to stability over feature expansion has had a very positive impact.  I give a lot of credit for this effort to the leadership of the project technical leads (PTLs), Canonical‘s drive to include OpenStack in the 12.04 LTS and the Rackspace Cloud drive to deploy Essex.  My team at Dell has also been part of this trend by focusing so much effort on making OpenStack production deployable (via Crowbar).

Overall, I am seeing a broad-based drive to minimize disruption.

2. Culture Trend: Friendly but some tension.

Companies at both large and small ends of the spectrum are clearly jockeying for position.  I think the market is big enough for everyone; however, we are also bumping into each other.  Overall, we are putting aside these real and imagined differences to focus on enlarging the opportunity of having a true community cloud platform.  For example, the OpenStack Foundation investment formation has moneyed competitors jostling for position to partner together.

However, it’s not just about paying into the club; OpenStack’s history is clearly about execution.  Looking back to the original Austin Summit sponsors, we’ve clearly seen that intent and commitment are different.

3. Discussion Trend: Small Groups Effective

The depth & quality of discussions inside sessions was highly variable.  Generally, I saw that large group discussions stayed at a very high level.  The smaller sessions required deep knowledge of the code to participate and seemed more productive.  We continue to have a juggle between discussions that are conceptual or require detailed knowledge of the code.  If conceptual, it’s too far removed.  If code, it becomes inaccessible to many people.

This has happened at each Summit and I now accept that it is natural.  We are using vision sessions to ensure consensus and working sessions to coordinate deliverables for the release.

I cannot over emphasize importance of small groups and delivery driven execution interactions: I spent most of my time in small group discussions with partners aligning efforts.

4. Deployment Trend: Testing and Upstreams matter

Operations for deploying OpenStack is a substantial topic at the Summit.  I find that to be a significant benefit to the community because there are a large block of us who were vocal advocates for deployability at the very formation of the project.

From my perspective at Dell, we are proud to see that wide spread acknowledgement of our open source contribution, Crowbar, as the most prominent OpenStack deployer.   Our efforts at making OpenStack installable are recognized as a contribution; however, we’re also getting feedback that we need to streamline and simplify Crowbar.  We also surprised to hear that Crowbar is “opinionated.”   On reflection, I agree (and am proud) of this assessment because it matches best practice coding styles.  Since our opinions also drive our test matrix there is a significant value for our OpenStack deployment is that we spend a lot of time testing (automated and manual) our preferred install process.

There’s a push to reconcile the various Chef OpenStack cookbooks into a single upstream.  This seems like a very good idea because it will allow various parties to collaborate on open operations.  The community needs leadership from Opscode to make this happen.  It appears that Puppet Labs is interested in playing a similar role for Puppet modules but these are still emerging and have not had a chance to fragment.

No matter which path we take, the deployment scripts are only as good as their level of testing.   Unreliable deployment scripts have are less than worthless.

Seven Cloud Success Criteria to consider before you pick a platform

Posted on by
1

From my desk at Dell, I have a unique perspective.   In addition to a constant stream of deep customer interactions about our many cloud solutions (even going back pre-OpenStack to Joyent & Eucalyptus), I have been an active advocate for OpenStack, involved in many discussions with and about CloudStack and regularly talk shop with Dell’s VIS Creator (our enterprise focused virtualization products) teams.  And, if you go back ten years to 2002, patented the concept of hybrid clouds with Dave McCrory.

Rather than offering opinions in the Cloud v. Cloud fray, I’m suggesting that cloud success means taking a system view.

Platform choice is only part of the decision: operational readiness, application types and organization culture are critical foundations before platform.

Over the last two years at Dell, I found seven points outweigh customers’ choice of platform.

  1. Running clouds requires building operational expertise both at the application and infrastructure layers.  CloudOps is real.
  2. Application architectures matter for cloud deployment because they can redefine the SLA requirements and API expectations
  3. Development community and collaboration is a significant value because sharing around open operations offers significant returns.
  4. We need to build an accelerating pace of innovation into our core operating principles
  5. There are still significant technology gaps to fill (networking & storage) and we will discover new gaps as we go
  6. We can no longer discuss public and private clouds as distinct concepts.   True hybrid clouds are not here yet, but everyone can already see their massive shadow.
  7. There is always more than one right technological answer.  Avoid analysis paralysis by making incrementally correct decisions (committing, moving forward, learning and then re-evaluating).

OpenStack Meetup 4/12: Austin at Summit, DevStack Essex

Posted on by
Reply

Austin Stackers!  This Thursday is our April meetup at the Austin TechRanch.

Please RSVP so that we know how much food to get!  SUSE is this Month’s sponsor for food and my team at Dell continues to pickup the room rental.  We have 35 RSVPs as of Monday noon – this will be another popular meeting (last meeting minutes).

Topics for the meetup are:

With the Summit next week, I think it is very important that we pre-discuss Summit topics and priorities as a community.  It will help us be more productive individually and for our collective interests when we engage the larger community next week.

OpenStack vs CloudStack? It’s about open innovation.

Posted on by
3

Yesterday, I got a short drive in a “Kick-Ass” Fisker Karma.  As someone who converted a car to electric, it was a great treat to see the amazing quality, polish and sophistication of the Karma.  Especially since, six years ago, I had to build my own EV.  Today there is a diversity of choices ranging from the Nissan, GM, Tesla, Aptera, Fisker and others.  Yet even with all these choices, EVs are far from the main stream.

How does that relate to Cloud *aaS?  It’s all about innovation cycles and adoption.

Cloud platforms (really, IaaS software) have transformed from DIY to vibrant projects in the last few years; however, we still don’t know what the finished products will look like – we are only at the beginning of the innovation cycle.

With yesterday’s Citrix’s “CloudStack joins Apache” announcement painted as a shot against OpenStack, it is tempting to get pulled into a polarized view of the right or wrong way to implement cloud software (NetworkWorld,  JavaWorld, CloudPundit).  I think that feature by feature comparisons miss the real dynamics of the cloud market.

The question is not about features today, it is about forward velocity tomorrow.  There are important areas needing technology development (network, storage, etc) in the cloud infrastructure space.  

So the real story, expressed eloquently by Thierry Carrez, is about open collaboration and the resulting pace of innovation.  That means that I consider all the cloud platforms in the market to be immature because we are still learning the scope of the “cloud” opportunity.

The critical question is how the various cloud projects will maintain growth and adopt innovation.  Like the current generation of EVs, we must both prove value in production and demonstrate our ability to learn and improve.

The Citrix decision to submit CloudStack to the Apache Foundation underscores this point: success of projects is about attracting collaboration and innovation.

From the perspective of building innovation and attracting developers, the tension between OpenStack and CloudStack is very real.

OpenStack’s global reach

Posted on by
3

The global reach of OpenStack has been clear from the very first “Austin” conference where we had participants from Europe and Asia; however, non-US adoption appears to be accelerating lately.

While this post is motivated by the Dell launch of my team‘s OpenStack-Powered Cloud Solution into select Europe (UK & Germany) and Asia (China) markets, it is just an indication of the overall acceleration.  This presence is bolstered by our relationships abroad (video from Martin Standtler @ Canonical).

Two weeks ago, our Essex Deploy Day gathered world wide participation that continues 24/7 on the Crowbar list and Skype channel.   Internally, I also see substantial interest and customer demand from customers and partners all over the Asia-Pacific region

I’m excited to see Dell (and the broader community) formalize international support for OpenStack.

Post Script: Hidden in that same release is some mention of our coming support for the latest  (12th!) generation of Dell servers (the R720xd and C6220).  I’m a software guy, but I have to admit that these new servers are very cool cloud nodes because they’ve got a sweet mix of fast CPU, high RAM limits and bodacious spindle counts.  These fit the hardware profile that we identified in the “bootstrapping open source clouds” white paper refresh.

OpenStack Austin: What we’d like to see at the Design Summit

Posted on by
2

Last week, the OpenStack Austin user group discussed what we’d like to see at the upcoming OpenStack Design Summit. We had a strong turnout (48?!).

  1. To get the meeting started, Marc Padovani from HP (this month’s sponsor) provided some lessons learned from the HP OpenStack-Powered Cloud. While Marc noted that HP has not been able to share much of their development work on OpenStack; he was able to show performance metrics relating to a fix that HP contributed back to the OpenStack community. The defect related to the scheduler’s ability to handle load. The pre-fix data showed a climb and then a gap where the scheduler simply stopped responding. Post-fix, the performance curve is flat without any “dead zones.” (sharing data like this is what I call “open operations“)
  2. Next, I (Rob Hirschfeld) gave a brief overview of the OpenStack Essex Deploy Day (my summary) that Dell coordinated with world-wide participation. The Austin deploy day location was in the same room as the meetup so several of the OSEDD participants were still around.
  3. The meat of the meetup was a freeform discussion about what the group would like to see discussed at the Design Summit. My objective for the discussion was that the Austin OpenStack community could have a broader voice is we showed consensus for certain topics in advance of the meeting.

At Jim Plamondon‘s suggestion, we captured our brain storming on the OpenStack etherpad. The Etherpad is super cool – it allows simultaneous editing by multiple parties, so the notes below were crowd sourced during the meeting as we discussed topics that we’d like to see highlighted at the conference. The etherpad preserves editors, but I removed the highlights for clarity.

The next step is for me to consolidate the list into a voting page and ask the membership to rank the items (poll online!) below.

Brain storm results (unedited)

Stablity vs. Features

API vs. Code

  • What is the measurable feature set?
  • Is it an API, or an implementation?
  • Is the Foundation a formal-ish standards body?
  • Imagine the late end-game: can Azure/VMWare adopt OPenStack’s APIs and data formats to deliver interop, without running OpenStack’s code? Is this good? Are there conversations on displacing incumbents and spurring new adoption?
  • Logo issues

Documentation Standards

  • Dev docs vs user docs
  • Lag of update/fragmentation (10 blogs, 10 different methods, 2 “work”)
  • Per release getting started guide validated and available prior or at release.

Operations Focus

  • Error messages and codes vs python stack traces
  • Alternatively put, “how can we make error messages more ops-friendly, without making them less developer-friendly?”
  • Upgrade and operations of rolling updates and upgrades. Hot migrations?

If OpenStack was installable on Windows/Hyper-V as a simple MSI/Service installer – would you try it as a node?

  • Yes.

Is Nova too big?  How does it get fixed?

  • libraries?
  • sections?
  • make it smaller sub-projects
  • shorter release cycles?

nova-volume

  • volume split out?
  • volume expansion of backend storage systems
  • Is nova-volume the canonical control plane for storage provisioning?  Regardless of transport? It presently deals in block devices only… is the following blueprint correctly targeted to nova-volume?

https://blueprints.launchpad.net/nova/+spec/filedriver

Orchestration

  • Is the Donabe project dead?

Discussion about invitations to Summit

  • What is a contribution that warrants an invitation
  • Look at Launchpad’s Karma system, which confers karma for many different “contributory” acts, including bug fixes and doc fixes, in addition to code commitments

Summit Discussions

  • Is there a time for an operations summit?
  • How about an operators’ track?
  • Just a note: forums.openstack.org for users/operators to drive/show need and participation.

How can we capture the implicit knowledge (of mailing list and IRC content) in explicit content (documentation, forums, wiki, stackexchange, etc.)?

Hypervisors: room for discussion?

  • Do we want hypervisor featrure parity?
  • From the cloud-app developer’s perspective, I want to “write once, run anywhere,” and if hypervisor features preclude that (by having incompatible VM images, foe example)
  • (RobH: But “write once, run anywhere” [WORA] didn’t work for Java, right?)
  • (JimP: Yeah, but I was one of Microsoft’s anti-Java evangelists, when we were actively preventing it from working — so I know the dirty tricks vendors can use to hurt WORA in OpenStack, and how to prevent those trick from working.)

CDMI

Swift API is an evolving de facto open alternative to S3… CDMI is SNIA standards track.  Should Swift API become CDMI compliant?  Should CDMI exist as a shim… a la the S3 stuff.

OpenStack Essex Deploy Day: First Steps to Production

Posted on by
7

One March 8th, 70 people from around the world gathered on the Crowbar IM channel to begin building a production grade OpenStack Essex deployment. The event was coordinated as meet-ups by the Dell OpenStack/Crowbar team (my team) in two physical locations: the Nokia offices in Boston and the TechRanch in Austin.

My objective was to enable the community to begin collaboration on Essex Deployment. At that goal, we succeeded beyond my expectations.

IMHO, the top challenge for OpenStack Essex is to build a community of deploying advocates. We have a strong and dynamic development community adding features to the project. Now it is time for us to build a comparable community of deployers. By providing a repeatable, shared and open foundation for OpenStack deployments, we create a baseline that allows collaboration and co-development. Not only must we make deployments easy and predictable, we must also ensure they are scalable and production ready.

Having solid open production deployment infrastructure drives OpenStack adoption.

Our goal on the 8th was not to deliver finished deployments; it was to the start of Essex deployment community collaboration. To ensure that we could focus on getting to an Essex baseline, our team invested substantial time before the event to make sure that participants had a working Essex reference deployment.

By the nature of my team’s event leadership and our approach to OpenStack, the event was decidedly Crowbar focused. I feel like this is an acceptable compromise because Crowbar is open and provides a repeatable foundation. If everyone has the same foundation then we can focus on the truly critical challenges of ensuring consistent OpenStack deployments. Even using Crowbar, we waste a lot of time trying to figure out the differences between configurations. Lack of baseline consistency seriously impedes collaboration.

The fastest way to collaborate on OpenStack deployment is to have a reference deployment as a foundation.

Success By The Numbers

This was a truly international community collaborative event. Here are some of the companies that participated:

Dell (sponsor), Nokia (sponsor), Rackspace, Opscode, Canonical, Fedora, Mirantis, Morphlabs, Nicira, Enstratus, Deutsche Telekom Innovation Laboratories, Purdue University, Orbital Software Solutions, XepCloud and others.

PLEASE COMMENT here if I missed your company and I will add it to the list.

On the day of the event, we collected the following statistics:

  • 70 people on Skype IM channel (it’s not too late to join by pinging DellCrowbar with “Essex barclamps”).
  • 14+ companies
  • 2 physical sites with 10-15 people at each
  • 4 fold increase in traffic on the Crowbar Github to 813 hits.
  • 66 downloads of the Deploy day ISO
  • 8 videos capture from deploy day sessions.
  • World-wide participation

For over 70 people to spend a day together at this early stage in deployment is a truly impressive indication of the excitement that is building around OpenStack.

Improvements for Next Deploy Day

This was a first time that Andi Abes (Boston event lead), Rob Hirschfeld (Austin event lead) or Jean-Marie Martini (Dell event lead) had ever coordinated an event like this. We owe much of the success to efforts by Greg Althaus, Victor Lowther and the Canonical 12.04/Essex team before the event. Also, having physical sites was very helpful.

We are planning to do another event, so we are carefully tracking ways to improve.

Here are some issues we are tracking.

  • Issues with setting up a screen and voice share that could handle 70 people.
  • Lack of test & documentation on Crowbar meant too much time focused on Crowbar
  • Connectivity issues distributed voice
  • Should have started with DevStack as a baseline
  • more welcome in the comments!

Thank you!

I want to thank everyone who participated in making this event a huge success!

OpenStack Essex Deploy Day 3/8 – Get involved and install with us

Posted on by
1

My team at Dell has been avidly tracking the ups, downs, and breakthroughs of the OpenStack Essex release.  While we still have a few milestones before the release is cut, we felt like the E4 release was a good time to begin the work on Essex deployment.  Of course, the final deployment scripts will need substantial baking time after the final release on April 5th; however, getting deployments working will help influence the quality efforts and expand the base of possible testers.

To rally behind Essex Deployments, we are hosting a public work day on Thursday March 8th.

For this work day, we’ll be hosting all-day community events online and physically in Austin and Boston.  We are getting commitments from other Dell teams, partners and customers around the world to collaborate.  The day is promising to deliver some real Essex excitement.

The purpose of these events is to deliver the core of a working OpenStack Essex deployment.  While my team is primarily focused on deploys via Crowbar/Chef, we are encouraging anyone interested in laying down OpenStack Essex to participate.  We will be actively engaged on the OpenStack IRC and mailing lists too.

We have experts in OpenStack, Chef, Crowbar and Operating Systems (Canonical, SUSE, and RHEL) engaged in these activities.

This is a great time to start learning about OpenStack (or Crowbar) with hands-on work.  We are investing substantial upfront time (checkout out the Crowbar wiki for details) to ensure that there is a working base OpenStack Essex deploy on Ubuntu 12.04 beta.  This deploy includes the Crowbar 1.3 beta with some new features specifically designed to make testing faster and easier than ever before.

In the next few days, I’ll cut a 12.04 ISO and OpenStack Barclamp TARs as the basis for the deploy day event.  I’ll also be creating videos that help you quickly get a test lab up and running.  Visit the wiki or meetup sites to register and stay tuned for details!

OpenStack Essex Events (Austin & Boston 3/8, WW Hack Day 3/1, Docs 3/6)

Posted on by
1

The excitement over the OpenStack Essex release is building!  While my team has been making plans around the upcoming design summit in SF,  there is more immediate action afoot.

Tomorrow (3/1), numerous sites are gathering around a World Wide Essex Hack Day on 3/1.  If you want to participate or even host a hack venue, get on the list and IRC channel (details).

My team at Dell is organizing a community a follow-up OpenStack Essex Install Day next week (3/8) in both Austin and Boston.  Just like the Hack Day, the install fest will focus on Essex release code with both online and local presence.  Unlike the Hack Day, our focus will be on deployments.  For the Dell team, that means working on the Essex deployment for Crowbar.  We’re still working on a schedule and partner list so stay tuned.  I’m trying to webcast Crowbar & OpenStack training sessions during the install day.

The hack day will close with the regularly scheduled 3/8 OpenStack Austin Meetup (6:30pm at Austin TechRanch).  The topic for the meetup will be, …. wait for it …., the Essex Release.  Thanks go to HP and Dell for sponsoring!

It’s important to note that Anne Gentle is also coordinating an OpenStack Essex Doc Day on 3/6.

To recap:

Wow… that should satisfy your Essex cravings.