I have been tracking the progress the Copper ARM-based server from design to implementation internally.  Now, I’m excited to see it getting some deserved attention.

The Copper platform is really cool because the cost, power, and density ratios of the nodes are unparalleled.  This makes it an ideal platform for distributed mixed compute/store workloads like Hadoop.  The nodes in the platform have excellent RAM/CPU/Spindle ratios.

While Copper is driving huge density, it also drives forward the same hyperscale challenges that we’ve been trying to address with Crowbar; consequently, we’re already working to ensure that we can deploy and manage Copper with Crowbar at scale.

Copper and Crowbar make a natural team and we’re excited to be part of today’s announcement:

Dell is staging clusters of the Dell “Copper” ARM server within the Dell Solution Centers and with TACC so developers may book time on the platforms. Dell also will deliver an ARM-supported version of Crowbar, Dell’s open-source management infrastructure software, to the industry in the future.

Congratulations to the Copper team!

Already know the event logistics? Skip back to my OSED observations post.

Some important general notes:

1. We are RECORDING everything and will link posts from the event page.
2. There is HOMEWORK if you want to get ahead by installing OpenStack yourself.
3. For last minute updates about the event, I recommend that you join the Crowbar Listserver.

Content Logistics work like this.

1. Everything will be available ONLINE. We are also coordinating many physical sites as rally points.
2. Introductory: FOUR 3-hour sessions for people who do not have OpenStack or Crowbar experience. These sessions will show how to install OpenStack using Crowbar, discuss DevOps and showcase companies that are in the OpenStack ecosystem. They are planned to have 2 European slots (afternoon & evening), 3 US slots (morning, afternoon & evening), and 1 Asian slot (morning).
3. Expert: ON-GOING deep technical sessions for engineers who have OpenStack and/or Crowbar experience. There will be one main screen and voice channel in which we are planning to highlight and discuss these topics in blocks throughout the day. We have a long list of topics to discuss and will maintain an ongoing Google Hangout for each topic. Depending on interest, we will jump back and forth to different hangouts.

Intro/Overview Session Logistics work like this

We’re planning FOUR introductory sessions throughout the day (read ahead?). Each session should be approximately 3 hours. The first hour of the sessions will be about OpenStack Essex and installing it using Crowbar. After some Q&A, we’re going to highlight the OpenStack ecosystem. The schedule for the ecosystem is in flux and will likely shift even during the event.

The Session start times for Overview & Ecosystem content

* There are no planned live venues at this time/region. You are always welcome to join online!

Experts Track Logistics

Note: we expect experts to have already installed OpenStack (see homework page). Ideally, an expert has already setup a build environment.

We have a list of topics (Essex, Quantum, Networking, Pull from Source, Documentation, etc) that we plan to cover on a 30-60 minute rotation.

We will cover the OpenStack Essex deploy at the start of each planned session (9am, Noon, 3pm & 8pm EDT). Before we cover the OpenStack deploy, we’ll spend 10 minutes setting (and posting) the agenda for the next three hours based on attendee input.

Even if we are not talking about a topic on the main channel, we will keep a dialog going on topic specific Google hangouts. The links to the hangouts will be posted with the Expert track agenda.

If you want to read more about the event, check out my event logistics post (link pending).

I do not apologize for my promotion of the Dell-lead open source Crowbar as the deployment tool for the OpenStack Essex Deploy. For a community to focus on improving deployment tooling, there must be a stable reference infrastructure. Crowbar provides a fast and repeatable multi-node environment with scriptable networking and packaging.

I believe that OpenStack benefits from a repeatable multi-node reference deployment. I’ll go further and state that this requires DevOps tooling to ensure consistency both within and between deployments.

DevStack makes trunk development more canonical between different developers. I hope that Crowbar will help provide a similar experience for operators so that we can truly share deployment experience and troubleshooting. I think it’s already realistic for Crowbar deployments to a repeatable enough deployment that they provide a reference for defect documentation and reproduction.

Said more plainly, it’s a good thing if a lot of us use OpenStack in the same way so that we can help each out.

My team’s choice to accelerate releasing the Crowbar barclamps for OpenStack Essex makes perfect sense if you accept our rationale for creating a community baseline deployment.

Crowbar is Dell-lead, not Dell specific.

One of the reasons that Crowbar is open source and we do our work in the open (yes, you can see our daily development in github) is make it safe for everyone to invest in a shared deployment strategy. We encourage and welcome community participation.

PS: I believe the same is true for any large scale software project. Watch out for similar activity around Apache Hadoop as part of our collaboration with Cloudera!

My team at Dell is about to turn another release of Crowbar. Version 1.3 released 5/14 (focused on Cloudera Apache Hadoop) and our original schedule showed several sprints of work on OpenStack Essex. Upon evaluation, we believe that the current community developed Essex barclamps are ready now.

The healthy state of the OpenStack Essex deployment is a reflection of 1) the quality of Essex and 2) our early community activity in creating deployments based Essex RC1 and Ubuntu Beta1.

We are planning many improvements to our OpenStack Essex and Crowbar Framework; however, most deployments can proceed without these enhancements.  This also enables participants in the 5/31 OpenStack Essex Deploy Day.

By releasing a core stable Essex reference deployment, we are accelerating field deployments and enabling the OpenStack ecosystem. In terms of previous posts, we are eliminating release interlocks to enable more downstream development. Ultimately, we hope that we are also creating a baseline OpenStack deployment (link pending).

We are also reducing the pressure to rush more disruptive Crowbar changes (like enabling high availability, adding multiple operating systems, moving to Rails 3, fewer crowbarisms in cookbooks and streamlining networking). With this foundational Essex release behind us (we call it an MVP), we can work on more depth and breadth of capability in OpenStack.

One small challenge, some of the changes that we’d expected to drop have been postponed slightly. Specifically, markdown based documentation (/docs) and some new UI pages (/network/nodes, /nodes/families). All are already in the product under but not wired into the default UI (basically, a split test).

On the bright side, we did manage to expose 10g networking awareness for barclamps; however, we have not yet refactored to barclamps to leverage the change.

This addresses the concern that our first Essex hack day was America’s daytime only so it was difficult for time zones east of GMT to participate.

We are working with Dell teams in Asia-Pac to setup more information to support Japan, China, Korea and Australia.

This picture, taken by Dan Choquette (my team too!), is from Toyko DevOpsDays.

Here’s the official overview:

The OpenStack Deploy hack-a-thon focuses on automation for deploying OpenStack Essex with Dell Crowbar and Opscode Chef. This is a day-long, world-wide event bringing together developers, operators, users, ecosystem vendors and the open source cloud curious. (read below: We are looking for global sites and leaders to extend the event hours!)

OpenStack is the fastest growing open source cloud infrastructure project with broad market adoption from major hardware and software vendors. Crowbar is an Apache 2 licensed, open infrastructure deployment tool and is one of the leading multi-node deployers for OpenStack and Hadoop.

Learn first-hand how OpenStack and Crowbar can make it easy to deploy and operate your own cloud environments.

The Deploy day will offer two individual parallel tracks with something for both experts and beginners:

• Newbies n00bs will learn the basics of OpenStack, Crowbar and DevOps and how they can benefit your organization. We’ll also have time for ecosystem vendors to discuss how they are leveraging OpenStack.
• Experts l33ts will take a deep dive into new features of OpenStack Essex and Crowbar, and learn how Crowbar works under the hood, which will enable them to extend the product using Crowbar Barclamps.

Why now? We’ve validated our OpenStack Essex deployment against the latest release bits from Ubuntu. Now it’s time to reach out to the OpenStack and Crowbar communities for training, testing and collaborative development.

Join the event!  We’re organizing information on the Crowbar wiki.  (I highly recommend you join the Crowbar list to get access to support for prep materials).  You can also reach out to me via the @DellCrowbar handle.

We’d love to get you up to speed on the basics and dive deep into the core.

I live demo’ed using a single command window with knife to manage both physical and cloud infrastructure.    That’s freaking cool!  (and thanks to Matt Ray for helping to get this working)

It’s no surprise that I’m already a DevOps advocate and Opscode enthusiast, there were aspects of the conference that are worth reiterating:

• Opscode is part of the cadre of leaders redefining how we operate infrastructure.  The energy is amazing.
• The acknowledgement of the “snowflake” challenge where all Ops environments are alike, but no two are the same.
• A tight integration between Operations and lean delivery because waterfall deployments are not sustainable
• Opscode’s vision is rooted in utility.  You can be successful without design and then excel when you add it.  I find that refreshing.
• There was a fun, friendly (“hug driven development?!”) and laid back vibe.  This group laughed A LOT.
• For a first conference, Opscode did a good job with logistics and organization.
• I saw that the back rooms and hallways are buzzing with activity.  This means that people are making money with the technology.

Crowbar + Chef installs & manages OpenStack Essex (Live Demo, 45 minutes):

Ignite Talk about Dell Crowbar History (5 minutes)

1.3 Release Highlights (branch “elefante”)

• Introduction of Cloudera 3.7 Hadoop Barclamp
• New Operating System versions: Ubuntu 11.10, RHEL 6.2, Centos 6.2
• Upgrade the Sledgehammer image to Centos 6.2
• Alias & Group Feature (Alias is linked into DNS & Chef Search)
• Barclamp import from UI
• Pre-populate Node names & descriptions
• Export of logs & database snapshot from UI
• For the Dell Additions: Support for 12g Hardware Models (720xd & 720) via WSMAN

1.4 Previews already in the tree (“essex-hack” branch)

We’ve been working in advance for the 1.4 Release on the Essex-Hack branch.

• Ubuntu 12.04 Support
• OpenStack Essex Packages (from Ubuntu)

Dieter Plaetinck

…I think this tool deserves more attention and should be added to your devops toolchain for the cloud (triple buzzword bonus!!!)…

Hosting News

…As an alternative to proprietary, licensed software models, Dell continues to see heavy customer interest in the OpenStack-Powered Cloud Solution, which integrates the OpenStack cloud operating system, cloud-optimized Dell PowerEdge C servers, the Dell-developed Crowbar software framework, and services…

Sys-Con Post by Mirantis

…Finally, there is Dell and Crowbar. Dell’s approach to riding the OpenStack wave is, perhaps, the most creative. Crowbar is neither a hardware appliance nor an enterprise version of OpenStack. It’s a configuration management tool built around OpSource’s Chef, designed specifically to deploy OpenStack on Dell servers. Crowbar effectively serves as the glue between the hardware and any distribution of OpenStack (and not only OpenStack)…

Robert Booth w/ Zenoss

Well if you care to only give them the best then introduce them to a set of tools that will drastically change the way they do business in a way they want to. Introduce them to Dell’s Crowbar and OpsCode Chef and you will make their job easier, faster and possibly put a stop to the finger pointing! No longer will they have to pull out the IT secret decoder ring to understand what the dev team put in the deployment docs.

My interest lies in the trend lines of OpenStack.  For details about sessions, I recommend Stefano Maffulli‘s  excellent link aggregation post for the Summit.

1. Technology Trend: Practical with Potential.

OpenStack started with a BIG vision to become the common platform for cloud API and operations.  That vision is very much alive and on-track; however, our enthusiasm for what could be is tempered by the need to build a rock solid foundation.  The drive to stability over feature expansion has had a very positive impact.  I give a lot of credit for this effort to the leadership of the project technical leads (PTLs), Canonical‘s drive to include OpenStack in the 12.04 LTS and the Rackspace Cloud drive to deploy Essex.  My team at Dell has also been part of this trend by focusing so much effort on making OpenStack production deployable (via Crowbar).

Overall, I am seeing a broad-based drive to minimize disruption.

2. Culture Trend: Friendly but some tension.

Companies at both large and small ends of the spectrum are clearly jockeying for position.  I think the market is big enough for everyone; however, we are also bumping into each other.  Overall, we are putting aside these real and imagined differences to focus on enlarging the opportunity of having a true community cloud platform.  For example, the OpenStack Foundation investment formation has moneyed competitors jostling for position to partner together.

However, it’s not just about paying into the club; OpenStack’s history is clearly about execution.  Looking back to the original Austin Summit sponsors, we’ve clearly seen that intent and commitment are different.

3. Discussion Trend: Small Groups Effective

The depth & quality of discussions inside sessions was highly variable.  Generally, I saw that large group discussions stayed at a very high level.  The smaller sessions required deep knowledge of the code to participate and seemed more productive.  We continue to have a juggle between discussions that are conceptual or require detailed knowledge of the code.  If conceptual, it’s too far removed.  If code, it becomes inaccessible to many people.

This has happened at each Summit and I now accept that it is natural.  We are using vision sessions to ensure consensus and working sessions to coordinate deliverables for the release.

I cannot over emphasize importance of small groups and delivery driven execution interactions: I spent most of my time in small group discussions with partners aligning efforts.

4. Deployment Trend: Testing and Upstreams matter

Operations for deploying OpenStack is a substantial topic at the Summit.  I find that to be a significant benefit to the community because there are a large block of us who were vocal advocates for deployability at the very formation of the project.

From my perspective at Dell, we are proud to see that wide spread acknowledgement of our open source contribution, Crowbar, as the most prominent OpenStack deployer.   Our efforts at making OpenStack installable are recognized as a contribution; however, we’re also getting feedback that we need to streamline and simplify Crowbar.  We also surprised to hear that Crowbar is “opinionated.”   On reflection, I agree (and am proud) of this assessment because it matches best practice coding styles.  Since our opinions also drive our test matrix there is a significant value for our OpenStack deployment is that we spend a lot of time testing (automated and manual) our preferred install process.

There’s a push to reconcile the various Chef OpenStack cookbooks into a single upstream.  This seems like a very good idea because it will allow various parties to collaborate on open operations.  The community needs leadership from Opscode to make this happen.  It appears that Puppet Labs is interested in playing a similar role for Puppet modules but these are still emerging and have not had a chance to fragment.

No matter which path we take, the deployment scripts are only as good as their level of testing.   Unreliable deployment scripts have are less than worthless.