7 takeaways from DevOps Days Austin

Posted on May 2, 2013 by Rob H

I spent Tuesday and Wednesday at DevOpsDays Austin and continue to be impressed with the enthusiasm and collaborative nature of the DOD events. We also managed to have a very robust and engaged twitter backchannel thanks to an impressive pace set by Gene Kim!

I’ve still got a 5+ post backlog from the OpenStack summit, but wanted to do a quick post while it’s top of mind.

My takeaways from DevOpsDays Austin:

DevOpsDays spends a lot of time talking about culture. I’m a huge believer on the importance of culture as the foundation for the type of fundamental changes that we’re making in the IT industry; however, it’s also a sign that we’re still in the minority if we have to talk about culture evangelism.
Process and DevOps are tightly coupled. It’s very clear that Lean/Agile/Kanban are essential for DevOps success (nice job by Dominica DeGrandis). No one even suggested DevOps+Waterfall as a joke (but Patrick Debois had a picture of a xeroxed butt in his preso which is pretty close).
Still need more Devs people to show up! My feeling is that we’ve got a lot of operators who are engaging with developers and fewer developers who are engaging with operators (the “opsdev” people).
Chef Omnibus installer is very compelling. This approach addresses issues with packaging that were created because we did not have configuration management. Now that we have good tooling we separate the concerns between bits, configuration, services and dependencies. This is one thing to watch and something I expect to see in Crowbar.
The old mantra still holds: If something is hard, do it more often.
Eli Goldratt’s The Goal is alive again thanks to Gene Kims’s smart new novel, The Phoenix project, about DevOps and IT (I highly recommend both, start with Kim).
Not DevOps, but 3D printing is awesome. This is clearly a game changing technology; however, it takes some effort to get right. Dell brought a Solidoodle 3D printer to the event to try and print OpenStack & Crowbar logos (watch for this in the future).

I’d be interested in hearing what other people found interesting! Please comment here and let me know.

Do Be Dense! Dell C8000 unit merges best of bladed and rackable servers

Posted on September 19, 2012 by Rob H

“Double wide” is not a term I’ve commonly applied to servers, but that’s one of the cool things about this new class of servers that Dell, my employer, started shipping today.

My team has been itching for the chance to start cloud and big data reference architectures using this super dense and flexible chassis. You’ll see it included in our next Apache Hadoop release and we’ve already got customers who are making it the foundation of their deployments (Texas Adv Computing Center case study).

If you’re tracking the latest big data & cloud hardware then the Dell PowerEdge C8000 is worth some investigation.

Basically, the Dell C8000 is a chassis that holds a flexible configuration of compute or storage sleds. It’s not a blade frame because the sleds minimize shared infrastructure. In our experience, cloud customers like the dedicated i/o and independence of sleds (as per the Bootstrapping clouds white paper). Those attributes are especially well suited for Hadoop and OpenStack because they support a “flat edges” and scale out design. While i/o independence is valued, we also want shared power infrastructure and density for efficiency reasons. Using a chassis design seems to capture the best of both worlds.

The novelty for the Dell PowerEdge C8000 is that the chassis are scary flexible. You are not locked into a pre-loaded server mix.

There are a plethora of sled choices so that you can mix choices for power, compute density and spindle counts. That includes double-wide sleds positively brimming with drives and expanded GPU processers. Drive density is important for big data configurations that are disk i/o hungry; however, our experience is the customer deployments are highly varied based on the planned workload. There are also significant big data trends towards compute, network, and balanced hardware configurations. Using the C8000 as a foundation is powerful because it can cater to all of these use-case mixes.

That reminds me! Mike Pittaro (our team’s Hadoop lead architect) did an excellent Deploy Hadoop using Crowbar video.

Interested in more opinions about the C8000? Check out Barton George & David Meyer.

Dell is open sourcing Crowbar Apache Hadoop barclamps!

Posted on November 8, 2011 by Rob H

I’m very excited to announce that my team at Dell will be open sourcing our Apache Hadoop Crowbar barclamps by the end of the month.

This release raises the bar on open Hadoop deployments by making them faster, scalable, more integrated and repeatable.

These barclamps were developed in conjunction with our licensed Dell | Cloudera Solution. The licensed solution is for customers seeking large scale and professionally supported big data solutions. The purpose of the open barclamps (which pull the open source parts from the Cloudera distro) is to help you get started with Hadoop and reduce your learning curve. Our team invested significant testing effort in ensuring that these barclamps work smoothly because they are the foundational layer of our for-pay Hadoop solution.

Included in the Hadoop barclamp suite are Hadoop Map Reduce, Hive, Pig, ZooKeeper and Sqoop running on RHEL 5.7. These barclamps cover the core parts of the Hadoop suite. Like other Crowbar deployments (see OpenStack), the barclamps automatically discover the service configurations and interoperate. One of our team members (call him Scott Jensen) said it very simply “I can deploy a fully an integrated Hadoop cluster in a few hours. That friggin’ rocks!” I just can’t put it more eloquently than that!

I’ll post again when we flip the “open” bit and invite our community to dig in and help us continue to set the standards on open Hadoop deployments.

For more perspectives on this release, check out posts by Barton George (just for devs), Joseph George (About Hadoop) and Aurelian Dumitru

Barton posted these two videos of me talking about the release too:

Hadoop & Crowbar:

Dev’s Only Short:

Notes from 10/27 OpenStack Austin Meetup (via Stephen Spector)

Posted on October 30, 2011 by Rob H

Stephen Spector (now a Dell Services employee!) gave me permission to repost his excellent notes from the first OpenStack Austin (#OSATX) Meetup Group.

Here are his notes:

[Stephen] wanted to update everyone on the Austin OpenStack Meetup last night at the Austin TechRanch sponsored by Joseph and Rob (that’s me!) of the Dell OpenStack team (I think I got that right?). You can find all the tweets from the event at https://twitter.com/#!/search/%23osatx as we created a new hashtag for tweeting during the event, #osatx.

Here are some highlights from the event:

About 60 or so attendees with a good amount from Dell (Barton George, Logan McCloud)and Rackspace, Opscode (Matt Ray), Puppet Labs, SUSE talked about their OpenStack commitment (http://t.co/bBnIO7xv), and Ubuntu folks as well
Jon Dickinson who is the Project Technical Lead for Swift (Object Storage) was there and presented information on the current Swift offering; It is interesting to note that Swift releases continuously when most of OpenStack releases during the 6 month development cycle like Nova (Compute)
Stephen and Jim Plamondon from Rackspace presented information on the overall community and talked about the announcement yesterday from Internap about their Compute public cloud and the information on the MercadoLibre 600 Node Compute cloud running their business:

“With 58 million users of MercadoLibre.com and growing rapidly, we need to provide our teams instant access to computing resources without heavy administrative layers. With OpenStack, our internal users can instantly provision what they need without having to wait for a system administrator,” said Alejandro Comisario, Infrastructure Senior Engineer, MercadoLibre, the largest online trading platform in Latin America. “With our success running OpenStack Compute in production, we plan to roll OpenStack Diablo out more broadly across the company, and have appreciated the community support in this venture, especially through the OpenStack Forums, where we are also global moderators.”

Discussion on the OpenStack API Issue which is a significant open issue at this time – should OpenStack focus on creating an API specification and then let multiple implementations of that API move forward or build 1 implementation of the API as official OpenStack (see my post for more on this).
Greg Althaus gave a demo of the Nova Dashboard
Future Meetings

Three organizations have offered to help host (pizza $ and TechRanch space $) but we always need more! You can offer to sponsor via the meetup site.
There will be future OpenStack Austin Meetups so sign up for the group and you’ll be notified automatically.

Pictures…

Continue reading →

Shout, chat and whisper with Dell at OpenStack Design Summit & Conference

Posted on September 30, 2011 by Rob H

My team at Dell has been very (very) busy delivering a lot of great materials for the Fall 2011 OpenStack Design Summit & Conference in Boston MA next week.

Our motto for this conference is “DOING IS DOING” or, perhaps, “DIABLO IS DOING.”

You can count on Dell to be walking the walk with deliverables that advance OpenStack. In fact, you can watch what we’re doing because it’s posted live as we work with the community to build it.

First, we’ll have our Crowbar demo rack showcasing LIVE MULTI-NODE DIABLO DEPLOYMENTS and some IMPORTANT FEATURE AND COMMUNITY ADDITIONS. No spoilers here – you’ll have to come by. Of course, it’s in the git hub too, but we’ve put a bow on it.

Second, there’s a DEPLOYMENT BLUE PRINT discussion about getting better interlocks between OpenStack development and deployment. We really need to reduce the pain and lag between adding great features and using those features.

Next, we’ve got a limited audience CONCEPT SNEAK PEEK for something from our labs that we think is very interesting and we’d like to get input about. Unfortunately, we’re very limited with space & time for this whisper session so you’ll need to contact OpenStack@Dell.com to request an invitation.

Finally, at the Conference, you can see OUR TEAM IN ACTION:

Thurs 11:30 – Dell Keynote by John Igoe
Thurs 3:30 – Private Cloud Panel w/ Rob Hirschfeld
Thurs 4:30 – Hardware Infrastructure w/ Rob Hirschfeld & Greg Althaus
Friday 11:00 – Deployments w/ Greg Althaus
Friday 3:15 – Crowbar!! w/ Scott Jensen (yes, he does it with the !!)
Friday 4:15 – KVM & OpenStack

More conference posts: JB Gorge & Barton George.

WHIR Webinar Notes: Prying Open the Cloud with Dell Crowbar & OpenStack

Posted on September 1, 2011 by Rob H

Panelists: Me (@zehicle) & Joseph B. George (@jbgeorge), Director, Cloud and Big Data Solutions, Dell

Moderator: Liam Eagle (@theWHIR) , Editor-in-Chief, Web Host Industry Review

Wow, this Webinar was an hour of OpenStack insights (see the whole thing). If you don’t have the hour then you can use my time line nodes to jump to what you want to hear.

2:50: Presentation Starts (introductions are over)
3:40: Joseph coins the word “dynormous” for dynamic & large scale clouds
4:40: Customers want to know how they are going to maintain a cloud
4:50: Customers don’t want a 9 month cycle for features, want it faster. DevOps gives us the flexibility to meet our customer needs as quickly as they want to.
7:11: Massive scalability… their (Rackspace & NASA) business is about scale
8:00: Rackspace and NASA started from the beginning to build a community
8:50: We have the data that this has staying power
10:10: We see a lot of companies joining in the community
11:56: Shout out to Opscode Chef
12:40: From bare metal to a fully functioning cloud in under 2 hours. Crowbar allows you to introduce new elements into the environment
13:40: Crowbar leverages our experience with cloud deployments
14:33: Dell was the only provider there from day 1. We have the most experience.
17:27: DevOps Poll
18:40: DevOps is a significant trend that you should consider. Hosters have a lot of operational chops.
19:34: There are a lot of right ways to do cloud. You need to pick what’s best for your business model
20:23: We could get hardware and software, but operational expertise was missing.
21:33: We’re more making the complexity of a cloud go away. We are getting our customers a head start. We are chipping away at the learning curve.
22:05: The cloud is always ready, never finished. Cloud is an ongoing operational environment: DevOps!
23:30: Crowbar bakes a lot of operational experience into the deployment.
25:17: Core tenant of DevOps: there is no single OpenStack image. Cloud is too complex. We build it in layers.
26:26: Before you deploy, you can change the configuration.
27:30: Barclamps are modules that execute a function. We are inviting community participation
28:40: Crowbar process view – Crowbar is a “PXE state machine” is a very simplified description.
29:57: You can go through a tuning cycle where you can get it working, make sure it’s right, flush and reset. That ensures you have an automated system.
30:34: Screen shots with descriptions
33:25: Event the core state machine that runs Crowbar is deployed as a barclamp
35:00: You can download OpenStack and install it yourself from our github. We don’t want to talk about OpenStack, we want to DO OpenStack.
Poll Results (see to the right)
38:00: Online resources
40:00: Question 1: Timeline for RHEL. Answer: RHEL is part of Hadoop, will make it into OpenStack by end of year (or sooner based on market demand)
42:17: Question 2: What led Dell to get involved in OpenStack? Answer: It’s about experience. We like being able to fix and change if we needed. There is a lot of active community
45:30: Question 3: How does a hardware maker play with open source software? Answer: It’s a solution for us. We wanted to make sure that people cloud deploy the software. Adding DevOps takes it to another level.
48:00 Question 4: What elements of Diablo are most exciting? Answer: Keystone (centralized authentication) is a big deal. Networking changes that “bust the top” of the networking hurdles.
50:25: Question 5: Where is OpenStack going long term? Answer: We’re pleasantly surprised about how much it’s picked up. We’ll see more standards in the community. We have high hopes for OpenStack and have invested heavily. We’ll see more as-a-service capabilities to build on a common infrastructure: both open and commercial.
52:47: Question 6: What’s the biggest barrier to operating at scale? Answer: learning how to operate is the biggest hurdle. We took a learning approach to help customers get started. We are hosting a training with Rackspace.
55:00: Question 7: Where does Dell and Rackspace overlap? Answer: We see Rackspace Cloud Builders that the premier experts. Dell Services is involved with all of it. Dell takes the phone call and deals with our customers directly.

Crowbar source released, includes OpenStack Cloud install

Posted on July 26, 2011 by Rob H

I’m delighted to announce (official version) that my team at Dell has opened the Crowbar source under the Apache 2 license. This action is part of the broader Dell OpenStack Cloud Solution which includes OpenStack install packages, Crowbar, reference hardware architectures, and services/consulting to support deployments.

There are two important components to this news:

Dell is officially offering our OpenStack Solution and helping advance the community’s ability to implement OpenStack quickly and consistently.
Dell is releasing the Crowbar code (which is included in the solution) as open source.

Both are significant items; however, my focus here is on the Crowbar release.

Crowbar started as a Dell OpenStack installer project and then grew beyond that in scope. Now it can be extended to work with other vendors’ kits and other solutions bits.

We are contributing Crowbar to the community because we believe that everyone benefits by sharing in the operational practices that Crowbar embodies. These are rooted in Opscsode Chef (which Crowbar tightly integrates with) and the cloud & hyper-scale proven DevOps practices that are reflected in our deployment model.

Where to get it?

Code: Github (Apache 2 license)

Community: Dell Crowbar listserv,

Information: wiki on Crowbar Github, Crowbar User’s Guide and RobHirschfeld.com web site.

What’s included?

A comprehensive set of barclamps to set up an OpenStack cloud.

Crowbar UI and Remote APIs to make it easy to set up your cloud

Automated testing scripts for community members doing continuous integration with OpenStack.

Build scripts so you can create your own Crowbar install ISO

Switch discovery so you can create Chef Cookbooks that are network aware.

Open source Chef server that powers much of Crowar’s functionality

What’s not included?

Non-open source license components (BIOS+RAID config) that we could not distribute under the Apache 2 license. We are working to address this and include them in our release. They are available in the Dell Licensed version of Crowbar.

Dell Branded Components (skin + overview page). Crowbar has an OpenSource skin with identical functionality.

Pre-built ISOs with install images (you must download the open source components yourself, we cannot redistribute them to you as a package)

Important notes:

Crowbar uses Chef Server as its database and relies on cookbooks for node deployments. It is installed (using Chef Solo) automatically as part of the Crowbar install.
Crowbar has a modular architecture so individual components can be removed, extended, and added. These components are known individually as barclamps.
Each barclamp has its own Chef configuration, UI sub-component, deployment configuration, and documentation.

On the project roadmap:

Hadoop support
Additional operating system support (specifically RHEL)
Barclamp version repository
Network configuration
We’d like suggestions! Please comment!

Sites for more information: Joseph George, Barton George (launch day), Dell

Demo Redux: OpenStack installer SXSW demo of Chef + Crowbar

Posted on March 11, 2011 by Rob H

If you missed the OpenStack installer demo at Cloud Connect Event then you’ll have another chance to see us go from bare iron to provisioning VMs in under 30 minutes at SXSW on Monday 3/14 from 2-4 pm at Kung Fu Saloon.

Note: Rackspace rented out the Kung Fu Saloon all day Monday, and are doing various events — from live webinars to a Scoble tweetup to a happy hour and more VIP after hours event.

The demo will be orchestrated by Greg Althaus from my team at Dell. Greg is the primary architect for Crowbar and responsible for some of it’s amazing capabilities including the Chef integrations, network discovery and rockin’ PXE state machine. Dell Cloud Evanglist, Barton George, will also be on hand.

Of course, our friends from Opscode & Rackspace will be there too – this is Rackspace’s party (they are a Platinum SXSW sponsor)

More more information (outside of this blog, of course), check out http://www.Dell.com/OpenStack.

Rob Hirschfeld

On Computing, Containers, Cloud & Tech Culture

Category Archives: Barton George