With Dell ARM-based “Copper” servers, Crowbar footprint grows May 29, 2012
Posted by Rob H in Crowbar, Dell, DevOps, Hadoop.Tags: ARM, Crowbar, Dell, DevOps
add a comment
One of my team at Dell’s most critical lessons from hyperscale cloud deployments was the DevOps tooling and operations processes are key to success. Our crowbar project was born out of this realization.
I have been tracking the progress the Copper ARM-based server from design to implementation internally. Now, I’m excited to see it getting some deserved attention.
The Copper platform is really cool because the cost, power, and density ratios of the nodes are unparalleled. This makes it an ideal platform for distributed mixed compute/store workloads like Hadoop. The nodes in the platform have excellent RAM/CPU/Spindle ratios.
While Copper is driving huge density, it also drives forward the same hyperscale challenges that we’ve been trying to address with Crowbar; consequently, we’re already working to ensure that we can deploy and manage Copper with Crowbar at scale.
Copper and Crowbar make a natural team and we’re excited to be part of today’s announcement:
Dell is staging clusters of the Dell “Copper” ARM server within the Dell Solution Centers and with TACC so developers may book time on the platforms. Dell also will deliver an ARM-supported version of Crowbar, Dell’s open-source management infrastructure software, to the industry in the future.
Congratulations to the Copper team!
OSED OMG: OpenStack Essex Deploy Day!! A day-long four-session two-track International Online Conference May 29, 2012
Posted by Rob H in OpenStack, Crowbar, DevOps, CloudOps, Open source, Meetup, Dell, Operations.Tags: Crowbar, essex, Logistics, OpenStack
add a comment
Curious about OpenStack? Know it, but want to tune your Ops chops? JOIN US on Thursday 5/31 (or Friday 6/1 if you are in Asia)!
Already know the event logistics? Skip back to my OSED observations post.
Some important general notes:
- We are RECORDING everything and will link posts from the event page.
- There is HOMEWORK if you want to get ahead by installing OpenStack yourself.
- For last minute updates about the event, I recommend that you join the Crowbar Listserver.
Content Logistics work like this.
- Everything will be available ONLINE. We are also coordinating many physical sites as rally points.
- Introductory: FOUR 3-hour sessions for people who do not have OpenStack or Crowbar experience. These sessions will show how to install OpenStack using Crowbar, discuss DevOps and showcase companies that are in the OpenStack ecosystem. They are planned to have 2 European slots (afternoon & evening), 3 US slots (morning, afternoon & evening), and 1 Asian slot (morning).
- Expert: ON-GOING deep technical sessions for engineers who have OpenStack and/or Crowbar experience. There will be one main screen and voice channel in which we are planning to highlight and discuss these topics in blocks throughout the day. We have a long list of topics to discuss and will maintain an ongoing Google Hangout for each topic. Depending on interest, we will jump back and forth to different hangouts.
Intro/Overview Session Logistics work like this
We’re planning FOUR introductory sessions throughout the day (read ahead?). Each session should be approximately 3 hours. The first hour of the sessions will be about OpenStack Essex and installing it using Crowbar. After some Q&A, we’re going to highlight the OpenStack ecosystem. The schedule for the ecosystem is in flux and will likely shift even during the event.
The Session start times for Overview & Ecosystem content
| Region | EDT | Session 1 | Session 2 | Session 3 | Session 4 |
| Europe (-5) | -5 | 3pm | 6pm | * | * |
| Americas Eastern | 0 | 10am | 1pm | 4pm | * |
| Americas Central | +1 | 9am | Noon | 3pm | * |
| Americas Mtn | +2 | * | 11am | 2pm | 7pm |
| Americas West | +3 | * | 10am | 1pm | 6pm |
| Asia (Toyko) | +10 | * | * | * | 6/1 10 am |
* There are no planned live venues at this time/region. You are always welcome to join online!
Experts Track Logistics
Note: we expect experts to have already installed OpenStack (see homework page). Ideally, an expert has already setup a build environment.
We have a list of topics (Essex, Quantum, Networking, Pull from Source, Documentation, etc) that we plan to cover on a 30-60 minute rotation.
We will cover the OpenStack Essex deploy at the start of each planned session (9am, Noon, 3pm & 8pm EDT). Before we cover the OpenStack deploy, we’ll spend 10 minutes setting (and posting) the agenda for the next three hours based on attendee input.
Even if we are not talking about a topic on the main channel, we will keep a dialog going on topic specific Google hangouts. The links to the hangouts will be posted with the Expert track agenda.
We need an OpenStack Reference Deployment (My objectives for Deploy Day) May 29, 2012
Posted by Rob H in OpenStack, Crowbar, DevOps, CloudOps, Open source, Dell, Cloudera, Operations.Tags: Crowbar, Dell, DevStack, OpenStack, Reference
1 comment so far
I’m overwhelmed and humbled by the enthusiasm my team at Dell is seeing for the OpenStack Essex Deploy day on 5/31 (or 6/1 for Asia). What started as a day for our engineers to hack on Essex Cookbooks with a few fellow Crowbarians has morphed into an international OpenStack event spanning Europe, Americas & Asia.
If you want to read more about the event, check out my event logistics post (link pending).
I do not apologize for my promotion of the Dell-lead open source Crowbar as the deployment tool for the OpenStack Essex Deploy. For a community to focus on improving deployment tooling, there must be a stable reference infrastructure. Crowbar provides a fast and repeatable multi-node environment with scriptable networking and packaging.
I believe that OpenStack benefits from a repeatable multi-node reference deployment. I’ll go further and state that this requires DevOps tooling to ensure consistency both within and between deployments.
DevStack makes trunk development more canonical between different developers. I hope that Crowbar will help provide a similar experience for operators so that we can truly share deployment experience and troubleshooting. I think it’s already realistic for Crowbar deployments to a repeatable enough deployment that they provide a reference for defect documentation and reproduction.
Said more plainly, it’s a good thing if a lot of us use OpenStack in the same way so that we can help each out.
My team’s choice to accelerate releasing the Crowbar barclamps for OpenStack Essex makes perfect sense if you accept our rationale for creating a community baseline deployment.
Crowbar is Dell-lead, not Dell specific.
One of the reasons that Crowbar is open source and we do our work in the open (yes, you can see our daily development in github) is make it safe for everyone to invest in a shared deployment strategy. We encourage and welcome community participation.
PS: I believe the same is true for any large scale software project. Watch out for similar activity around Apache Hadoop as part of our collaboration with Cloudera!
Quick turn OpenStack Essex on Crowbar (BOOM, now we’re at v1.4!) May 28, 2012
Posted by Rob H in OpenStack, Lean, Crowbar, Hadoop, Dell, Process Interlock.Tags: hadoop, OpenStack, Crowbar, Interlock, v1.4
1 comment so far
Don’t blink if you’ve been watching the Crowbar release roadmap!
My team at Dell is about to turn another release of Crowbar. Version 1.3 released 5/14 (focused on Cloudera Apache Hadoop) and our original schedule showed several sprints of work on OpenStack Essex. Upon evaluation, we believe that the current community developed Essex barclamps are ready now.
The healthy state of the OpenStack Essex deployment is a reflection of 1) the quality of Essex and 2) our early community activity in creating deployments based Essex RC1 and Ubuntu Beta1.
We are planning many improvements to our OpenStack Essex and Crowbar Framework; however, most deployments can proceed without these enhancements. This also enables participants in the 5/31 OpenStack Essex Deploy Day.
By releasing a core stable Essex reference deployment, we are accelerating field deployments and enabling the OpenStack ecosystem. In terms of previous posts, we are eliminating release interlocks to enable more downstream development. Ultimately, we hope that we are also creating a baseline OpenStack deployment (link pending).
We are also reducing the pressure to rush more disruptive Crowbar changes (like enabling high availability, adding multiple operating systems, moving to Rails 3, fewer crowbarisms in cookbooks and streamlining networking). With this foundational Essex release behind us (we call it an MVP), we can work on more depth and breadth of capability in OpenStack.
One small challenge, some of the changes that we’d expected to drop have been postponed slightly. Specifically, markdown based documentation (/docs) and some new UI pages (/network/nodes, /nodes/families). All are already in the product under but not wired into the default UI (basically, a split test).
On the bright side, we did manage to expose 10g networking awareness for barclamps; however, we have not yet refactored to barclamps to leverage the change.
Join us 5/31 for a OpenStack Deploy Hack-a-thon (all-day, world-wide online & multi-city) May 23, 2012
Posted by Rob H in Crowbar, Dell, DevOps, Meetup, OpenStack, Opscode.Tags: Chef, cloud, Crowbar, Dell, DevOps, essex, Hack-a-thon, OpenStack, OpsCode
1 comment so far
An OpenStack Deploy Hack-a-thon is like 3-liter bottle of distilled open source community love. Do you want direct access to my Dell team of OpenStack/Crowbar/Hadoop engineers? Are you just getting started and want training about OpenStack and DevOps? This is the event for you!
Here’s the official overview:
The OpenStack Deploy hack-a-thon focuses on automation for deploying OpenStack Essex with Dell Crowbar and Opscode Chef. This is a day-long, world-wide event bringing together developers, operators, users, ecosystem vendors and the open source cloud curious. (read below: We are looking for global sites and leaders to extend the event hours!)
OpenStack is the fastest growing open source cloud infrastructure project with broad market adoption from major hardware and software vendors. Crowbar is an Apache 2 licensed, open infrastructure deployment tool and is one of the leading multi-node deployers for OpenStack and Hadoop.
Learn first-hand how OpenStack and Crowbar can make it easy to deploy and operate your own cloud environments.
The Deploy day will offer two individual parallel tracks with something for both experts and beginners:
Newbiesn00bs will learn the basics of OpenStack, Crowbar and DevOps and how they can benefit your organization. We’ll also have time for ecosystem vendors to discuss how they are leveraging OpenStack.Expertsl33ts will take a deep dive into new features of OpenStack Essex and Crowbar, and learn how Crowbar works under the hood, which will enable them to extend the product using Crowbar Barclamps.
Why now? We’ve validated our OpenStack Essex deployment against the latest release bits from Ubuntu. Now it’s time to reach out to the OpenStack and Crowbar communities for training, testing and collaborative development.
Join the event! We’re organizing information on the Crowbar wiki. (I highly recommend you join the Crowbar list to get access to support for prep materials). You can also reach out to me via the @DellCrowbar handle.
We’d love to get you up to speed on the basics and dive deep into the core.
Hungry for Operational Excellence? ChefConf 2012 satisfies! May 23, 2012
Posted by Rob H in Crowbar, DevOps, Events, Lean, Matt Ray, OpenStack, Opscode.Tags: Chef, Crowbar, DevOps, essex, Ignite, Knife, OpenStack, OpsCode
1 comment so far
Since my team at Dell sponsored the inaugural ChefConf, we had the good fortune to get a handful of passes and show up at the event in force. I was also tapped for a presentation (Chef+Crowbar gets Physical+OpenStack Cloud) and Ignite session (Crowbar history).
I live demo’ed using a single command window with knife to manage both physical and cloud infrastructure. That’s freaking cool! (and thanks to Matt Ray for helping to get this working)
It’s no surprise that I’m already a DevOps advocate and Opscode enthusiast, there were aspects of the conference that are worth reiterating:
- Opscode is part of the cadre of leaders redefining how we operate infrastructure. The energy is amazing.
- The acknowledgement of the “snowflake” challenge where all Ops environments are alike, but no two are the same.
- A tight integration between Operations and lean delivery because waterfall deployments are not sustainable
- Opscode’s vision is rooted in utility. You can be successful without design and then excel when you add it. I find that refreshing.
- There was a fun, friendly (“hug driven development?!”) and laid back vibe. This group laughed A LOT.
- For a first conference, Opscode did a good job with logistics and organization.
- I saw that the back rooms and hallways are buzzing with activity. This means that people are making money with the technology.
Crowbar + Chef installs & manages OpenStack Essex (Live Demo, 45 minutes):
Ignite Talk about Dell Crowbar History (5 minutes)
Crowbar v1.3 Release adds Cloudera Hadoop & RHEL/Centos 6.2. Preps v1.4 for Essex & Ubuntu 12.04 May 15, 2012
Posted by Rob H in Cloudera, Crowbar, Hadoop, OpenStack.2 comments
I’m very pleased to post we’ve cut the 1.3 Crowbar release!
1.3 Release Highlights (branch “elefante”)
- Introduction of Cloudera 3.7 Hadoop Barclamp
- New Operating System versions: Ubuntu 11.10, RHEL 6.2, Centos 6.2
- Upgrade the Sledgehammer image to Centos 6.2
- Alias & Group Feature (Alias is linked into DNS & Chef Search)
- Barclamp import from UI
- Pre-populate Node names & descriptions
- Export of logs & database snapshot from UI
- For the Dell Additions: Support for 12g Hardware Models (720xd & 720) via WSMAN
1.4 Previews already in the tree (“essex-hack” branch)
We’ve been working in advance for the 1.4 Release on the Essex-Hack branch.
- Ubuntu 12.04 Support
- OpenStack Essex Packages (from Ubuntu)
Crowbar’s emergence as a DevOps enabled Cloud Provisioner May 9, 2012
Posted by Rob H in Crowbar, Dell, DevOps, Opscode.Tags: Crowbar, DevOps, Mirantis, Sys-Con, Zenoss
1 comment so far
I’m going to be talking Crowbar & OpenStack at Chef Conf next week. While I’m always excited to wave the Crowbar flag, it’s humbling to see our vision for an open source based cloud provisioner picking up momentum in the community.
Dieter Plaetinck
…I think this tool deserves more attention and should be added to your devops toolchain for the cloud (triple buzzword bonus!!!)…
Hosting News
…As an alternative to proprietary, licensed software models, Dell continues to see heavy customer interest in the OpenStack-Powered Cloud Solution, which integrates the OpenStack cloud operating system, cloud-optimized Dell PowerEdge C servers, the Dell-developed Crowbar software framework, and services…
Sys-Con Post by Mirantis
…Finally, there is Dell and Crowbar. Dell’s approach to riding the OpenStack wave is, perhaps, the most creative. Crowbar is neither a hardware appliance nor an enterprise version of OpenStack. It’s a configuration management tool built around OpSource’s Chef, designed specifically to deploy OpenStack on Dell servers. Crowbar effectively serves as the glue between the hardware and any distribution of OpenStack (and not only OpenStack)…
Robert Booth w/ Zenoss
Well if you care to only give them the best then introduce them to a set of tools that will drastically change the way they do business in a way they want to. Introduce them to Dell’s Crowbar and OpsCode Chef and you will make their job easier, faster and possibly put a stop to the finger pointing! No longer will they have to pull out the IT secret decoder ring to understand what the dev team put in the deployment docs.
Four OpenStack Trends from Summit: Practical, Friendly, Effective and Deployable May 9, 2012
Posted by Rob H in Canonical, Crowbar, Meetup, Open source, OpenStack, OpenStack Design Summit, Opscode, Puppet, RackSpace.Tags: Canonical, Crowbar, Culture, Dell, OpenStack, OpsCode, Puppet
add a comment
With the next OpenStack Austin meetup on Thursday (sponsored by Puppet), I felt like it was past time for me to post my thoughts and observations about the Spring 2012 OpenStack design conference. This was my fifth OpenStack conference (my notes about Bexar, Cactus, Diablo & Essex). Every conference has been unique, exciting, and bigger than the previous.
My interest lies in the trend lines of OpenStack. For details about sessions, I recommend Stefano Maffulli‘s excellent link aggregation post for the Summit.
1. Technology Trend: Practical with Potential.
OpenStack started with a BIG vision to become the common platform for cloud API and operations. That vision is very much alive and on-track; however, our enthusiasm for what could be is tempered by the need to build a rock solid foundation. The drive to stability over feature expansion has had a very positive impact. I give a lot of credit for this effort to the leadership of the project technical leads (PTLs), Canonical‘s drive to include OpenStack in the 12.04 LTS and the Rackspace Cloud drive to deploy Essex. My team at Dell has also been part of this trend by focusing so much effort on making OpenStack production deployable (via Crowbar).
Overall, I am seeing a broad-based drive to minimize disruption.
2. Culture Trend: Friendly but some tension.
Companies at both large and small ends of the spectrum are clearly jockeying for position. I think the market is big enough for everyone; however, we are also bumping into each other. Overall, we are putting aside these real and imagined differences to focus on enlarging the opportunity of having a true community cloud platform. For example, the OpenStack Foundation investment formation has moneyed competitors jostling for position to partner together.
However, it’s not just about paying into the club; OpenStack’s history is clearly about execution. Looking back to the original Austin Summit sponsors, we’ve clearly seen that intent and commitment are different.
3. Discussion Trend: Small Groups Effective
The depth & quality of discussions inside sessions was highly variable. Generally, I saw that large group discussions stayed at a very high level. The smaller sessions required deep knowledge of the code to participate and seemed more productive. We continue to have a juggle between discussions that are conceptual or require detailed knowledge of the code. If conceptual, it’s too far removed. If code, it becomes inaccessible to many people.
This has happened at each Summit and I now accept that it is natural. We are using vision sessions to ensure consensus and working sessions to coordinate deliverables for the release.
I cannot over emphasize importance of small groups and delivery driven execution interactions: I spent most of my time in small group discussions with partners aligning efforts.
4. Deployment Trend: Testing and Upstreams matter
Operations for deploying OpenStack is a substantial topic at the Summit. I find that to be a significant benefit to the community because there are a large block of us who were vocal advocates for deployability at the very formation of the project.
From my perspective at Dell, we are proud to see that wide spread acknowledgement of our open source contribution, Crowbar, as the most prominent OpenStack deployer. Our efforts at making OpenStack installable are recognized as a contribution; however, we’re also getting feedback that we need to streamline and simplify Crowbar. We also surprised to hear that Crowbar is “opinionated.” On reflection, I agree (and am proud) of this assessment because it matches best practice coding styles. Since our opinions also drive our test matrix there is a significant value for our OpenStack deployment is that we spend a lot of time testing (automated and manual) our preferred install process.
There’s a push to reconcile the various Chef OpenStack cookbooks into a single upstream. This seems like a very good idea because it will allow various parties to collaborate on open operations. The community needs leadership from Opscode to make this happen. It appears that Puppet Labs is interested in playing a similar role for Puppet modules but these are still emerging and have not had a chance to fragment.
No matter which path we take, the deployment scripts are only as good as their level of testing. Unreliable deployment scripts have are less than worthless.
Dell Team at the OpenStack Spring 2012 Summit April 16, 2012
Posted by Rob H in Andi Abes, Crowbar, Dell, DevOps, Greg Althaus, Joseph George, Matt Ray, OpenStack, OpenStack Design Summit, Opscode.add a comment
It’s OpenStack Summit time again for my team at Dell and there’s deployment in the air. It’s been an amazing journey from the first Austin summit to Folsom today. Since those first heady days, the party has gotten a lot more crowded, founding members have faded away, recruiters became enriched as employees changed email TLDs and buckets of code was delivered.
Throughout, Dell has stayed the course: our focus from day-one has been ensuring OpenStack can be deployed into production in a way that was true to the OpenStack mission of community collaboration and Apache-2-licensed open source.
We’ve delivered on the making OpenStack deployable vision by collaborating broadly on the OpenStack components of the open source Crowbar project. I believe that our vision for sustainable open operations based on DevOps principles is the most complete strategy for production cloud deployments.
We are at the Folsom Summit in force and we’re looking forward to discussions with the OpenStack community. Here are some of the ways to engage with us:
-
Demos
- During the summit (M-W), we’ll have our Crowbar OpenStack Essex deployments running. We kicked off Essex development with a world-wide event in early March and we want more people to come and join in.
- During the conference (W-F), we’ll be showing off application deployments using enStratus and Chef against our field proven Diablo release.
-
Speakers
- Thursday 1:00pm, OpenStack Gains Momentum: Customers are Speaking Up by Kamesh Pemmaraju (Dell)
- Friday 9:50am, Deploy Apps on OpenStack using Dashboard, Chef and enStratus by Rob Hirschfeld (Dell), Matt Ray (Opscode) and Keith Hudgins (enStratus).
- Friday 11:30am, Expanding the Community Panel
including Joseph George (Dell)
-
- This fun round trip road trip from Rackspace & Dell HQs in Austin to the summit and home again promises to be an odyssey of inclusion. Dell OpenStack/Crowbar engineer Andi Abes (@a_abes). Follow @RoadstackRV to follow along as they return home and share their thoughts about the summit!
-
Parties
My team has been in the field talking to customers and doing OpenStack deployments. We are proud to talk about it and our approach.
Mostly importantly, we want to collaborate with you on our Essex deployments using Crowbar. Get on our list, download/build crowbar, run the “essex-hack” branch and start banging on the deploy. Let’s work together to make this one rock solid Essex deploy.