Greg Althaus at 11/15 Austin Cloud User Group meeting (annotated 90 min audio)

Greg Althaus did a 90 minute Crowbar deep dive at this week’s Austin Cloud User Group.  Brad Knowles recorded audio and posted it so I thought I’d share the link and my annotations.  There are a lot more times to catch up with our team at Dell in Austin, Boston, and Seattle.

Video Annotations –  (##:## time stamp)

  • 00:00 Intros & Meeting Management
  • 12:00 Joseph George Introduction / Sponsorship
  • 16:00 Greg Starts – why Crowbar
  • 19:00 DevOps slides
  • 21:00 What does Crowbar do for DevOps
    • make it easier to manage
    • make it easier to repeat
  • 24:00 What’s included – how we grow / where to start
  • 27:20 Starting to show crowbar – what’s included as barclamps
    • pluggable / configuration
    • Barclamps!
  • 28:10 What is a barclamp
    • discussion about the barclamps in the base
  • 34:30: We ❤ Chef. Puppet vs Chef
  • 36:00 Why barclamps are more than cookbooks
  • 36:30 State machine & transitions
  • Q&A Section
    • 38:50 Reference Architectures
    • 43:00 Barclamps work outside of Crowbar?
    • 44:15 Hardware models supported
    • 47:30 Storage Queston
    • 49:00 HA progress
    • 53:00 Ceph as a distributed cloud on all nodes
    • 56:20 Deployer has a map of how to give out roles
  • 58:00 Demo Fails
  • 58:30 Crowbar Architecture
  • 62:00 How Crowbar can be extended
  • 63:00 Workflow & Proposals
  • 65:40 Meta Barclamps
  • 71:10 Chef Environments
  • 73:40 Taking OpenStack releases and Environments
  • 75:00 The case for remove recipes
  • 77:33 Git Hub Tour
  • 79:00 Network Barclamp deep dive
  • 84:00 Adding switch config (roadmap topic)
  • 86:30 Conduits
  • 87:40 Barclamp Extensions / Services
  • 89:00 Questions
    • 89:20 Proposal operations
    • 93:30 OpenStack Readiness & Crowbar Design Approach
    • 93:10 Network Teaming
    • 94:30 Which OS & Hypervisors
    • 96:30 Continuous Integration & Tools
    • 98:40 BDD (“cucumberesque”) & Testing
    • 99:40 Build approach & barclamp construction
  • 100:00 Wrap up by Joseph

Dell is open sourcing Crowbar Apache Hadoop barclamps!

I’m very excited to announce that my team at Dell will be open sourcing our Apache Hadoop Crowbar barclamps by the end of the month.

This release raises the bar on open Hadoop deployments by making them faster, scalable, more integrated and repeatable.

These barclamps were developed in conjunction with our licensed Dell | Cloudera Solution. The licensed solution is for customers seeking large scale and professionally supported big data solutions. The purpose of the open barclamps (which pull the open source parts from the Cloudera distro) is to help you get started with Hadoop and reduce your learning curve. Our team invested significant testing effort in ensuring that these barclamps work smoothly because they are the foundational layer of our for-pay Hadoop solution.

Included in the Hadoop barclamp suite are Hadoop Map Reduce, Hive, Pig, ZooKeeper and Sqoop running on RHEL 5.7. These barclamps cover the core parts of the Hadoop suite. Like other Crowbar deployments (see OpenStack), the barclamps automatically discover the service configurations and interoperate. One of our team members (call him Scott Jensen) said it very simply “I can deploy a fully an integrated Hadoop cluster in a few hours. That friggin’ rocks!” I just can’t put it more eloquently than that!

I’ll post again when we flip the “open” bit and invite our community to dig in and help us continue to set the standards on open Hadoop deployments.

For more perspectives on this release, check out posts by Barton George (just for devs), Joseph George (About Hadoop) and Aurelian Dumitru

Barton posted these two videos of me talking about the release too:

Hadoop & Crowbar:

Dev’s Only Short:

Crowbar OpenStack deployment video (15 mins): Diablo + Keystone + Dashboard

This week at the OpenStack Design Summit and Conference in Boston, my team unveiled the Diablo+ Crowbar deployments. The OpenStack deployment that’s included with Crowbar reflects a collaborative effort between Dell, Opscode, and Rackspace and pulls packages from the Rackspace repository. It was important for us to use the Rackspace repos so that we could include integrated Keystone and Dashboard components that were omitted from the Diablo (current) release. Our decision to include these Essex (coming) components is based on customer feedback.

Since some of you cannot make it to the show and see the demo in person, we’ve captured it as a video for your enjoyment. The OpenStack deployment is available in our open source distribution. We are currently in QA for the overall solution so expect additional refinement as we progress towards our next OpenStack solution release.

REMINDER: Dell Hardware is NOT required to use Crowbar for OpenStack.  The open source version has everything you need – the BIOS and RAID barclamps are optional (but handy).

Building Crowbar post-modularization (15 minute how-to video)

Note: I’m putting build ISOs and Sledgehammer TARs on crowbar.zehicle.com if you don’t want to follow these steps then download the ISO. We are updating the ISO daily, so don’t assume that you have that latest!

To build Crowbar, you need a Linux machine and access to the internet. The video shows how you can use an Ubuntu 10.10 Rackspace Cloud Server.  We build Crowbar inside our firewall on our PCs too. No matter how you do it, Crowbar is full of fuzzily delicious cloud bits.

For up-to-date instructions, see the Crowbar wiki Build ISO page.


Video of Crowbar Install & Introduction (two 15 min videos)

These Crowbar videos are the first two in a series of how to setup and use your own local Crowbar dev environment (here’s more info & the ISO).   I used VMware Workstation, but any virtual hosts that support Ubuntu 10.10 will work fine.  We use ESX, KVM and Xen for testing too.

Creating this environment is the basis for learning Crowbar, experimenting with OpenStack and creating your own barclamps.  It’s also a handy way to play with Opscode Chef since it includes a stand alone Chef server.

Some items from the install video that I want to re-emphasize:

  • The admin server REQUIRES >1 GB of RAM (more is better)
  • All other nodes REQUIRE > 512 MB of RAM (more depending on what you install)
  • At least TWO NICS are required.
  • You must disable DHCP on the virtual network because Crowbar has a DHCP server and they will conflict.
  • The login for the Crowbar admin server is openstack/openstack.  You must “sudo -i” before you run the install script.
  • The default url for Crowbar is http://192.168.124.10:3000  (crowbar/crowbar)

Installing Crowbar (15 mins)

Introduction to Crowbar (15 mins)

9/30 Bonus Video: How to change the default networking

Continue reading