Tag Archives: Barclamp

Crowbar modules (aka barclamps) perform many functions and enable multi-vendor hardware

Posted on by
14

10/18 Update:

More recent information about Barclamps can be found at https://robhirschfeld.com/2011/09/14/details-of-crowbar-changes/.  We’ve also created videos showing how you can create your own barclamps.

Original Post

Just after we’d started deep Crowbar development, Andi Abes, Paul Webster and Victor Lowther joined the Dell Crowbar+OpenStack team.  They immediately started to dig into our Swift, BIOS/RAID, and Network components.  They also started to bump into each other in our original code base.  It quickly became apparent that we needed to modularize Crowbar.

Restructuring Crowbar into modules has proved essential as a method for safe community collaboration.

Greg Althaus coined the name “barclamps” during the modularization rearchitecture.  A barclamp is a class extension of the Crowbar ServiceObject that allows Crowbar to identify the Chef components used by the barclam

p (name p

attern in Chef is bc-template-[barclamp]) and provides capabilities that are specific to each barclamp.

  • In the simplest case, the barclamp is a minimal wrapper that just provides naming hooks for your Chef cookbooks.  This makes it very easily to adapt existing Chef work to work with Crowbar.
  • In more complex cases, the barclamp will help identity how nodes are allocated, interacts with other barclamps, extends the provisioner state machine and provides custom user interfaces.
  • In most cases, the barclamp’s generic integration and UI are sufficient.
Initially, barclamps were entirely exposed via REST using the ServiceObject.  We quickly wrapped those into a CLI for our continuous integration system.  Lately, we’ve expressed them in the user interface.
At launch, you’ll find all but two in the open source repository.  Unfortunately, we were not able to include BIOS and RAID barclamps in the open version because they use licensed components – we are working to correct this.  They are available in the Dell licensed version.
When looking at the barclamps, it is critical to understand that even the most core Crowbar functionality is expressed as a barclamp.
This exposure of Crowbar internals as barclamps is important because it
  1. helps modularize the code and
  2. reflects the deep integration between Chef and Crowbar.

Consequently, the core logic of the state machine, networking configuration, and provisioning are all exposed in barclamps.  This makes it possible to modify and extend the most basic Crowbar operations; however, there are currently no guards against breaking these barclamps either!

The following list includes all the barclamps that we’ve created for Crowbar.
Barclamp   Function  Included
Crowbar The roles and recipes to set up the barclamp framework.  Yes
Deployer Initial classification system for the Crowbar environment (aka the state machine)  Yes
Provisioner The roles and recipes to set up the provisioning server and a base environment for all nodes  Yes
Network Instantiates network interfaces on the crowbar managed systems. Also manages the address pool.  Yes
NTP Common NTP service for the cluster. An NTP server or servers can be specified and all other nodes will be clients of them.  Yes
DNS manages the DNS subsystem for the cluste  Yes
Logging centralized logging system based on syslog  Yes
IPMI Integrates with IP management to allow direct hardware control bypassing the operating system.  Yes
RAID LSI Licensed components.  Cannot be included in open source release at this time.  No
BIOS
 PowerEdge C series: Dell License component.  Cannot be included in open source release at this time.  No
Ganglia Optional: a common Ganglia service for the cluster that can be used by other barclamps  Yes
Nagios Optional : common monitoring service for the cluster that can be used by other barclamps  Yes
Nova OpenStack: installs and configures the Openstack Nova (Cactus Release) component. It relies upon the network and glance barclamps for normal operation.  Yes
Swift OpenStack: part of Openstack (Cactus Release) , and provides a distributed blob storage  Yes
Glance OpenStack: Glance service (Cactus Release, Nova image management) for the cloud  Yes
Test provides a shell for writing tests against  Yes

Modularizing Crowbar via Barclamps – Dell prepares to open source our #OpenStack installer

Posted on by
7

My team at Dell is working diligently to release Crowbar (Apache 2) to the community.

  • We have ramped up our team size (Andi Abes was spotted recently posting on the Swift list).
  • We are collaborating with partners like Rackspace, Opscode and Citrix
  • We brought in UI expertise (Jon Roberts) to improve usability and polish.
  • We are making sure that the code is integrated with our Dell OpenStack Solution (DOSS).
  • We are lining up customers for real field trials.

The single most critical aspect of Crowbar involves a recent architectural change by Greg Althaus to make Crowbar much more modular.  He dubbed the modules “barclamps” because they are used to attach new capabilities into the system.  For example, we include barclamps for DNS, discovery, Nova, Swift, Nagios, Gangalia, and BIOS config.  Users select which combination to use based on their deployment objectives.

In the Crowbar architecture, nearly every capability of the system is expressed as a barclamp.  This means that the code base can be expanded and updated modularly.  We feel that this pattern is essential to community involvement.

For example, another hardware vendor can add a barclamp that does the BIOS configuration for their specific equipment (yes! that is our intent).  While many barclamps will be included with the open source release to install open source components, we anticipate that other barclamps will be only available with licensed products or in limited distribution.

A barclamp is like a cloud menu planner: it evaluates the whole environment and proposes configurations, roles, and recipes that fit your infrastructure.  If you like the menu, then it tells Chef to start cooking.

Barclamps complement the “PXE state machine” aspect of Crowbar by providing logic Crowbar evaluates as the servers reach deployment states.  These states are completely configurable via the provisioner barclamp; consequently, Crowbar users can choose to change order of operations.  They can also add barclamps and easily incorporate them into their workflow where needed.

Barclamps take the form of a Rails controller that inherits from the barclamp superclass.  The superclass provides the basic REST verbs that each barclamp must service while the child class implements the logic to create a “proposal” leveraging the wealth of information in Chef.  Proposals are JSON collections that include configuration data needed for the deployment recipes and a mapping of nodes into roles.

Users are able to review and edit proposals (which are versioned) before asking Crowbar to implement the proposal in Chef.  The proposal is implemented by assigning the nodes into the proposed roles and allowing Chef to work it’s magic.

Users can operate barcamps in parallel.  In fact, most of our barclamps are designed to operate in conjunction.

Reminder: It is vital to understand that Crowbar is not a stand-alone utility.  It is coupled to Chef Server for deployment and data storage.  Our objective was to leverage the outstanding capabilities and community support for Chef as much as possible.

We’re excited about this architecture addition to Crowbar and encourage you to think about barclamps that would be helpful to your cloud deployment.