jump to navigation

Crowbar modules (aka barclamps) perform many functions and enable multi-vendor hardware July 25, 2011

Posted by Rob H in Crowbar, Greg Althaus.
Tags: , , , , , , , , ,
trackback

10/18 Update:

More recent information about Barclamps can be found at http://robhirschfeld.com/2011/09/14/details-of-crowbar-changes/.  We’ve also created videos showing how you can create your own barclamps.

Original Post

Just after we’d started deep Crowbar development, Andi Abes, Paul Webster and Victor Lowther joined the Dell Crowbar+OpenStack team.  They immediately started to dig into our Swift, BIOS/RAID, and Network components.  They also started to bump into each other in our original code base.  It quickly became apparent that we needed to modularize Crowbar.

Restructuring Crowbar into modules has proved essential as a method for safe community collaboration.

Greg Althaus coined the name “barclamps” during the modularization rearchitecture.  A barclamp is a class extension of the Crowbar ServiceObject that allows Crowbar to identify the Chef components used by the barclam

p (name p

attern in Chef is bc-template-[barclamp]) and provides capabilities that are specific to each barclamp.

  • In the simplest case, the barclamp is a minimal wrapper that just provides naming hooks for your Chef cookbooks.  This makes it very easily to adapt existing Chef work to work with Crowbar.
  • In more complex cases, the barclamp will help identity how nodes are allocated, interacts with other barclamps, extends the provisioner state machine and provides custom user interfaces.
  • In most cases, the barclamp’s generic integration and UI are sufficient.
Initially, barclamps were entirely exposed via REST using the ServiceObject.  We quickly wrapped those into a CLI for our continuous integration system.  Lately, we’ve expressed them in the user interface.
At launch, you’ll find all but two in the open source repository.  Unfortunately, we were not able to include BIOS and RAID barclamps in the open version because they use licensed components – we are working to correct this.  They are available in the Dell licensed version.
When looking at the barclamps, it is critical to understand that even the most core Crowbar functionality is expressed as a barclamp.
This exposure of Crowbar internals as barclamps is important because it
  1. helps modularize the code and
  2. reflects the deep integration between Chef and Crowbar.

Consequently, the core logic of the state machine, networking configuration, and provisioning are all exposed in barclamps.  This makes it possible to modify and extend the most basic Crowbar operations; however, there are currently no guards against breaking these barclamps either!

The following list includes all the barclamps that we’ve created for Crowbar.
Barclamp   Function  Included
Crowbar The roles and recipes to set up the barclamp framework.  Yes
Deployer Initial classification system for the Crowbar environment (aka the state machine)  Yes
Provisioner The roles and recipes to set up the provisioning server and a base environment for all nodes  Yes
Network Instantiates network interfaces on the crowbar managed systems. Also manages the address pool.  Yes
NTP Common NTP service for the cluster. An NTP server or servers can be specified and all other nodes will be clients of them.  Yes
DNS manages the DNS subsystem for the cluste  Yes
Logging centralized logging system based on syslog  Yes
IPMI Integrates with IP management to allow direct hardware control bypassing the operating system.  Yes
RAID LSI Licensed components.  Cannot be included in open source release at this time.  No
BIOS
PowerEdge C series: Dell License component.  Cannot be included in open source release at this time.  No
Ganglia Optional: a common Ganglia service for the cluster that can be used by other barclamps  Yes
Nagios Optional : common monitoring service for the cluster that can be used by other barclamps  Yes
Nova OpenStack: installs and configures the Openstack Nova (Cactus Release) component. It relies upon the network and glance barclamps for normal operation.  Yes
Swift OpenStack: part of Openstack (Cactus Release) , and provides a distributed blob storage  Yes
Glance OpenStack: Glance service (Cactus Release, Nova image management) for the cloud  Yes
Test provides a shell for writing tests against  Yes

Comments»

1. #Crowbar source released, includes #OpenStack Cloud install (#apache2 #opschef #Dell #cloud) « Rob Hirschfeld's Blog - July 25, 2011

[...] comprehensive set of barclamps to set up an OpenStack [...]

2. How OpenStack installer (crowbar + chefops) works (video from 3/14 demo) « Rob Hirschfeld's Blog - August 5, 2011

[...] 4+ months following this post, we substantially refactored the code make make it more modular (see Barclamps), better looking, and multi-vendor/multi-application (Hadoop & RHEL).  If you want more [...]

3. Collaboration between Dell Crowbar & VMware Cloud Foundry – unleashes your inner cloud « Rob Hirschfeld's Blog - August 17, 2011

[...] single sprint can deliver magic: when I signed up to document how to create a Crowbar module (aka a barclamp) two weeks ago, I had no idea that it would add a new flavor to Crowbar . I’m proud to announce [...]

4. Canonical, Dell, EnStratus and Opscode Join Cloud Foundry | SiliconANGLE - August 24, 2011

[...] A Crowbar module, also known as a barclamp, is coming to Cloud Foundry. Crowbar automates the installation and configuration of open-source [...]

5. Crowbar near-term features: increasing DevOps mojo and brewing Diablo « Rob Hirschfeld's Blog - August 24, 2011

[...] Diablo Barclamps. Expect to see individual barclamps for various components like Keystone, Dashboard, Glace, Nova, Swift, [...]

6. Crowbar modularized: latest changes that make clouds even easier to create, update, and maintain « Rob Hirschfeld's Blog - September 19, 2011

[...] this version, each Crowbar barclamp is an independent delivery unit that can be integrated before, while or after installing [...]

7. So you want to create a Crowbar barclamp? Here’s what you have to know… « Rob Hirschfeld's Blog - September 19, 2011

[...] major objective of our recent modularization refactoring was to make it easier to the community to contribute barclamps. The Crowbar CloudOps approach is to build up a full cloud deployment using layers. So each [...]

8. My Bookmarks » New Cookbooks from Opscode for OpenStack Automation and Deployment - October 4, 2011

[...] It makes use of Chef Cookbooks (also known as “barclamps” here) that do everything from network provisioning to direct hardware control. [...]

9. New Cookbooks from Opscode for OpenStack Automation and Deployment — NetworqScience Blog - October 4, 2011

[...] It makes use of Chef Cookbooks (also known as “barclamps” here) that do everything from network provisioning to direct hardware control. [...]

10. New Cookbooks from Opscode for OpenStack Automation and Deployment - ReadWriteCloud - October 4, 2011

[...] It makes use of Chef Cookbooks (also known as "barclamps" here) that do everything from network provisioning to direct hardware control. [...]

11. New Cookbooks from Opscode for OpenStack Automation and Deployment | Technology NZ - October 4, 2011

[...] It makes use of Chef Cookbooks (also known as "barclamps" here) that do everything from network provisioning to direct hardware control. [...]

12. Windows Azure and Cloud Computing Posts for 10/4/2011+ - Windows Azure Blog - October 4, 2011

[...] It makes use of Chef Cookbooks (also known as "barclamps" here) that do everything from network provisioning to direct hardware control. [...]

13. Crowbar source released, includes OpenStack Cloud install « Rob Hirschfeld's Blog - October 10, 2011

[...] comprehensive set of barclamps to set up an OpenStack [...]

14. Dell Crowbar Project: Open Source Cloud Deployer expands into the Community « Rob Hirschfeld's Blog - October 18, 2011

[...] Barclamps are a very significant architecture pattern for Crowbar: [...]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 980 other followers