Tag Archives: Diablo

OpenStack Keystone makes smart & bold move to improve quality

Posted on by
3

Just after the OpenStack
Essex 3 milestone, Ziad Sawalha of Rackspace announced a major shift in the Keystone code base. I applaud the clarity of Ziad’s email but want to restate my understanding of the changes here rather than simply parrot him.

These changes improve Keystone and OpenStack in several ways.

The Keystone team is keeping the current APIs while swapping their implementation. They recommend switching back to an implementation based on the Rackspace Cloud Builder’s Keystone Light code base. I say switching back because my team at Dell has some experience with the Keystone Light (KSL) code. KSL was used with our first Diablo release work while legacy Keystone (Diablo Keystone?) was being readied for release. Upon reflection, the confusion around Keystone readiness for Diablo may have been an indicator to some disconnects that ultimately contributed to last week’s decision.

This is not an 11th hour rewrite. Keystone Light (now Essex Keystone?) offers

  • An existing code base that has been proven in real deployments
  • Stronger identity pluggability, better EC2 compatibility and higher production readiness
  • An existing testing framework and proven extensibility and flexibility
  • Plus, the team has committed to ensure a simple migration path

Beyond the code and Keystone, making a change like this takes confidence and guts.

This change is not all sunshine and rainbows. Making a major change midway through the release cycle introduces schedule and delivery risk. Even though not fully graduated to core project status, Keystone is already an essential component in OpenStack. People will certainly raise valid questions about production readiness and code churn within the project. Changes like these are the reality for any major project and doubly so for platforms.

The very fact that this change is visible and discussed by the OpenStack community shows our strength.

Acknowledging and quickly fixing a weakness in the OpenStack code base is exactly the type of behavior that the community needs to be successful and converge towards a great platform. The fact that maintaining the API is a priority shows that OpenStack is moving in the direction of more API based standards. While the Keystone change is not a recommendation for dual implementations (the Diablo Keystone fork will likely die out), it should help set the stage for how the community will handle competing implementations. If nothing else, it is a strong argument for maintaining API tests and compliance.

The Keystone change is a forward looking one. Our Crowbar team will investigate how we will incorporate it.  As part of OpenStack, the new Keystone code will (re)surface for the Essex deployment and that code will be part of the Dell OpenStack-Powered Cloud.  This work, like the previous, will be done in the open as part of the OpenStack barclamps that we maintain on the Crowbar github.

Austin OpenStack Meetup (January Minutes) + OpenStack Foundation Web Cast!

Posted on by
2

Sorry for the brevity… At the last Austin OpenStack meetup, we had >60 stackers!  Some from as far away as Portland and Boston (as in Oregon and Massachusetts).

Notes:

  • Suse introduced their OpenStack beta and talked about their Suse Studio that can deploy images against the OpenStack APIs
  • I showed off DevStack.org code that can setup the truck of OpenStack (now Essex) in about 10 minutes on a single node.  Great for developers!
  • I showed an OpenStack Diablo Final deployment from Crowbar.  I focused mainly on Dashboard and used our reference architecture (see below) as illustration of the many parts.
  • Matt Ray suggested everyone watch the webcasts about the OpenStack Foundation (Thurs 6pm central  & Friday 9am central)
  • We planned the next few meetups.
    • For February, we’ll talk about Swift and Dashboard.
    • For March, we’ll talk about Essex and DevStack to prep for the next design summit (in SF).
    • For April, we’ll debrief the conference

Thank you Suse and Dell (my employer) for sponsoring!   The next meetup is sponsored by Canonical.

Crowbar 1.2 released includes OpenStack Diablo Final

Posted on by
17

With the holiday rush, I neglected to post about Monday’s Crowbar v1.2 release (ISO here)!

The core focus for this release was to support the OpenStack Diablo Final bits (which my employer, Dell, includes as part of the “Dell OpenStack Powered Cloud Solution“); however, we added a lot of other capability as we continue to iterate on Crowbar.

I’m proud of our team’s efforts on this release on both on features and quality.  I’m equally delighted about the Crowbar community engagement via the Crowbar list server.  Crowbar is not hardware or operating system specific so it’s encouraging to hear about deployments on other gear and see the community helping us port to new operating system versions.

We driving more and more content to Crowbar’s Github as we are working to improve community visibility for Crowbar.  As such, I’ve been regularly updating the Crowbar Roadmap.  I’m also trying to make videos for Crowbar training (suggestions welcome!).  Please check back for updates about upcoming plans and sprint activity.

Crowbar Added Features in v1.2:

  • Central feature was OpenStack Diablo Final barclamps (tag “openstack-os-build”)
  • Improved barclamp packaging
  • Added concepts for “meta” barclamps that are suites of other barclamps
  • Proposal queue and ordering
  • New UI states for nodes & barclamps (led spinner!)
  • Install includes self-testing
  • Service monitoring (bluepill)

Looking forward

Dell has a long list of pending Hadoop and OpenStack deployments using these bits so you can expect to see updates and patches matching our field experiences.  We are very sensitive to community input and want to make Crowbar the best way to deliver a sustainable repeatable reference deployment of OpenStack, Hadoop and other cloud technologies.

OpenStack Deployments Abound at Austin Meetup (12/9)

Posted on by
7

I was very impressed by the quality of discussion at the Deployment topic meeting for Austin OpenStack Meetup (#OSATX). Of the 45ish people attending, we had representations for at least 6 different OpenStack deployments (Dell, HP, ATT, Rackspace Internal, Rackspace Cloud Builders, Opscode Chef)!  Considering the scope of those deployments (several are aiming at 1000+ nodes), that’s a truly impressive accomplishment for such a young project.

Even with the depth of the discussion (notes below), we did not go into details on how individual OpenStack components are connected together.  The image my team at Dell uses is included below.  I also recommend reviewing Rackspace’s published reference architecture.

Figure 1 Diablo Software Architecture. Source Dell/OpenStack (cc w/ attribution)

Notes

Our deployment discussion was a round table so it is difficult to link statements back to individuals, but I was able to track companies (mostly).

  • HP
    • picked Ubuntu & KVM because they were the most vetted. They are also using Chef for deployment.
    • running Diablo 2, moving to Diablo Final & a flat network model. The network controller is a bottleneck. Their biggest scale issue is RabbitMQ.
    • is creating their own Nova Volume plugin for their block storage.
    • At this point, scale limits are due to simultaneous loading rather than total number of nodes.
    • The Nova node image cache can get corrupted without any notification or way to force a refresh – this defect is being addressed in Essex.
    • has setup availability zones are completely independent (500 node) systems. Expecting to converge them in the future.
  • Rackspace
    • is using the latest Ubuntu. Always stays current.
    • using Puppet to setup their cloud.
    • They are expecting to go live on Essex and are keeping their deployment on the Essex trunk. This is causing some extra work but they expect it to pay back by allowing them to get to production on Essex faster.
    • Deploying on XenServer
    • “Devs move fast, Ops not so much.”  Trying to not get behind.
  • Rackspace Cloud Builders (RCB) is running major releases being run through an automated test suite. The verified releases are being published to https://github.com/cloudbuilders (note: Crowbar is pulling our OpenStack bits from this repo).
  • Dell commented that our customers are using Crowbar primarily pilots – they are learning how to use OpenStack
    • Said they have >10 customer deployments pending
    • ATT is using OpenSource version of Crowbar
    • Need for Keystone and Dashboard were considered essential additions to Diablo
  • Hypervisors
    • KVM is considered the top one for now
    • Libvirt (which uses KVM) also supports LXE which people found to be interesting
    • XenServer via XAPI are also popular
    • No so much activity on ESX & HyperV
    • We talked about why some hypervisors are more popular – it’s about the node agent architecture of OpenStack.
  • Storage
    • NetApp via Nova Volume appears to be a popular block storage
  • Keystone / Dashboard
    • Customers want both together
    • Including keystone/dashboard was considered essential in Diablo. It was part of the reason why Diablo Final was delayed.
    • HP is not using dashboard
OpenStack API
  • Members of the Audience made comments that we need to deprecate the EC2 APIs (because it does not help OpenStack long term to maintain EC2 APIs over its own).  [1/5 Note: THIS IS NOT OFFICIAL POLICY, it is a reflection of what was discussed]
  • HP started on EC2 API but is moving to the OpenStack API

Meetup Housekeeping

  • Next meeting is Tuesday 1/10 and sponsored by SUSE (note: Tuesday is just for this January).  Topic TBD.
  • We’ve got sponsors for the next SIX meetups! Thanks for Dell (my employeer), Rackspace, HP, SUSE, Canonical and PuppetLabs for sponsoring.
  • We discussed topics for the next meetings (see the post image). We’re going to throw it to a vote for guidance.
  • The OSATX tag is also being used by Occupy San Antonio.  Enjoy the cross chatter!

OpenStack Seattle Meetup 11/30 Notes

Posted on by
6

We had an informal OpenStack meetup after the Opscode Summit in Seattle.

This turned out to be a major open cloud gab fest! In addition to Dell OpenStack leads (Greg and I), we had the Nova Project Technical Lead (PTL, Vish Ishaya, @vish), HP’s Cloud Architect (Alex Howells, @nixgeek), Opscode OpenStack cookbook master (Matt Ray, @mattray). We were joined by several other Chef Summit attendees with OpenStack interest including a pair of engineers from Spain.

We’d planned to demo using Knife-OpenStack against the Crowbar Diablo build.  Unfortunately, the knife-openstack is out of date (August 15th?!).  We need Keystone support.  Anyone up for that?

Highlights

There’s no way I can recapture everything that was said, but here are some highlights I jotted down the on the way home.

  • After the miss with Keystone and the Diablo release, solving the project dependency problem is an important problem. Vish talked at length about the ambiguity challenge of Keystone being required and also incubated. He said we were not formal enough around new projects even though we had dependencies on them. Future releases, new projects (specifically, Quantum) will not be allowed to be dependencies.
  • The focus for Essex is on quality and stability. The plan is for Essex to be a long-term supported (LTS) release tied to the Ubuntu LTS. That’s putting pressure on all the projects to ensure quality, lock features early, and avoid unproven dependencies.
  • There is a lot of activity around storage and companies are creating volume plug-ins for Nova. Vish said he knew of at least four.
  • Networking has a lot of activity. Quantum has a lot of activity, but may not emerge as a core project in time for Essex. There was general agreement that Quantum is “the killer app” for OpenStack and will take cloud to the next level.  The Quantum Open vSwitch implementaiton is completely open source and free. Some other plugins may require proprietary hardware and/or software, but there is definitely a (very) viable and completely open source option for Quantum networking.
  • HP has some serious cloud mojo going on. Alex talked about defects they have found and submitted fixes back to core. He also hinted about some interesting storage and networking IP that’s going into their OpenStack deployment. Based on his comments, I don’t expect those to become public so I’m going to limit my observations about them here.
  • We talked about hypervisors for a while. KVM and XenServer (via XAPI) were the primary topics. We did talk about LXE & OpenVZ as popular approaches too. Vish said that some of the XenServer work is using Xen Storage Manager to manage SAN images.
  • Vish is seeing a constant rise in committers. It’s hard to judge because some committers appear to be individuals acting on behalf of teams (10 to 20 people).

Note: cross posted on the OpenStack Blog.

Reminder: 12/8 Meetup @ Austin!

Missed this us in Seattle? Join us at the 12/8 OpenStack meetup in Austin co-hosted by Dell and Rackspace.

Based on our last meetup, it appears deployment is a hot topic, so we’ll kick off with that – bring your experiences, opinions, and thoughts! We’ll also open the floor to other OpenStack topics that would be discussed – open technical and business discussions – no commercials please!

We’ll also talk about organizing future OpenStack meet ups! If your company is interested in sponsoring a future meetup, find Joseph George at the meetup and he can work with you on details.

Crowbar community support and 111111 sprint plan

Posted on by
Reply

The Dell Crowbar team is working to improve road map transparency. In the last few weeks, the Crowbar community has become more active on our lists, testing builds, and helping with documentation.

We love the engagement and continue to make supporting the list a priority.

Participation in Crowbar, OpenStack and Hadoop has been exceeding our expectations and we’re working to implement more community support and process. Thank you!!!

Our next steps:

  1. I’ve committed to post sprint plans and summary pages (this is the first)
  2. New Crowbar Twitter account
  3. I’m going to setup feature voting on the Crowbar Facebook page (like to vote)
  4. Continue to work the listserv and videos. We need help converting those to documentation on the crowbar wiki.
  5. Formalize collaborator agreements – we’re working with legal on this
  6. Exploring the option of a barclamp certification program and Crowbar support
  7. Moving to a gated trunk model for internal commits to improve quality
  8. Implementing a continuous integration system that includes core and barclamps. This will be part of our open source components.

We are working towards the 1.2 release (Beta 1) . That release is focused on supporting OpenStack but includes enhancements for upgrades, Hadoop, and additional OS support.

Our Sprint 111111 plan.

Source: Crowbar Wiki: [[sprint 111111]]

  • Theme: OpenStack Diablo Final release candidate.
  • Core Work: Refine Deployment for Nova, Glance, Nova Dashboard (horizon), keystone, swift
  • New additions: mySQL barclamp, Nova HA networking, kong
  • Crowbar internals: expose error states for proposals, allow packages to be included with barclamps to make upgrades easier, barclamp group pages
  • Operating system: added CentOS
  • Documentation: we’ve split the user guides into distinct books so Crowbar, OpenStack, and Hadoop each have their own user guide.
  • Pending action: expose the Hadoop barclamps
  • OS note: OpenStack is being tested (at Dell) against Ubuntu 10.10 only. Hadoop was tested against RHEL 5.7 and we expect it to work against CentOS also.

Crowbar OpenStack deployment video (15 mins): Diablo + Keystone + Dashboard

Posted on by
12

This week at the OpenStack Design Summit and Conference in Boston, my team unveiled the Diablo+ Crowbar deployments. The OpenStack deployment that’s included with Crowbar reflects a collaborative effort between Dell, Opscode, and Rackspace and pulls packages from the Rackspace repository. It was important for us to use the Rackspace repos so that we could include integrated Keystone and Dashboard components that were omitted from the Diablo (current) release. Our decision to include these Essex (coming) components is based on customer feedback.

Since some of you cannot make it to the show and see the demo in person, we’ve captured it as a video for your enjoyment. The OpenStack deployment is available in our open source distribution. We are currently in QA for the overall solution so expect additional refinement as we progress towards our next OpenStack solution release.

REMINDER: Dell Hardware is NOT required to use Crowbar for OpenStack.  The open source version has everything you need – the BIOS and RAID barclamps are optional (but handy).

Shout, chat and whisper with Dell at OpenStack Design Summit & Conference

Posted on by
2

My team at Dell has been very (very) busy delivering a lot of great materials for the Fall 2011 OpenStack Design Summit & Conference in Boston MA next week.

Our motto for this conference is “DOING IS DOING” or, perhaps, “DIABLO IS DOING.”

You can count on Dell to be walking the walk with deliverables that advance OpenStack.  In fact, you can watch what we’re doing because it’s posted live as we work with the community to build it.

First, we’ll have our Crowbar demo rack showcasing LIVE MULTI-NODE DIABLO DEPLOYMENTS and some IMPORTANT FEATURE AND COMMUNITY ADDITIONS.  No spoilers here – you’ll have to come by.  Of course, it’s in the git hub too, but we’ve put a bow on it.

Second, there’s a DEPLOYMENT BLUE PRINT discussion about getting better interlocks between OpenStack development and deployment.  We really need to reduce the pain and lag between adding great features and using those features.

Next, we’ve got a limited audience CONCEPT SNEAK PEEK for something from our labs that we think is very interesting and we’d like to get input about.  Unfortunately, we’re very limited with space & time for this whisper session so you’ll need to contact OpenStack@Dell.com to request an invitation.

Finally, at the Conference, you can see OUR TEAM IN ACTION:

  • Thurs 11:30 – Dell Keynote by John Igoe
  • Thurs 3:30 – Private Cloud Panel w/ Rob Hirschfeld
  • Thurs 4:30 – Hardware Infrastructure w/ Rob Hirschfeld & Greg Althaus
  • Friday 11:00 – Deployments w/ Greg Althaus
  • Friday 3:15 – Crowbar!! w/ Scott Jensen (yes, he does it with the !!)
  • Friday 4:15 – KVM & OpenStack

More conference posts: JB Gorge & Barton George.

Dell Crowbar to deploy OpenStack Diablo Cloud

Posted on by
5
Direction in the Cloud

Photo by JB George

This week, some of the Crowbar/Dell OpenStack-Powered Cloud team, plus Matt Ray from Opscode, have been working with our partners at Rackspace in San Antonio (see Opscode post about collaboration). Our target is to have Crowbar deliver a core Diablo deployment by the October 2011 design conference (sponsored in part by Dell). This is a collaborative effort and we invite community participation – we are trying to be open and communicative (via the Crowbar listserv) while also respecting that there is a mountain of work if we are to meet deadlines.

We are doing the work in the open on the Crowbar Github so you have access to the very latest capabilities and it also means that the head the Crowbar may be unstable while we add capabilities. We feel like this is an important trade off because it allows us to keep up with the rapid pace of development in OpenStack (and other projects). This is the motivation for the recent modularization work and will continue to be a feature driver for Crowbar enhancements because it allows Crowbar users to easily bring in updated bits.

 

Crowbar modularization work begins

Posted on by
3

I shared the following with the Crowbar listserv and wanted to post it for the larger audience.  If you want the latest on Crowbar then subscribe!

We’ve been getting questions and defects (thanks Matt Ray) about how we are going to allow you to update and add barclamps to Crowbar.  We’re working on that exact issue right now – you can watch me on the “modules” branch of the github.

NOTE TO CROWBAR FOLLOWERS: we are moving some items around in the repo!  There are “cactus” and “v1.0” tags in place so you can still build the current trees after we start the refactor.

We’ve got some big plans that I’ll outline on the list and earlier posts.

Right now, we’re working to modularize barclamps so that each one is in its own github repo.   This will allow you to pull in barclamps at build time or live on site. We’re also creating import/update routines that work for live systems to make it easier to develop barclamps.  Once again, that’s on the github modules branch. These will be exposed as rake barclamp:create[“foo”] and rake barclamp:install[“../foo”] type commands and I’ve committed to create some “how to make barclamps” videos.

That work is a prelude for a hard push on OpenStack Diablo before the design conference.  All that work will also be done in the github but the Diablo barclamps will be in independent repos from the Crowbar framework.

If you want to get started early.  80% of a barclamp effort is around the Chef Cookbooks.  Keith Hudgins with DTO did a great job writing up barclamps here: http://kb.dtosolutions.com/wiki/Deploying_the_cloudfoundry_barclamp.  We’re changing some of it to make it much easier and more modular.