Tag Archives: ubuntu

OpenStack Seattle Meetup 11/30 Notes

Posted on by
6

We had an informal OpenStack meetup after the Opscode Summit in Seattle.

This turned out to be a major open cloud gab fest! In addition to Dell OpenStack leads (Greg and I), we had the Nova Project Technical Lead (PTL, Vish Ishaya, @vish), HP’s Cloud Architect (Alex Howells, @nixgeek), Opscode OpenStack cookbook master (Matt Ray, @mattray). We were joined by several other Chef Summit attendees with OpenStack interest including a pair of engineers from Spain.

We’d planned to demo using Knife-OpenStack against the Crowbar Diablo build.  Unfortunately, the knife-openstack is out of date (August 15th?!).  We need Keystone support.  Anyone up for that?

Highlights

There’s no way I can recapture everything that was said, but here are some highlights I jotted down the on the way home.

  • After the miss with Keystone and the Diablo release, solving the project dependency problem is an important problem. Vish talked at length about the ambiguity challenge of Keystone being required and also incubated. He said we were not formal enough around new projects even though we had dependencies on them. Future releases, new projects (specifically, Quantum) will not be allowed to be dependencies.
  • The focus for Essex is on quality and stability. The plan is for Essex to be a long-term supported (LTS) release tied to the Ubuntu LTS. That’s putting pressure on all the projects to ensure quality, lock features early, and avoid unproven dependencies.
  • There is a lot of activity around storage and companies are creating volume plug-ins for Nova. Vish said he knew of at least four.
  • Networking has a lot of activity. Quantum has a lot of activity, but may not emerge as a core project in time for Essex. There was general agreement that Quantum is “the killer app” for OpenStack and will take cloud to the next level.  The Quantum Open vSwitch implementaiton is completely open source and free. Some other plugins may require proprietary hardware and/or software, but there is definitely a (very) viable and completely open source option for Quantum networking.
  • HP has some serious cloud mojo going on. Alex talked about defects they have found and submitted fixes back to core. He also hinted about some interesting storage and networking IP that’s going into their OpenStack deployment. Based on his comments, I don’t expect those to become public so I’m going to limit my observations about them here.
  • We talked about hypervisors for a while. KVM and XenServer (via XAPI) were the primary topics. We did talk about LXE & OpenVZ as popular approaches too. Vish said that some of the XenServer work is using Xen Storage Manager to manage SAN images.
  • Vish is seeing a constant rise in committers. It’s hard to judge because some committers appear to be individuals acting on behalf of teams (10 to 20 people).

Note: cross posted on the OpenStack Blog.

Reminder: 12/8 Meetup @ Austin!

Missed this us in Seattle? Join us at the 12/8 OpenStack meetup in Austin co-hosted by Dell and Rackspace.

Based on our last meetup, it appears deployment is a hot topic, so we’ll kick off with that – bring your experiences, opinions, and thoughts! We’ll also open the floor to other OpenStack topics that would be discussed – open technical and business discussions – no commercials please!

We’ll also talk about organizing future OpenStack meet ups! If your company is interested in sponsoring a future meetup, find Joseph George at the meetup and he can work with you on details.

Crowbar community support and 111111 sprint plan

Posted on by
Reply

The Dell Crowbar team is working to improve road map transparency. In the last few weeks, the Crowbar community has become more active on our lists, testing builds, and helping with documentation.

We love the engagement and continue to make supporting the list a priority.

Participation in Crowbar, OpenStack and Hadoop has been exceeding our expectations and we’re working to implement more community support and process. Thank you!!!

Our next steps:

  1. I’ve committed to post sprint plans and summary pages (this is the first)
  2. New Crowbar Twitter account
  3. I’m going to setup feature voting on the Crowbar Facebook page (like to vote)
  4. Continue to work the listserv and videos. We need help converting those to documentation on the crowbar wiki.
  5. Formalize collaborator agreements – we’re working with legal on this
  6. Exploring the option of a barclamp certification program and Crowbar support
  7. Moving to a gated trunk model for internal commits to improve quality
  8. Implementing a continuous integration system that includes core and barclamps. This will be part of our open source components.

We are working towards the 1.2 release (Beta 1) . That release is focused on supporting OpenStack but includes enhancements for upgrades, Hadoop, and additional OS support.

Our Sprint 111111 plan.

Source: Crowbar Wiki: [[sprint 111111]]

  • Theme: OpenStack Diablo Final release candidate.
  • Core Work: Refine Deployment for Nova, Glance, Nova Dashboard (horizon), keystone, swift
  • New additions: mySQL barclamp, Nova HA networking, kong
  • Crowbar internals: expose error states for proposals, allow packages to be included with barclamps to make upgrades easier, barclamp group pages
  • Operating system: added CentOS
  • Documentation: we’ve split the user guides into distinct books so Crowbar, OpenStack, and Hadoop each have their own user guide.
  • Pending action: expose the Hadoop barclamps
  • OS note: OpenStack is being tested (at Dell) against Ubuntu 10.10 only. Hadoop was tested against RHEL 5.7 and we expect it to work against CentOS also.

Technical details of pending Crowbar changes

Posted on by
2

We’re testing a HUGE batch of changes to Crowbar before we commit them. The changes support the barclamp modularization work and also include the addition of RHEL and network barclamp update.

You may be eager to dig in; however, disruptiveness of these changes means that we are taking extra time to make sure that the build and install still work.

Here’s what you’ll see when we commit the changes:

  • Changes in naming to be more generic
    • Crowbar server user/pass is now crowbar/crowbar (was openstack/openstack)
    • Rails app path now crowbar_framework (was openstack_manager )
  • The pre-split barclamps (/change-image/dell/barclamps/*) have been moved into individual github repos (barclamp-*).
    • Barclamps are pulled into the build using “git submodule”
    • Chef scripts for barclamps are no longer copied and comingled together in the chef directory. They remain in their source directories (default /opt/dell/barclamps)
  • Inside the barclamps, you’ll find
    • A crowbar configuration file to direct the barclamp installer including localization and menu extensions.
    • Path changes to better align with the destination paths (command_line -> bin, app ->crowbar_framework)
    • App views moved under subdirectories
  • Changes to installation scripts
    • Barclamp installation changed to a ruby library so it can do more and be used individually outside of the install process. This allows barclamps to be imported or updated after installation.
    • Changes to create accommodate multiple operating systems
  • Addition of a “redhat-5.6-extra” directory with the RHEL 5.6 installation build components.
    • The RHEL version installs Opcode Chef Server 0.10 (Ubuntu is still 0.9 – community help here?)
  • Crowbar framework Rails app runs under Rainbow instead of Apache.
  • The code for the framework and the barclamp installer has been moved into the crowbar barclamp.
    • The installer bootstraps the crowbar barclamp to install itself.
  • The network barclamp has been substantially changed – that will require additional documentation. Features include
    • Concept of “conduits” that are constructed on nodes to be shared between barclamps
    • Ability to map adapters in a general way to deal with inconsistent enumeration
    • Mapping conduits to adapters allows for new teaming and multiple teaming configurations

We’ll post to the Crowbar listserv when changes. They will be posted to Crowbar HEAD. If you want the current build, we have created a “v1.0” tag.

Crowbar build using Ubuntu 10.10 vm on Rackspace Cloud from Github Repo

Posted on by
11

Our OpenStack team at Dell (especially Victor Lowthor) has been working hard with the public Crowbar repos to make it possible for the community to build their own version of a Crowbar ISO.   When you build the ISO, you’ll be downloading a whole bunch (that’s the technical term) of open source licensed components to make it work: we’re trying to maintain a list of licenses on the Github wiki.

To make sure that it was possible for mortals, I signed up for a Ubuntu 10.10 VM (512 Mb RAM, $0.03/hr) at RackSpace Cloud.  I did this from a non-Dell to ensure that it was as independent from our source as possible.

Once I had my vm, there were just a few steps to follow (these are NOT verbatim):

  • apt-get install debootstrap, mkisofs, git, build-essential packages
  • git clone git://github.com/dellcloudedge/crowbar.git
  • Got the results from a sledgehammer build (a fresh sledgehammer tarball) and extracted it into $HOME/.crowbar-build-cache/tftpboot, which is where build_crowbar.sh expects to find it cached.
    • NOTE: I’m not ready to document sledgehammer builds yet, but I will tell you that you’d need a CentOS VM.
  • In the crowbar directory, ran ./build_crowbar.sh
  • The build will pull down all the packages that you need and cache them to the VM.  Subsequent builds will be much faster!

The end result of the build is an “openstack-dev.iso” that will install Crowbar with the OpenStack barclamps (here’s how to do it on VMs).  Just for fun, I copied _my build_ output ISO off the build VM and to my web server.

Please let me know if you have problems with this process, we want people to try Crowbar!

$$ Note: Turn off your VM when you’re done so you don’t incur extra expenses.  Since this process only took about 2 hours, the whole build cost me less than a dime.  Which is good, since I was building it on “my own dime” anyway.

Network World on Ubuntu Cloud

Posted on by
Reply

My team at Dell is working on solutions around this cloud strategy.  I like the approach that Canonical & Ecalyptus are taking concerning the use of open source (KVM), ad hoc API standards (Amazon), and flexible storage configurations (DAS or SAN).

Looking at usage trends, stateless server designs (as we get closer to PaaS) will allow us to rethink how we architect hypervisor based clouds.  Of course, this requires us to rethink application architectures and the OS choices that we make to run them. 

Thanks for BartonGeorge.net for the link  that got this thought started.  Network World says…

“Ubuntu Enterprise Cloud provides tight integration between Ubuntu and Ecalyptus and a series of CLI tools (made even more simple by apps like HybridFox with gives them a GUI) that follows along Amazon’s construction. Work done for Ubuntu Enterprise Cloud ends up being somewhat reusable if you’re transporting your work to Amazon.”