Tomorrow (3/1), numerous sites are gathering around a World Wide Essex Hack Day on 3/1. If you want to participate or even host a hack venue, get on the list and IRC channel (details).
My team at Dell is organizing a community a follow-upOpenStack Essex Install Day next week (3/8) in both Austin and Boston. Just like the Hack Day, the install fest will focus on Essex release code with both online and local presence. Unlike the Hack Day, our focus will be on deployments. For the Dell team, that means working on the Essex deployment for Crowbar. We’re still working on a schedule and partner list so stay tuned. I’m trying to webcast Crowbar & OpenStack training sessions during the install day.
Matt Ray from Opscode presented some of the work with Chef and OpenStack. He talked about the three main chef repos floating around. He called out Anso’s original cookbook set that is the basis for the Crowbar cookbooks (his second set), and his final set is the emerging set of cookbooks in OpenStack proper. The third one is interesting and what he plans to continue working on to make into his public openstack cookbooks. These are an amalgamation of smokestack, RCB, Anso improvements, and his (Crowbar’s).
He then demoed his knife plugin (slideshare) to build openstack virtual servers using the Openstack API. This is nice and works against TryStack.org (previously “Free Cloud”) and RCB’s demo cloud. All of that is on his github repo with instructions how to build and use. Matt and I talked about trying to get that into our Crowbar distro.
There were some questions about flow and choice of OpenStack API versus Amazon EC2 API because there was already an EC2 knife set of plugins.
Ziad Sawalha talks about Keystone
Ziad Sawalha is the PLT (Project Technical Lead) for Keystone. He works for Rackspace out of San Antonio. He drove up for the meeting.
He split his talk into two pieces, Incubation Process and Keystone Overview. He asked who was interested in what and focused his talk more towards overview than incubation.
Some key take-aways:
Keystone comes from Rackspace’s strong, flexible, and scalable API. It started as a known quantity from his perspective.
Community trusted nothing his team produced from an API perspective
Community is python or nothing
His team was ignored until they had a python prototype implementing the API
At this point, comments on API came in.
Churn in API caused problems with implementation and expectations around the close of Diablo.
Because comments were late, changes occurred.
Official implementation lagged and stalled into arriving.
API has been stable since Diablo final, but code is changing. that is good and shows strength of API.
Side note from Greg, Keystone represents to me the power of API over Code. You can have innovation around the implementation as long all the implementations have a fair ground work to plan under which is an API specification. The replacement of Keystone with the Keystone Light code base is an example of this. The only reason this is possible is that the API was sound and documented. (Rob’s post on this)
Ziad spent the rest of his time talking about the work flow of Keystone and the API points. He covered the API points.
Client to Keystone, Keystone to Client for initial auth token
Client to Middleware API for the services to have a front.
Middleware to Keystone to verify and establish identity.
Middleware to Service to pass identity
Not many details other then flow and flexibility. He stressed the API design separated protocol from actions and data at all the layers. This allows for future variations and innovations while maintaining the APIs.
Ziad talked about the state of Essex.
RBAC (aka Role Based Access Control)
Code replacement Keystone Light
Federation was planned but will most likely be pushed to G
Federation is the ability for multiple independent Keystones to operate (bursting use case)
Dependent upon two other federation components (networking and billing/metering)
In typing up my notes from the sessions, I ended up with so much information that it made more sense to break them into independent blog posts. Wow – that’s a lot of value from a free meetup!eetup was ideal for us. While we showed up in force, so did many other Stackers including people from HP, Nicira, Suse, Havard, Voxel, RedHat, ESPN and many more! The turnout for the event was great and I’m taking notes that Austin may need to upgrade our pizza and Boston may need to upgrade their cookies (just sayin’).
The Quantum session by David Lapsley from Nicira talked about the architecture and applications of Quantum. I think that Quantum is an exciting incubated project for OpenStack; however, it is important to remember that Essex stands alone without it. I believe this fact gets forgotten in enthusiasm over Quantum’s shiny potential.
The OpenStack session by Rob Hirschfeld from Dell (me!) talked about the importance of governance for OpenStack and how the Foundation will play a key role in transitioning it from Rackspace to a neutral party. There are many feel-good community benefits that the Foundation brings; however, the collaborators’ ROI is driver for creating a strong foundation. There is nothing wrong with acknowledging that fact and using it to create a more sustainable OpenStack.
Your’s truly (Rob Hirschfeld) gave the presentation about the OpenStack Foundation. To readers of this blog, it’s obvious that I’m a believer in the OpenStack mission; however, it’s not obvious how creating a foundation helps with that mission and why OpenStack needs its own. As one person at the meetup put it, “Why not? Every major project needs a foundation!”
Governance does not sound sexy compared to writing code and deploying clouds, but it’s very important to the success of the project.
Here are my notes without the poetic elocution I exuded during the meetup…
What: Creating a neutral body to govern OpenStack. Rackspace has been leading OpenStack. This means that they own the copyrights, name and also pay the people who organize the community. They committed (to executives at Dell and others) that they would ultimately setup a standalone body to govern the project before the project was public and endorsed by those early partners. Dell (my employer), Citrix, Accenture and NASA were some of biggest names at the Austin conference launch.
Why: A neutral body is needed because a lot of companies are committing significant time and money to the project. They cannot risk their investments on Rackspace good will alone. This may mean many things. It could be they don’t like Rackspace direction or they feel that Rackspace is not investing enough.
When: Right now and over the next few releases. You should give feedback right now on the OpenStack Foundations mission. The actual foundation will take more time to establish because it requires legal work and funding commitments.
Who: The community – all stakeholders. This is important stuff! While trying to standup a financially independent Foundation, which requires moneys, the little guys are not left out. There is a clear realization and desire to enable independent developers and contributors and small players to have a seat at the table.
How Much: The amounts are unclear, but establishing a foundation will require a significant ongoing investment from highly involved and moneyed parties (Rackspace, Dell, Cisco, HP, Citrix, NTT, startups?, etc). The funding will pay salaries for people dedicated to the community doing the things that I’ll discuss below. Overall, the ROI for those investments must be clear!
The foundation does “governance.” But, what does that mean? Here is a list of vitally important work that the foundation is responsible for.
Branding – Protecting, certifying, and promoting the OpenStack brand is important because it ensures that “OpenStack” has a valuable and predictable meaning to contributors and users. A strong the brand also means a stronger temptation for people to abuse the brand by claiming compatibility, participation and integration.
API – Many would assume that the OpenStack API is the very heart of the project and there is merit to this position. As more and more OpenStack implementations emerge, it is essential that we have a body that can certify which implementations (and even which versions of the implementation!) are valid. This is a substantial value to the community because API integrity ensures project continuity and helps the ecosystem monetize the project. Note: my opinion differs from others here because I think we should favor API over implementation
Community – The OpenStack community is not an accident. It is the function of deliberate actions and choices made by Rackspace and supported by key contributors. That community requires virtual and physical places to coalesce and leaders to organize and manage those meeting places. The excellent conferences, wikis, blogs, media awareness, documentation and meetups are a product of consistent community management.
Arbitration – An open source community is a family and siblings do not always get along. Today, Rackspace must be very careful about balancing their own interests because they are like the oldest sibling playing the parent role – you can get away with it until something serious happens. We need a neutral party so that Rackspace can protect their own interests (alternate spin: because Rackspace protects their own interests at the expense of the community).
Leadership – OpenStack today is a collection of projects with individual leadership. We will increasingly need coordinated leadership as the number of projects and users increases. Centralized leadership is essential because the good of the project as a whole may mean sacrifices within individual projects. It may even mean that some projects chose to leave the OpenStack tent. Stewarding these challenges will require a new level of leadership.
Legal – This is a function of all the above but also something more. From a legal stand point, OpenStack be able to represent itself. There is a significant amount of intellectual property being created. It would be foolish to overlook that this property is valuable and needs adequate legal representation.
I used “vitally important” to describe the above items. Is that an exaggeration? Our goal is collaboration and that requires some infrastructure and rules to make it sustainable. We must have a foundation that encourages innovation (multiple implementations) and collaboration (discourages forking). Innovation and collaboration are the heartbeat of an open source project.
The foundation is vitally important because collaboration by competitors is fragile.
In addition to the core areas above, the foundation needs to handle routine tactical items such as:
Delivering on milestones & releases
Moving new subprojects into OpenStack
Electing and maintaining Project Policy Board
Electing and maintaining Project Technical Leads
Ensuring adherence and extensions to the current bylaws
At the end of the day, OpenStack monetization is the central value for the Foundation.
In order for the OpenStack project, and thus its foundation, to flourish, the contributors, ecosystem, sponsors and users of the project must be able to see a reasonable return (ROI) on their investment. I would love to believe that the foundation is allow about people banding together to solve important problems for the benefit of all; however, it is more realistic to embrace that we can both collaborate and profit simultaneously. Acknowledging the pragmatic self-interested view allows us to create the right incentives and processes as embodied by the OpenStack foundation.
I’ll be in BOSTON THIS WEDNESDAY 2/1 for the OpenStack Meetup there. We’re going to be talking about Quantum and the OpenStack Foundation. I suspect that Keystone will come up too (but that’s the subject of another post). Of course, it’s not just your humble blogger: the whole Dell CloudEdge OpenStack/Crowbar team will be on hand! So put on your cloud geek hat and take a trip to Harvard for the meetup!