Once this initial commit is approved by the DefCore Committee (expected at DefCore Scale.8 Meeting 3/25 @ 9 PT), we’ll be ready for broader input by the community using the standard OpenStack Gerrit review process. If you are not comfortable with Gerrit, we’ll take your input anyway that you want to give itexcept via telepathy (we’ve already got a lot on our minds).
The DefCore Process documents the rules (who, what, when and where) that will govern how we create the DefCore Guidelines. By design, it has to be detailed and specific without adding complexity and confusion. The whyof DefCore is all that work we did on principles that shape the process.
This process reflects nearly a year of gestation starting from the June 2014 DefCore face-to-face. Once of the notable recent refinements was to organize material into time phases and to be more specific about who is responsible for specific actions.
To make review easier, I’ve reposted the draft. Comments are welcome here and on the patch (and here after it lands).
A little while back, Art Fewell and I had two excellent discussions about general trends and challenges in the cloud and scale data center space. Due to technical difficulties, the first (funnier one) was lost forever to NSA archives, but the second survived!
The video and transcript were just posted to Network World as part of Art’s on going interview series. It was an action packed hour so I don’t want to re-post the transcript here. I thought selected quotes (under the video) were worth calling out to whet your appetite for the whole tamale.
.. partnering with a start-up was really hard, but partnering with an open source project actually gave us a lot more influence and control.
Then we got into OpenStack, … we [Dell] wanted to invest our time and that we could be part of and would be sustained and transparent to the community.
Incumbents are starting to be threatened by these new opened technologies … that I think levels of playing field is having an open platform.
…I was pointing at you and laughing… [you’ll have to see the video]
docker and containerization … potentially is disruptive to OpenStack and how OpenStack is operating
You have to turn the crank faster and faster and faster to keep up.
Small things I love about OpenStack … vendors are learning how to work in these open communities. When they don’t do it right they’re told very strongly that they don’t.
It was literally a Power Point of everything that was wrong … [I said,] “Yes, that’s true. You want to help?”
…people aiming missiles at your house right now…
With containers you can sell that same piece of hardware 10 times or more and really pack in the workloads and so you get better performance and over subscription and so the utilization of the infrastructure goes way up.
I’m not as much of a believer in that OpenStack eats the data center phenomena.
First thing is automate. I’ve talked to people a lot about getting ready for OpenStack and what they should do. The bottom line is before you even invest in these technologies, automating your workloads and deployments is a huge component for being successful with that.
Now, all of sudden the SDN layer is connecting these network function virtualization .. It’s a big mess. It’s really hard, it’s really complex.
The thing that I’m really excited about is the service architecture. We’re in the middle of doing on the RackN and Crowbar side, we’re in the middle of doing an architecture that’s basically turning data center operations into services.
What platform as a service really is about, it’s about how you store the information. What services do you offer around the elastic part? Elastic is time based, it’s where you’re manipulating in the data.
RE RackN: You can’t manufacture infrastructure but you can use it in a much “cloudier way”. It really redefines what you can do in a datacenter.
That abstraction layer means that people can work together and actually share scripts
I definitely think that OpenStack’s legacy will more likely be the community and the governance and what we’ve learned from that than probably the code.
We had a great discussion about OpenStack, Ops and Crowbar. I appreciate Niki’s insightful questions and an opportunity to share my opinions. I feel that we covered years of material in just 1 hour and I appreciate the opportunity to appear on the podcast.
51:00 We should be encouraging people to use OpenStack for its use cases
51:30 Existential question for OpenStack: are we a suite or product. The community is split here
51:30 In comparing with Amazon, does OpenStack have to implement it or build an ecosystem to compete
53:00 As soon as you make something THE OpenStack project (like Heat) you are sending a message that the alternates are not welcome
54:30 OpenStack ends up in a trap if we pick a single project and make it the way that we are going do something. New implementations are going to surface from WITHIN the projects and we need to ready for that.
55:15 new implementations are coming, we have to be ready for that. We can make ourselves vulnerable to splitting if we do not prepare.
56:00 API vs Implementation? This is something that splits the community. Ultimately we to be an API spec but we are not ready for that. We have a lot of work to do first using the same code base.
56:50 DefCore has taken a balanced approach using our diversity as a strength
57:20 Bylaws did not allow for enough flexibility for what is core
We often act as if coding, and especially open source coding, is a unique activity and that’s hubris. Most human activities follow common social patterns that should inform how we organize open source projects. For example, research papers are very social and community connected activities. Especially when published, written compositions are highly interconnected activities. Even the most basic writing builds off other people’s work with due credit and tries create something worth being used by later authors.
Here are seven principles to good writing that translate directly to good open source development:
Research before writing – take some time to understand the background and goals of the project otherwise you re-invent or draw bad conclusions.
Give credit where due – your work has more credibility when you acknowledge and cross-reference the work you are building on. It also shows readers that you are not re-inventing.
Follow the top authors – many topics have widely known authors who act as “super nodes” in the relationship graph. Recognizing these people will help guide your work, leads to better research and builds community.
Find proof readers – All writers need someone with perspective to review their work before it’s finished. Since we all need reviewers, we all also need to do reviews.
Rework to get clarity – Simplicity and clarity take extra effort but they pay huge dividends for your audience.
Don’t surprise your reader – Readers expect patterns and are distracted when you don’t follow them.
Socialize your ideas – the purpose of writing/code is to make ideas durable. If it’s worth writing then it’s worth sharing. Your artifact does not announce itself – you need to invest time in explaining it to people and making it accessible.
Thanks to Sean Roberts (a Hidden Influences collaborator) for his contributions to this post. At OSCON, Sean Roberts said “companies should count open source as research [and development investment]” and I thought he’s said “…as research [papers].” The misunderstanding was quickly resolved and we were happy to discover that both interpretations were useful.
We want community input! The Board is going discuss and, hopefully, approve the matrix at our next meeting on 7/22. After that, the Board will be focused on defining Designated Sections for Havana and Ice House (the TC is not owning that as previously expected).
The DefCore process is gaining momentum. We’ve reached the point where there are tangible (yet still non-binding) results to review. The Refstack efforts to collect community test results from running clouds is underway: the Core Matrix will be fed into Refstack to validate against the DefCore required capabilities.
Now is the time to make adjustments and corrections!
In the next few months, we’re going to be locking in more and more of the process as we get ready to make it part of the OpenStack by-laws (see bottom of minutes).
If you cannot make these meetings, we still want to hear from you! The most direct way to engage is via the DefCore mailing list but 1×1 email works too! Your input is import to us!
OpenCrowbar community contributors are offering a “100 Node Challenge” by volunteering to setup a 100+ node Crowbar system to prove out the v2 architecture at scale. We picked 100* nodes since we wanted to firmly break the Crowbar v1 upper ceiling.
The goal of the challenge is to prove scale of the core provisioning cycle. It’s intended to be a short action (less than a week) so we’ll need advanced information about the hardware configuration. The expectation is to do a full RAID/Disk hardware configuration beyond the base IPMI config before laying down the operating system.
The challenge logistics starts with an off-site prep discussion of the particulars of the deployment, then installing OpenCrowbar at the site and deploying the node century. We will also work with you about using OpenCrowbar to manage the environment going forward.
Sound too good to be true? Well, as community members are doing this on their own time, we are only planning one challenge candidate and want to find the right target.
We will not be planning custom code changes to support the deployment, however, we would be happy to work with you in the community to support your needs. If you want help to sustain the environment or have longer term plans, I have also been approached by community members who willing to take on full or part-time Crowbar consulting engagements.
Let’s get rack’n!
* we’ll consider smaller clusters but you have to buy the drinks and pizza.