Start-ups are Time Machines for Big Companies [and open source is a worm hole]

My time at Dell (ended 10/2014) forced me to correct one of the most common misconceptions I hear about big companies – that they cannot innovate. My surprise at Dell was not the lack of innovation, but it’s overabundance. Having talked to many colleagues at big companies, I find the same pattern is everywhere. It’s not that these companies lack amazing and creative ideas, it’s that they have so many that it’s impossible for them to filter and promote them.

Innovation at a big company is like a nest of baby chicks all fighting for a worm but the parent bird can’t decide which chicks to feed.

In order for an idea to win at a big company, it generally has to shout so loudly and promise so extravagantly that it’s setup to fail right out of incubation. Consequently, great ideas are either never launched or killed in adolescence. Of course YMMV, but I’ve seen this pattern repeated throughout the tech industry.

TeamTimeCar.com-BTTF DeLorean Time Machine-OtoGodfrey.com-JMortonPhoto.com-07.jpg

“TeamTimeCar.com-BTTF DeLorean Time Machine-OtoGodfrey.com-JMortonPhoto.com-07″ by Terabass. Licensed under CC BY-SA 4.0 via Wikimedia Commons

What big companies really need is a time machine. That way, they can retroactively pick the right innovation and nurture it into a product that immediately benefits from their customer base, support infrastructure and market presence.

Money is a time machine.

With enough money, they can go backwards in time and unwind the decision to not invest in that innovative idea or team. It’s called purchasing a company. Sure, there’s a significant cash premium but that’s easier than stealing more plutonium for your DeLorean. In my experience, it’s behaviorally consistent for companies to act quickly on large outlays for retroactively correct decisions while being unwilling to deal with the political and long-term planning aspects of incubation.

I’ve come to embrace this cycle of innovation with an interesting twist: the growth of open source business models enables a new degree of cross innovation between start-ups and big companies. With open source, corporate locked innovators can exercise their ideas with start-ups and start-ups can leverage the talent and financial depth of big companies.

That’s like creating temporal worm holes in the venture-time continuum. Now that sounds like a topic for a future post… thoughts?

DefCore Process 9 Point Graphic balances Community, Vendor, Goverance

I’ve been working on the OpenStack DefCore process for nearly 3 years and our number #1 challenge remains how to explain it simply.

10 days ago, the DefCore committee met face-to-face in Austin to work on documenting the process that we want to follow (see Guidelines).  As we codify DefCore, our top priority is getting community feedback and explaining the process without expecting everyone to read the actual nut-and-bytes of the process.

I think of it as writing the DefCore preamble: “We, the community, in order to form a more perfect cloud….”

defcore 9 pointsI don’t think we’ve reached that level of simplicity; however, we have managed to boil down our thinking into nine key points.  I’m a big Tufte fan and believe that visualizations are essential to understanding complex topics.  In my experience, it takes many many iterations with feedback to create excellent graphics.  This triangle is my first workable pass.

An earlier version of these points were presented to the OpenStack board in December 2014 and we’ve been able to refine these during the latest DefCore community discussions.

We’re interested in hearing your opinions.  Here are the current (2015-Feb-22) points:

  1. COMMUNITY INVOLVEMENT
    1. MAPPING FEATURE AVAILABILITY: We are investing in gathering data driven and community involved feedback tools to engage the largest possible base for core decisions.   This mapping information will be available to the community.
    2. COMMUNITY CHOSEN CAPABILITIES :  Going forward, we want a community process to create, cluster and describe capabilities.  DefCore bootstrapped this process for Havana.  Further, Capabilities are defined by tests in Tempest so test coverage gaps (like Keystone v2) translate into Core gaps that the community will fill by writing tests.
    3. TESTS AS TRUTH: DefCore could expand in the future, but uses Tempest as the source of tests for now.  Gaps in Test will result in DefCore gaps.  We are hosting final documents in the Gerrit, using the OpenStack review process to ensure that we work within the community processes.
  2. VENDOR
    1. CLEAR RESULTS (PASS-FAIL): Vendors must pass all required core tests as defined by this process.  There are no partial results.  Passing additional tests is encouraged but not required.
    2. VENDORS SELF-TEST: Companies are responsible for running tests and submitting to the Foundation for validation against the DefCore criteria.  Approved vendor reports will be available for the community.
    3. APPEAL PROCESS / FLAGGED TESTS: There is a “safety valve” for vendors to deal with test scenarios that are currently difficult to recreate in the field.  We expect flags to be temporary.
  3. GOVERNANCE
    1. SCORING BASED ON TRANSPARENT PROCESS (DEFCORE): The 2015 by-laws change requires the Board and TC to agree to a process by which the Foundation can hold OpenStack Vendors accountable for their use of the trademarks.
    2. BOARD IS FINAL AUTHORITY: The Board is responsible for approving the final artifacts based on the recommendations.  By having a transparent process, community input is expected in advance of that approval.
    3. TIMELY GUIDANCE: The process is time sensitive.  There’s a need for the Board to produce DefCore guidance in a timely way after each release and then feed that result into the next cycle.  The guidance is expected to be drafted for review at each Summit and then approved at the Board meeting three months after the draft is posted.

OpenStack DefCore Accelerates & Simplifies with Clear and Timely Guidelines [Feedback?]

Last week, the OpenStack DefCore committee rolled up our collective sleeves and got to work in a serious way.  We had a in-person meeting with great turn out
with 5 board members, Foundation executives/staff and good community engagement.

defcore timelineTL;DR > We think DefCore deliverables should be dated milestone guidelines instead tightly coupled to release events (see graphic).

DefCore has a single goal expressed from two sides: 1) defining the “what is OpenStack” brand for Vendors and 2) driving interoperability between OpenStack installations.  From that perspective, it is not about releases, but about testable stable capabilities.  Over time, these changes should be incremental and, most importantly, trail behind new features that are added.

For those reasons, it was becoming confusing for DefCore to focus on an “Icehouse” definition when most of the capabilities listed are “Havana” ones.  We also created significant time pressure to get the “Kilo DefCore” out quickly after the release even though there were no “Kilo” specific additions covered.

In the face-to-face, we settled on a more incremental approach.  DefCore would regularly post a set of guidelines for approval by the Board.  These Guidelines would include the required, deprecated (leaving) and advisory (coming) capabilities required for Vendors to use the mark (see footnote*).  As part of defining capabilities, we would update which capabilities were included in each component and  which components were required for the OpenStack Platform.  They would also include the relevant designated sections.  These Guidelines would use the open draft and discussion process that we are in the process of outlining for approval in Vancouver.

Since DefCore Guidelines are simple time based lists of capabilities, the vendors and community can simply reference an approved Guideline using the date of approval (for example DefCore 2015.03) and know exactly what was included.  While each Guideline stands alone, it is easy to compare them for incremental changes.

We’ve been getting positive feedback about this change; however, we are still discussing it and appreciate your input and questions.  It is very important for us to make DefCore simple and easy.  For that, your confused looks and WTF? comments are very helpful.

* footnote: the Foundation manages the OpenStack brand and the process includes multiple facets.  The DefCore Guidelines are just one part of the brand process.

Art Fewell and I discuss DevOps, SDN, Containers & OpenStack [video + transcript]

A little while back, Art Fewell and I had two excellent discussions about general trends and challenges in the cloud and scale data center space.  Due to technical difficulties, the first (funnier one) was lost forever to NSA archives, but the second survived!

The video and transcript were just posted to Network World as part of Art’s on going interview series.  It was an action packed hour so I don’t want to re-post the transcript here.  I thought selected quotes (under the video) were worth calling out to whet your appetite for the whole tamale.

My highlights:

  1. .. partnering with a start-up was really hard, but partnering with an open source project actually gave us a lot more influence and control.
  2. Then we got into OpenStack, … we [Dell] wanted to invest our time and that we could be part of and would be sustained and transparent to the community.
  3. Incumbents are starting to be threatened by these new opened technologies … that I think levels of playing field is having an open platform.
  4. …I was pointing at you and laughing… [you’ll have to see the video]
  5. docker and containerization … potentially is disruptive to OpenStack and how OpenStack is operating
  6. You have to turn the crank faster and faster and faster to keep up.
  7. Small things I love about OpenStack … vendors are learning how to work in these open communities. When they don’t do it right they’re told very strongly that they don’t.
  8. It was literally a Power Point of everything that was wrong … [I said,] “Yes, that’s true. You want to help?”
  9. …people aiming missiles at your house right now…
  10. With containers you can sell that same piece of hardware 10 times or more and really pack in the workloads and so you get better performance and over subscription and so the utilization of the infrastructure goes way up.
  11. I’m not as much of a believer in that OpenStack eats the data center phenomena.
  12. First thing is automate. I’ve talked to people a lot about getting ready for OpenStack and what they should do. The bottom line is before you even invest in these technologies, automating your workloads and deployments is a huge component for being successful with that.
  13. Now, all of sudden the SDN layer is connecting these network function virtualization ..  It’s a big mess. It’s really hard, it’s really complex.
  14. The thing that I’m really excited about is the service architecture. We’re in the middle of doing on the RackN and Crowbar side, we’re in the middle of doing an architecture that’s basically turning data center operations into services.
  15. What platform as a service really is about, it’s about how you store the information. What services do you offer around the elastic part? Elastic is time based, it’s where you’re manipulating in the data.
  16. RE RackN: You can’t manufacture infrastructure but you can use it in a much “cloudier way”. It really redefines what you can do in a datacenter.
  17. That abstraction layer means that people can work together and actually share scripts
  18. I definitely think that OpenStack’s legacy will more likely be the community and the governance and what we’ve learned from that than probably the code.

My OpenStack Vancouver Session Promotion Dilemma – please, vote outside your block

We need people to promote their OpenStack Sessions, but how much is too much?

Megaphone!Semi-annually, I choose to be part of the growing dog pile of OpenStack summit submissions.  Looking at the list, I see some truly amazing sessions by committed and smart community members.  There are also a fair share of vendor promotions.

The nature of the crowded OpenStack vendor community is that everyone needs to pick up their social media megaphones (and encourage some internal block voting) to promote their talks.   Consequently, please I need to ask you to consider voting for my list:

  1. DefCore 2015 
  2. The DefCore Show: “is it core or not” feud episode
  3. Mayflies: Improve Cloud Utilization by Forcing Rapid Server Death [Research Analysis] (xref)
  4. It’s all about the Base. If you want stability, start with the underlay [Crowbar] 
  5. State of OpenStack Product Management

Why am I so reluctant to promote these excellent talks?  Because I’m concerned about fanning the “PROMOTE MY TALKS” inferno.

For the community to function, we need for users and operators to be heard.  The challenge is that the twin Conference/Summit venue serves a lot of different audiences.

In my experience, that leads to a lot of contributor navel gazing and vendor-on-vendor celebrations.  That in turn drowns out voices from the critical, but non-block-enabled users and operators.

Yes, please vote those sessions of mine that interest you; however, please take time to vote more broadly too.  The system randomized which talks you see to help distribute voting too.

Thanks.

Want CI Consul Love? OK! Run Consul in Travis-CI [example scripts]

If you are designing an application that uses microservice registration AND continuous integration then this post is for you!  If not, get with the program, you are a fossil.

Inside The EngineSunday night, I posted about the Erlang Consul client I wrote for our Behavior Driven Development (BDD) testing infrastructure.  That exposed a need to run a Consul service in the OpenCrowbar Travis-CI build automation that validates all of our pull requests.  Basically, Travis spins up the full OpenCrowbar API and workers (we call it the annealer) which in turn registers services in Consul.

NOTE: This is pseudo instructions.  In the actual code (here too), I created a script to install consul but this is more illustrative of the changes you need to make in your .travis.yml file.

In the first snippet, we download and unzip consul.  It’s in GO so that’s about all we need for an install.  I added a version check for logging validation.

before_script:
  - wget 'https://dl.bintray.com/mitchellh/consul/0.4.1_linux_amd64.zip'
  - unzip "0.4.1_linux_amd64.zip"
  - ./consul --version

In the second step, we setup the consul service and register it to itself in the background.  That allows the other services to access it.

script: 
  - ../consul agent -server -bootstrap-expect 1 -data-dir /tmp/consul &

After that, the BDD infrastructure can register the fake services that we expect (I created an erlang consul:reg_serv(“name”) routine that makes this super easy).  Once the services are registered, OpenCrowbar will check for the services and continue without trying to instantiate them (which it cannot do in Travis).

Here’s the pull request with the changes.

Erlang Consul Client

OpenCrowbar has been using Consul more and more deeply.  We’ve reached the point where we must register services on Consul to pass automated tests.

Consequently, I had to write a little Consul client in Erlang.

The client is very basic, but it seems to perform all of the required functions.  It relies on some other libraries in OpenCrowbar’s BDD but they are relatively self-contained.  Pulls welcome if you’d like to help build this out.

Here’s the code and the API reference.   Check in OpenCrowbar/BDD for latest updates and dependencies.