Docker-Machine Crowbar Driver Delivers Metal Containers

I’ve just completed a basic Docker Machine driver for OpenCrowbar.  This enables you to quickly spin-up (and down) remote Docker hosts on bare metal servers from their command line tool.  There are significant cost, simplicity and performance advantages for this approach if you were already planning to dedicate servers to container workloads.

Docker Machine

The basics are pretty simple: using Docker Machine CLI you can “create” and “rm” new Docker hosts on bare metal using the crowbar driver.  Since we’re talking about metal, “create” is really “assign a machine from an available pool.”

Behind the scenes Crowbar is doing a full provision cycle of the system including installing the operating system and injecting the user keys.  Crowbar’s design would allow operators to automatically inject additional steps, add monitoring agents and security, to the provisioning process without changing the driver operation.

Beyond Create, the driver supports the other Machine verbs like remove, stop, start, ssh and inspect.  In the case of remove, the Machine is cleaned up and put back in the pool for the next user [note: work remains on the full remove>recreate process].

Overall, this driver allows Docker Machine to work transparently against metal infrastructure along side whatever cloud services you also choose.

Want to try it out?

  1. You need to setup OpenCrowbar – if you follow the defaults (192.168.124.10 ip, user, password) then the Docker Machine driver defaults will also work. Also, make sure you have the Ubuntu 14.04 ISO available for the Crowbar provisioner
  2. Discover some nodes in Crowbar – you do NOT need metal servers to try this, the tests work fine with virtual machines (tools/kvm-slave &)
  3. Clone my Machine repo (Wde’re looking for feedback before a pull to Docker/Machine)
  4. Compile the code using script/build.
  5. Allocate a Docker Node using  ./docker-machine create –driver crowbar testme
  6. Go to the Crowbar UI to watch the node be provisioned and configured into the Docker-Machines pool
  7. Release the node using ./docker-machine rm testme
  8. Go to the Crowbar UI to watch the node be redeployed back to the System pool
  9. Try to contain your enthusiasm :)

2015, the year cloud died. Meet the seven riders of the cloudocalypse

i can hazAfter writing pages of notes about the impact of Docker, microservice architectures, mainstreaming of Ops Automation, software defined networking, exponential data growth and the explosion of alternative hardware architecture, I realized that it all boils down to the death of cloud as we know it.

OK, we’re not killing cloud per se this year.  It’s more that we’ve put 10 pounds of cloud into a 5 pound bag so it’s just not working in 2015 to call it cloud.

Cloud was happily misunderstood back in 2012 as virtualized infrastructure wrapped in an API beside some platform services (like object storage).

That illusion will be shattered in 2015 as we fully digest the extent of the beautiful and complex mess that we’ve created in the search for better scale economics and faster delivery pipelines.  2015 is going to cause a lot of indigestion for CIOs, analysts and wandering technology executives.  No one can pick the winners with Decisive Leadership™ alone because there are simply too many possible right ways to solve problems.

Here’s my list of the seven cloud disrupting technologies and frameworks that will gain even greater momentum in 2015:

  1. Docker – I think that Docker is the face of a larger disruption around containers and packaging.  I’m sure Docker is not the thing alone.  There are a fleet of related technologies and Docker replacements; however, there’s no doubt that it’s leading a timely rethinking of application life-cycle delivery.
  2. New languages and frameworks – it’s not just the rapid maturity of Node.js and Go, but the frameworks and services that we’re building (like Cloud Foundry or Apache Spark) that change the way we use traditional languages.
  3. Microservice architectures – this is more than containers, it’s really Functional Programming for Ops (aka FuncOps) that’s a new generation of service oriented architecture that is being empowered by container orchestration systems (like Brooklyn or Fleet).  Using microservices well seems to redefine how we use traditional cloud.
  4. Mainstreaming of Ops Automation – We’re past “if DevOps” and into the how. Ops automation, not cloud, is the real puppies vs cattle battle ground.  As IT creates automation to better use clouds, we create application portability that makes cloud disappear.  This freedom translates into new choices (like PaaS, containers or hardware) for operators.
  5. Software defined networking – SDN means different things but the impacts are all the same: we are automating networking and integrating it into our deployments.  The days of networking and compute silos are ending and that’s going to change how we think about cloud and the supporting infrastructure.
  6. Exponential data growth – you cannot build applications or infrastructure without considering how your storage needs will grow as we absorb more data streams and internet of things sources.
  7. Explosion of alternative hardware architecture – In 2010, infrastructure was basically pizza box or blade from a handful of vendors.  Today, I’m seeing a rising tide of alternatives architectures including ARM, Converged and Storage focused from an increasing cadre of sources including vendors sharing open designs (OCP).  With improved automation, these new “non-cloud” options become part of the dynamic infrastructure spectrum.

Today these seven items create complexity and confusion as we work to balance the new concepts and technologies.  I can see a path forward that redefines IT to be both more flexible and dynamic while also being stable and performing.

Want more 2015 predictions?  Here’s my OpenStack EOY post about limiting/expanding the project scope.

9 scenarios to have prepared for a College Interview [from someone who does interviews]

This quick advice for preparing for a college interview is also useful for any interview: identify three key strengths and activities then prepare short insightful stories that show your strengths in each activity.  Stories are the strongest way to convey information.

I’ve been doing engineering college interviews since 2013 for my Alma mater, Duke University.  I love meeting the upcoming generation of engineers and seeing how their educational experiences will shape their future careers.  Sadly, I also find that few students are really prepared to showcase themselves well in these interviews.  Since it makes my job simpler if you are prepared, I’m going to post my recommendation for future interviews!

It does not take much to prepare for a college interview: you mainly need to be able to tell some short, detailed stories from your experiences that highlight your strengths.

In my experience, the best interviewees are good at telling short and specific stories that highlight their experiences and strengths.  It’s not that they have better experiences, they are just better prepared at showcasing them.  Being prepared makes you more confident and comfortable which then helps you control of how the interview goes and ensures that you leave the right impression.

1/9/15 Note: Control the interview?  Yes!  You should be planning to lead the interviewer to your strengths.  Don’t passively expect them to dig that information out of you.  It’s a two-way conversation, not an interrogation.

Here’s how it works:

  1. Identify three activities that you are passionate about.  They do not have to represent the majority of you effort.  Select ones that define who you are, or caused you to grow in some way.  They could be general items like “reading” or very specific like “summer camp 2016.”  You need to get excited when you talk about these items.  Put these on the rows/y-axis of a 3×3 grid (see below).
  2. Identify three attributes that describe you (you may want help from friends or parents here).  These words should be enough to give a fast snapshot of who you are.  In the example below, the person would be something like “an adventure seeking leader who values standing out as an individual.”  Put these attributes on the columns/x-axis of your grid as I’ve shown below.
  3. Come up with the nine short stories (3-6 sentences!) for the intersections on the grid where you demonstrated the key attribute during the activity.  They cannot just be statements – you must have stories because they provide easy to remember examples for your interview.  If you don’t have a story for an intersection, then talk about how you plan to work this in the future.

Note: This might feel repetitive when you construct your grid but this technique works exceptionally well during an hour-long interview.  You should repeat yourself because you need reinforce your strengths and leave the interviewer with a sure sense of who you are.

Interview Grid

Sample Grid – Click to Enlarge

 

Remember: An admissions, alumni or faculty interview is all about making a strong impression about who you are and, more importantly, what you will bring to the university.

Having a concrete set of experiences and attributes makes sure that you reinforce your strengths.  By showing them in stories, you will create a much richer picture about who you are than if you simply assert statements about yourself.  Remember the old adage of “show, don’t tell.”

Don’t use this grid as the only basis for your interview!  It should be a foundation that you can come back to during your conversations with college representatives.  These are your key discussion points – let them help you round out the dialog.

Good luck!

PS: Google your interviewer!  I expect the candidates to know me before they meet me.  It is perfectly normal and you’d be crazy to not take advantage of that.

unBIOSed? Is Redfish an IPMI retread or can vendors find unification?

Server management interfaces stink.  They are inconsistent both between vendors and within their own product suites.  Ideally, Vendors would agree on a single API; however, it’s not clear if the diversity is a product of competition or actual platform variation.  Likely, it’s both.

From RedFish SiteWhat is Redfish?  It’s a REST API for server configuration that aims to replace both IPMI and vendor specific server interfaces (like WSMAN).  Here’s the official text from RedfishSpecification.org.

Redfish is a modern intelligent [server] manageability interface and lightweight data model specification that is scalable, discoverable and extensible.  Redfish is suitable for a multitude of end-users, from the datacenter operator to an enterprise management console.

I think that it’s great to see vendors trying to get on the same page and I’m optimistic that we could get something better than IPMI (that’s a very low bar).  However, I don’t expect that vendors can converge to a single API; it’s just not practical due to release times and pressures to expose special features.  I think the divergence in APIs is due both to competitive pressures and to real variance between platforms.

Even if we manage to a grand server management unification; the problem of interface heterogeneity has a long legacy tail.

In the best case reality, we’re going from N versions to N+1 (and likely N*2) versions because the legacy gear is still around for a long time.  Adding Redfish means API sprawl is going to get worse until it gets back to being about the same as it is now.

Putting pessimism aside, the sprawl problem is severe enough that it’s worth supporting Redfish on the hope that it makes things better.

That’s easy to say, but expensive to do.  If I was making hardware (I left Dell in Oct 2014), I’d consider it an expensive investment for an uncertain return.  Even so, several major hardware players are stepping forward to help standardize.  I think Redfish would have good ROI for smaller vendors looking to displace a major player can ride on the standard.

Redfish is GREAT NEWS for me since RackN/Crowbar provides hardware abstraction and heterogeneous interface support.  More API variation makes my work more valuable.

One final note: if Redfish improves hardware security in a real way then it could be a game changer; however, embedded firmware web servers can be tricky to secure and patch compared to larger application focused software stacks.  This is one area what I’m hoping to see a lot of vendor collaboration!  [note: this should be it’s own subject – the security issue is more than API, it’s about system wide configuration.  stay tuned!]

To thrive, OpenStack must better balance dev, ops and business needs.

OpenStack has grown dramatically in many ways but we have failed to integrate development, operations and business communities in a balanced way.

My most urgent observation from Paris is that these three critical parts of the community are having vastly different dialogs about OpenStack.

Clouds DownAt the Conference, business people were talking were about core, stability and utility while the developers were talking about features, reorganizing and expanding projects. The operators, unfortunately segregated in a different location, were trying to figure out how to share best practices and tools.

Much of this structural divergence was intentional and should be (re)evaluated as we grow.

OpenStack events are split into distinct focus areas: the conference for business people, the summit for developers and specialized days for operators. While this design serves a purpose, the community needs to be taking extra steps to ensure communication. Without that communication, corporate sponsors and users may find it easier to solve problems inside their walls than outside in the community.

The risk is clear: vendors may find it easier to work on a fork where they have business and operational control than work within the community.

Inside the community, we are working to help resolve this challenge with several parallel efforts. As a community member, I challenge you to get involved in these efforts to ensure the project balances dev, biz and ops priorities.  As a board member, I feel it’s a leadership challenge to make sure these efforts converge and that’s one of the reasons I’ve been working on several of these efforts:

  • OpenStack Project Managers (was Hidden Influencers) across companies in the ecosystem are getting organized into their own team. Since these managers effectively direct the majority of OpenStack developers, this group will allow
  • DefCore Committee works to define a smaller subset of the overall OpenStack Project that will be required for vendors using the OpenStack trademark and logo. This helps the business community focus on interoperability and stability.
  • Technical leadership (TC) lead “Big Tent” concept aligns with DefCore work and attempts to create a stable base platform while making it easier for new projects to enter the ecosystem. I’ve got a lot to say about this, but frankly, without safeguards, this scares people in the ops and business communities.
  • An operations “ready state” baseline keeps the community from being able to share best practices – this has become a pressing need.  I’d like to suggest as OpenCrowbar an outside of OpenStack a way to help provide an ops neutral common starting point. Having the OpenStack developer community attempting to create an installer using OpenStack has proven a significant distraction and only further distances operators from the community.

We need to get past seeing the project primarily as a technology platform.  Infrastructure software has to deliver value as an operational tool for enterprises.  For OpenStack to thrive, we must make sure the needs of all constituents (Dev, Biz, Ops) are being addressed.

Three critical ingredients for digital age relationships. [Collaborate Series 8/8]

Translation: Are you ready to apply these lessons?

This post is the final post in an collaborative eight part series by Brad Szollose and I about how culture shapes technology.

End of LineDuring this blog series, we’ve explored how important culture is in the work place.  The high tech areas are especially sensitive because they disproportionately embrace the millennial culture which often causes conflicts.

Our world has changed, driven by technology, new thinking, and new methodologies yet we may be using 20th century management techniques on 21st century customers and workers. There is an old business axiom that states, “If you can’t measure it, you can’t manage it.”  Yet how much of our process, interaction, successes, and failures never wind up on a spreadsheet, yet impact it?

Customers don’t leave bad companies; they leave companies that miss the mark when it comes to customer engagement. To better serve our customers we need to understand and adapt to the psychology of a new customer … one who has been trained to work as a Digital Native.

What would that look like? Tech people who interact with patience, collaboration, deep knowledge, and an openness to input, adapting to a customer’s needs in real-time. Wouldn’t that create a relationship that is second to none and unbreakable? Wouldn’t that be a leg up on the competition?

By understanding that new business culture has been influenced by the gaming experience, we have a deeper understanding of what is important to our customer base. And like a video game, if you cling to hierarchy, you lose. If you get caught up in linear time management, you lose. If you cling to bottlenecks and tradition you lose.

Three key takeaways: speed, adaptation, and collaboration

Those three words sum up today’s business environment. By now, you should not be surprised that those drivers are skills honed in video games.

We’ve explored the radically different ways that Digital Natives approach business opportunities. As the emerging leaders of the technological world, we must shift our operations to be more open, collaborative, iterative, and experience based.

Rob challenges you to get involved in his and other collaborative open source projects. Brad challenges you to try new leadership styles that engage with the Cloud Generation. Together, we challenge our entire industry to embrace a new paradigm that redefines how we interact and innovate. We may as well embrace it because it is the paradigm that we’ve already trained the rising generation or workers to intuitively understand.

What’s next?

Brad and Rob collaborated on this series with the idea of extending the concepts beyond a discussion of the “digital divide” and really looking at how culture impacts business leadership.  Lately, we’ve witnessed that the digital divide is not about your birthday alone.  We’ve seen that age alone does not drive the all cultural differences we’ve described here.  Our next posts will reflect the foundations for different ways that we’ve seen people respond to each other with a focus on answering “can digital age workers deliver?”

Like the conclusion?  Reading the rest of the series! 1: Intro > 2: ToC > 3: Video Reality > 4: Authority > 5: On The Game Training > 6: Win by Failing > 7: Go Digital Native > 8: Three Takeaways

 

4 item OSCON report: no buzz winner, OpenStack is DownStack?, Free vs Open & the upstream imperative

Now that my PDX Trimet pass expired, it’s time to reflect on OSCON 2014.   Unfortunately, I did not ride my unicorn home on a rainbow; this year’s event seemed to be about raising red flags.

My four key observations:

  1. No superstar. Past OSCONs had at least one buzzy community super star.  2013 was Docker and 2011 was OpenStack.  This was not just my hallway track perception, I asked around about this specifically.  There was no buzz winner in 2014.
  2. People were down on OpenStack (“DownStack”). Yes, we did have a dedicated “Open Cloud Day” event but there was something missing.  OpenSack did not sponsor and there were no major parties or releases (compared to previous years) and little OpenStack buzz.  Many people I talked to were worried about the direction of the community, fragmentation of the project and operational readiness.  I would be more concerned about “DownStack” except that no open infrastructure was a superstar either (e.g.: Mesos, Kubernetes and CoreOS).  Perhaps, OSCON is simply not a good venue open infrastructure projects compared to GlueCon or Velocity?  Considering the rapid raise of container-friendly OpenStack alternatives; I think the answer may be that the battle lines for open infrastructure are being redrawn.
  3. Free vs. Open. Perhaps my perspective is more nuanced now (many in open source communities don’t distinguish between Free and Open source) but there’s a real tension between Free (do what you want) and Open (shared but governed) source.  Now that open source is a commercial darling, there is a lot of grumbling in the Free community about corporate influence and heavy handedness.   I suspect this will get louder as companies try to find ways to maintain control of their projects.
  4. Corporate upstreaming becomes Imperative. There’s an accelerating upstreaming trend for companies that write lots of code to support their primary business (Google is a primary example) to ensure that code becomes used outside their company.   They open their code and start efforts to ensure its adoption.  This requires a dedicated post to really explain.

There’s a clear theme here: Open source is going mainstream corporate.

We’ve been building amazing software in the open that create real value for companies.  Much of that value has been created organically by well-intentioned individuals; unfortunately, that model will not scale with the arrival for corporate interests.

Open source is thriving not dying: these companies value the transparency, collaboration and innovation of open development.  Instead, open source is transforming to fit within corporate investment and governance needs.  It’s our job to help with that metamorphosis.