The Process Interlock Dilemma – where Roadmaps get lost and why Waterfalls suck

Note: This is part 1 of a 3 part series. I have been working on this series for nearly six months in an attempt to make this subtle but extremely expensive problem understandable. Rather than continue to polish the posts, I will post series for your enjoyment. I hope that it is enlightening, humorous or (ideally) both. Comments are welcome!

I’ve been struggling to explain a subtle process fail that occurs every day at my company (Dell) and also at every company I’ve ever worked with or for. I call this demon “Process Interlock” and it is the invisible bane of projects big and small. It manifests by forcing well-meaning product managers and engineering directors to make trade-offs that they know are wrong because of schedule commitments. It means that product quality consistently drops to the bottom of the list in favor of getting in that one promised feature. It shows up when customers get products late because of prospect who decided not to buy demanded a feature a year ago. These are the symptoms of the process interlock dilemma.

Process Interlock occurs when another team depends on your team for a future feature.

That sounds pretty innocuous right? It makes sense that other teams, customers and partners should be able to ask you about your roadmap and then build your delivery schedule into their plans. That is the perfectly logical request that happens inside my group every single day. Unfortunately, that exact commitment is what creates the problem because it locks your team’s velocity into the future and eliminates agility.

Note: I was reading chapter 11 in Eric Ries’ Lean Startup as was surprised to find him making very similar arguments but from a different perspective.

To hopefully help explain, I’m inventing a hypothetical project from the car division of the G.Mordler company. GM plans to add time travel as an option for their 2016 product line. They believe that there is a big market in minivan’s that can solve the proverbial “are we there yet problem” by simply skipping over the boring part of the trip. The trans-dimensional mommy mobile (or Trans Ma’am) will be part of a refresh of their 2014 model. The addition of a time circuit and power generator developed two internal divisions, Alpha and Omega, support a critical marketing event for the company so timing is important.

Let’s examine four outcomes of how these two divisions turn their assumed schedules into rigidly locked conundrum.

Scenario 0: Ideal Case.

Alpha makes the fusion power supply and Omega is making the time circuits. Based on experimental data, Omega’s design calls for 3.14 Gigawatts to operate their time capacitor; however, Alpha’s available design is limited to 0.73 Gigawatts. Alpha expects to reach 3.5 Gigawatts in 9 months when their supplier releases an updated nitrogen cooled super conductor. Based on that commitment, Omega has enough information to make an informed decision about their timeline. Since Alpha commits to deliver in 12 months (9 for the new part + 3 for development), Omega expects to deliver a working time circuit in 20 months (12 for the supply + 8 for development). In this example, there are 3 levels of Process Interlock: Alpha interlocks with the supplier and then Omega interlocks with Alpha. From a PERT schedule perspective, the world is now under control! It’s a brand new day and the birds are singing…

Scenario 1: Meet Schedule w/ Added Cost

Unfortunately, we now have a highly interlocked schedule. In the best case scenario (the one where we meet the schedule), Alpha has just signed up to meet an aggressive delivery timeframe. They have to put heavy pressure on the supplier to deliver their part which causes the supplier to increase the price for the cooler component. When their product manager identifies available alternative markets (such as power generating pet waste incineration), they are not able to purse the opportunities because they cannot risk the schedule impact of redirecting engineers. Meanwhile, Omega understands that a critical part is missing for 12 months and decides to reduce staffing while waiting for the needed part. In the process, they lose a key engineer who could have optimized the manufacturing process to half the production defect rate. Overall, the project meets schedule but at added cost, reduced quality and missed opportunities. This happened because the interlocks eliminated flexibility in the schedule for upstream and downstream participants. GM meets the launch window for the Trans Ma’am but high costs for the upgrade limit sales.

Scenario 2: Meet Schedule w/ Lost Features

A more likely “on schedule” alternative is that Alpha’s supplier cuts some corners to meet the aggressive deadline; consequently, power generation for Alpha is not reliable. This issue is not revealed by load testing in Alpha’s labs or short time travel testing by Omega. Instead, the faulty generators fail in integration field testing accidentally sending a DOT test driver home during rush hour traffic. Fixing the problem requires a redesign of the power plant. The new design does not fit into space allowed by the Trans Ma’am design team causing the entire program, while delivered “on time,” to be considered a failure and not shipped. GM misses the launch window for the Trans Ma’am.

Scenario 3:
Miss Schedule

In the most likely scenario the project is late. The schedule for Alpha slips because supplier requires an extra three months to meet the Alpha’s specs. In a common turn of fate, the supplier’s specs would be sufficient for Alpha to proceed; however, Alpha’s risk manager bumped up the cooling requirements by 20% in order to ensure they had wiggle room in their own design. Because of the supplier contract requiring delivery per spec, the supplier could not ship a workable but contractually unacceptable product. Since the part is delayed, Alpha has to slip the schedule to Omega. Compounding the problem, Alpha’s manager is optimistic that it will work out and does not alert Omega until 2 weeks before the deadline. Omega, who has been testing their circuits using liquid sodium cooled nuclear fission power plants, attempts to make up the schedule delay by imposing 20 hour Mountain Dew fueled work days. The aggressive schedule results in quality issues for the time circuits so that they can only be used during Mountain-time rebroadcasts of Seinfeld. After an unsuccessful bid to purchase the Denver cable TV station KDEV, GM misses the launch window for the Trans Ma’am.

I realize these examples are complicated, but I hope they humorously illuminate the problem.

In part 2, I’ll show an alternate approach for GM that addresses the process interlock.

Post Script

Of course, for this example, the entire project plan is a moot point since we’re talking about time machines! I’m offering two likely endings for the scenarios above:

The Pragmatists’ Ending: Once the project is finally complete, the manager simply drives the car back to the beginning of the project. Over white Russian martinis and sushi, her future self explains how the painful delivery schedule cost her the best years of her life causing her to quit. Her replacement cannot maintain funding for the project so it is eventually scraped by G.Mordler six months before the working pieces can be assembled.

The Realists’ Ending: Once the project is finally complete, the manager simply drives the car back to the beginning of the project. Over lemonade vodka tonics and tapas, her future self provides a USB stick with the critical design data needed to complete the project on time and budget. When she examines the data, the resulting time paradox creates a rift in the Einstein-Jacob space-time fabric thus ending the universe.

Agile Analogy: sprints are way points on a road trip

Happy 111111!  I’m working on a BIG AGILE post discussing the “interlock dilemma” that challenges big companies (like my employer Dell) as we become more lean in our development approaches.  That thought exercise turned up an analogy that is worth sharing.

We use sprints like we are driving on a long road trip.  As we travel, we want to stop at regular intervals to:

  • make sure we’re still going in the right direction (check the map)
  • see if we’re going to fast (overheating the engine)
  • see if we need to go faster (storms behind us)
  • avoid traffic (market is congested)
  • linger if there’s something interesting around (customers?!)
  • abandon the whole trip (kids are fighting in the back seat)
  • change our destination (saw a cool billboard)
  • pick-up a hitch hiker (partnering)

It just does not make sense to drive forward blindly hoping everything works out.  We need to inject decision points into our journey so that we take the right path.  And we have to remember, the right path is rarely that exact one that we started on!

If your product journey is predictable enough to navigate without frequent checks then your problem is not unique enough to generate much value.

The Tao of Agile: focus on delivery while still dreaming BIG

This post is a continuation of the Agile Strategy post.

So, how do we get into the right frame of mind for roadmapping?

You must embrace the Tao of Planning.

There are two conflicting principles behind roadmapping: you must keep thinking out of the box while keeping work deliverable. Neither of these principles is difficult in isolation. The challenge is the keep them in balance and to make sure that the whole team is included.

For my team, we struggle to find group times when we can do some big thinking. The challenge is not the thinking – it’s the TEAM aspect of working on strategy together. Our sprint planning needs to focus on the “keeping work deliverable” objective; consequently, there is precious little time in planning to have big ideas. To make the meeting duration manageable, planning meetings should have a tactical focus. Unfortunately, that leaves a strategy gap.

So, where does a team go to dream?

I wish I had a clear answer to this problem. Ideally, sprint review meetings should extend into deep thinking about where things could go. Strategy during Review is a natural extension because a review mindset should be forward looking. Reviews help us think about how we’re going to use what we delivered and the audience should bring external perspectives. If we could do this then it would be very empowering and exciting during review.

That’s why it’s important to celebrate, play, reflect and pause. All work and no play leaves a team that makes very dull products

Note: the Agile decorations that I use are: Sprint Planning (commits that plan) -> Stand-up (daily sync meeting -> Review (demo/sprint close) -> Retrospective / Hats (team feedback, improvement).

Agile takes discipline: having a strategy means saying “no” more than saying “yes”

With the Crowbar release behind us, it’s time for my team at Dell to do some Capital “P” Planning. Planning for us includes both tactical (next release) and strategic (the releases beyond the one after next), but each type of planning looks very different. I’m going to call it “roadmapping” because planning means something specific and tactical in Agile.

I love roadmapping but I’m a pain to roadmap with because I’m a ruthless prioritizer.

When I sit down for roadmapping, I always do it from a 1 to N list without ties. That means that when marketing asks for a new feature (double the foo on the bar!) we put it on the list relative to other work that needs to get done. If you add something at the top then something else will fall off the bottom. Effectively, we’re using the list to say no to a lot of great ideas. This is essential because “the great is the enemy of the good (Voltaire).” It’s hard, but that’s the cold reality of delivering product.

The most important part of strategy is figuring out what to push down to make room for the precious few yes items.

Successful roadmapping is negotiating the splitting of big ideas into smaller ones. Decomposition is a circular process because one compromise may require another, but one change may force a cascading assumption fault. If you get too emotionally committed to one feature or subset then you’re going to slow down the process. It’s vital to approach roadmapping in free fall.

As always, my advice is to not mix meeting objectives. If you need more strategy then you’ve got to make time for it.

Interested in more…stay tuned for Agile Tao: balancing tactics & strategy

Avoid false agreements and saying no with a yes. #TeamDeath

caution

One of my favorite things about Agile is how it helps teams get committed toward a shared goal.  There are so many distractions and confusions, that we need to double down ways to help people get and then stay on the same page.  In some cases, it comes down to something as simple as word choice!

First, I feel like I need some explanation…

There comes a time in any disagreement when the team needs everyone to get on the same page even if they don’t agree.  As a rule, this should be a relatively small window (maybe 20 minutes max) because the team can defer issues by having a sprint long spike* or exploration story that collects more information to settle arguments down the road. 

Personal Experience Note: A team should NEVER spend much time arguing about the mid or long-term future!  It’s just not worth the time to convince someone that your vision is more compelling.  It’s more efficient to accept that there are MULTIPLE VALID FUTURES and that the team needs to watch to see which one(s) is  taking shape.  There is no need to be “right” about the future.

So, back to the fake agreement phrases that effective teams avoid.

#1 “Yes, but…”

This statement really means “Will you shut up already?  I don’t agree.”  The speaker says “yes” to acknowledge the first person has finished; however, it does not mean that they agree.  The confusing thing is the speaker typically does not even realize that they are sending you into a discussion death spiral. 

Anytime someone says “but” then they are disagreeing.   Just for fun, trying have discussions where people are not allowed to say but – it creates a whole new positive dynamic.

#2 “I don’t disagree”

This statement really means “You are full of shit and my opinion is more right.”  The speaker is trying to avoid addressing your points directly and refocus discussion on their opinion.  Agreement means that everyone believes the same thing.  There are many ways to not agree and only one way to agree.

This is one of my pet peeves because the speaker thinks they are rewarding you with some back-handed pat on the head.  In reality, they shutting your ideas down without validation or acknowledgement.

There are many such statements that waste team time and mask disagreement.  If you have some that bug you, please comment on this post and add to the dialog.  I’m sure that I won’t disagree with any of them!

* Spike stories are time bounded stories that have specific research or opinion deliverables.  They are intended to collect enough information that the team can take action and move forward.   Sometimes these are also called “time box” stories.

Substituting Action for Knowledge – adopting “ready, fire, aim” as a strategy (and when to run like hell)

Today my mother-in-law (a practicing psychiatrist) was bemoaning the current medical practice of substituting action for knowledge. In her world, many doctors will make rapid changes to their patients’ therapy. Their goal is to address the issues immediately presented (patient feels sad so Dr prescribes antidepressants) rather than taking time to understand the patients’ history or make changes incrementally and measure impacts. It feels like another example of our cultural compulsion to fix problems as quickly as possible.

Her comments made me question the core way that I evangelize!

Do Lean and Agile substitute action for knowledge? No. We use action to acquire knowledge.

The fundamental assumption that drives poor decision-making is that we have enough information to make a design, solve a problem or define a market. Lean and Agile’s more core tenet is that we must attack this assumption. We must assume that we can’t gather enough information to fully define our objective. The good news, is that even without much analysis we know a lot! We know:

  • roughly what we want to do (road map)
  • the first steps we should take (tactics)
  • who will be working on the problem (team members)
  • generally how much effort it will take (time & team size)
  • who has the problem that we are trying to solve (market)

We also know that we’ll learn a lot more as we get closer to our target. Every delay in starting effectively pushed our “day of clarity” further into the future. For that reason, it is essential that we build a process that constantly reviews and adjusts its targets.

We need to build a process that acquires knowledge as progress is made and makes rapid progress.

In Agile, we translate this need into the decorations of our process: reviews for learning, retrospectives for adjustments, planning for taking action and short iterations to drive the feedback loop.  Agile’s mantra is “ready, fire, aim, fire, aim, fire, aim, …” which is very different from simply jumping out of a plane without a parachute and hoping you’ll find a haystack to land in.

For cloud deployments, this means building operational knowledge in stages.  Technology is simply evolving too quickly and best practices too slowly for anyone to wait for a packaged solution to solve all their cloud infrastructure problems.  We tried this and it does not work: clouds are a mixture hardware, software and operations.  More accurately, clouds are an operational model supported by hardware and software.

Currently, 80% of cloud deployment effort is operations (or “DevOps“).

When I listen to people’s plans about building product or deploying cloud, I get very skeptical when they take a lot of time to aim at objects far off on the horizon.  Perhaps they are worried that they will substitute action for knowledge; however, I think they would be better served to test their knowledge with a little action.

My MIL agrees – she sees her patients frequently and makes small adjustments to their treatment as needed.  Wow, that’s an Rx for Agile!

Dell to spin bare iron into OpenStack gold

I’m at the CloudConnect conference today supporting my team’s initial OpenStack foray.   Our announcement part of the Rackspace Cloud Builders announcement.

Tonight (3/8), we’re at the Rackspace Launch with a pony rack of servers (6 nodes) where we will run a LIVE DEMO of our cloud installer (codename “Crowbar”).  The initial offer includes my hyperscale white paper and our cloud foundation kit.

Interested in the details?  Here are background posts that talk about the Lean/Agile process we use, what is Crowbar, and my write up about hyperscale (“flat edge”) data centers.

Added 3/9: Links to articles about the release:

Here’s what Dell is saying about OpenStack on Dell.com/openstack:

Dell is one of the original partners in the OpenStack community, which has now grown to more than 50 companies and participants. To accelerate adoption of this powerful platform, Dell has worked to develop an effortless out-of-box OpenStack experience with:
  • Optimized PowerEdge™ C-based hardware configurations
  • A technical whitepaper that details the design of an OpenStack hyperscale cloud on PowerEdge C server technology
  • An OpenStack installer that allows bare metal deployment of OpenStack clouds in a few hours (vs. a manual installation period of several days)

Read more about the steps to design an OpenStack hyperscale cloud in a Dell technical whitepaper entitled “Bootstrapping OpenStack Clouds.”

Interested?  Contact OpenStack@Dell.com.