Very Costly Accounting and why I value ship ready code

Or be careful what you measure because you’ll get it.

One of Agile’s most effective, least understood and seldom realized disciplines is that teams should “maintain ship-readiness” of their product at all times.  Explaining the real value of this discipline in simple terms eluded me for years until I was talking to an accountant on a ski lift.

Before I talk about snow and mountain air inspired insights, let me explain what ship-ready means.  It means that you should always be ready to deliver the output from your last iteration to your most valuable customer.   For example, when my company, BEware, finished our last iteration we could have delivered it to our top customer, Fin & Key, without losing our much needed beauty sleep.   Because we maintain ship-readiness, we worked to ensure that they could upgrade, have complete documentation, and high quality without spending extra cycles on release hardening.

The version that we’d ship to Fin & Key would probably not have any new features enabled (see tippy towers & Eric Ries on split testing) , but it would have fixed defects and the incremental code for those new features baked in.  While we may decide to limit shipments to fixed times for marketing reasons, that must not keep us from always being ready to ship.

Meanwhile, back at 8,200 feet, my accountant friend was enrolled in Cost Accounting 101.  To fulfill my mission as an Agile Subversive, I suggested reading  “The Goal” by Eli Goldratt which comes out very strongly against the evils of CA.  Goldratt’s logic is simple – if you want people to sub-optimize and ignore the overall system productivity then you’d assign costs to each sub-component of your system.  The result in manufacturing is that people will always try to keep the most expensive machine 100% utilized even if that causes lots of problems elsewhere and drives up costs all over the factory.

Cost Accounting’s process of measuring on a per cost basis (instead of a system basis) causes everyone to minimize the cost at their area rather than collaborate to make the system more efficient.  I’ve worked in many environments where management tried to optimize expensive developer time by off loading documentation, quality, and support to different teams.  The result was a much less effective system where defects were fixed late (if ever), test automation was never built, documentation was incomplete, and customer field issues lingered like the smell of stale malt beverage in a frat house.

No one wanted these behaviors, yet they were endemic because the company optimized developer time instead of working product.

Agile maintains ship readiness because it becomes Engineering’s primary measurement.  Making sellable product the top priority focuses team on systems and collaboration.  It may not be as “efficient” to have a highly paid developer running tests; however, it does real economic harm if the developer continues to write untested code ahead of your ability to verify it.  Even more significantly, a developer who plays a larger part in test, documentation, and supports is much more invested in fixing problems early.

If your company wants ship product then find a metric that gets you to that goal.  For Agile teams, that metric is percent of iterations delivering ship ready product.  My condolences if your team’s top values are milestones completed, bugs fixed, or hours worked.

Death by Ant Bytes

Or the Dangers of Incremental Complexity

Products are not built in big bangs: they are painfully crafted layer upon layer, decision after decision, day by day.  It’s also a team sport where each member makes countless decisions that hopefully help flow towards something customers love.

In fact, these decisions are so numerous and small that they seem to cost nothing.  Our judgment and creativity to builds the product crumb by drop.  Each and every morning we shows up for work ready to bake wholesome chocolaty goodness into the product.   It’s seeming irrelevance of each atomic bit that lulls us into false thinking that every addition is just a harmless Pythonesque “wafer thin”  bite.

That’s right, not all these changes are good.  It’s just as likely (perhaps more likely) that the team is tinkering with the recipe.  Someone asks them to add a pinch of cardamom today, pecans tomorrow, and raisins next week.  Individually, these little changes seem to be trivial.  Taken together, they can delay your schedule at best or ruin your product at worst.

Let me give you a concrete example:

In a past job, we had to build an object model for taxis.  At our current stage, this was pretty simple: a truck has a name, a home base, and an assigned driver.  One of our team independently looked ahead and decided individually that he should also add make, model, MPG, and other performance fields.  He also decided that assignments needed a whole new model since they could date range (start, end) and handle multiple drivers.  Many of you are probably thinking all this was just what engineers are supposed to do – anticipate needs.  Read on…

By the time he’d built the truck model, it had taken 5x as a long and resulted in 100s of lines of code.  It got worse the very next week when we built the meter interface code and learned more about the system.  For reporting requirements, MPG and performance fields had to be handled outside the taxi model.  We also found that driver assignments were much more naturally handled by looking at fare information.   Not only had we wasted a lot of time, we had to spend even more time reversing the changes we’d put in.

One of my past CEOs called this a “death by ant bites” and “death of a million cuts.”

It’s one of the most pernicious forms of feature creep because every single one of the changes can be justified.  I’m not suggesting that all little adds are bad, but they all cost something.   Generally, if someone says they are anticipating a future need, then you’re being bitten by an ant.

You need to make sure that your team is watching each other’s back and keeping everyone honest.  It’s even better to take turns playing devil’s advocate on each feature.  It’s worth an extra 10 minutes in a meeting to justify if that extra feature is required.

PS: Test Driven Design (TDD) repels ants because it exposes the true cost for those anticipatory or seemingly minor changes.  That “10 minute” feature is really a half day of work to design, test, integrate, and document.  If it’s not worth doing right, then it’s not worth adding to the product.

Wearing the Cape

Or Team Death by Heroism

I remember the day that put on the hero’s cape and I killed our team.

A few years ago I would have felt really good about saving the deadline and exposing the deadweight we’d been carrying on the team.   I’d get a good performance review, a bonus, and some expectation that I’d be picked first on the corporate playground baseball team.  Everyone was oblivious that I just cost the company a lot of time and money.

So with all this goodness raining down on me like gumballs in an Adam Sander movie, how does wearing the cape cause team death?  Let us count the ways:

  1. I’m going to do my job badly.  A Hero only focuses on winning the day.  Like Batman saving the Gotham, we aren’t worried about the collateral damage.  We don’t care about the unfortunate drivers in the way of my super awesome armored tank-bike!  Can you afford to have documentation, cross-training and automated tests become collateral damage?
  2. My teammates are going to feel pretty crummy.  Who gets to be Aquaman on you team?   I’m telling them that they could not do their job so I’ve got to do my job AND their job.  There’s no way to sugar coat that ego buster and their not likely to offer much help after that.
  3. Decisions will be one-sided.  Without a team, the hero has no balancing ideas and will go racing off into obvious traps and dead-ends.  On my latest hero project, I worked all weekend and was told on Monday that there was a check-box on the operating system that would have done the same thing!  Doh.  Why didn’t my teammate give me a heads up on Friday?  Why should he bother – it was better cinema to watch me ricochet all over the project.
  4. Heroes require drama, so nothing will ever stabilize.  It’s silly to dash and keep saving the day if everything’s working pretty well.  Once I’ve put on the cape, I’m much more likely to invent crises to solve because most of the work that is needed to ship a product is pretty drama-free.  Some of the heros I’ve met just leap from job to job faster than a speeding bullet.

Looking for some kryptonite?  It’s not that hard to find:

  1. Stop rewarding heroics with praise, money, pizza, or beer.
  2. Make the hero take a long vacation, transfer them, or fire them.  Really, you can live without them.
  3. If you’ve got a hero, pair them with another person (they can respect) to minimize the damage.
  4. Have heros take on the grunt work instead of letting them cherry pick the best work.
  5. Stop the team.  If you believe team needs a hero then you need to reevaluate your deadlines, feature list, or external messaging.  Generally, an emergent hero is a symptom of a larger problem.

Wonder team powers, activate!

Tippy Towers

or “Synergistic Nondistruptive Integration”

Back in my Space Suit Architect days, I was a fan of “green field” or second system redesign.  It made sense that we should use our hard earned knowledge of the application domain to “build the product right this time instead of rushed crapware.”  Since then, I’ve come to appreciate a more pragmatic approach that I call “Tippy Towers.”

Let’s get specific with an example…

My company, BEware, has a shipping product, Alpha, that works pretty well even though it’s all written in this obsolete language AWFOL-88.  Yeah, it’s got problems scaling, only runs on Vista, and it’s hard to add new features; however, it’s earning revenue and the sales team can sell it.

The engineering team is really hot to recreate Alpha as a new product, Omega, using Ruby on Rails (yeah, baby!) and marketing is behind the move because they’re told it’s the only way to add features.  Side note: in my experience, these claims are always exaggerated but generally true (look for a future post about working out of order).

In a green field approach, everyone agrees to stop working on Alpha and focus all work on Omega.  The company enters the tunnel of death and hopefully dies of starvation before a buggy v1.0 version Omega ships to the companies few remaining customers.

Here’s the plan.  Since we’re astronauts, our design is to build the new hotness Omega first and the later bring over the boring old features of Alpha (like an installer).  As good Agile engineers, we’ll build the most critical use cases into Omega first.  We’ll identify early customers that only need a subset of the Alpha features.  Since these are a known subset and core features, we quickly build some kick-ass next-generation hypen-inducing wonderware.

The early trial customers love the stripped down Omega product and start demanding more of the Alpha features.  Now the Alpha tower begins to tip over because we need Omega to duplicate the all features that are working adequately in Alpha.  Customers can do a comparison between Alpha and Omega and conversations start saying “I would buy Omega but I need to green cross fade accounting API from Alpha.” 

At this point, our Alpha tower is free falling.   Omega has enough features that it is hurting the sales for Alpha while customers wait for Omega.  Worse, abandoned Alpha is falling behind in the market.  At this point, we’re going to spend many months porting working Alpha features from Omega.  This is not value added work, its quality optional crisis management where we rush Omega to be “feature complete” ASAP.

Our alternative is to see Alpha as a tippy tower that we want to keep standing.  Each feature we pull from Alpha to Omega can makes it more unstable.  It takes more effort to support both; however, if Omega can leverage Alpha and vice versa then we have the luxury to focus on new features and better designs.

In a Tippy Tower mindset, Alpha is valuable.  The existing code base is field proven, generating revenue, and shipping.  Even if Omega holds promise, it’s worth our time to keep Alpha selling.  It may slow down Omega, but it ultimately gives the company more time to get Omega right.

Let’s take the Tippy Tower concept a farther.  I argue that Alpha and Omega should be designed to co-exist.  For example, Omega’s new web interface could be displayed on a form in Alpha’s UI.   It may take some time to enable a browser in Alpha, but Alpha users would have immediate access to Omega’s new capability.  In this case, we’ve done work that helps create Omega and also added uplift to our shipping product.

In an ideal Tippy Tower case, those legacy Alpha features just stay in the Alpha code base and die a natural death because Omega replaced them with a better paradigm. The fact that we keep Alpha from tipping over kept us from having to port them and kept us from creating more rushed crapware.

One last note: sales people love this approach – they always have something to sell and customers to reference.

Screening Recruits for Agile Savvy

We’re hiring new managers and developers into my team and its important (to me) that we find people who will embrace our Agile processes.

Sadly, many people experience with the fluffy Agile decorations and not its core disciplines; consequently, interviewees will answer “yes, I’ve done Agile” and not really (IMHO) know what they are saying.

So I wanted to craft some questions that will help identify good Agile candidates even if they have no experience (or negative experience) with the process.

  • Explain a time that you did not agree with a design decision that was being made. [Good candidates will tell you that they had a healthy debate about it, made sure they were heard, and then supported the team decision.  Excellent candidates will give you a specific case where they were wrong and the outcome was better their suggestion.]
  • How have you handled the trade-off between shipping quality software and getting a release done on time? [Good candidates will be pragmatic about the need to release but own quality as their responsibility.  Excellent candidates will talk about implementing TDD and automation so that quality can be maintained throughout a release cycle.]
  • How have you made changes to your work habits based on retrospectives? [Good candidates will tell you about items where they had to acknowledge other people’s suggestions and change their behavior.  Excellent candidates will be excited about having ownership in their team’s continuous improvement and can give examples.]
  • Why are sprint reviews important? [Good candidates will say that it’s important for a team to show progress to other groups.  Excellent candidates will tell you that it’s how a team shows that it is meeting its commitments and getting feedback to improve the product.]
  • Is it possible to achieve the objective to be “ship ready” at the end of each sprint? [Good candidates will say that ship ready is a great target but only practical in the last sprints before a release.  Excellent candidates will explain that being ship ready is a core driver for the process that ensures the team is focused on priorities, quality, and breaking work into components.]
  • Tell me about the best performing team that you’ve been part of. What made it a great team? [Good candidates will tell you about having quality people or a very tight focus.  Excellent candidates will tell you have the shared goals of the team and how people gave up individual recognition to accomplish team objectives.]
  • What does it mean to for a team to be transparent? [Good candidates will talk about status reports and documentation.  Excellent candidates will talk about being willing to take risks and fail fast.]

If they can’t pass theses questions then go buy a lifeboat.  You’ll want it for that that waterfall you’re going to be riding down shortly.

Don’t confuse the decorations for the process!

Or “Icing is pretty, but you need a cake too!”

Good Agile Process books will explain in chapter 1 that the stand-ups, iterations, demos, retrospectives, long planning meetings, et cetera are all “decorations” for the process.  This is an important concept that gets completely lost when they spend the next 43 chapters talking about how the decorations are used to support the process.

The decorations are not the process!

Having stand-ups does not make you agile.  Doing work in sprints does not make you agile.  Using story- cards does not make you agile.  Doing retrospectives where people share honest feedback does not make you agile (but could help you become agile if you actually take the feedback).

Agile is a business discipline.

Agile teams have the discipline to focus on delivering a product that generates revenue.  Management supporting agile teams has the discipline to reward team work.  Product owners have the discipline to provide crisp actionable priorities.  Everyone has the discipline to be transparent and willing to adapt as the environment evolves. 

These disciplines are hard to build and maintain, but they are fundamentally valuable.  They are honest and practical.  They are revolutionary.

Now, if you embrace the agile disciplines then the decorations will reinforce your efforts like mocha fudge icing on a double-dutch chocolate groom’s cake.

Unfortunately, if you lack the discipline then the decorations become a whip used to micromanage teams into a long frustrating death march.  Sadly, I’m finding that this experience is the more common one in the industry. 

Bon appetite!

Won to End (1..N) Ranking

Or I can buy a new toothbrush in Newark

Our team had a breakthrough moment last week, we changed our MRD* into a prioritized (1 to N, no ties) list and immediately identified some artificial work grouping that would have jeopardized the whole release.  Within 5 minutes, we’d done more to rescue our release than the 4 weeks prior.

Working on a traditional MRD is packing to go on a vacation.  You’ve made great plans, you’ve read all the brochures, and you can visualize yourself on the sparkling white sand surrounded beach chair relaxing next to your partner under gently swaying palms. 

Paradise sounds great, but the cold fact of today is that you’re late for your flight and you have to pack everything into a single suit case. 

Remember the McCallister household in Home Alone?   They were so busy packing and rushing out that they left behind sweet defenseless Macaulay Culkin.  Treating an MRD like a suitcase makes you so focused on the little details and the deadlines that you and your team will likely miss the big picture.  And once you’ve started your team may not react to market changes because they’ve already packed their mental bags:  “honey, we can’t have dinner with the President tonight because you promised that we’d play tennis.”

Another frustration I have with MRDs is that they are classically adversarial.  “Why did you pack that?  If you’re bringing that yoga matt then I’ll need a polo helmet!”  The fault is not in the people, the very nature of the document creates antagonism because the document is considered a monolithic whole.

It does not have to be like this.

Creating product direction should be more like planning a road trip than preparing for a transoceanic flight.  We know where we want to go, the essential things we need and who’s in the car: that’s enough to get started.  We’ll discuss the roadmaps as we go so we can explore the interesting sites (“geysers, cool!”) and toss out the bone headed detritus (“infrared beachcombing metal detector?”) along the way.

It takes trust and discipline to free fall into this type of planning.

When my team ranked our traditional MRD items into a 1 to N list, the conversation immediately became more focused, birds began to sing, and we each lost 5 pounds.  Several things happened when we changed the list. 

First, we looked at the deliverable as a system, not just a collection of features.   “Yeah, mayo is not much good without the bread, keep those together”

Then we started talking about what the customer needed (not what we had to deliver).  “Rob, we know you like sweet hot banana peppers but those are really optional compared to slicing the bread.”

Finally, we found it much easier to compare items against each other (“yeah, the ham feature is more important than tofu move that higher” instead of “we can’t ship this product without both ham and tofu!”)

Once we had the list ranked, it was obvious to everyone which features were required for the release.   Our discussion focused on the top priorities and engineering was able to focus on the most import items first.

Spoonful of Zietgiest

Do you want to have a winning team?  Bring a spoonful of zietgiest to your next meeting!

For me, Zietgiest is about how group dynamics influence how we feel about technology and make decisions.  It’s like meme, but I like Zeitgiest more because of its lower vowel ratio (Hirschfeld = 2:10).

Yesterday, my release team meeting had negative Zeitgiest.  Locally everyone was checking email while the remote speaker flipped through dense powerpoint slides.  It was like watching my divorced aunt’s family vacation slides via Webex.  We needed a spoonful of zietgiest!  That’s how I found myself explaining some of our challenges with the phrase “turd in the punchbowl” and getting people paying more attention to the real work.  A small positive spark and faked enthusiasm changed the momentum.  Yeah, it was fake at first and then become real zeitgiest when the other attendees picked up on the positive vibe.

The idea of seeding zietgiest is critical for everyone on teams.  It’s like the William James expression,feeling follows action: if you act happy then you’ll shake off the blues and start to feel happy.  Yes, this is 100% real.  The same applies for groups.  We can choose to ride or steer the zietgiests.

There’s no reason to endure low energy meetings when you can get out your spoon and stir things up.

Free Fall

Or elimination of pre-discussion bias

Learning Free Fall was the hardest of all the Agile comeuppance meted out to me.  Of course, it was not called free fall by the team teaching me about it.  Instead, it was the sinking feeling that I was always arguing the wrong side of every issue.  I was supposed to be the hot architect but my ideas kept getting shot down by the team.  They weren’t wrong per se, but they were not as good as the team could figure out during a discussion.

Meetings went like this:

Rob: “Hey guys, I’ve got this great idea that all of our SQL queries should leverage the zeitgeist feature because it will improve performance 25%.”

Team: “Um, yeah but zeitgeist requires us to perform green joins on the red data.”

Rob: “That’s a good point.  I thought that most of the data was yellow.”

Team: “That’s true.  What about meme queries?  They’re like zeitgeist but work for yellow and red data.”

Rob: “I really thought that zeitgeist was the way to go.  I think we can make that work if we limit the red data.”

Team: “Zeitgeist was a good suggestion, but we’ll go with meme.”

While my ideas helped the team, I could not get over the fact they did not go with my original idea.  Even worse, I would keep fighting for my idea after the team had obviously moved past it.  It slowed down decisions, made meeting more contentious, and caused our optometrist bills to skyrocket from excessive eye rolling.

All that changed when I learned how to trust my team and free fall during meetings.  Free falling (queue Tom Petty) is decoupling the problem from your idea of how it should be solved.  If you bring the problem to your team without a solution then you’ll get everyone’s brain working on ideas for the solution.  When I stopped being emotionally committed to the solution, I was much more ready to listen to other ideas.  More importantly, it was obvious to everyone else that I was more willing to listen so they were more willing to contribute and less interested in arguing.

When I started the discussion without having picked a solution, it was obvious to everyone that their ideas would be heard. 

We had more ideas, better discussion, and better results.  Generally, the team result was much better than my original idea and (bonus!) the team was always more willing to implement them after a free fall discussion.

Free fall takes some practice.  Find a buddy to help you stay on course by kicking you under the table when you’re reaching for your parachute.  Trust me – it’s like EPO for your team (and it’s legal!).