Big Data to tame Big Government? The answer is the Question.

Today my boss at Dell, John Igoe, is part of announcing of the report from the TechAmerica Federal Big Data Commission (direct pdf), I was fully expecting the report to be a real snoozer brimming with corporate synergies and win-win externalities. Instead, I found myself reading a practical guide to applying Big Data to government. Flipping past the short obligatory “what is…” section, the report drives right into a survey of practical applications for big data spanning nearly every governmental service. Over half of the report is dedicated to case studies with specific recommendations and buying criteria.

Ultimately, the report calls for agencies to treat data as an asset. An asset that can improve how government operates.

There are a few items that stand out in this report:

  1. Need for standards on privacy and governance. The report calls out a review and standardization of cross agency privacy policy (pg 35) and a Chief Data Officer position each agency (pg 37).
  2. Clear tables of case studies on page 16 and characteristics on page 11 that help pin point a path through the options.
  3. Definitive advice to focus on a single data vector (velocity, volume or variety) for initial success on page 28 (and elsewhere)

I strongly agree with one repeated point in the report: although there is more data available, our ability to comprehend this data is reduced. The sheer volume of examples the report cites is proof enough that agencies are, and will be continue to be, inundated with data.

One short coming of this report is that it does not flag the extreme storage of data scientists. Many of the cases discussed assume a ready army of engineers to implement these solutions; however, I’m uncertain how the government will fill positions in a very tight labor market. Ultimately, I think we will have to simply open the data for citizen & non-governmental analysis because, as the report clearly states, data is growing faster than capability to use it.

I commend the TechAmerica commission for their Big Data clarity: success comes from starting with a narrow scope. So the answer, ironically, is in knowing which questions we want to ask.

Screening Recruits for Agile Savvy

We’re hiring new managers and developers into my team and its important (to me) that we find people who will embrace our Agile processes.

Sadly, many people experience with the fluffy Agile decorations and not its core disciplines; consequently, interviewees will answer “yes, I’ve done Agile” and not really (IMHO) know what they are saying.

So I wanted to craft some questions that will help identify good Agile candidates even if they have no experience (or negative experience) with the process.

  • Explain a time that you did not agree with a design decision that was being made. [Good candidates will tell you that they had a healthy debate about it, made sure they were heard, and then supported the team decision.  Excellent candidates will give you a specific case where they were wrong and the outcome was better their suggestion.]
  • How have you handled the trade-off between shipping quality software and getting a release done on time? [Good candidates will be pragmatic about the need to release but own quality as their responsibility.  Excellent candidates will talk about implementing TDD and automation so that quality can be maintained throughout a release cycle.]
  • How have you made changes to your work habits based on retrospectives? [Good candidates will tell you about items where they had to acknowledge other people’s suggestions and change their behavior.  Excellent candidates will be excited about having ownership in their team’s continuous improvement and can give examples.]
  • Why are sprint reviews important? [Good candidates will say that it’s important for a team to show progress to other groups.  Excellent candidates will tell you that it’s how a team shows that it is meeting its commitments and getting feedback to improve the product.]
  • Is it possible to achieve the objective to be “ship ready” at the end of each sprint? [Good candidates will say that ship ready is a great target but only practical in the last sprints before a release.  Excellent candidates will explain that being ship ready is a core driver for the process that ensures the team is focused on priorities, quality, and breaking work into components.]
  • Tell me about the best performing team that you’ve been part of. What made it a great team? [Good candidates will tell you about having quality people or a very tight focus.  Excellent candidates will tell you have the shared goals of the team and how people gave up individual recognition to accomplish team objectives.]
  • What does it mean to for a team to be transparent? [Good candidates will talk about status reports and documentation.  Excellent candidates will talk about being willing to take risks and fail fast.]

If they can’t pass theses questions then go buy a lifeboat.  You’ll want it for that that waterfall you’re going to be riding down shortly.