...copyright Elena Yatzeck, 2010-2017

Monday, November 29, 2010

Collocating for Project Inception: It Costs Too Much NOT To Do It


One reality of modern software development is that teams are not collocated.  Not around the same table, not in the same building, not in the same city, and often not in the same time zone.
  • In some cases, companies are taking advantage of exchange rates or pay rates, and locate teams where they can buy more hours for each unit of their currency.  
  • In other cases, companies cultivate a better work-life balance for their employees, and stop enforcing policies around grueling commutes and time clock punching.
Communications tend to deteriorate even more once you let Briana work from home in Wisconsin and Tad work from his mom's place in Hawaii, and everyone has to get on a conference call with Australia at 4:30pm, CST, since a third of the team isn't even in a time zone which is offset by a full hour from you.  Especially if Tad has small barking dogs and a cockatoo.

So here's one incredibly good thing you can do, in the face of this potentially toxic communications cocktail:  start every project with a requirements workshop where in-person attendance is mandatory (and funded) for the product owner, leads from analysis, development, QA, and operations teams.  Sponsors and subject matter experts should be readily available in person as well.

This get-together will require an initial outlay of funds which will hugely repay itself over time.  In her 2004 book, Requirements By Collaboration, Ellen Gottesdiener quotes these helpful statistics from Capers Jones about return on investment for such gatherings.  According to Jones's empirical measurements of large-scale software projects, a facilitated workshop of this type will:
  • Reduce the risk of scope creep from 80 percent to 10 percent, and to 5 percent when combined with prototyping (Patterns of Software Systems Failure and Success, 1996)
  • Cut requirements creep in half (Software Assessments, Benchmarks and Best Practices, 2000)
  • Provide a 5 percent to 15 percent overall savings in time and effort for the project as a whole (Patterns)
  • Reduce defects delivered in software by 20 percent (Patterns)
  • Reduce project failure and cancellation rates by about 50 percent (Patterns)
  • Provide a 10-to-1 return on investment ($10 for every $1 invested; Patterns)
If your CFO is still skeptical, you may suggest running an internal experiment on two projects (try just one with an in-person requirements workshop) and measure for yourself!

Saturday, November 20, 2010

Comprehensive Documentation Strikes Back

IT community members catechized on the Agile Manifesto will recall that the original signers placed a higher value on "working software" than on "comprehensive documentation." But is working software in isolation our ultimate goal?  I think that as the Agile community has moved in the direction of delivering value, rather than just delivering working software, we are now newly realizing we need more than just rich face-to-face conversations at white boards.  We need accessible archives populated by well-structured documents which have been written in full sentences.  It's time to even the balance.

An early Tech Writer
Scott Ambler has done the Agile community a huge service with frequent and comprehensive surveys to determine what people are actually doing in the field, and how well it's working (see this Ambysoft page for a full list of his surveys and results).  As Ambler reviewed his results in 2008, he was surprised to find that Agile projects are actually more likely, and not less likely, than their traditional counterparts to produce "comprehensive documentation" such as "user manual, operations documentations, support documentation, system overview documentation."  Ambler explains this result by saying:
So, I was thinking about this a bit; and what I think was going on is one of the things that we do know, is that Agilists are actually more likely to deliver than traditionalists. That type of documentation is something that you would write towards the end of the project as things stabilize. So, I think what’s going on is Agilists are more likely to get to the end, therefore, they are more likely to do the type of work that you would do at the end, which is finish up this documentation. That’s my theory at least. I can’t back that up with real numbers but that’s what the trend appears to… (both quotations from Ambler, 2008)
One of the joys of early Agile development was its return to a mode of working which is natural for humans: small groups, close proximity, verbal communication, shared whiteboard drawing, project management through tactile movement of cards or sticky notes around on a big visual chart.  And what about actually being able to sit and talk issues through completely in one sitting rather than futilely exchanging emails in stolen moments between other meetings?  The success and happiness of these early teams is literally hard-wired into our DNA, if you believe in socio-biology, and it is intuitively beguiling to the point that today (again by Ambler's count) over 76% of American corporations are at least dabbling in Agile techniques, despite very few on-the-ground measures of what the return on investment might be.

The importance of wide-band face-to-face communication as part of the software development process is undeniable.  But I believe this heady success has led some people to confuse what is "necessary" with what is "sufficient."  As Agile teams embraced this rich and necessary form of communication, proponents began to assert that immediate, personal, and primarily verbal or visual communcations forms were better than their written alternative in most or all settings and contexts.  See for example this widely distributed chart on the "effectiveness" of communications methods.  It is completely brilliant, and worth reproducing even if I weren't quibbling with it:


Anyone who has done any greenfield software development at all will attest to the validity of this chart in expressing the best way to do "modeling," if that is defined as the creative and iterative process of determining what needs to be done, and how it should be done.

Anyone who has ever done production support of a legacy, or "brownfield" application of any vintage, however, will likely find some fault with the lower curve on the diagram, describing the best way to write lasting documentation.  And in fact, even greenfield developers may disagree that incorporating the visual picture is the sole predictor of either "richness" of communication or overall "effectiveness."
  • If you have a "system down" situation, do you want to bring up a video (if you can find it) and watch it through to figure out what might be going wrong?
  • How many times have you silently mouthed the words "Oh thank goodness!" or the equivalent when someone dug up an old yellowed computer printout for you that had the complete set of metadata translations typed out along with pithy descriptions of what the terms meant in context at the time the application was originally written?
  • And truthfully, how many times have you been grateful that someone took comprehensive minutes during your brainstorming meeting to put context around your architectural diagram, capture intent, and record immediate action items?  How handy was that email to you?  If you were there, do you now want to watch the whole meeting again on video, complete with incomprehensible low-volume discussions you can't hear, nose-blowing, groin-area adjustments, and long-winded tangents from people who aren't usually that annoying (or who are)?  If you weren't able to attend the meeting, how much will you get from the video?
When you introduce a time parameter into the evaluation, the graph looks a little different:
I don't mean to trivialize the achievement of early Agile adopters who have brought people back into the process, nor do I propose to claim that comprehensive written documentation is more important than written code.  Using what Jim Highsmith calls "And" Leadership, I'm saying that we should keep both the baby AND the bath water.

No, wait.

I mean we shouldn't throw out the use of written communication for post-facto documentation just because we want to get rid of big speculative requirements drafting.  For the long term, written communication can do some things that video can't do, and even in-person communication can't.  When you've had that really long, contentious meeting, it's extremely helpful for the consensus you've reached to be neatly summarized in words and whatever pictures are necessary.  Otherwise, you're going to have the argument again tomorrow, in two weeks, and/or next year.

Over time, the delivery of business value depends almost as much on the delivery of comprehensive documentation as it does on the delivery of working software.  Let the light saber wars begin.

Tuesday, November 16, 2010

Agile Business Analyst Is Not An Oxymoron

I just read a very angry blog entitled:  Business Analysts And The Million Dollar Question - What Would You Say You Do Here?  The author quotes Scott Ambler's famous line, "Remember, 'BA' is also the abbreviation for band-aid" and he goes on to say that if you hire a typical BA, chances are high that "you're not just wasting your money; you are, as a matter of fact, risking your project by introducing an additional monkey that'll have to be subdued and sedated as your project moves on and the rest of your team starts doing some real work."  Okay, then.

Tom Smykowski, Office Space BA
The Agile Manifesto itself seems to be taking a direct swipe at Business Analysts when it talks about valuing "working software over comprehensive documentation" and "customer collaboration over contract negotiation."  Somehow, Business Analysts have ended up as a proxy for everything bad about the "Waterfall" development method:  Big Upfront Design, Big Documentation, Bad Communication, Silos, Cholera, Blunder-headed Filling Out of Templates, the works.  Oh, wait, not cholera.  But you can see how it would accidentally get into the list--that's what happens when you start to write things down unnecessarily.  Next thing you know, you're gold plating.

But let's take another look at this.  When we're pragmatic, rather than dogmatic, we tend to find that business analysts are proving to be as vital for agile project teams as they were for the first struggling dev-only teams in the dark days before they invented "waterfall."  Let's break this down a little bit.
  1. Even On Agile Teams, someone still needs to facilitate discussion, record consensus once reached, and analyze the business/technical flow of dataThe best business analysts were never the people who doggedly filled out every template to completion--BAs at their best have always facilitated discussions, promoted development ideas to "the business," and urged people to talk directly to each other.  Blaming BAs for "Big Upfront Design" is no more appropriate or helpful than deriding developers as pocket-protector wearing individuals with thick glasses who only communicate in binary.
  2. Time Has Told.  Although the Manifesto and Agile Founding Fathers originally thought of all agile team members as developer generalists (or--dare I say it--old fashioned "programmer/analysts"?), time and experience have dis-proven the hypothesis that BAs aren't needed for agile.  Modern Analyst quotes Alistair Cockburn in 2009 saying:  “the early attempts at Agile development tried to do away with the Business Analyst” because of the potential distortion of communication with a go-between. However, given the complexity of organizations, the disparate business languages that various units may use and the time constraints of subject matter experts, "more recent variations are finding good use for someone with deep business knowledge, who has time to spend talking with the programmers.” 
  3. Corollary:  Time Is At A Premium.  As Cockburn says, some agile teams may gain benefits from a division of labor whereby business analysts stalk, cage, and interview disparate organizational stakeholders, always time consuming practices, in order to present a coherent set of stories to developers, who can then focus on the actual development.  This is not because the developers are not capable of understanding the stakeholders, but because facilitating consensus is a different activity than writing actual code. 
  4. People Have Preferences.  As actual agile teams have been born and gone out into the world, it turned out that everyone didn't want to be a developer.  Speaking generally, some people wanted to work with people, some people wanted to write algorithms, and some people wanted to develop psychotically completest edge cases to guarantee quality.  So although agile teams have often offered everyone the opportunity to do a little bit of everything, gradually the roles of BA, Dev, and QA have emerged as personal expressions of interest by team members.  One could argue that things have now gone too far, and people have hardened into their job titles, but so long as teams (or companies) offer people the chance to continue to learn and grow, it isn't necessary to jump to the conclusion that the presence of BAs or QAs on an agile team is caused by HR departments blindly filling out a template somewhere.
There has been a flood of conversation among my peers recently about the "role of the user experience designer" versus the "role of the BA," and it is beginning to degenerate in a manner very reminiscent of this Dev versus BA conversation.  One colleague posited that the BA position should be eliminated in favor of a combined BA/UX position on each project.  Why must this always be so zero sum?  Some people have excellent design ability, some people have excellent skills for breaking down functionality into easily digested pieces.  Not everyone has both of these skills, and even if they do, not everyone enjoys doing both types of work equally.  So far as I know, the last human who ever knew "all there is to know" was Goethe, and that was a long time ago.  There's a lot more to know these days, a lot more mistakes to be made, and a lot more experience to be hard won.

So to me, the bottom line in all of this discussion is that the fundamental unit of agile production is the team.  The team should combine a number of diverse skills, analytical, communicative, creative, quality-minded, dev/ops, and good-design-informed.  As you put together your team, you should make sure that you have all of the skills needed, from some combination of interests, experience, and skills of the team members.  People will have a combination of generalist and specialist capabilities, and people should hopefully be emotionally as well as analytically intelligent.  But you know what?  It's absolutely fine to start by setting up a ratio of one BA, one QA, and two pairs of Devs, and then correct for missing skills.

It is certainly better to do so than to start tinkering with theories like "<my job title here> can be trained to do it all, and the rest of you are monkeys."

Wednesday, November 10, 2010

Quantifying Manual Test Technical Debt

Forced by circumstances (and an especially pragmatic client), I've recently been asking peers and "the blogosphere" the apparently naive question, "is it important to do automated testing and clear up what Mike Cohn calls 'manual test technical debt?'"

Theoretically, if you have no Big Upfront Design and you also have no automated test suite, you're pretty much just building a big unplanned mess and you'll never be able to change anything for fear of breaking something else.  I understand that, and so does my pragmatic client.  But can this issue be quantified?  Aside from feeling somewhat inadequate when reading agile theory, I mean.

The reason it's important to ask this question is that setting up and maintaining automated testing is expensive.  Unlike many agile practices, which can be set off with some sharpies and a clean wall, automated testing requires a large scale capital expenditure to bring one or more testing packages into an organization, and then some significant strategy development, staging, and training of resources to put those new packages into use and to keep them running.

You need some compelling financial case to show the likely return on an investment which may be half a million dollars for software alone in a medium sized corporation.  So how much is it worth to clean up your manual test technical debt?

There are some related questions out there with quantified answers.  Gartner and CAST
have recently published estimates of how much technical debt is out there globally and on a per-application basis ($500B and $1M, respectively). "Technical debt" is defined by these studies as "the cost of dealing with delayed and deferred maintenance of the application portfolio."  As most people who worry about technical debt note, this "cost to fix" is not the same as what it costs the business to have a software emergency of some kind, or to be slow in rolling out enhancements to the software.  And of course this "cost to fix" is not exactly the same as the cost accrued by an organization due to the lack of automated testing.  But it's a handy statistic to be able to throw around.

Within your own organization, you may want to look at the following, in terms of making your business case:
  • Regression Testing costs:  this may be your easiest way to quantify a return on investment.  What does it cost you to manually regression test your most expensive features?  How much would it cost you to set up automated testing on those features?  It is likely that you'll have some parts of your application where the one-time cost of setting up an automated regression test will pay for itself within the year, in terms of making your manual testers available to do other important testing of new features as they are written.
  • Software development costs:  if you're in a position to do so, quantify the number of points your project teams deliver per iteration, to get a "cost per point."  If your teams are routinely interrupted to fix production bugs and put in patches, you should be able to quantify the cost of these unplanned interruptions, in terms of undelivered feature points.  The costs will likely add up to tens of thousands of dollars quite easily.
  • Insurance analogycost of unexpected down time for your application.  Calculate the cost to your company for down time to your application.  Even if down time is not catastrophic, since manual work-arounds are available, you should be able to quantify the value of the time of staff diverted to doing manual work-arounds during software down time.  If automated tests give you better quality software and reduced down time, a large-scale investment in automated testing may be justified.
  • Business losses due to lack of speed in adding new features:  this may be harder to readily quantify, but as your software gets harder to change, you will lose the agility you intended to gain when your company first took on agile software development techniques.  You may want to put together one or two hypothetical or real cases of changes which were much more expensive to deliver due to the cost of manual regression testing than they would have been, had automated testing been in place.  Depending on what your software does, you should be able to assign a business value to software flexibility, in terms of business lost due to delayed time to market.
  • Total software replacement cost:  at some point your software will become partially or completely impossible to repair, due to the cost of regression testing the whole.  What is the cost of starting all over?  On the other hand, if your company was planning to totally replace the software during some short time horizon, then maybe your manual testing is fine.
It's all very well and good for us zealots to say things like "without automated testing, you Just Aren't Agile."  But if you're looking to get your CFO to release significant funds, you'll want a case relevant to the company bottom line, not just an alignment with some invisible Agile Correctness measure.

Monday, October 25, 2010

And We are His Sisters and His Cousins and His Aunts...!

I don't know whether you have begun to participate in the trend of communicating through "skits" rather than Keynote or MS-PowerPoint presentations, but I would just like to recommend if you do it, you should dramatize "Jeapordy" with questions relative to your message ("What is ... VELOCITY?").  That is easy to do, and always a winner.  If "Jeapordy" is not possible on your budget, then next best is to have a large group of normally dignified people traipse around on stage in herd formation, as is so often done in Gilbert and Sullivan.


I couldn't help but notice this morning that on our small commuter jet from Chicago to Atlanta, there were 50 seats, only six of which were in first class.  As I sat in the airport lounge waiting to board, I noticed there were 38 of us waiting to be upgraded.  38!  And only one unfilled seat.  So eventually, we all tried to board, and that's when I noticed we were all pulling carry-on sized black rolling luggage along with a smaller bag.  We each dutifully tagged the bag with a green label and left it at plane-side.  We all crowded onto the plane and resented the people in the six first-class seats.  After all, virtually the whole plane had been placed on the list!  Then we all herded off the plane and waited for our nearly identical black rolling luggage to be delivered back to us.

There was something so lemming-like in this operation, I couldn't help but be charmed.

Thursday, October 14, 2010

The Agile Business Case

Many agile teams have never seen a business case, ever, and they may even be proud of it.

    Our mantra is that we deliver "business value," not just "software," quicker, better, and faster, but if so, we certainly don't spend a lot of time reporting on value delivery, and in fact we may be scornful about "analysis paralysis."  As software developers, we consider ourselves to be doing quite well if we can deliver the software every two weeks (or continuously).  And this is particularly if we've enabled this frequent high-quality delivery through automated testing and automated build-and-release techniques.  We've reduced business risk by making results visible more often, and allowing the business to change direction more frequently.  We assert that along the way of course we're also delivering value.  But how would we prove it?

    I've recently posited that we shouldn't even think of doing agile projects without capturing and recording story value points during iterations.  But this is only one piece of a larger puzzle.  From a business value point of view, we have five main opportunities to identify business value and tune projects to deliver on it faster, better, and cheaper.
    1. At investment time:  "Someone from the business" (let's call her the "product owner") makes the business decision to start the project and ponies up money for it.  <== We can optimize project value here by ensuring the project is based on some actual business case whose predictions are quantifiable in some way, and can be measured during and beyond project delivery.
    2. During story creation:  At a project's inception, a product backlog or master story list is built, outlining the pieces of business functionality to be written in sentences in the format "as a <>, I need <>, so that <>" <== We can maximize value here by ensuring stories are indeed designed to capture individual pieces of valuable functionality, and that value is captured at analysis time.  I'm frightened to report that I've had the business owner of a project create stories like: "as the system, I need a user interface layer, so that I can input and output values to the database."  Creating stories which capture end-to-end business value in small pieces is very hard to do in practice.
    3. During story prioritization:  The product owner prioritizes the stories (a process which can be repeated before each iteration, as the business climate changes) <== We can and should insist that we have a full product backlog which is being revisited by the business owners of the initiative before every iteration, and we should be keeping tabs on what has actually been delivered, as it is delivered.
    4. While we are delivering the software every two weeks (or continuously)! <== If we were using techniques such as a "value burn-up" as opposed to an "effort burn-up," we could actually see the value accruing over the course of the project.  We should do this!
    5. As the business collects returns on the investment:  the new software translates into some actual benefit for the business (the business sells the software, or the software improves the business's operation in some key way). <== If long-term statistics on ROI are collected, we could quantify actual delivered value over time.
    In our Agile IT writings and trainings, we tend to relegate steps 1, 2, 3, and 5 to specialized "product owner" or "business analysis" functions which require "soft skills."  Often we decide to put our training dollars into learning how to do pair programming, automated testing, or continuous release management, and we simply decide to let the business carry on as it always has.  I think this is a mistake.  At the end of the day, we're not paid to deliver software--we're paid to deliver value.  It's not just strategically superior for us to insist on meticulously optimizing and measuring the value our projects deliver--in a tough economy, it's simple self-preservation.

    Sunday, October 10, 2010

    Agile Lessons from the Army Leadership Field Manual

    You will be amazed to learn that Army Field Manual number 6-22 is entitled Army Leadership, Competent, Confident and Agile.  If it weren't for the small word, "Army," at the beginning of this title, any aspiring lean/agile leader might eagerly browse through this readily available text for tips and tricks to warm the heart and increase the team velocity.  In case there's any suspense, I'm going to suggest that despite the title, you do exactly that.

    Of course, in the world of the Army Field Manual, command-and-control, like gravity, isn't a guideline--it's the law.  And we in the agile/lean community typically frown on command-and-control right along with "waterfall," "big modeling upfront," and using pens smaller than sharpies.  See, for example, this impassioned description from the "Energized Work" blog:

    Command and control elicits compliance to enforced processes through management by policy. Focusing on process fixates management on the means rather than the result. Emphasis is placed on building hierarchies, formalising roles, and people are viewed as resources. All this amounts to bureaucracy and adds no value. Hierarchies introduce protracted decision-making. Decisions made up the hierarchy typically don't involve the people who will be tasked with fulfilling the decisions. And consequently, the support from these people for the decisions is absent. These decisions aren't easily reversed. All this nonsense doesn't exactly set the project up for success. I can't think of a better way to demotivate people and reduce productivity. 

    Interestingly, however, although FM 6-22 is uncompromising in the matter of the leader's authority and concomitant responsibility for the results achieved by her team, it describes a philosophy and practice which would add a lot to any agile work environment.  It turns out that although the ideal Army leader works within a command-and-control framework, her leadership style is the opposite of that outlined by the Energized Work blogger.  For example:
    • People versus process:  in FM 6-22, leaders (and soldiers) are defined by character, knowledge, and application, or "BE-KNOW-DO" for short:  results come first from who you are, second from what you know, and third from what you do.  FM 6-22, in fact, is uncompromising about the fact that people are always more important than process:  "Respect for the individual is the basis for the rule of law—the very essence of what the Nation stands for. In the Army, respect means treating others as they should be treated. This value reiterates that people are the most precious resource and that one is bound to treat others with dignity and respect." (4-18)
    • Means versus results:  FM 6-22 outlines ten ways for leaders to influence their teams which are better than simply issuing a direct order.  "Compliance-focused influence is not particularly effective when a leader’s greatest aim is to create initiative and high esteem within the team."  A good leader is a person focused on results, not process, and who knows how best to work with her team to achieve that result (options include "pressure," "legitimate requests," "exchange," "personal appeals," "collaboration," "rational pursuasion," "apprising," "inspiration," "participation," and "relationship building.) (7-3 to 7-17)
    • Developing the team:  agile theory is pretty silent on this matter, and indeed in practice, companies love to hire young people infatuated with software but who won't throw their weight around when it comes to making decisions.  FM 6-22 puts "developing the team" front and center as a responsibility of a good leader, and provides an entire chapter on how to choose your team and how to counsel each individual to help her advance.
    Certainly, FM 6-22 occupies a world foreign to what most of us experience in the "trenches" of corporate America, where good leadership can literally mean life or death to a large number of people.  Certainly I have no knowledge about how the ideals of FM 6-22 are applied on the ground by actual Army leaders.  Likely, they are as bad at "command and control" as our own corporate commanders. Certainly we in the agile/lean community embrace an egalitarian philosophy that says the person doing the work knows more about the problems at hand than a Vice President eight levels above her.  Certainly we don't want to put as much energy into organizing our hierarchies as the Army has done over the past 200 years--picture what that would do to our deadlines.

    But this field manual is a gold mine for suggesting how to be a leader, how to scale leadership techniques, how to motivate, counsel, and talk with other people.  I can't do it justice in a blog.  Please think about reading it.

    Wednesday, September 29, 2010

    Schrödinger's Agile Leader

    The "agile team leader" seems at first glance to be a sort of Schrödinger's cat:  if a project team is self-governing, then who is leading it?  If there's a leader, is it self-governing?  Is everyone a leader?  Is "the team" a leader?  What happens to individual responsibility and career development, and who watches over it?  Who makes the decisions?  What does an agile executive look like?  As Martin Proulx captures the issue in the title of his excellent August Analytical-Mind blog posting, "I don't feel so good, I'm a people manager in an Agile organization."  It's interesting to think about.



    One way to break this question down is to recap some standard and helpful distinctions in vocabulary.  In the growing field of "leadership training," it has become well understood that a "manager" is not the same as a "leader."  See for example this ChangingMinds.org blog entry.  As managers aspire to become executives, they are sometimes prepared for this increased responsibility by undergoing a specific leadership training which talks about three total roles:
    • Doer:  gets things done
    • Manager:  ensures things get done right
    • Leader: ensures the right things get done
    If we apply these terms to the problem of an "agile leader," within the context of an "agile team," we can see immediately that on an agile team, all team members are asked to play the combined roles of "doer" and "manager" for the team, and that being the "leader" is a different role.  In Scrum, the "Scrum Master" would to some degree ensure that things get done correctly, but a Scrum Master would not typically tell a BA, a Dev, or a QA how to do his or her job, but rather help be the conscience of the team in terms of making sure agreed upon Scrum practices were being followed.

    In Scrum, the person who decides "what" the software should do would be the Product Owner, and the team might take guidance on "how" to implement the software from a technical lead or software architect.  So on the face of it, the Product Owner and Tech Lead could be considered the leaders.  But hold that thought for a second.

    A final, helpful piece of vocabulary is the concept of the "Executive."  Jim Highsmith, who I'm excited to say has joined ThoughtWorks this month, has this helpful list of things agile "executives" can do:
    • aligning agile transformation efforts to business strategy
    • helping teams understand and deliver on business, product, and project objectives
    • creating an agile performance management system
    • facilitating a decentralized, empowered, collaborative workplace
    • fostering an adaptable product line and product architecture
    • creating an agile proficiency framework
    • creating both proactive and reactive organizational adaptation processes
    • understanding the agile development process
    • creating guidelines, training; and support for agile processes, practices, and tools.
    But are we there yet?  I think not.  On the ground, knowing the goals, as the PO and the tech lead do for a Scrum team, or even knowing what the strategic goals should be, as Highsmith's agile executive does, is not the same as leadership either. 

    I habitually turn to Joe Vranich for advice when I am in need of wisdom, and when I talked with him yesterday, he passed along this quotation from Peter Drucker to me:

    “Only three things happen naturally in organizations: friction, confusion, and under-performance. Everything else requires leadership.”

    Drucker's name doesn't come up frequently in the agile theory texts I've been reading lately, for a number of reasons (see this lovely Business Week eulogy from 2005), but doesn't this quotation really capture the essence of the problem?  A leader addresses sources of friction, confusion, and under-performance and does something about them.  So coming full circle, in an August 24 post on the Cutter site, Highsmith, too, talks about how leadership comes down to having the ability to reduce ambiguous situations to their essentials:

    Agile leaders have the ability to cleave through this ambiguity, to focus on a decision when everyone else is floundering, to clarify direction when everyone else
    sees confusion. This requires an ample supply of thought and analysis, but probably an even greater supply of guts.

    In the end, I posit, leadership is a fundamentally human excellence.  It is not something which can be captured on a card wall or a burn-up, or guaranteed through daily 15-minute stand-ups.  Leadership is a human trait, and if there's something like a prototypical "Agile leader" we want to be nurturing, that person is going to be deft at working with that other prototypical Schrödinger's cat, the "Agile enterprise" which never quite settles upon whether to be a philosophy or a set of techniques.

    Tuesday, September 21, 2010

    Agile Fun

    A colleague of mine just sent out a link to an exquisite Schumpeter blog in the Economist entitled "Down With Fun."  You must read this blog!  It skewers the concept of enforced workplace giddiness for strategic gain.  In my view, the blogger is unnecessarily grumpy that today's workplace has eliminated the kind of fun they have on "Mad Men," which includes smoking, drinking, and vigorous workplace fornication, but that's just me.

    From The Economist, 9/16/2010

    What about fun, though?  Should we simply give up on workplace fun?  Is my employer, ThoughtWorks, wrong to make "fun" one of its core values?  The Economist warns that "...as soon as fun becomes part of a corporate strategy it ceases to be fun and becomes its opposite—at best an empty shell and at worst a tiresome imposition."  And indeed sometimes I feel a twinge of irritation as I leap away from a fast-moving oncoming scooter on the carpeting, duck from the path of an incoming beach ball, or put headphones on to avoid the guitar playing ("Kumbay-THIS, you long-haired jerk!" I think to myself, cranking the Mahler). 

    The Economist blogger states, "if it's fun, it needn't be compulsory."  I would argue something much stronger than this:  if it is compulsory, it is no longer fun.

    In 1905, Freud wrote an extremely long and tedious book entitled Der Witz which can completely spoil jokes for you for the rest of your life, if you read it.  So I hesitate to delve too deeply into the true nature of fun, but this is for a good cause.  There's a super cool web site called Visual Thesaurus which actually visualizes synonyms for you in a little map.  Here's what VT does with "fun:"


    On the map, you can see that fun has a dotted line but somewhat distant relationship to "activity," which might include management-purchased oversized My Little Pony plushies in the corporate greeting area and management sponsored conga-line dancing and hat wearing.  VT shows fun much closer to "play" and "playfulness" in common usage.  And as Wikipedia, that font of information correct and incorrect says quite clearly, "Play refers to a range of voluntary, intrinsically motivated activities that are normally associated with pleasure and enjoyment."[1]

    The key words here are "voluntary" and "intrinsically motivated."  And they take us right back to the basis of Lean manufacturing and Agile software development:  harnessing the intrinsic knowledge, good will, and power of the people closest to the work for greatest progress and gain. 

    I once participated in a Rally training exercise which has stuck with me ever since, whereby we divided into pairs, the "boss" and the "staff person," and the boss had to direct the staff person through a maze.  We timed it, and it turned out when the boss had to provide turn-by-turn instructions to the staff person, it took 2-3 times as long as when the person was empowered to maneuver on her own to an agreed-upon destination.

    And indeed my observation is that when you get a group of knowledgeable people in a room, get command-and-control management out of the way, and let the group progress organically towards a well-understood goal, even an evolving one, that is a very fun thing.  If the freedom to take charge of a goal allows people to bring remote-activated tanks, harmonicas, and even handballs into the workplace as well, so be it.  But the gadgets and the behaviors are only symptoms, and you have to be doing the work in the environment to know whether it's fun or just window dressing.

    Thursday, September 16, 2010

    Value Chains and Value Streams: Strategic Maps Are Better For Project Inception

    I was working on some training materials this week, and I needed to whip up as-is and to-be business process maps.  These would be used to help trainees visualize how the software they were going to build in an agile way during the training would transform the fictional business for which it was being written.

    I got completely mired down in this seemingly simple task, and I realized it is because the more you think about it, the more the business process mapping for the purpose of planning a software project is just not simple.

    BPMN 2.0

    If you're like me, you are probably thinking, "well gee, just throw together some circles, rectangles, and arrows, and you're done, what's the big deal?"  Indeed, I started down this track, and in the name of being professional, I spent an afternoon this week familiarizing myself with Business Process Modeling Notation (BPMN, currently at version 2.0).  I became fairly adept at BPMN, which is very nice for creating process flowcharts, and tried out a very cool tool, Lombardi Blueprint, which keeps track of all the arrows for you (as MS-Visio and OmniGraffle do not).  Blueprint also ensures that you create "legal" process flows, which means that each process has a single starting point and a single ending point, and within the process, each box has one or more entering arrows, and only one exit arrow.

    Armed with a new knowledge of circles, rectangles, arrows, and a very nice "clock" widget, I quickly mapped all of the business processes relevant to the software process, and thought how handy the flow charts were going to be when planning scenarios and stories for the software
    But that's when it hit me--what had I done?!  My "business analysis" was predisposed to think of the business process as a set of manipulations of database objects which could potentially be automated, and the moment I had chosen the BPMN flow-chart as the tool I wanted to use to map the business, I lost focus on the actual business problems which were spurring the need for new software in the first place.  I was describing "as-is" and "to-be" from a systems perspective without providing any particular way to notate why "to be" was going to be better than "as is," except perhaps by showing some rectangles being eliminated through software creation.

    Here's where I entered the world Dan Woods describes aptly as "Stalking and Capturing a Business Process."  I didn't want to start with procedural details--I wanted to understand in plain words what the company was trying to do, and where the pain points were.  I wanted to talk about value and strategy.  For this fictional widget company, it turns out, there's an inventory problem customers run into when trying to buy widgets.  You might guess that when you see the clock in the diagram, but the problem doesn't leap out at you.  I wondered whether other maps might do a better job at visualizing pain points.  I found two, which I'll race through here--I think I might need to write something longer one day, to do this topic justice.  (who knew?  I need to write a book!)


    Value Stream Mapping

    Mary and Tom Poppendieck lay out Value Stream Mapping in Chapter 4 of their awesome book, Implementing Lean Software Development.  For their purposes, mapping should start with a customer demand, and work back from there to show all the steps that lead the company to meet the customer demand.  A business process map looks something like this, and can be developed with a process the Poppendiecks describe, and which is also nicely laid out step-by-step in Ron Periera's Lean Six Sigma blog:

    Note that this value stream map focuses on things which occur which impact the cost and timing of delivering goods or services to the customer, focusing on the customer.  Although it may be necessary, in Periera's example, to eventually create and describe an object model which includes customers and peanut butter sandwiches, creation of a flowchart which traces the movement of those objects is secondary to this first business process map, which looks primarily at strategic value and what may be getting in the company's way in delivering it.

    Value Chain Mapping

    Value Chain Mapping was first developed by a Harvard professor, Michael Porter (and in fact you can find out a lot about it by doing an internet search on "Porter Value Chain." )  Value Chain Mapping provides a more rigorous checklist of topics to evaluate in mapping business processes, and is likely appropriate when looking at a larger company than my widget store or Periera's peanut butter sandwich factory.  Porter formalizes the different areas which come into play with any company of size, in delivering a product or service to a customer.  Like value stream modeling, value chain modeling looks at the process that takes raw materials and converts them into something beyond what the customer could build for himself without help, and evaluates where and how value is added.  Here's Porter's scheme:



    There's quite a good free web site describing value chain analysis here.

    The Bottom Line

    I apologize for flying through this quickly, but I wanted to put a bit of an outline out for people who are interested--the bottom line is that the type of mapping you do will heavily impact the way you think about problems.  As usual, I'm pushing tools which may help us to think more about strategic business value, at least at first, during project inception, and less about the systematic steps to improve it.

    Author's note:  It's no coincidence that BPMN is embraced first and foremost by software engineering companies like Oracle who can handily map it to Business Process Engineering Language (BPEL), and thereby create the framework for a Service Oriented Architecture (SOA) software solution.  Further, it's no coincidence that the BPEL created this way is generally described as "not human readible," and that nobody has come up with a way to map BACK from BPEL to BPMN.  I would like to think that it simply stands to reason that the governing body which owns BPEL and BPMN (and UML, for that matter) has the acronym OMG.

    Friday, September 10, 2010

    Using Agile to Optimize Value of Projects

    I was asked recently for guidelines on "how to use value points in agile projects".  I was glad to get the question, since some people, like the blogger who writes "Agile 101," say stuff like: value points are "not appropriate or particularly necessary in all cases."  Gah!  The Agile 101 author actually goes on to do quite a nice job describing how to use value points, and I recommend you visit the blog, but I would like to explain why I don't think all of this is an optional nicety.

    If you were CEO of your company and you were being briefed on your project every two weeks, what do you want to hear from your agile team?
    1. "We delivered 14 value points this iteration" (which translates into some specific thing the company values, in terms of cost avoidance, expected revenue within a portfolio, increased quality, or faster time to market)
    2. "We delivered 14 story points this iteration" (which translates directly into "2 weeks of effort")
    I think the answer is pretty clear.

    Jim Highsmith first published details of this proposition in this blog entry on the Cutter Consortium site, and expands on it in the second edition of his watershed book, Agile Project Management.  In his book, he also expands on this concept of the "Agile Triangle" of Value, Quality, and Constraints, which replaces the traditional Project Management Triangle of Cost, Schedule, and Scope.

    From Highsmith, Agile Project Management, Second Edition, 2009.
    Highsmith's proposition is that when your company embarks on a project, you are making a financial investment in order to produce something of value to the company.  That value has two dimensions:
    • Value (extrinsic quality), which must be determined by the people in your company who are going to use the new software to create revenue for your company.  They will sell the software, or sell the things the software help them to build, and they should have an idea from market analysis and company strategic goals why they want to spend money on the project at all, and how much money the project is worth to company owners (stockholders, public bodies, or a private individual).
    • Quality (intrinsic quality), which must be determined by your software architects.  They will maintain the software you are building over time, and their goal should be to minimize the cost of this maintenance over time.
    Product managers can predict the revenue your company will gain from completing each feature of the project, and architects can predict the revenue your company will lose over the projected lifespan of the project by cutting corners in design at the outset.  "Cut corners" in design are what Highsmith and others call "technical debt."  Bad software is a hidden cost waiting to hit your company in the future, when you suddenly want to do something new and find you can't do it without starting all over, because what you built isn't flexible.

    So as a company invests in a project, it should maximize the immediate and long term value from that investment, while operating within the constraints of cost, time, and potential features to be included.

    Okay, you say, I feel motivated today.  I want to maximize my project's value, not just track the cost of the effort to deliver it at full scope.  How does that work?

    Here are some mechanics:
    • For each project, assign an IT Lead to monitor what Jim Highsmith and others call "technical debt," so that as the software is developed, expected costs of maintenance do not cut into the financial values being tracked by the Product Owner.
    • For each project, assign a Product Owner, a decision maker whose job it is to predict, record, and then update the business value parameters for the project.  These consist of:
      • Project value:  the overall value of a planned software project to your company (predict at project inception, and update before each release planning meeting as needed).  This can be a financial return on investment, or some other appropriately quantified measure, such as increased speed to market, increased quality, or avoidance of future costs.  Cost avoidance could include such factors as regulatory compliance to avoid fines or building software to allow something new to be done in an automated way, rather than taking on new staff. 
      • Feature value:  the relative value of each requested software feature which is planned as part of the overall project (predict during project inception and update before each release planning meeting as needed).  Feature value should be determined through a method such as planning poker, which allocates value to specific features on a relative basis.  So the Product Owner should gather the right people together to think about the expected high level features of the project, and assign a "points" value to each feature.  Once a sum of points is established, that can be mapped to the overall predicted financial value of the project as a whole, and each value point is worth some fraction of that overall value.
      • Story value:  the amount of feature value delivered by each story, where each feature is divided into stories which can be delivered in 1-5 days.  The rule is that the story must provide some recognizable business value on its own.  (Assign as stories are carved out of features, during release and before or during sprint planning).  After the team determines which features should be targeted for a release, the first features to be delivered within a release should be divided into stories which deliver some proportion of the feature's business value in their own right, end to end.  Stories should be things which can be completed within a sprint.
    You can readily see that once you are tracking value at project, feature, and story level, you can do a whole bunch of excellent things, each of which optimizes your company's bottom line:
    • Plan your portfolio:  now that you know the overall expected return on investment of a project, you can weigh that project's risks and potential returns and determine when to make it part of your active portfolio.
    • Speed up and maximize the return on investment for all of your projects:  each project is executed as a series of releases which have the most valuable features in the earliest release, and the most valuable work delivered earliest within each release.
    • Graphically track the value delivered by the project during each sprint and for whole releases, using a "value burn-up," (exactly like an "effort burn-up," but measuring something a lot more interesting to your CEO)
    • STOP your project once you get decreasing returns on investment, in favor of starting a different project in the portfolio which has higher-value features still to be delivered.
    • Collect returns on the working software through the expected lifespan of the design and beyond.
    One final point.  Agile software development is premised on avoiding "Big Upfront Design" (BUFD), and so projects are started with some minimal inception phase to lay out an initial architecture design and an initial effort estimate.  All designs and estimates are expected to change over the course of the project as more is learned.   

    Value points should not take years to estimate any more than your effort points do, even though they result from different techniques.  Do not leap out of the frying pan of the BUFD into the fire of Big Upfront Value Modeling (BUVM, sadly not pronouncable). 

    Your decision-making Product Owner should try hard to limit the time she spends estimating its value.  Just as you shouldn't "start coding" on day 1 of an agile project, you shouldn't "pick a number out of a hat" as your business case.  But neither should you over-analyze, since you expect to be enhancing and revising your business case as the project proceeds.  And of course the market is going to be changing during that time anyway!  So as the project unfolds, the Product Owner should plan to revise the overall predicted project value and value of desired features on a regular basis, and the whole team should plan to adjust release plans appropriately as they go.

    The main point is that value points, like their better known cousins, effort (or "story" points), can and should be estimated, written down, adjusted, and used for planning and reporting purposes by project teams.  They can be estimated in Planning Poker, burned up, burned down, and written in the upper-left-hand corner of your story cards.  ONLY value points, however, can tell you how well the project is returning on investment.  And that's why I don't think they're ever optional.

    Wednesday, September 8, 2010

    Slaying the Agile Scaling Lycanthrope with A Barrage of Silver Bullets, er Maturity Models

    Scott Ambler became my new hero in an instant today when I reached the conclusion of his dazzling Scaling Agile: An Executive Guide, published on his Agility@Scale blog site in February, 2010, where he refers to corporate productivity challenges as "lycanthropes." [p. 19, if you want to skip right to it.]

    I had been hoping to write about scaling agile to large organizations today and will doubtlessly do so immediately, heavily referencing Ambler, whose blog you really must visit early and often.  But I hadn't heard the word lycanthrope before, and I just had to pause and marvel at the aptitude of the metaphor, in a way I haven't done since I encountered Dan North and Martin Fowler describing the relationship between business and technical people in a software endeavor as the "Yawning Crevasse of Doom." 

    If agile enthusiasts gain any traction at all in this world, and apparently we are (Ambler quotes a survey showing that 76% of American corporations were at least experimenting with agile techniques in 2009), I would like to think it's because we are so frequently able to reap a fiscally significant corporate harvest from the hours of effort we've invested in the colorful worlds of J.R.R. Tolkien, StarCraft II (Wings of Liberty) and DragonQuest IX (Sentinels of the Starry Skies).

    Back to the point, however, as agilists, we seem to have moved on since Martin Fowler's 2003 keynote on "Why Scaling Agile is the Last Thing You Want to Do."  Although his advice to "scale down projects" before "scaling up agile" is still true, by 2009, would-be organizational transformers had produced so many models to describe agile at scale that InfoQ had to publish a roundup.  The models annotated are either prescriptive, like CMMI or PMI, or purposefully heuristic and descriptive, as proposed by Ross Pettit

    Now that the dust has settled a little and Ambler has produced his own model, I thought it might be time for a look at what he and two other practitioners offer, and how their models can help:
    • Ambler's Agility Scaling Model (ASM), referenced above.  Ambler's bottom line advice is going to be to retain IBM's Global Services and Rational teams to embed experts into your organization and help with the specifics of applying the model to your situation.  The framework itself is awesome in its own right, however, suggesting an evolution from:
      • "Core Agile Development" (a team iterating) to 
      • "Disciplined Agile Delivery" (an IT organization delivering a business solution from ideation to money-making production), and finally to 
      • "Agile at Scale" which describes a whole IT organization operating in an agile manner for all projects across all necessary continents.
    A blurry approximation of Ambler's original.
    •  An earlier and also highly regarded framework has been developed by Ahmed Sidky and Greg Smith, and is included in their excellent book, Becoming Agile in an imperfect world.  The model, called the Sidky Agile Measurement Index (SAMI), is one which Sidky would ideally like you to engage his company, X2A Consulting (recently acquired by ubroadcast, inc), to help apply to your organization, but Sidky and Smith provide detailed information about how to do this for yourself in Appendix A of their book, "Readiness assessment tables by practice."  The book as a whole provides a helpful reference narrative describing steps you might go through in your own organizational transformation.  Like the ASM, the SAMI does not lend itself to intra-blog rendering at readable scale, but here is roughly what it looks like:
    From Becoming Agile in an imperfect world--read the original!
    •  Michael Spayd has an interesting perspective on corporate agile conversions, and has developed methods for implementing XP within the framework of the CMMI, for companies who need to do so.  More recently, however, he has developed an agile-specific Seven-Layer Framework which usefully lays out the different layers of the organization which need to be addressed in order to successfully transform it.  Spayd, like Ambler and Sidky, can offer guidance on use of the framework through his company, CollectiveEdge Coaching.  Spayd's framework is useful in the way it helps you devise appropriate strategies for transformations of different demographics within a corporate environment:
      • Individual
      • Team
      • Management
      • Program
      • Strategy
      • Leadership
      • Organization
    From Spayd's Agile 2009 Presentation -- read it in full!

    The available published 2010 maturity models have in common that they do not provide a point-by-point guideline as to how to change your organization.  As my brief descriptions above suggest, the owners of these models want you to hire them, and there are other companies out there who have their own proprietary models as well.  And that is a good thing. 

    Returning, as is my wont, to the Agile Manifesto, I suggest that organizational transformation, like so much in life, is going to be about a relationship between an owner of a corporate productivity problem of some size and one or more people who are going to help that person solve it.  We in the agile community, both suppliers and demanders of advice, are going to want to educate ourselves on different ways to think about the problem, and apply different lenses to evaluating how things are going.  In the end, maturity models won't increase corporate productivity, just as surely as silver bullets don't kill werewolves.  You need people to slay both figurative and literary lycanthropes.

    Monday, August 30, 2010

    MegaAgileTron and Sharpie Man Move To A Smaller Tent

    A friend of mine works for a company that did some one-stop shopping and hired the vendor of their agile project management software to also run their IT organization transformation.  Handily, the transformation consisted of implementing the tool, which I'll call MegaAgileTron, and then providing training on the tool.  Apparently, at my friend's Friday's stand-up meeting, one of the people on his team said proudly "Wow, I have a lot of work to do today in [MegaAgileTron]!" And everyone nodded enthusiastically about how agile they have all become.  As my friend said, "it wasn't funny at the time, but now that I think about it..."

    Agile project management tracking software has become its own industry, and the gadget-oriented CIO may be tempted to think that the best way to take her organization to the next level is to buy, install, and provide training on a new project management tool.  This approach, once taken, has the handy side effect that she can now easily prove the "success of the agile transition" by grabbing some readily accessible metrics, like "how many people are using the tool?" or "what projects are being tracked in the tool?" or, in the extreme case of my friend using MegaAgileTron, "how many hours a day are people spending in the tool?"  And really, that's fine with MegaAgileTron, especially for the short term, because at the end of the day, they're in business to sell licenses.

    On the other hand, agile zealots, with whom I uneasily align myself at all times, sometimes lose track of the fundamental fact that what you're trying to do in your organization is become more effective, not become more agile, although of course we all like to think that these two are correlated.  But if purists believe it is "more agile" to limit oneself to a wall covered messily with sharpie-annotated sticky notes than to install a tool, then they will advocate violently for the sticky notes and the sharpies.  I am aware to my sorrow of a project on which purists were asked to update both the card wall and the electronic tool, and refused to do it, because "double-entry isn't agile," even though for their particular situation, double entry was the least evil of the available options to keep the project moving forward.

    So as MegaAgileTron takes on Sharpie man for the banner of agile process purity, I have to ask myself, "why?"  Why does talk of "agile" slide so quickly into talk of processes and tools?  I think the answer is that it's harder to talk about something big like "IT's return on investment to the business" than something small like "what did I do when I clocked into work today."

    An effective organizational transformation to agile techniques and philosophy requires an old-fashioned quantifiable business case which should be revisited by a high level corporate steering body on a regular basis for results.  "Make a good business case" is not an agile concept--it's a business concept upon which agile depends on many levels, from justifying an agile pilot to justifying any given project within the agile portfolio.  Building a good business case is hard, and requires the best attention of your best business strategists.  Tool-buying decisions and decisions about how an individual project team plan to conduct business can be done with far less rigor in most corporate contexts, since these decisions get made by "the tool department" or the individual line manager.  With time, passion, and energy at a premium in all corporations, the business case doesn't always get made, and that's to the detriment of the transformation.

    If you're toying with the idea of "going agile," please work to understand how you will measure the actual business value of changing your IT organization, before you do much investment in highly paid internationally acclaimed speakers, professional services, tool-buying, or training on tools.

    In its simplest form, the Agile Manifesto would ask you whether post-transition, you now get "Working Software" out of your IT organization:
    • faster, 
    • cheaper, 
    • and/or with more reliable quality. 
    Or if you're taking a longer view, you might also measure whether your organizational transformation has impacted:
    • executive confidence (does the process seem more predictable than the old process, allowing for executive flexibility as the project team learns more?), 
    • customer satisfaction (do they like you better now that they get to participate in your process?), 
    • sales (overall, has your company reputation improved, and with it, your overall new sales?), and
    • employee turnover (do your employees like the new process, and flee less?).
    If you are writing software for the internal use of your own company, rather than sale to customers, you may also want to measure:
    • how productive are the users of the software, in terms of the goods or services THEY produce, before and after the change to the IT organization.
    You will also want to factor in the cost of change itself--your experiments with agile will have their own cost, so in your measurements, you will want to make sure to ask whether things are taking longer or being more expensive for your company due to the use of agile, in a way that will remain relatively constant over time, or whether the effect is a short term effect of being in transition.

    Luminaries like Jim Highsmith and Michael Mah have long since made a rigorous, quantified case for how agile software development has paid off in businesses like yours in the past.  Read their work and ponder its impact on your business.  But as you actually proceed with your own transformation, let me just issue the simple reminder that you want to keep focused on the health of the business as you proceed, and not get lost in the sideshow antics of the process people.  Choose tools to use based on the business problem you're trying to solve.

    [Disclaimer:  I am necessarily agnostic with regards to tools, since I think they should be chosen to fit your actual business need, but my firm, ThoughtWorks, does sell one, Mingle, which seems to have been quite good in a lot of contexts.]

    Friday, August 27, 2010

    Et tu, Toyota?

    I was gripped by anxiety today when I found out that Toyota is recalling another 1.13 million vehicles, and even more if you count Canada.

    I worry for Toyota owners of course, American and Canadian alike.  But professionally, I feel even more worried for us advocates of lean and agile principles who have depended on Toyota to provide a one-word motivation to our audiences.  For decades, Toyota has been our hero, and even if we quickly disavow the company now as having any major role to play with lean, the questions about Toyota’s quality and processes are going to come back to haunt us, and we need to be prepared.

    So today’s recall news brings agile presenters (and would-be Toyota owners) together to ask three new common, and newly urgent, questions:

    • Is there actually a problem?  Do friends let friends buy Toyota any more?  Or is this nothing more than an emotional media circus?
    • If there is a problem, was it caused by Toyota’s adherence to lean principles?  If so, what do we need to adjust in our teaching of lean?  If not, what do we need to do in describing how Toyota fits into our world view as agile/lean advocates
    • What should our go-forward plan be, as consumers, and as advocates for lean/agile principles, in light of this year’s Toyota-related events?
    Is there a problem? 

    Well, as agilistas of all races, colors, creeds, and sexual orientation are prone to say, “it depends.”

    Thesis 1:  this is media hype, like what happened to the Audi 5000 ten years ago.  Jamie Flinchbaugh provides this excellent summary of the journalistic and blogosphere debates around the Toyota Affair.  He points to a valuable article by Business Week’s Ed Wallace on how media coverage of this type of incident has fanned the public’s tendencies to prefer the hype around an emotional story to verifiable facts, which tend to be less interesting.

    Thesis 2:  Toyota has an actual long-term problem with their electronic throttling system.  Today’s recall supports a theory the New York Times advanced already in February that Toyota has had problems in their electronic throttling system design which go back to 2002 or even 1998, and what we saw in 2010 was not just driver error, oversized accelerator pedals or loose floor mats.  The smoking gun here is that when the first recall broke in January, Toyota brought consumers in to have their accelerated pedals shortened and mats tightened, they also did a quiet computer reflash on the vehicles to allow the braking system to override the throttle, which it didn’t previously do.

    The ABC story which provided the most details about the throttling problems, however, included footage which was clearly doctored, which suggests that whatever the truth of the matter, hype is part of the equation as well.

    If there is a problem, was it caused by Toyota’s adherence to lean principles?

    Let’s assume there is a long-standing problem with Toyota’s electronic throttling system.  Or even a shorter-standing problem with floor mats.  To what extent should we blame lean?  The Wall Street Journal’s headline article on January 30 discussing the Toyota affair was entitled “How Lean Manufacturing Can Backfire.” 

    Mark Graban of LeanBlog observes that the WSJ seems to make frequent ill-informed attacks on agile/lean techniques, particularly “just in time” manufacturing, and if he's right, this is no exception.  If, indeed, the use of large quantities of standard foot pedals was the root cause of the problem, as they explain, that’s hardly a manufacturing choice Toyota can be said to have made from lean principles, which calls for small, quality batches.

    Moreover, if Toyota knew about problems with their foot pedals, floor mats or electronic throttling systems and didn’t act on them, then Toyota was definitely NOT following fundamental principles of agile and lean that call for “failing fast” and “building in quality.”  Toyota is famous for providing a literal “emergency stop” cable which could be pulled by any person on the production line who finds any fault in the product, and which should result in an immediate halt to production until the problem has been fixed.  As Gad Allon says eloquently, there were enough quality problems being reported to Toyota as early as 2002 for someone to have done something much earlier.  He calls this “The Andon Cord that Wasn’t Pulled

    Toyota, in fact, persists in denying that there’s a problem with the throttling system, even while issuing soft fixes on it, including in this webinar which provides Extreme Scientific Detail.

    So Where Does That Leave Us?  Can We Still Use Toyota Quality to Motivate Agile Converts?  And Should We Buy Toyota?

    Any long-term admirer of Toyota and what it stands for may be disappointed in the recent quality problems with the cars, and with what seems like clear violations of the agile/lean philosophy that should have prevented these problems.  However, as many others have said quite eloquently, Toyota is still quite exemplary in most ways, under most circumstances, compared to the competition, particularly when it comes to using lean processes to leverage generally higher quality. 
    • Long-time Toyota watchers predict that Toyota will continue to beat out their competition in finding and removing these defects, in the long run.
    • And as Marty Larivieri says, the fact that the WSJ is using the example of Toyota to show that lean principles lead to lower quality could certainly mean that “apparently someone who is too young to remember the Chevy Citation and its X-body siblings is now old enough to write for a major newspaper.”   
    All the same, in my slides, I am temporarily going to start talking about Toyota the way people talk about their favorite rock band, and explain “I only like their old stuff.”

    Monday, August 23, 2010

    The Agile Ronin

    I was recently cheered and inspired by Jonathan Rasmussen's post on the "agile samurai," also the title of his recent book.  In his blog, he suggests that the the samurai, unlike the "rice pickers," are the ones who:

    • say what needs to be said
    • call BS when they see it
    • laugh in the face of unrealistic schedules and expectations
    • tackle all the hard, complex, thorny stuff no one else wants to wade into
    • are technically excellent at their craft
    • take pride in their work
    • and are comfortable in their own skins
    I liked this description, but it made me nervous in some ways as well.

    So since I'm prone to zealous pursuit of metaphors, to the point of finding and reading two or even three results of a Google search as I look for enlightenment, I investigated real-life samurais and found that they were actually constrained significantly in their behavior, living by a creed of "loyalty to one's master, self discipline and respectful, ethical behavior."  Such a person might think twice about pushing back on authority in a corporate context.

    In fact, it appears that the bold warrior implied by Rasmussen's description is most likely not a run of the mill samurai, but rather a ronin, the "masterless samurai," who lived by his own ethical code but had no permanent master.  Prior to the awesome 1998 movie with Jean Reno, ronin were already famous in Japan through the mythology that grew out of the eighteenth century heroics of the "47 Ronin." Wikipedia says that the story "was popularized in Japanese culture as emblematic of the loyalty, sacrifice, persistence, and honor that all good people should preserve in their daily lives."  But this story also holds some cautions for people who value, say, continued employment.
    From http://japanesetex-style.blogspot.com/2011/02/47-ronin-kisshoji-temple-osaka.html
    Without giving the full plot away, the incident involved 46 principled men revolting against all established authority to avenge their leader, the 47th man, who committed ritual seppuku at the request of his boss, the shogun, rather than apologize after he physically attacked a person who severely annoyed him.  Before he died he announced his only regret was that he swung and missed.  Of course, once the 46 successfully killed Kira, the annoying person who caused all the ruckus in the first place, they were also immediately requested to fall on their own swords, literally, which they did.  

    In the aftermath, commentators complained about a number of things, including that the 46 should have slit their own bellies immediately upon taking their revenge, rather than waiting for orders, since they were being cowardly if they hoped for a non-death sentence.  Others argued that they shouldn't have waited a full year to take their revenge since the annoying person at the root of the problem was 60 and could have died in the interim.  As in so many real life situations, an odd situation seems to have spun out of control and then been very weirdly interpreted after the fact.

    At any rate, at the end of the day, the implied hypertext behind the concept of the "agile samurai" is probably a little disruptive to the central message of the book and the blog.  It is sometimes good to remember that discretion is sometimes the better part of valor.

    Friday, August 20, 2010

    The Good Silo -- Servant Leadership, Self-Organization and Yes, Old Fashioned Middle Management

    Please be honest.  Does the concept of "self organizing teams" frighten you at all?  Do you picture everyone in your organization trooping into some large room with no chairs and being asked to arrange themselves into color-coded "pods" and then come up with plans for total quality management, preferably presented as a skit with lame props to upper management, who, naturally, will not be participating in the pods in any in-person sort of way?

    If you're in charge of an IT organization of, say, 250 people, and you want to really shake things up the way British Telecom did, you would be fully within your rights to ask:  so how do we structure this?  And you should demand an answer from your transformation coach more specific than "well, we'll set up a bunch of collocated self-organizing teams" which is implicitly followed by the silent phrase ("and your middle management will hate it, hate us, and hate you, but mostly hate us, and won't THAT be dramatic??").

    I'm going to break a pretty strong agile taboo here and suggest that you just go ahead and set up a new hierarchy.  Yes, that's right.  If you don't already have them, you should hurry to set up old-fashioned line of business silos.  Certainly I'll allow for piloting first and the like, although strictly speaking, I don't think it's necessary.  And certainly, I'll quickly explain why these will be "good" silos, not "bad" silos, if you'll bear with me for a moment.  For now, let's just say that the best way to structure your teams is going to be to start by having everyone focus on the customer.

    This structure will naturally create the need for multiple levels of leadership, and give each of your middle managers something quite meaningful to do:
    • Somebody should be on point for each line of business.
    • Everyone else should be organized into a tree structure attached to this person where managers are held responsible for their area of authority, and wherein no manager is responsible for more than 10 people.  Really, six would be optimal.
    • The nodes of the hierarchy should be project teams working off of a shared program backlog for the whole line of business.
    That's it, you're done with the hierarchy.  But now comes the hard part:  servant leadership and, yes, self-organization.

    When you publish your org chart, borrowing from the Swedish as is so often the case, you are going publish it upside-down, at least philosophically, or even literally as this Houston, TX, construction firm does:


    As symbolized by the chart, you will start your agile improvements by recognizing that the most important people in your organization are the project team members who are delivering the software to your customers.  This transaction is the reason your company is in business.  These teams are cross-functional and self-organizing, once they are set up, and through their interactions directly with customers, they will tell you what you need to do.

    Meanwhile, each manager is in place to carry water and remove obstacles for the team above them.  Sure, then as now, as you move further away from the project team level, you will encounter higher salaries and fancier titles, but to some degree, these will now be justified, because you need skills that are harder to find in the marketplace as you move down this chain.  And what are these skills?
    • Monitoring the team for problems and protecting it from bullies, removing them where necessary.  The more points in the hierarchy, the more challenging it is to figure out who the bullies are to remove them.
    • Coaching, including providing salary and bonus related financial incentives to individual team members, based on how well they are working within the structure.
    • Representing the team's actual needs on occasions when attendance by everyone would be a pointless free-for-all.
    • Encouraging and organizing cross-team and cross-boundary meetings at all levels, so that within and across lines of business, the whole corporate structure can take advantage of mutually known best practices to continue to do the best thing for the customer.
    The stress in this new structure is on non-cynically using one's authority to consistently tune the individuals and interactions which are the most important part of the agile endeavor, and keeping everyone focused on the customer.

    The result of the new structure is that although each line of business has a boundary, the members of these hierarchies are given incentives to work together, within and through their boundaries, to do the best they can for the customer.

    Your job, as the owner of the new structure, is to monitor it to ensure that it behaves as though it is upside down, primarily by observing the human interactions which are occurring within it on a day to day basis.

    When you hear "servant leadership," you may be even more frightened than you were about self-organization.  Perhaps you think of socialism, or maybe a "rap session" someone made you attend in your youth.  Maybe you flash back to when they tore the desks out of the library at your university and filled it with orange bean bag chairs, and you all started calling it "the Romper Room."  And even though you're super liberal on the political spectrum, or maybe you're not, maybe that scares you.  If so, okay, fine, we're all scared.  But let's get over it and start building some silos.