Now that the world has heard of Agile [1], they think–incorrectly–that the pieces of Agile they like best can be cherry-picked and used in isolation. Unless it is combined with Lean Thinking, agile software development can achieve only a fraction of its potential. Agile software teams are not sustainable for very long if they are islands in a sea of waterfall projects. The presence of agile teams creates a new and incompatible dynamic within a waterfall company. Agile adoption programs conducted without a thorough understanding of this dynamic will continue to have a very high mortality rate, especially in larger organizations. This artcle examines four change processes that must occur simultaneously for agile adoption to succeed.
High Mortality of Agile Adoption
Why do we see such a wide variation in agile adoption programmes across companies? Why is it that we so often see agile pilot projects able to deliver better quality software in significantly less time, and yet the companies involved do not replicate this success across the enterprise? Every year a growing number of agile conferences offer new experience reports showing high quality projects delivered in 30% to 50% of the time they would have needed using previous methods. From the agile coach community (no verifiable statistics, unfortunately) I know that a very large percent of companies fail in their attempt to take agile the rest of the way to being used enterprise-wide.
Often when a company has piloted a few successful agile teams, managers decide that they can mostly go on as before and just "cherry-pick" some of the Agile practices and tools. The "mostly go on as before" part means conducting product development as a push system, rather than as a lean pull system. So rather than bother with having a product pwner (the person who pulls value from the development team on behalf of the business), they let analysts decide via committees what features to have the agile team build. Testing is difficult for realistic data and error handling scenarios so rather than invest the effort, they settle for superficial testing that is too labour-intensive to be sustained. These are just a couple typical compromises that pave the way for failure.
There are actually four change initiatives that must be managed successfully if a business is to sustain and spread the success of agile pilot teams throughout the enterprise. Even more critical is that they must occur simultaneously. These pillars of agile adoption are:
- Teams must be able to produce bug-resistant software sustainably
- Teams must consist of empowered, engaged people
- Work flow to the agile teams must be controlled via a ‘pull' system
- Lean portfolio management must be used to control work flow for the organization
Unless all four of these change initiatives are running successfully, characteristic problems arise. They did for the Comet team—an agile development group at an insurance company. Let's look in on this agile team a year after they started their agile adoption initiative.
Agile Entropy–An Example
The Comet team is one of several agile teams that were set up to pilot agile adoption at this insurance company. Their story here is a composite of events that I and other agile coaches have seen. They had external help for a total of eight months: Scrum training, and additional technical training in the use of agile test frameworks and help in cleaning up their build process. Their consultant Scrum Master was on site most of the time for the first three months and then present every other week until disengagement when they felt confident to continue on their own.
Comet Team from Agile Adoption to One Year On
When the consultants completed their work, the eight members of Comet team were running three-week iterations. Their velocity had initially jumped around but was stable and gradually increasing. They were holding retrospectives at the end of every iteration and checking a few days into the next iteration to make sure they were really following the new resolutions they made in the retrospective.
Their product owner was 50% allocated to that role and was meeting with stakeholders a week prior to the end of each iteration to finalise stories for the backlog. The team's ongoing mission was to maintain and enhance risk analysis software for use by the underwriters. The work they turned out was the best quality the company had seen. The team knew that there were still bugs in the legacy code but at least they were not adding many new ones. As they improved their tests they were cleaning out more and more old bugs.
A year later the consultant Scrum Master saw Tony, the Comet team's lead developer, at an industry conference and asked him how things were going. Tony said he wished they had never gotten into agile at all! They were no longer doing real iterations, just moving the work along in a more or less continuous stream. They had dropped doing the retrospectives after Joanne, who facilitated them so well, moved on to other work. They continued them awhile but they were just making a quick list of what worked, didn't work, and what they'd change, without delving into the tougher issues. The biggest problem with the retrospectives was that things they most needed to change required cooperation from another department that was always overworked. So there seemed no point in making the same useless resolutions over and over. The other department was not responding and wasn't going to. They were invited to send someone to the retrospective but they never did.
Dissension Within Team
The biggest letdown in Tony's mind was that agile had opened the door to turning software development into a sweatshop. That's the word he used—"sweatshop". Before the conversion the developers had their own cubicles, and in the enthusiasm of early agile they'd given them up in favour of a team room. But they were happy for eight months with the team room–why the problem now? They were arguing a lot now, and being together all the time in a team room only increased the tension.
Stressors
They were arguing a lot because of disagreements over how much time to devote to cleaning up quick fixes they'd had to put into the code versus getting new functionality in place that their customers needed. At least that's when the arguing began. Now there were many running arguments on all sorts of things. The quick fixes got rushed into place when new features activated a couple latent bugs in the code that their agile test framework didn't yet cover. Those bugs got a set of insurance policies written with incorrect risk assessments, forcing the legal department to sort out a remedy. That brought criticism on the project manager who'd been fine with team autonomy so far. But now he was being criticized by his manager for not having been firm enough with the team.
Repercussions on Team, Product Owner, and Managers
The product owner had cut way back on the time he devoted to his role because the team was delivering so predictably month after month, before the problem with the bugs and the Legal department. He felt that at some point agile should just be able to run on autopilot; besides there were many other demands on his time. He was feeling caught-out when Legal had to come to the rescue. At first he defended the team, understanding their explanation about the test framework. But he became dismayed by their incessant arguing and no longer wanted to be in the product owner role.
Both the product owner and the project manager were starting to wish for the good old days when you just had the analysts write a specification and hand it over to a virtual team - one where the project manager rotated people in and out of a dispersed team and assigned all work tasks. It was getting easy to forget the headaches associated with that, when these new problems seemed so intractable.
Team velocity was steadily decreasing. So managers pressured the team to speed up. With each iteration, the team responded by promising more and delivering less. Often nearly half the stories committed to were not completed in the sprint. In an effort to work faster, they no longer helped each other out, working solo on the tasks they claimed for themselves. Everyone missed the teamwork they used to have but didn't know how to get it back again.
Tony was using the conference to renew his contacts elsewhere. He had decided to leave the company.
Diagnosis
It is clear that in this agile conversion this team learned how to operate using agile practices - to an extent - but managers did not learn how to monitor this new Agile system and use the leverage points that could have kept the situation from spiraling downward. Pleased by the agile turnaround, they expected it to just remain on course of its own accord.
Agile systems don't automatically stay on track. No system does; they all need some degree of monitoring and adjustment to counter entropy. In an organization, the "Agile system" consists of the core team, plus the stakeholders [2] who've chartered that team, and others who manage or interact with the core team. As part of an agile adoption programme, managers need to learn the critical role they must play in fighting entropy by exercising control in a new way.
Let's look at a simple analogy. If enforcement of traffic laws were suspended, it wouldn't be long before there would be chaos on the roads. People who want to follow the laws would have to respond to those who don't and the whole thing would spiral downward. Likewise in an organization it is for managers to understand the laws - the key leverage points - and use them correctly to keep the agile system running with minimal ad hoc intervention.
Let's take it one step further. Suppose there has been a breakdown–as we've seen with the Comet team—what are managers to do? An accident, flooding, or downed trees can create a big traffic jam on a road. Regardless of the cause, the individual motorists are powerless to move their vehicles once they are caught in the gridlock. No amount of pressuring or punishing them makes any sense. The first thing the police do is not to pull cars out of the jam, but to halt the inflow of more cars. They block off the routes into the gridlock, thus preventing it from growing. Next, they free cars out the other side by clearing the blockage.
The jam is a system problem, not a problem of the individual motorists. Although each driver has responsibility for controlling their vehicle properly, that won't prevent or solve traffic jams. Likewise many of the problems that software teams experience are really system problems of their company. Executives and managers govern that system and have a special role to play during and after a change to agile.
What should be done in the case of the Comet team? The set of problems they encountered can be viewed in terms of the four change pillars. For each one, we'll look at the leverage points that could've stopped agile entropy.
Problems With Pillar #1: Sustainable Bug-Free Code
Agile teams must be capable of delivering bug-resistant software. That doesn't mean zero bugs every time, but it does mean that they should have negligible impact. Troubleshooting and fixing bugs should not take up a noticeable amount of an agile team's time. Agile technical practices are designed to achieve this goal, and keep the code base in a fit condition for further work. Agile technical practices shift the mindset from:
Specify feature; code feature; integrate feature; then do ‘infinite' test—fix looping, till resources depleted
To:
Specify feature via tests; code feature; integrate feature continuously so code is always releasable
Agile teams must be able to estimate their work and—in the absence of blockages—be able to deliver on their estimates every time. Managers are partners with the teams. They fulfill their part by promptly removing blockages, and by providing the team with resources to fully test their code during development.
Technical Debt and Latent Bugs
Let's look at Comet's bug issue. Activation of bugs in the legacy code via new features added to the code base was the initiation point for the unraveling. "Technical Debt" is a term covering hidden problems in the code: unnecessary complexity, unclear naming, excessive coupling, inflexible architecture, and latent bugs. Perfectly running software can have these breeding grounds for bugs. Technical Debt shows up when you try to make changes to the software. It makes bugs far easier to create, even for careful developers.
All software has technical debt. But code written the agile way with strong unit tests has much less debt and therefore far fewer bugs–typically one or two orders of magnitude fewer. In Comet's case, technical debt was getting removed incrementally. This is normal. Removing it all in one go would mean having to focus on that alone for quite some time without building new features. That would be unacceptable to the project sponsors.
Without their unit tests fully constructed, the possibility of a critical bug getting through does exist. Even if the unit tests were fully constructed, it is always possible to overlook something in the design of the unit tests. Human error cannot be eliminated completely by Agile or any other methodology. With that said, agile teams do routinely achieve bug rates on the order of 0.2 defects per function point, or roughly two bugs per month for a team of five people. [3]
Who Owns Bug Risk?
Management in this case needed to own the risk of bugs. If you view the organization as a system for producing software-intensive products and services, then the maintenance and enhancement of that system is the job of managers. Owning the bug risk means understanding that technical debt is a precondition for bugs and that technical debt is not there by accident. It is a property of the system that can be controlled.
The bug risk can never be taken to 0% but it can be lowered to any level that one cares to pay for. This is always a cost-benefit decision that is ultimately up to management, not the team. Managers decide how good a development environment to provide. Even if a perfect testing environment is provided, there is no limit to the number of agile unit tests that can be written for a given code base. [4] You can make the tests as fine-grained as you need. But they will require effort (cost) to maintain.
There is a constant tension between how much technical debt you allow into the code vs. how much effort you put into your agile testing strategy to combat it. This cannot be managed effectively from outside the team. It requires a deep and constant familiarity with all aspects of the code. Only a cross-functional core team will have the skill and judgment to do this efficiently. This is why testers need to be part of the team, and why the team needs autonomy to decide how the work, and the testing, will be done. [5]
Agile technical practices give the team new skills to take the bug risk far closer to zero than they could have before. Still, it is for managers to own the fact that accidents can happen, and they must be prepared to untangle the resulting traffic jam if an accident does occur. This is what Deming names "common cause variation", variation intrinsic to the system that is not due to any mistakes made by those working within the system.
Particularly in software development, insufficient early investment takes a very heavy toll. The problem is that it's hard to say definitively just how much specification detail is enough, or how much requirements analysis is enough, etc. agile teams solve this problem by following a simple rule: In every iteration deliver tested, working code that customers can directly evaluate. That means deliver whole usable features, not mere technical components.
Takeaway Lesson
Every agile team needs to be able to produce bug-free code. To the extent that you have bugs, your project is out of control. The product owner and line managers of the Comet team members should have owned the bug risk by defending the decision to pay down the technical debt incrementally. If they had fully explained the risk and its mitigation plan to higher-ups beforehand, the incident with the Legal department needn't have been so damaging to the agile adoption programme.
Problems With Pillar #2: Empowerment
Partnership between team and stakeholders is fundamental. If one side can never say ‘no' to the other then it is not a partnership–it is a master-slave relationship where neither side can ultimately get their needs met. Partnership is what opens up the entire domain knowledge of a whole team and places it fully in service to the organization. Only autonomous teams can take full ownership of a goal, relieving managers of tedious unnecessary control tasks.
Two things are needed for a core team to step up to their role in this partnership:
- Autonomy over their work, with a practical way to make decisions as a group
- Mentoring to continuously improve their skills in the technical work itself
Empowerment of agile teams is not optional; it is crucial for the high degree of focus and dedication necessary to produce bug-free code sustainably. Mere process rules that a company may institute cannot make people engage to the degree necessary for producing such high quality software. Their commitment, creativity, passion, and energy are needed - not just their obedience to rules! No externally imposed discipline is any match for self-discipline. This is the key to agile (and to Lean Thinking), which is completely missed by so many would-be adopters. [6]
Let's look at how issues with empowerment surfaced for the Comet team. There are many problems stemming from this, and in my coaching experience that is common– empowerment is widely viewed as a "nicety" that can be skipped or, even worse, as a step toward anarchy.
Erosion of "Soft" Skills as First Danger Signal
The first danger sign was the disappearance of retrospectives. Group learning and team unity of purpose must be renewed periodically. This is a very high-leverage activity that solves a great many thorny problems while they are still tiny. The line manager(s) of those on the team should have noted that Joanne was carrying the load for retrospective facilitation, and should have recognized the importance of that skill by helping others get trained to do it well also.
Facilitation is only part of the picture; good data is needed too. Teams should experiment in every iteration with ways to improve their work, recording data on each experiment. Data plus an efficient way to make team based decisions should have prevented the team's arguments from getting out of control. Comet team's first argument was over how much effort to spend cleaning up the quick fixes in the code when customers were clamouring for new features. A wise technical manager knows that new features built on top of quick fixes just creates a bigger cleanup job to be done later. If this wasn't being grasped by the product owner, then it was a point where a technical manager could've reinforced the team's message.
Too often good facilitation is dismissed as a "soft skill" not as important as the technical skills, and something we can do without in a pinch.
Missing Cooperation Between Departments
Failure to act on issues raised in retrospectives is another problem. The team had repeatedly raised issues requiring the cooperation of another department, but it was not forthcoming. By letting this issue sit, managers sent a message to the team that it wasn't important. This contributed to the team's feeling that retrospectives were pointless. A deeper root of this problem is insufficient larger commitment to agile, spanning departments. This is a system problem that the core team cannot solve. Managers need to take the lead here.
In an agile adoption programme, friction between the agile teams and the surrounding organization will occur. The organization evolved to support waterfall teams with handoffs between siloed job functions. It will have to change if the agile teams are to survive. Management's chief job in an agile adoption programme is to watch for the friction points and then adapt the organization to the agile teams.
Misalignment of Accountability and Control - ‘Blame' culture
The organization was trying to hold the project manager responsible for the team's actions. The team has to be responsible for its actions. There was nothing the project manager could've done to make the team avoid those bugs. When things go wrong there is often an impulse to place blame. The need to deflect blame is what lies behind much of the overly-detailed planning in waterfall projects. Deming placed "Drive Out Fear" as one of the principles in his fourteen Points for Management. The project manager needed support from those sponsoring the agile adoption programme. By allowing others to place blame on him they let fear grow.
In the case of the bugs that occurred, they were unforeseeable since they were latent in the existing code. Probably the only way to have prevented them was if the company had invested in more thorough testing designed to catch a scenario of that type. But it is very common (and possibly justified) for companies to decide not to invest in doing that. In that case, the team were operating within a system that had set this trap for them. Once the bugs had repercussions in the Legal department, the organization's blame reflexes kicked in. Too bad they didn't have the benefit of a good retrospective to identify these dynamics and find a constructive way to address them!
Failure to Sustain Agile Feedback Mechanisms
Another key leverage point that managers neglected is the extremely vital feedback loop between team and stakeholders. A strong product owner actively engaged with both the team and the stakeholders is necessary for agile teams. Like well-run retrospectives, this prevents a host of problems that are very difficult to address in firefighting mode. Once a team loses the trust of their stakeholders everything is an uphill battle. Although projects vary in their demands on product owners, it is always wise to invest in keeping that feedback loop healthy. Give them time to perform the role.
Comet team's product owner was feeling pressure to minimize the time devoted to his role. The executive sponsoring the agile adoption programme (the agile champion) should have been making sure that people in key roles were allowed the necessary time. A product owner's manager won't be willing to cooperate unless they have bought-in to their own role in the agile adoption programme. The programme must have sponsorship high enough to span business and technology departments.
A reminder from the agile champion would be all that's needed to keep the product owner properly engaged, if the right agile sponsorship groundwork had been laid.
Summary of Empowerment issues
In summary, Comet's management needed to do these things to prevent the problems stemming from insufficient empowerment:
- Recognise the importance of facilitation skills and grow those skills
- Act on issues arising from retrospectives that are outside team's control
- Support the project manager when he/she gets inappropriate pressure to direct the team
- Support the need of product owners for time to perform duties of their role
Takeaway Lesson
Without the right empowerment, the goal of sustainable bug-free code cannot even be achieved. A properly empowered agile team is the means to generating sustainable bug-free code.
Problems With Pillar #3: Team's Work Stream
An agile team ‘pulls' work from the product backlog into a time-boxed iteration. After a couple iterations they know how much they can do in a given period; they have established a velocity. A common problem for managers new to agile is that they want the team to do more than the amount their velocity indicates. So they pressure the team to give artificially low estimates, or to work long hours, and so on. [7]
Velocity as an emergent property
An essential idea of a pull system is it recognizes that the team cannot control everything that has a bearing on their velocity. The team is part of a bigger, interconnected system. That system can produce almost bug-free code at a certain rate, full stop. That's the velocity. It might be stated as so many story points per week. If faster output is needed then the interconnections between the agile core team and the rest of the system need to be understood and improved to generate a higher velocity.
Velocity is an emergent property of the system. It cannot be directly commanded. If you get fifty story points this week by working twelve-hour days, when the usual was forty points, you are not working sustainably. You're simply robbing future production to get a peak now. Perhaps a tool to do data profiling would result in higher velocity. Tools are an example of an interconnection between the core team and the organizational system. So is training. If the team is given training to help them write better agile unit tests, that could improve velocity in a sustainable way. So could having a good team workspace. Each idea has to pass the sustainability test—if you could do it indefinitely then it's a valid way to influence velocity.
A very common mistake is to try to drive velocity directly, through pressure.
Diagnosis of Comet's Work Stream Problem
Comet team's velocity eventually started decreasing. When more coding is done on top of quick fixes, the code base quickly becomes brittle, and very hard to work with. Changes produce new bugs and they take time to fix, hence a slower speed. An increasing bug rate tempts people to do even more quick fixes leading to a downward spiral.
In frustration over the developing problems, Comet's managers reacted by pressuring the team to get more work done. Direct pressure cannot solve this–it intensifies the problems. The team responded by overly-optimistic "commitments", and by refusing to devote any time to helping each other with tasks. Undoubtedly, that further slowed their progress.
The team could not, in a short period of time, clean all bugs out of the legacy code. It was not within their control to do that; it was part of the landscape they had to cope with. They also could not quickly cover all the legacy code with agile unit tests. It had been agreed that this would be done incrementally along with creating new code. Deming said in his Fourteen Points for Management "the bulk of the causes of low quality and low productivity belong to the system and thus lie beyond the power of the work force", and that applies perfectly here.
In this instance, managers needed to recognise that the velocity was slowing because the group had decided to go with using quick fixes, and they were not taking the time to clean these out of the code. The question technical managers needed to ask is "why are we using an unsustainable technique to keep code production going at this rate?" Was the team simply bowing to inappropriate pressure to keep velocity up? That's the surest way into trouble. Or did the team really believe the quick fixes would be harmless? If so then they should have seen that wasn't the case and reversed course. Retrospectives provide a good way for managers to gain early insight into issues like these before they are compounded.
Lessons Learned for Keeping Team's Work Stream Healthy
Agile teams should receive their priorities through the product owner, and no one else. Sometimes standards groups, regulatory representatives, or others expect to direct team activities. They are stakeholders in the team's outputs and should work through the product owner. When a team has too many bosses they are set up to fail. Note that the product owner does not assign tasks to other team members. The product owner collaborates with them and is an information resource for them.
Pressure on a team to make optimistic estimates is often non-verbal, and it may come from many sources. Teams want to deliver, and when they err it is always on the optimistic side. It doesn't take very much pressure to move them onto shaky ground. Learning to push back in an appropriate way is one of the last skills agile teams acquire, so new agile teams are especially vulnerable to this mistake.
Wise managers will watch for times when a team commits to do much more work than they usually accomplish, and they will ensure that the team know they'll be supported if they need to push back on demands made of them. A commitment that cannot be fulfilled is of no help to the organization but can become a magnet for blame. Managers must understand that they have a responsibility to regulate the flow of work to agile teams. It is a simple, effective way to help teams keep their credibility.
The worst thing an agile team can do is erode trust with their customers. Each iteration should build up that trust.
Agile teams that form spontaneously "under the radar" are very often undermined because management has not acknowledged any obligation to regulate their workload. They are flooded with too much to do, no time for learning new tools or for maintaining a unit test infrastructure, and they can only hold the line so long.
Takeaway Lesson
If teams have mastered bug-resistant code and are empowered, they can still be destroyed by the failure of management to regulate the flow of work to them, i.e. allow them to pull the work in at their sustainable pace. Managers at every level have got to buy into the idea that they must never jam more work into a pipeline than its proven capacity. The good news is there are legitimate ways managers can increase that capacity.
Problems With Pillar #4: organization's Work Stream
One of the worst things that can happen to a good Agile team is to be assigned to a weakly supported project. That will mean blockages don't get removed. People and other resources get taken away when needed by other projects. The team repeatedly fail to deliver on their commitments and they lose the trust of their product owner.
Need Lean Project Portfolio Management
The previous three pillars of agile adoption are covered by the popular agile methodologies, but this issue of governing the work stream at the organizational level takes us over to Lean territory. Unless a lean discipline is used to decide what work an organization undertakes, it runs the risk of spreading its energy too thinly. Weakly supported projects will thrash and waste resources. A company that regularly completes 40 to 50 projects a year should not have 500 active projects in their portfolio!
Projects that have a direct business need are easiest to do as agile because there is a strong pull by a customer. Necessary infrastructure projects are harder to gain sponsorship for. There is less will to allocate a dedicated team, and pay for the tools they'll need. There is no easy answer here, but if the project is truly needed then it is the responsibility of the business to understand what it will accomplish and be a sponsor for it.
In the example of the Comet team, they were one of several agile teams. There isn't enough context to tell for sure whether theirs was a weakly supported project. Probably not, as their software was for use by the insurance underwriters to determine risks. This illustrates that it's not enough to just look at individual agile teams–you need to understand how groups of agile teams function within an organization.
Agile Habitat
One agile pilot project can be sustained by almost any company. It will place more or less demands on various other departments and infrastructure, but can be coped with. As the number of agile teams increases, pressure on certain resources (testing environments, product owner attention, team rooms, etc.) reaches a point where something has to give. Either the agile teams will be reined in and forced to make more and more compromises in their roles and practices, or else managers will recognize that they have to make the organization into a better habitat for agile teams.
A perfectly skilled and competent agile team may still fail. It's the same problem as when healthy animals are released into a habitat that is being decimated. They don't have a real chance to survive even though each individual knows how to survive. The highest priority projects should go to agile teams. Portfolio managers will have to regularly kill off those projects that cannot deliver or that are vying with Agile projects for resources.
Takeaway Lesson
If teams have mastered bug-free code, are empowered, and their work stream is properly controlled, they can still be destroyed by the failure of managers to match the organization's work flow to its sustainable capacity. One more time: Managers at every level have got to buy into the idea that they must never jam more work into a pipeline than its proven capacity.
Lean Thinking and Agile belong together for creating software-intensive products and services. They are actually different names for parts of one unified concept.
Conclusion
The Comet team got into quite a difficult situation even though they were in good shape when the external coaching ended. Once problems accumulate to this degree, it is very difficult to sort them out - too much so for an organization still getting used to agile. The key is to prevent these problems by understanding the control points in an agile system.
Too often it is easy to dismiss those control points when they involve so-called "soft" skills like good facilitation, or when people feel too much time pressure to clean up a "quick fix". Agile adoption must be led by executives and managers who thoroughly understand its dynamics. Otherwise, the normal frictions that arise will completely erode the agile adoption programme.
Agile is not just new software development techniques, or new project management techniques. It is that and more - it is an overall management philosophy for an organization. Agile points the way to applying Lean Thinking to software-intensive parts of product development, and of business process improvement. Comet's managers had adopted lean-agile practices but had not yet fully grasped its management philosophy.
The biggest gains of all come from management innovations. Management innovation is very hard to copy because it is almost invisible from the outside. Practices are easy to see and copy but copying them seldom produces sustainable benefits within an incompatible system of management. Lean businesses are radically different from traditional businesses in almost every respect. This is why it is so difficult to move an organization fully to lean-agile methods. It's also why big rewards still await those who get there first.
[1] The term "Agile" is used to mean any approach that addresses lean software development together with lean management. Scrum plus Extreme Programming does that. So do DSDM and Crystal, to mention a few.
[2] The term "stakeholders" is intended to include customers, or those who represent them, representatives of other groups within the company that are affected by the agile project, and groups whose cooperation is needed by the agile team.
[3] "Embedded Agile Project by the Numbers With Newbies", paper presented at Agile 2006 conference by N. Van Schooenderwoert. Available at http://www.leanagilepartners.com/publications.html. The author found many other agile teams who had similar successes but had not produced detailed metrics. They typically said that they don't track bugs because they're so rare - maybe one in a month. They sometimes could not recall when they last had a bug.
[4] There are many factors involved beyond just the number of tests. The quality of tests matters, as well as how easy or difficult it is to change them. The key point is that there is no natural, obvious stopping point when creating tests. They can be as fine or coarse grained as desired.
[5] Independent testing external to the team is a separate matter, and is not being ruled out by these statements. In a lean organization, each work group should routinely produce defect-free outputs. Ideally, final inspection or quality control should only find defects that could not have been prevented by the team. One of Deming's management principles is "Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place."
[6] Quotation: "Only after American carmakers had exhausted every other explanation for Toyota's success - an undervalued yen, a docile workforce, Japanese culture, superior automation - were they finally able to admit that Toyota's real advantage was its ability to harness the intellect of 'ordinary' employees." Source is "Management Innovation" by Gary Hamel, Harvard Business Review, February, 2006.
[7] There are techniques to achieve higher velocity, such as using multiple agile teams for the project, changing composition of the team, etc. These are beyond the scope of this article.
User Comments
Over the last few years, many organizations have adopted adaptive project management methods like Agile to increase the efficiency of their project management function. Among the different Agile frameworks, Scrum in particular has become extremely popular in most of the organizations. http://www.scrumstudy.com/
I think there is another phenomenon, which is that agile creates internal conflicts between team transparency and manager opacity. I wrote a little about it here http://senexrex.com/agile-managers, arguing that we need to develop agile managers (who employ even more talents than "good managers").