The benefits of agile software methods, including faster time to market, better responsiveness to changing customer requirements and higher application quality are undeniable to those who have mastered these practices. Agile practices, however, have been defined and recommended primarily to small team environments where co-location, ready access to interactive customers and small team size are the defining rule. Are the benefits of agility to be denied to those larger software enterprises who don't share these simple paradigms? Or can the industry learn from these practices and apply some of the core principles to large scale development of applications that require 100, 200 or even 1,000 distributed team members to achieve?
In the last few years, many of us have been engaged daily in bringing higher degrees of agility to these larger scale enterprises and these companies have demonstrated that they can indeed benefit from application of agile principles. In Part I of this article, we will describe seven best practices of agility that natively scale to the enterprise level. In Part II, we will describe an additional set of seven organizational best practices that a company can master to achieve even more of the benefits of software agility on an enterprise scale.
Over the last decade, a variety of agile methods have emerged. Ranging from Extreme Programming through the methods of Scrum, DSDM, Feature Driven Development, Lean Software Development, and Test-Driven Development to the iterative and incremental methods as espoused by the Rational Unified Process and its agile variants, the basic principles of software agility have now been effectively applied in thousands of projects. As we have experienced these various methods in practice we have come to understand that, while the methods themselves vary, the core practices themselves have more in common than they are different. Moreover, we have also seen that, once mastered, many of these core practices scale natively and can be applied directly to the enterprise level where larger and more distributed teams and some degree of outsourcing is the norm. This should provide some comfort to those CIOs, VPs of Development and other agents of organizational change who look to adopt these methods to improve the software productivity of their larger enterprise.
Seven Agile Team Practices that Scale
Specifically, we observed seven principles common to all agile methods that can be readily applied throughout the enterprise.
1. Iteration foundation. If there is one single practice that characterizes the agile methods and differentiates them from the sequential waterfall processes that have been applied in the past, it is the ability of the team to create tested working code in a small time box. While not easy to master, teams that learn this skill fundamentally change the nature of their software practices. There is no going back; once mastered this basic paradigm could be applied at the small team feature or component level, and with some work, at the system-level as well. In so doing, the basic building block of agile is achieved and the foundation of agility serves the business and development needs that follows.
2. The definebuildtest component team. In order to build working code in a short time box, teams must reorganize to contain the three basic skills necessary to deliver software: a) product owners, who serve as customer proxies and define the solution, b) team members who can create the code with a minimum of distractions and an absolute minimum of multiplexing other projects, and c) integral test, test automation and QA functions. This definebuildtest component team, or the fractal software unit, if you will, is the cornerstone organizational unit of agility. While eliminating the functional barriers that may have prevented teams from organizing this way in the past is not a trivial undertaking, the process does scale in that there is no limit to the number of such teams that can be formed!
3. Smaller and more frequent releases. A further principle of agility requires continuous feedback from the customer or the marketplace as to the fitness of the application for its intended purpose. In many IT shops this can be accomplished by pushing the software to the users at the end of each iteration. For independent software vendors however, releasing software to the marketplace may be a significant undertaking that involves resources from other teams and other departments. Achieving ready feedback pushes all these teams to smaller and more frequent releases. At enterprise scale, release cycles typically move from as long as 12 to 18 months to as short as three to four months. Again, while not a trivial undertaking, once mastered, the teams become addicted to the faster cycle of market feedback and often push for faster and faster releases to help them evolve their products more quickly.
4. Two-level planning. While agile eschews detailed up-front, long-range planning, a subtle fact is that agile is a very disciplined and (yes, even) planned process that simply breaks the planning cycle into shorter chunks (in a manner similar to building software in smaller iterations). Two types of planning occur. At the release level, which is the preparation for deployment and/or shipment to end users, application or product roadmaps define a series of releases that are sketched out using "broad brush strokes": themes and high-level features, based on a prioritized backlog (a list of things the application needs to do.) At the iteration level, which is a shorter, time-boxed increment of new functionality, more detailed planning occurs. Iteration planning breaks the objectives of the iteration into smaller tasks typically of no more than a day or two. This improves estimating and helps the team more readily achieve their commitment to the iteration. Together, these two types of planning provide the long and short view necessary for the enterprise to communicate its objectives to both the project team and to its executive stakeholders and customers.
5. Concurrent testing. In agile, all code is tested code. There is no other kind. No longer do the developers create "mounds of theoretical working code" as testing is integral to the development process. Unit testing, acceptance testing, and performance and reliability level testing are driven to occur inside each iteration. In some agile models, such as XP and Test Driven Development, the teams are encouraged write the test first in which case the test also serves as a ready proxy for the system requirements, but a proxy that persists in the future as regression tests to continually assess readiness of the code as the solution evolves in future iterations.
6. Continuous integration. Another roadblock on the way to the agile team's performance is the necessity of continuous integration. Like many of the practices we described above, continuous integration is not new and has been applied by many teams for some time. At scale, however, continuous integration has created challenges as it can be difficult to assemble changes from a large number of components daily, create an adequate build environment, and to automate the tooling necessary to integrate, build and perform "smoke tests" frequently. In agile, this roadblock cannot stand. For example, I witnessed one team of approximately 300 people building a large-scale systems infrastructure application reduce the integrate-build-regression test cycle time from a month to less than a day. Initially, these teams were intimidated by the objective but as their agility increased, they recognize this bottleneck to productivity and correspondingly increased their resources and focus on this key activity. They are software professionals after all and this is a software problem.
7. Regular reflection and adaptation. A key agile manifesto principle (http://www.agilealliance.com/) is "At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly." In agile, this key practice is truly "the gift that keeps on giving" as the empowered, agile teams naturally address and eliminate the roadblocks and impediments that prevent them from continuously improving their productivity. Since this process is not prescriptive, it can give some discomfort to project management organizations, process organizations executives and the like, who tend towards documented, prescriptive and mandated processes. And yet, the reality is that these empowered teams will indeed improve their practices so long as management encourages this basic behavior and does not stand in the way.
Taken together these seven agile team practices that scale can be applied at most any enterprise-level. As we've said, we do not minimize the undertaking or suggest that there easy to adopt and apply; but rather that any enterprise that seriously addresses agility will be able to accomplish these practices to some degree and their localized component and feature teams will operate with higher degrees of productivity and quality.
About the Author
Dean Leffingwell is a software industry executive, entrepreneur, and part time methodologist/author who has spent his career trying to help software teams achieve their goals. He is the former founder and CEO of Requisite, Inc. makers of RequisitePro, and a former SVP at Rational Software. He now serves as a consulting methodologist to Rally Software and as advisor to a number of larger software enterprises. He is the lead author of the text: "Managing Software Requirements: A use Case Approach", Addison Wesley, 2003. This article is excerpted from his next book entitled; "Scaling Software Agility: Best Practices for the Enterprise", scheduled for publication in fall of 2006 from Addison Wesley.
