Even before the first "agile" methods were formalized, industry reports such as The Chaos Chronicles confirmed collaboration and reuse as significant contributors to IT project success.[1] Agile methods conspicuously bring their importance into focus.
On an Extreme Programming (XP) team, for instance, an absence of collaboration could manifest as non-implementation of pair programming, a core XP practice. Similarly, any self-proclaimed agile team lacking an actively engaged customer-user would likely raise more than a few eyebrows. Simply put, ignorance of collaboration and reuse opportunities, whether deliberate or unintentional, directly undermines a team's ability to deliver a continuous flow of value to the customer.
But how are collaboration and reuse encouraged across project boundaries in large, globally distributed organizations employing agile methods? Further, how can an organization maintain and enhance its agility as it grows?
Institutionalizing agile practices across an expanding enterprise poses a change management challenge. This is true even when individuals are philosophically aligned to the values and principles which underpin the process. Organization-wide continuous improvement programs that notably include an adoption and compliance function can play a vital role in driving the levels of collaboration and reuse necessary to achieving scale with agility.
Audits can directly contribute to collaboration and knowledge reuse amongst agile teams in three key ways:
- Actively introduce processes and best practices at the time of project inception to avoid projects "reinventing the wheel."
- Collect new ideas for processes/tools and best practices for existing processes/tools that can be incorporated into an organizational knowledge base.
- Disseminate new best practices across many project teams within an organization.
Establishing Defined Agile Processes
Agile methods do not lack discipline. Some practices, such as Test-Driven Development (TDD) with its implication of 100% unit test coverage, are among the most rigorous software engineering techniques used today. The fact that agile methods carry with them clear behavioral expectations means that they can be codified and shared amongst practitioners. Articulating the intent of an agile practice is in itself a form of process reuse that leads to efficiencies. This is clearly seen if one considers the kind of confusion that would ensue during an iteration planning session if every member of an XP team held fast to his own unique personal view of what constituted a good user story. Typically, this kind of conceptual and behavioral alignment is achieved on a local scale through the use of agile coaches. However, relying only upon agile coaches for the purpose of "spreading the gospel" imposes significant limitations in hyper-distributed organizations wherein every project team is {sidebar id=1} potentially housed in multiple, different locations.
At the start of every project at Sapient, we create a process adoption plan consisting of a set of organization-standard policies that capture the essence of Sapient|Approach, our project delivery methodology (see Figure 1). Working with project leaders, we tailor the policies to accommodate the project's unique circumstances. These conversations challenge project teams to think critically about what processes actually make sense for them and to justify their rationale. This activity results in a defined process that is owned by the project team - not by ivory-tower process junkies removed from the on-the-ground realities of the project - and drives a number of key outcomes:
- Organizational knowledge is cultivated around what constitutes the canonical set of agile best practices for the vast majority of IT projects undertaken. Exception cases where certain practices are unfeasible or fail or add commensurate value are also made visible.
- Project and organization leaders learn where peoples' understanding of best practice is lacking before the project gets substantially underway. When left to their own devices, projects run the risk of establishing processes that are either inadequate (thus leading to project-level sub-optimization) or are significantly divergent from organizational norms making it difficult for skills and experience obtained on one project to be effectively transferred to another (thus leading to organization-level sub-optimization). By identifying these gaps early, response plans utilizing mentorship, formal training or even process automation can be initiated while such actions are still likely to meaningfully influence the project's ultimate success.
- Teams buy into implementing defined processes, providing a set of baseline expectations against which performance can be gauged. Periodic assessments provide opportunities to identify improvements and extract best practices that can be leveraged elsewhere within the organization. Importantly, such assessments may be used to reward teams and reinforce good behaviors, but never for disciplinary purposes.
- Accountability assumed by project leaders for implementing defined processes goes hand-in-hand with a firm expectation that they will transfer best practices to their team members (primarily through just-in-time coaching). Thus, current leaders act as change catalysts by growing the next generation of agile project leaders within the organization.
Figure 1: A process adoption plan shown in the Sapient|Approach adoption management tool
Agile Audits?
Agile teams thrive upon a deep-rooted commitment to doing the right thing for the customer. Delivering on this promise requires that teams embrace changing requirements, stakeholders, and project constraints, and commonly such changes warrant modifying process accordingly. Merely defining process is therefore insufficient for the purpose of ensuring ongoing implementation of best practice. Similarly, project teams should not change process merely for the sake of doing so. Thus, periodic audits are appropriate for ensuring that teams hold themselves accountable to established process adoption commitments, deviate from those commitments only as necessary, and leverage innovations.
The mere mention of the word "audit" is often enough to send shivers down the spines of developers and project managers alike. More often than not, audits conjure up images of armies of grim-faced, clipboard-wielding consultant-drones that are in the best of times a slight distraction and at the worst of times a significant drain on the team's energy and productivity. The team's relationship with auditors is generally cool if not downright adversarial in the most extreme of cases. However, audits can prove themselves of value to both a project team as well as the organization-at-large if approached with a cooperative spirit. Re-casting audits as a program of retrospection, current-practice assessment and coaching helps to remove some of the stigma associated with this term.
A thread of recent posts to the Agile Management email group has significantly progressed this idea.[2] One contributor suggested that project managers regularly observe one another's projects and debrief on what was seen. Another described a scenario wherein a team claimed that it had "tried" pair programming and it had "failed" when in fact it had two people sitting in front of a computer at the same time but they were not following the "protocol" associated with this practice (e.g. writing unit tests first, switching roles within pairs, rotating pairs, asking pertinent questions about the expressiveness of the code). Both posts point to the value in obtaining an objective "second opinion" and suggest that doing so is compatible with the values enshrined in the Agile Manifesto and does not suggest a shift away from individuals and interactions in overwhelming favor of processes and tools.[3]
Closing The Feedback Loop
Team retrospectives are a common practice amongst agile teams. They are used to encourage collaboration and reuse by surfacing best practices and improvement opportunities within a project context. However, in practice, retrospectives generally do not facilitate the transfer of best practices across project teams or product groups. This is where audits (one component of a larger organizational governance program whose discussion is beyond the scope of this article) can play a meaningful role not only in helping to spread best practices beyond the context of a single project team, but by acting as a catalyst for organization-wide agile transformation.
As a memorable illustration, a number of Sapient project teams developing data-intensive web applications felt that implementing fully automated unit tests for their front-end code would be too onerous a task to justify the effort required. Our adoption and compliance program afforded us visibility into this lack of rigorous TDD. It also brought to our attention another team's innovative solution integrating StrutsTestCase with DDSteps and the project's continuous integration build. Thus, we were able to not only help these teams connect and prevent a significant process deficit from going unchecked, but we were also able to create a generalized solution to a common project need.
In Sapient Approach, iterations can vary between one and four weeks depending on a number of factors. As such, we have adopted a monthly project audit schedule, which roughly corresponds with the upper limit on iteration length. Given that most of our projects run at least six months in duration, this approach offers us ample opportunities to observe projects and inject positive change without subjecting projects to undue process volatility.
Summary Of Best Practices:
- Establish process adoption plans at the time of project inception or as soon as possible thereafter.
- Provide flexibility for project teams to deviate from accepted best practice when there exists a valid business justification for doing so.
- Ensure that the whole team feels that it owns its defined process.
- Use project leaders as catalysts for process adoption.
- Audit against intent and adequacy of implementation - not the literal step-by-step process.
- Conduct audits early and often to ensure a continuous flow of best practices to projects from the organization and vice-versa.
It is worth taking a moment to challenge the necessity of adoption and compliance programs in promoting and sustaining collaboration in agile organizations. Could there be conditions under which this need is obviated? In all likelihood, there exists a "tipping point" beyond which the number agile-aware practitioners engaged in project work reaches critical mass and the need for a formal mechanisms becomes moot. However, given that the number of experienced agile coaches today is relatively small in comparison to the industry as a whole, any large, multi-national organization will likely find this tipping point to be intolerably distant in the face of significant double-digit percentage growth.
[1] The Standish Group, The Chaos Chronicles v3.0, 2004.
[2] Agile Management email group, http://groups.yahoo.com/group/agilemanagement/
[3] Agile Manifesto, http://www.agilemanifesto.org/principles.html
About the Author
Erik M. Gottesman is a Senior Manager, Technology at Sapient, a global services firm that helps clients innovate their businesses in the areas of marketing, business operations, and technology. As a member of the Sapient|Approach team, Erik is responsible for Sapient's research and point of view on software methodologies, metrics, effective team structures, efficiency initiatives and overall process-improvement programs. While at Sapient, Erik has also delivered solutions for Hilton International, Nissan and major financial services firms. Apart from his publishing credits relating to software development, Erik's theoretical writings on artificial intelligence and music have appeared in the proceedings of various international symposia. Erik holds a Bachelor of Science in Electrical Engineering and a Bachelor of Fine Arts in Performing Arts Technology, both from the University of Michigan.
