One of the principles in the Agile Manifesto states, "At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly." This principle guides both aspiring and seasoned Agile teams in the pursuit of continuous improvement and can support whatever Agile adoption path an organization may choose. Kent Beck adds observations about this topic.
"'Continuous improvement is a bit of a misnomer. It means continuous awareness, responsiveness to feedback, and openness to improvement. When you know how to improve, then you improve. You make a change; observe the effects; then digest the change, turning it into a solid habit. Eventually you hit a plateau from which you can absorb more feedback and identify the next opportunity."
Extreme Programming Explained: Embrace Change (Beck amp; Andres 2005, p141) - Addison-Wesley Boston MA
This quote describes a process that we recognize as the ability to identify difference and learn through attentive repetition, either for individuals or groups. Through practice, we can get a feel for the contextual application of knowledge, rather than the acquisition of individual techniques." Practice is the only ingredient that the research group found to be connected to every other ingredient. Beck refers to continuous improvement in the context of an individual's learning process. "Change myself, then offer the fruits of that change to others." Practice is a process through which groups can also learn and change.
Although individuals contribute to the features developed by a project team, the product of their work (the features themselves) does not belong to an individual, but is the product of the entire group. Just as the music produced by a jazz ensemble includes the individual improvisations of each member, the music belongs to the group. To whom does the technical debt and code quality in software systems belong? Why do some environments not take advantage of the opportunity for iteration provided by Agile methods to turn releases into non-events?
We propose that items such as code quality, technical debt and release cycles have tacit ownership at the group level. This is unconscious, as group ownership is often minimally recognized in organizations. These unacknowledged group items are vulnerable to decisions and prioritization made by individuals without the group's voice, at the expense of the opportunity for the group to learn. Untapped group learning can be brought to awareness through group Practice thus supporting the pursuit of continuous improvement in these often challenging areas.
Challenges - Release Process
Large organizations are prone to centralize their release process in an attempt to protect their live systems from rogue updates, or their customers from a greater pace of change than they can absorb. A common implementation of this protection is to use an additional team for production releases independent of the development team. In reality the development team is invariably called in to resolve issues that surface during the release process. This illustrates tacit ownership of the release process that belongs to both release and project teams. The whole-group ownership of the release is not acknowledged. The resulting integration feedback loop is late, long and cumbersome and releases are big events involving multiple departments and stressful interactions with long hours.
In environments where a centralized release process is assumed to be the constraint and not up for discussion, Agile is stunted in its ability to provide continuous Practice in the software delivery. A common reason to avoid the process conversation is the excuse that "our complex business needs such a process." However, by not allowing Agile teams to release to production early and often, we prevent the Practice that would allow them to turn the release process into a non-event.
Challenges - Technical Debt
The centralized release process implementation can result in development and test environments with limited ability to test performance, integration, code coverage or product usability. This forces a lot of critical testing to be done during the release rather than the development process. The project team's best efforts to deliver a "safe" and tested feature set become little more than guesses. The developers have to attempt all their testing in the System Test environment. This is like developing blind as this environment lacks the connections to the system's external integrations.
The project team's group product consists of all the features and all the tests. Centralizing the release process fragments group ownership. Developers focus on unit and system tests and don't have access to accomplish the full set of critical tests. The release team can focus on the tests the developers can't execute but release engineers do not have access to the code. The result surfaces technical debt that neither team is empowered to own. When responsibilities for the group product are fragmented, no whole group exists to own that product. Practice by the whole ownership group could improve both the release process and catch technical debt earlier.
Challenges - Code Quality
Customers and product owners may press the project team for a wide feature set and an aggressive delivery date. This can compromise code quality, causing the developers to express concern. The two groups might have a different interpretation of the Agile principle for "minimizing the work not done". Customers might think that refactoring is unnecessary waste since this effort results in no visible changes to the application. They often challenge the value of what they might describe as "writing the code twice". However, once a feature is implemented, an Agile development team refactors the code to make it more efficient to run, cheaper to maintain and easier to extend. So who does the code quality belong to? The customers are paying for the work but the developers have a fiduciary duty to ensure that the customer receives maximized value. The code quality is part of the whole-group product. This disagreement shows that in this case the true communication never happened. The need is to resolve the disagreement and create collaboration.
Code quality is built into the software over time. If the opportunity for collaboration is limited to the final stages of release preparation, the ability to identify deficiencies in code quality is also limited. An occasional look at the system does not provide the opportunity to develop trust, align goals and expectations, and learn to communicate effectively as a team.
Practice to Enable Whole-Group Ownership
We would like to share our personal experience of one way in which we have turned the tacit group ownership of release process, technical debt and code quality into a conscious seamless part of the day-to-day interactions of a whole project team. In a financial risk management system like the one in this example, projects waited until a complete financial model was implemented in C. Once the traders validated this model, the developers could begin to extend their risk management system to use it. After this software development was complete, operations and support processes could be created. These sequential handoffs were an established project life cycle.
When market conditions suddenly invalidated an existing financial model, a business need arose to extend the risk management system considerably without knowing what replacement model to use. Using an invalid model put traders at great financial risk and under considerable stress to manually adjust every quoted price. This was a crisis that could not wait for a sequence of handoffs over the next 12-18 months.
Instead, by initiating communication with all the stakeholder groups simultaneously, it became clear that each had a part of the process and also a part of the solution. To make the communication among the stakeholders both practical and concrete, developers created sandbox versions of production for each conversation among traders, financial modelers, trading administrators and support engineers. The increased level and quality of communication led to the realization that the whole group owned the system.
Implementing the sandboxes removed the handoffs and enabled the participation of the whole team throughout the project. The whole group was able to take ownership of the technical debt, code quality and release process, since each of these sandboxes made these items visible to everyone involved. Most sandboxes were set to refresh their database daily from the live system, and would be extended with all the daily code changes from the development system as well. In addition, the financial modelers were evolving multiple analytic libraries, and the latest versions of their work were also applied to the sandboxes on a daily basis.
Traders and their administrative staff picked up any drop in code quality on the part of either the developers or the financial modelers immediately. Any technical debt that impacted the usability of the system or the daily generation of risk or profit and loss figures was also picked up immediately. Degradation in the computational performance of the application was obvious to everyone straight away. And of course, the release process was being tested on each sandbox every day.
An Environment for Group Learning
The previous example illustrates how software development can be a group learning process. This solution allowed for both Practice by the entire group and also for continuous improvement in several areas. For example, the financial modelers were able to share each other's work and thus were able to explore four or five times the normal number of candidate models to develop a solution, with each cycle of modeling providing input for continuous improvement. The traders were able to test numerous hypotheses about the market parameters that would influence the new models, thereby contributing improvement to their business model and trading strategy.
Given the crisis, the stakeholders were willing to do away with the traditional handoffs, include all the participants and create a shared sense of common purpose. This holistic project interaction initiated group Practice in which self-organization and group learning spontaneously emerged. When somebody suggested something that the group could do, the group provided a way for them to try it, improving the group's environment with each cycle. Eventually, the group was able to create its ideal environment for continuous improvement. This environment was the result of acknowledging the previously tacit group ownership of the risk management system. The Practice allowed a new self-organization to emerge.
Conclusion
When everyone is able to see and experience the progress made throughout the development process, they will offer and receive feedback without being solicited or prepared. This open group exchange is one result of Practice. To guide a collection of Agile practitioners and their project stakeholders towards Group Coherence, it is important to provide opportunities for the group to Practice ‘being a group'.
Without ‘being a group' the team has no access to the tacit knowledge. Once it accesses its shared understanding, the team develops a knack for group learning. Group learning is necessary for a team to embrace suggested improvements. Continuous improvement depends on group learning provided through Practice.
Guiding a group from practice to continuous improvement can allow practical results in such challenging areas as code quality and technical debt and can make releases so frequent, tested and drama-free that they become invisible.
About the Authors
This article was written collaboratively by Joanna Zweig and César Idrovo.
Joanna Zweig holds a Ph.D. in Integral Studies with an emphasis on Learning and Change in Human Systems (how groups learn and Change) from California Institute of Integral Studies in San Francisco, California and a Project management Institute (PMI) Project Management Professional (PMP) certification. Her research on Group Coherence was motivated by her practical experience in two collaborative professional fields where groups exceeded expectations and experienced enormous energy and success in their goals.
She is a project manager in information technology in large businesses for more than 15 years and a producer and director of theatrical productions for more than 20 years. Her passion for collaboration in creative groups helped her to formulate the idea of group coherence and carry out her four-year research project to find out about it. Her research on group coherence revealed a way to learn about capabilities of collective consciousness. She is currently an independent consultant and CEO of Integral System Response, Inc. http://www.groupcoherence.com
César Idrovo created his first hyper-productive team at JPMorgan London during 2000, in response to strong demand for his own work. He recruited a highly heterogeneous group and implemented "continuous collaboration" to achieve high team cohesion and tangible results. In several instances, his team's tactical solutions were adopted as strategic implementations and are still in use today.
From 2003 he has focused on formal Agile methodologies and adaptability to high rates of change in requirements, creating further hyper-productive teams. As a project and program manager, he has applied Scrum for managing, tracking, and delivering working software in time-boxed iterations and introduced Extreme Programming including 100% pair programming. He has worked on practical complementary techniques to Agile practices for management and executive levels such as Project Patterns and Adaptive Roadmap.
He also blogs at Nonlinear Enterprise where he is starting to exploring creative ways for large companies to deal with nonlinear behavior - and at times potentially explosive behavior - without losing the required adaptability to rapidly changing market conditions.
In 2008 he has given over 50% of his time to nonprofit initiatives, focusing mainly on BayAPLN.org, chartered for community research and education. It creates, fosters and supports a learning community of Agile leaders in the Bay Area. He contributes as a Community Builder, Organizer and Connector.
He holds a Masters in Engineering from Imperial College London and a Masters in Finance from London Business School.
