Agile Shu-Ha-Ri for Business Innovation

It’s common for organizations to put agile adoption initiatives under the dominating management paradigm from the old industrial age. Without realizing that agile represents an effort toward business innovation, leadership may consider agile to be yet another process standardization run by the program management office (PMO). But what organizations are finding is that agile is not just another process—it requires changes to our mindsets that some PMOs may not be ready for.

Alistair Cockburn's book Agile Software Development: The Cooperative Game introduced a learning pattern, Shu-Ha-Ri—originally from the Japanese martial art aikido—into the agile community. Martin Fowler interpreted the three levels as imitate, assimilate, and innovate.

What we have seen, however, is that it is easy to oversimplify Shu-Ha-Ri, which can slow or halt your agile adoption.

As an example, an American retailer asked for help from consultants to start their agile adoption. The PMO of the organization insisted that the focus of their adoption program be on the classroom training of organizational staff on agile. The PMO believed the online self-training system would adequately achieve levels Shu and Ha for their agile adoption. Instead, little to no behavioral change occurred, and this approach created a mockery of the agile adoption effort.

Ron Fox pointed out that in a martial arts class, Shu-Ha-Ri “in classical interpretation is a linear sequence which leads the student with minimal deviations down a path of learning. The student progresses from imitation [Shu], to reasoning [Ha], to creating [Ri].” However, even in martial arts, Fox warns that we need a more cyclical approach rather than linear. This applies even more crucially to agile transformations, as attempting to adopt agile in a linear manner will not work.

Worldviews and Intentions

Each of us has a worldview, built up over the years as a sum of our experiences and our perceptions. We also have intentions shaped by that worldview, which we may or may not show to our colleagues. Those worldviews and intentions are as important to a switch to agile as are the new terminology (speech) and practices (actions).

Conventional process standardization, like what a PMO typically does, is based on the principles from the early twentieth century of Frederick Taylor, who shaped much of the industrial age with his perspective on efficiency and measurement. With Taylor’s worldview, we assume that we know everything about the product we are manufacturing, from its design to its production processes. Therefore, standardization is not only technically and economically possible, but necessary to ensure simple replication of the mass production.

Cockburn explained the fundamental reasons an agile approach is needed for software development. There were two common issues with modern software development that traditional practices don’t address well: It is impossible to have complete experience in any particular discipline within software development, and it is impossible to have perfect communication on any subjects of software development. Cockburn named these issues as unknowable and uncommunicable.

While standardization was the answer to the knowable and communicable industrial product from Taylor’s era, the agile approach is the answer to the unknowable and uncommunicable challenges in modern software development. Both the unknowable and uncommunicable will require that Shu-Ha-Ri be a cyclic and dynamic process, not something done linearly.

In the retail example mentioned above, most of the managers and their teams didn’t have the opportunity to learn the new worldviews and intentions behind the words they used to explain their agile approach—both their speech and their resulting actions. Instead, they only learned the quick copies of the documented speech and actions from the online courses they took.

By taking this approach, the organization boxed their employees into the “Shu” stage of learning, without allowing them to adapt the processes to their own environment and (necessarily) shifting worldviews and intentions (Ri). Adopting agile isn’t about “doing agile” or “being agile;” it is about improving business results, and this cannot occur unless the changes go more deeply into worldviews and intentions.

When companies stop their learning process of agile at the levels of Shu or Ha, without Ri, they may only see limited benefit to their agile practice. However, the real limiting factor here is not the traditional exercise of process documentation, standardization, or best practices per se. It is the same worldview and intention inherited from the old industrial age that leads to the same mentality of process standardization in action.

Learn Ri from the Beginning

Most companies set up their initiatives for agile adoption because they’d like to gain the full benefit of agile practice. They may or may not realize the initiatives of agile adoption are presenting a new business innovation from the very beginning, which is more challenging than they expect.

In a linear view of Shu-Ha-Ri, that could create a dilemma. Before the companies entered into their learning cycle of Shu-Ha-Ri, they would have to do a Ri before they learned it, so that they could branch out from the existing ways of doing business to start the new journey to learn agile. Of course, this would be impossible if learning Shu-Ha-Ri were just mechanical steps without innovation.

This is actually a common issue in agile practice. With agile continuous improvement, Shu-Ha-Ri will always be cyclic and iterative—that is, at each and every moment of Ri, the agile team is expected to innovate so that the team can move away from previously structured practice to deliver better solutions with better business values.

Such an agile Ri can be hard. Each organization and each person carry certain worldviews and intentions. When we design, build, and manage our systems, our speech and actions will embody our views and intentions, all the way into our systems. 

Melvin Conway summarized such relations between design and communication in his Conway’s law:

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.

This thesis was taken from his paper “How Do Committees Invent?” In the conclusion, Conway points out:

Primarily, we have found a criterion for the structuring of design organizations: a design effort should be organized according to the need for communication.

This criterion creates problems because the need to communicate at any time depends on the system concept in effect at that time. Because the design which occurs first is almost never the best possible, the prevailing system concept may need to change. Therefore, flexibility of organization is important to effective design.

Both Conway’s law and Cockburn’s unknowable and incommunicable criteria show us why it can be hard to identify the limiting factors for us to be innovative outside our own experience box. Many existing structures in organizations or in our own experience, such as standard documentation processes, decision hierarchies, and disciplines of professional specializations, could easily become the constraints in our speeches, actions, and decisions. They will typically stop our Shu-Ha-Ri from being innovative.

Behind the slow progress at the retailer, it was the dominant worldview that still considered software development and agile adoption to be the same knowable and communicable product from Taylorism. The retailer failed to acknowledge that the new digital technologies have encountered much more unknowable and uncommunicable realities from a highly competitive global market. In terms of Conway’s conclusions, the existing organizational structure was not flexible enough to allow a multifunctional team, a new structure, to explore the new approach to more effective design.

The worldview and intention of agile fully acknowledge the existence of the unknowable and incommunicable. It is neither necessary nor possible to build complete knowledge and perfect communication.

What’s essential is to develop speech and actions to address the issues effectively and efficiently, as described in the agile Shu-Ha-Ri, that would start from not “the best possible” design and evolve it to be better and better through iterations.

The consultants suggested to the retail organization to start with a Ri to the existing organization structure: a newly created multi-functional team where the experience and communication would not have to rely so much on the handovers between existing silos, but on direct human-to-human interactions. It was felt that this was a necessary condition to start a cyclic and dynamic agile Shu-Ha-Ri for the entire organization.

With an agile Shu-Ha-Ri cycle of learning, the Ri must be experienced, learned, and created first. The Ri is the key factor to build the ability to innovate and create without constraints from previously learned structures of any kind, agile or not. Existing organizational structures are included as well.

This way, there is a world of opportunity for the agile practitioners and students to take clear, decisive action toward real, valuable transformative change worthy of the effort.