In the IT world, "paving cow paths" means automating a business process as is, without thinking too much about whether or not that process is effective or efficient. Often business process automation initiatives require figuring out entirely new ways of doing business processes–impossible prior to automation (for example, work flow automation and digital image processing)–defining more effective and efficient process highways. In this week's column, Jim Highsmith warns that when we pave the cow paths and ignore the highways, we do a disservice to our customers.
In looking at the focus of many Agile methodologies, we may forget important history lessons from the dreaded traditional development and blithely aid our clients and customers by paving more cow paths. While the Agile principle of early delivery of working software has created enormous benefits for many companies, there is an underlying assumption in many Agile methods that customers and users have done their homework; that they:
- Understand their business process,
- Have done the necessary business process analysis and rationalization, and
- Understand how automation might change their process.
The danger in this assumption is further increased by requirements definition approaches that begin at the micro level: individual stories, features, backlog items, or use cases. When a development team begins at an iteration and feature level, it may be laboring under the assumption that the user has the big picture. Many Agile methods are developer-user centric; they tend to leave out the critical business process analysis role. When developers who don't understand business processes interact with users who only understand the current business processes, and the "methodology" for requirements investigation doesn't look at business context, business process flow, and business information needs within a wider framework, trouble is just around the corner.
On the business side it generally appears that the Business Process Management (BPM) movement is somewhat divorced from IT, and virtually ignored by the Agile community. We write the BPM'ers off as the big process up-front crowd, and fail to see where we might help one another.
The BPM crowd complains about how long IT takes to implement projects. Large BPM projects have resulted in large failures. (Remember the rise and crash of the business process reengineering [BPR] movement in the early to mid 1990s?) What if the BPM analysts could help development teams better understand how business processes really work from the micro level as well as a contextual level? What if Agile development teams could show BPM teams how to greatly reduce their big up-front design (BUFD) mentality and implement new business processes incrementally rather than in a big bang. How many of the failed BPR projects of the '90s would have had much higher success rates using Agile methods?
The struggle over business process analysis (or product requirements definition or architecture work) is one of balancing anticipation and adaptation. In highlighting the dangers of big-up-front-design (BUFD) and waterfall development, Agilists seem to advocate no-up-front-design (NUFD) or no-up-front-requirements (NUFR) or no-up-front-architecture (NUFA). Most projects and teams need a balance between "big" and "no." This balancing act needs to take into account project complexity (size, distribution, etc.), uncertainty (risk, innovation need, etc.), and the cost of change at the project level and for each major component.
One of the inherent dangers of any form of iterative development is confusing iteration with oscillation. Good iterative development involves a successive convergence on a workable, adaptable system. Poor iterative development involves flailing around randomly searching for a solution-and mistaking this oscillation for iteration. Compounding the problem of iteration disguised as oscillation is the cost of change. Different components in any system have varying costs of change. So, for example, to change from an Oracle database management system (DBMS) to an SQL-Server DMS (one component of a software system) halfway through a project (a high-level evolution), and then to IBM's DB2 two months later, would be prohibitively expensive. Changing the data schema on a regular basis would not be as expensive. Incurring high-cost changes isn't evolutionary design-it's oscillation caused by poor planning and requirements specification on a high cost-of-change component-it tips the anticipation/adaptation balance too far towards adaptation.
Joshua Kerievsky's recent book, Refactoring to Patterns, addresses this issue in a somewhat different way. By discussing refactoring in the context of patterns, Kerievsky shows that just refactoring isn't enough. You have to refactor to a good design concept (a pattern). Examining the works of other Agilists indicates a similar pattern that supports both refactoring and evolutionary design (i.e. Scott Ambler's Agile Modeling and Martin Fowler's Patterns of Enterprise Architecture, Refactoring: Improving the Design of Existing Code). Ambler and Fowler propose developing a good conceptual design (anticipation) and then evolving it over the life of a project (adaptation). This is not the same as BUFD. It's conceptual design (fast, flexible, and informal) followed by evolutionary design (of which refactoring is a part), based on concrete feedback.
Getting back to business process analysis, I am in no way advocating a return to the huge up-front requirements definition disasters of prior years, but I do believe that many development efforts could benefit from better business process (or product) understanding by the development team. Some degree of business process understanding, rationalization, and automation potential needs to be anticipated in early planning so that the later details, like features and use cases, can be put into context. Furthermore, business process design should be evolutionary, just like the software design: develop an initial business process framework or skeleton, implement both the software and the improved business processes incrementally, and then adapt both the process and the software.
The prime Agile mantra is to deliver customer value. Paving the cow paths doesn't usually deliver the highest value, so maybe a little attention anticipating business process design can raise the value we deliver.
