I’ve never been hired to convert a strongly process driven software shop into an Agile-based one. I usually bring Process into a shop where the de facto methods are very similar to what is described as Agile, but without the effectiveness that Agile development strives for. On the other hand, I never choke a software shop with Process either. I’m also well aware of how effective Agile-type development can be. Throughout this article, I will compare and contrast a process-based software production, “Process,” with Agile-based production, “Agile.” I am primarily concerned with the declaration in the Agile Manifesto that Agile favors “Individuals and Interactions over Processes and Tools.”
I’ve jotted down a few conditions that from my experience suggest movement toward a more planned, process-based approach and when you can expect success from a more free-form approach based more on individuals reacting. Before I get flamed, the following are generalizations - I myself could come up with anecdotal exceptions to most of these. We also do not need to confine ouselves to the context of software development as discussions of Process versus Agile can apply equally well to physical manufacturing including rockets, baseballs and apple pies.
Conditions Favoring More Processed Based Development
- Larger teams
- Large project scope
- Greater complexity
- Average motivation (lower cost labor)
- Geographically dispersed team (impedes communication)
- Strong auditing requirements
- Software defects are expensive
- Complex Testing Requirements
- Business need for predictability in cost and revenue
Conditions Allowing Effective Agile Development
- Smaller teams
- Smaller project scope
- Very effective communication exists between team members
- Highly motivated, creative (usually expensive) resources
- Weak auditing requirements
- Software defects have less impact, or fast turnaround of defects mitigates the impact
- Easier testing
A Faster, Cheaper Failure
One of the myths frequently propagated about Agile development is that it is the one true way to turn business requirements around in the shortest amount of time. While this may be seen to be true anecdotally in cases of small software companies and web sites, it is not the case in the typical large, mission-critical enterprise application. NASA’s Agile flirtation with “faster, cheaper, better” became a “faster, cheaper failure” with two lost Mars probes. The aerospace and defense industries have learned from decades of experience that very heavy process is required when the expense of defects is measured in the loss of human lives.
Striking A Balance
Fortunately for most of us, our defects are not that expensive and spending $50,000 to prevent a $5,000 defect does not make sense. A more mundane example that I’ve experienced first-hand, is that of a large financial software organization that had significant business pressure to improve time-to-market. They were able to reduce their sofware release cycle from quarterly to once every three weeks both by application of enterprise CM tools and processes and by having a highly talented development staff. The high level processes that goverened the interaction between the different teams allowed the developers to develop, the testers to test and the infrastructure team to do the production installs. The change management team coordinated all of the activity according to a well-defined, well documented process.
Within the development team however, was a very Agile-like environment. Having well defined roles and responsibilities and a far-reaching overall process restricted the scope of developer’s activities to primarily development activities. The developers were largely kept away from the day-to-day business of production support, managing the QA build and testing. This no doubt increased the individual Agile developer’s individual effectiveness and many were glad to shed the excess, often stress-increasing responsibilities. Process and Agile living together in harmony.
In my consulting engagements, my goal is to help that organization be continuously and maximally productive within the required constraints.
Deciding On The Right Mix
Considering that you will likely have a mix of Process and Agile, the two most important decisions to make are:
One of the myths frequently propagated about Agile development is that it is the one true way to turn business requirements around in the shortest amount of time. While this may be seen to be true anecdotally in cases of small software companies and web sites, it is not the case in the typical large, mission-critical enterprise application. NASA’s Agile flirtation with “faster, cheaper, better” became a “faster, cheaper failure” with two lost Mars probes. The aerospace and defense industries have learned from decades of experience that very heavy process is required when the expense of defects is measured in the loss of human lives.Fortunately for most of us, our defects are not that expensive and spending $50,000 to prevent a $5,000 defect does not make sense. A more mundane example that I’ve experienced first-hand, is that of a large financial software organization that had significant business pressure to improve time-to-market. They were able to reduce their sofware release cycle from quarterly to once every three weeks both by application of enterprise CM tools and processes and by having a highly talented development staff. The high level processes that goverened the interaction between the different teams allowed the developers to develop, the testers to test and the infrastructure team to do the production installs. The change management team coordinated all of the activity according to a well-defined, well documented process.Within the development team however, was a very Agile-like environment. Having well defined roles and responsibilities and a far-reaching overall process restricted the scope of developer’s activities to primarily development activities. The developers were largely kept away from the day-to-day business of production support, managing the QA build and testing. This no doubt increased the individual Agile developer’s individual effectiveness and many were glad to shed the excess, often stress-increasing responsibilities. Process and Agile living together in harmony.In my consulting engagements, my goal is to help that organization be continuously and maximally productive within the required constraints.Considering that you will likely have a mix of Process and Agile, the two most important decisions to make are:
- How finely grained should you make your process?
- What kind of people do you have?
The scale or scope of the process is important. We certainly would not normally define and document a process down to the level of every key that should be typed and every decision to be made on a minute-by-minute basis. So, how “tight” should we make the process-part of our environment? There are no easy answers, but you may find that you can only apply so many changes to the environment at any one time before overloading the participants and adding stress and therefore expense. Even if your organization demands a tight process, you may have to start lighter and tighten it over time. Consider how easy it will be for a new person to learn the process.
The biggest impact of all is the people who define the culture, as Bob Aiello pointed out in last month’s Crossroads News (Behaviorally Speaking: Content Management - The argument against process! , Crossroads News, March 2003). I’ve worked with people who become almost furious unless they have a perfectly defined process prepared for them to follow, but I myself would go crazy with hour-by-hour process and would definitely prefer a more Agile type environment. (As you can see, I’m not bashing Agile!) If you put both of us together and force us to interact in the wrong way, at least one of us is likely to be unhappy. There is the danger that with the wrong mix of people, you will never strike that balance between Process and Agile.
Because I tend to work with larger financial organizations, my consulting prescription is often more Process and a little bit of Agile. Finding the balance for each organization within the constraints imposed by the business is part of the challenge and art of consulting.
Sean Blanton is Director of Consulting for Catalyst Systems Corporation. Sean has been with Catalyst for 6 years as a distributed platforms change and configuration management consultant and a trainer and product contributor for Openmake. He has a Ph.D. in Physics from the University of Chicago and a B.S. from Columbia University.
