The Goldilocks Parable: How Much Process Is Just Right

What are the guidelines?
The Capability Maturity Model-Integrated (CMMI ®) provides a useful—but hard to quantify—statement about process definition. The Organization Process Definition Process Area states, "These elements are described in sufficient detail that the process, when fully defined, can be consistently performed by appropriately trained and skilled people." (Emphasis mine.) In this statement, "elements" refer to process elements. This definition has utility whether you are using the CMMI for guidance or not.

The trick is figuring out what "sufficient," "consistently," and "appropriately" mean. I have seen the best and worst of process definition, unfortunately erring on the "too much" side of "sufficient" often enough that people take the view that process improvement can be categorized into the "Big P" and "little p" camps, and that "Big P" is always bad. This article will help you to quantify the above three terms, hopefully helping to define useful improvement processes for your organization no matter what dimension is chosen.

What is a process?
Simply defined, a process is "a sequence of steps to achieve a given purpose." They are useful when they prevent us from re-inventing the wheel when starting a project. They are especially useful when many people in different parts of an organization do the same tasks. It reduces friction (wasted energy), and frees up time and brain-power for solving the tough problems rather than having to invent a presentation or documentation method for the task.

Who are the "customers" of process?
The process users are the customers of the group that develop and document processes. They can be divided into three groups: expert, average, and novice. 

• The expert user, one with many years of experience, needs brief process descriptions, templates, and checklists. Templates and checklists are valuable for preventing omission and enable consistency across multiple process executions.
• The average user may want more guidance; a process step may still be new, or not frequently used. Access to guidelines or training material is valuable.
• The novice user will want step-by-step lists of activities, thinking these will ensure correct execution of the task.

Compounding the problem of over-documenting processes is that process writers may have little background in technical writing. A novice's description contains every step in the process, including "Open template, Save as..." One has to ask, if they need to be told how to open and save files, why are they even attempting the task?Resolving the Expert-Novice Gap
A multi-level approach to process documentation can help bridge the expert-novice gap. I divide the documentation into the following parts: 

1. The Process—a three to four page document listing that includes:
   • Input: things needed to start the task
   • Entry Criteria: training, events, or input attributes that must be satisfied to start
   • The Process Steps: concise description of the activity performed
   • Outputs: what is delivered
   • Exit Criteria: output attributes that must be satisfied to end the task
   • Measurements: data generated by the process that can be used to monitor and improve the process, or product developed by the process
2. The Procedure—used when a process step requires decomposition (not all do)
3. Work Instructions—step-by-step guidance when exact execution of the process is required
4. The Template—guide to producing the work product specified by the process, such as a requirements (e.g., the XP Story Card in Kent Beck's Extreme Programming Explained), design, or project plan document
5. Checklists—a list of error detection and detailed content that substitutes a list for fallible memory
6. Training—(two suggested levels:)
   • Expert: information that allows the seasoned project manager or developer with the requisite knowledge and skill to consistently execute the work according to the organization or company practice
   • Novice: extended training for someone who has never previously done the particular activity (e.g., new project manager, new language training)

Process versus Procedure versus Work Instruction
Getting these levels of detail correct enables useful process documentation for all classes of users. The high level process lists the major steps needed to execute the task. For project planning, they might be: 

1. Develop top level work breakdown structure from requirement
2. Estimate project size
3. Identify work products to be developed
4. Estimate effort
5. Identify people needed
6. Develop schedule

At the procedure level, the second step might be further described in a procedure that includes: 

1. Decompose elements of the WBS
2. Using historical data, estimate size of the piece parts
3. Lacking historical data, use a group activity like wideband Delphi to do estimation
4. Record assumptions and size for each estimated part

Processes and procedures should be six to seven steps and three to four pages of text. More than that and the level of detail might be too high. Templates should provide the detailed instructions, examples, and guidelines for work product development. Checklists should list sequential steps to be performed.

Detailed work instructions are usually needed with configuration management activities, especially when tools are not used and the project or organization depends on manual control of the configuration items. It may be necessary to list the steps for ownership, modification, and return to the configuration baseline in detail. As the process becomes more automated, the details are usually handled by the tooling (a tool itself forces sequential activity, with help to guide the user).

Determining the breakpoint between process, procedure, and work instruction is somewhat like Goldilocks saying "and this porridge is just right." Trying to get this perfect in an initial release is usually not possible. When a team of users and a process person writes the process, be careful that an overly detail-oriented person doesn't take you too far in his direction. I'd also suggest interviewing the process users from the first few projects before using them, and again after several repetitions through the process. That's why change requests were invented.

Templates
A good template provides the structure for a work product. It clearly identifies instructional content and mandatory and optional content in the final product. Beware of over-zealous auditing, where minor deviations from the order or content generate corrective action requests. Leave the minor stuff to verbal comments, or for needed observations that do not require corrective action. Unused sections should have an "N/A" to indicate the author did not forget the section. If a substantial deletion is reasonable, an explanation is usually in order.

Checklists
I once heard a developer state "using checklists is unprofessional," apparently he was thinking that relying on a memory aid was somehow beneath his dignity. I consider pilots professionals, and as much time as I spend on planes I am thankful they do not have this mindset!

Good checklists are based on common errors noted in the industry for the particular activity or work product, with modifications based on your organization's experience.

Training Myths
Training will never replace experience. It only expedites the learning curve (e.g., someone that has never written a line of code versus an expert in one language moving to a different but similar language versus one moving from a procedural to an event-driven language).Processes do not substitute for training or else you'll have a 500-pound process document.

An expert may not need detailed subject matter training, but still needs to learn how the process is applied in the organization to ensure consistent execution.

A Final Thought
The observant reader will note I've used a fair number of conditionals as there is no one right way to do this. A fair degree of common sense is needed to be sure you match the breadth and depth of your processes to the business needs of the organization.