People, Processes and Tools: The Three Pillars of Software Development

[article]
Summary:

Every project is dependent upon people, processes, and tools:  they are how the work gets done. These three essential elements are not equal, though, as each has its own strengths and weaknesses.   Each one provides a different value to our projects.

 

This three-part series explores the relative value of people, process, and tools. We will look at the strengths and weaknesses of and how they provide value to our projects. Is one of the three more important than the others? We will see, as we investigate people, processes, and tools.

People Are Critical
Can you imagine a project without people involved? Of course not. People are the heart of every project. Without them, the work would not get done. Why are people such an integral part of any project, though? Why do we put so much emphasis on them?

People are important to our projects because of the value they provide. It is people who are creative and who embrace a vision of the future that has not yet been realized. They also apply their intellect to turn that vision into reality. This creative energy is the life-blood of our projects and  is what makes the difference between something that might happen and the realization of a dream.

Creativity
For decades now, we have been anticipating artificial intelligence (AI): the dream that computers will someday be able to do the work of human beings. Though the capabilities of computers have increased dramatically each few years, the holy grail of AI has remained beyond our reach. The power of computers, while it serves to enhance our ability to perform work, can simply not replace us and won’t be able to in the foreseeable future.

The reason that humans are so indispensable is that we have capabilities that cannot be duplicated by machines. Primary among those capabilities that are unique to us is creativity. Simply put, creativity is the ability to create: to make something out of nothing. While we are unable to create in the physical sense, we have vast ability to create in the intellectual sense.

Any project that we embark on requires this sort of creativity. Starting with a goal or a need, we make intuitive leaps from concept to concept as we build on the things we know and our base of experience to establish a springboard into the unknown. We then launch ourselves from that springboard into new realms where ideas that had not been in our minds before are then captured. We realize new concepts, connections, and new ways to attack problems and challenges.

It is this creativity that allows us to move beyond what we knew and to capitalize on what we discover. Machines cannot do this. Software is unable to do this and human creativity is the key.

Vision
Creativity is most often prompted by vision, yet another uniquely human ability. Generally, vision refers to the ability to see. As we are using it here, though, it refers to our ability to see what is not yet there: to imagine. While machines are able to clearly see what actually exists (often better than humans can), the ability to see what might be is uniquely human.

We humans are able to envision what we might create and to then use that vision as the impetus for our actions and the motivation for our creative energy. Every project starts as a vision in someone's mind, and that vision is communicated from the champion to the rest of the people who work on the project. It becomes the guiding principle for all of the work that is done, and all of the creative energy that is expended.

Intellect
Finally, our combination of creativity and vision provides a special outlet for our intellect. Intellect is more than just what we know. The facts we carry around in our heads provide the basis for intellect, but intellect goes far beyond those mere facts. Intellect is our ability to understand relationships among those facts, and then abstract those relationships into principles that we can apply in new ways in new situations.

Our intellect is the enabler of creativity, as we bring our understanding of facts, relationships and abstract principles to bear on new challenges and problems.

People Are Limited
If people are so able, then why do we need anything else? Why worry about processes and spend money on tools? Why not just put our stock in people, move out of their way, and let them work their magic?

The problem is that people are not magic. Although they are essential to the success of our projects, they often also sow the seeds of failure. People are the source of most of the problems we experience on our projects. They make mistakes, and those mistakes waste our time and money. They forget things, and those omissions force us to go back and rework what has already been done. And they are imprecise. What is "good enough" for one person may not be precise enough for another, and is often insufficient for the machines we must interact with.

Errors
"To err is human." Indeed, just as with creativity, vision and intellect, our propensity to error is a defining attribute of being human. While we expect consistency from the machines we use, when people are involved, we expect errors. We don't double-check the compiler, because we fully expect it to produce correct results. We have little trust for the programs that people write until they have been tested, reviewed, examined, and tested again.

The errors that people commit have significant impact on our projects. First, because we expect errors, we plan for them by including reviews, audits and tests as part of our project plan. This time and expense is simply part of what we do on projects because we know that people make mistakes.

Additionally, when we detect the defects or other problems that people's errors cause, we must spend more time and money to correct those errors. This usually includes scrapping some work that has been done, and spending additional time and money on rework.

Omissions
In addition to the mistakes people make, they also forget things. We forget facts and information, but we also forget to do things that we are responsible for. When an activity includes many steps, it is not unusual for the person who must do that activity to omit one or more steps. This can happen because he or she is unfamiliar with the activity and is unaware of all of the steps that comprise it. It can also be an effect of familiarity, such as when a person's attention lapses and his or her normally complete actions fall short for a change. In either case, the omissions almost always result in problems for the project. Someone must often go back and fill in the missing work. In many cases, the entire job must be re-done, thus wasting the person's original effort.

Imprecision
Often, even when a person does all of the things that are necessary, the resulting work product can still lack the necessary precision. For example, an analyst may document the requirements for the system the project will build, but if the words and graphics in the specification do not communicate the stakeholders' needs with sufficient precision, then the system that is built may not satisfy those needs. Or, if the system architect does not analyze the impact of his or her design choices in sufficient detail, important ramifications of those choices may be missed until they show up as problems during construction (or worse, during testing)!

Human perception is quite tolerant of ambiguity and we readily fill in the gaps by making assumptions. We are often satisfied with work that lacks the level of precision that is actually needed by the circumstance.

People Alone are Insufficient
People are a critical part of every project, precisely because of the unique abilities they bring to the project. Their creativity, vision and intellect are what enable projects to build complex systems or derive new concepts, but their propensity to error, omission and imprecision put a serious damper on the results that can be achieved.

Clearly, to gain the full benefits that people bring to our projects, we must look for opportunities to mitigate for their shortcomings. And this is the reason for processes and tools. In the other two parts of this three-part series, we will look at each of those two enables of our most precious resource: our people.

Processes for People
Every project is dependent upon people, processes, and tools. They are how the work gets done. These three essential elements are not equal, however. Each has its own strengths and weaknesses, and provides a different value to our projects.

People Are Important, But Insufficient
People are a critical part of every project, precisely because of the unique abilities they bring to the project. Their creativity, vision and intellect are what enable projects to build complex systems or derive new concepts, but their propensity to error, omission and imprecision put a serious damper on the results that can be achieved.

People make mistakes, and those mistakes waste our time and money. They forget things, and those omissions force us to go back and rework what has already been done. They are also imprecise. What is "good enough" for one person may not be precise enough for another and is often insufficient for the machines we must interact with.

Clearly, to gain the full benefits that people bring to our projects, we must look for opportunities to mitigate for their shortcomings. And this is the reason for processes. Effective processes enable our most precious resource, our people, to work their magic!

The Nature of Processes
First, we must understand the basic nature of processes. People follow processes in many of the things we do. Our processes are the routines that help us to make our way through the day without having to stop and think through every step of everything we do each time we do it. These routines free up our minds to focus on the things that require our conscious attention by providing a program for those activities that we repeat regularly.

So, it is not a question of whether or not our people have (or should have) processes. Those processes exist, and people do follow them. Just as surely as they follow their morning routines, and their patterns for traveling to work, much of the project work they engage in is defined by processes that they follow. Most people may not call these things processes, and may not even be aware of these routines, but those things are processes nonetheless. Following processes is just part of being human.

So, why is the subject of process such a hot issue? Why do Agilists and others rail against them? It is because much of what is done in business in the name of process fails in the primary purpose of a process. It doesn't meet the needs of the people who must follow it. Our unconscious processes are invisible precisely because they meet our needs without getting in our way. A process becomes visible to us when it either fails to meet our needs, or it requires us to do things that are wasteful of our effort or time. These ineffective processes are what people complain about.

Meeting People's Needs
As we saw in the first article in this series, people have shortcomings that can impede their effectiveness at capitalizing on their creativity, vision and intellect. Processes meet our needs by compensating for those shortcomings or errors, omissions and imprecision. By doing so in a way that makes efficient use of our time and energy.

People make errors, so our processes must help us to mitigate for that propensity. Most of the time, a good process will help us to avoid making an error in the first place. A laundry process that includes matching socks before putting them away will prevent one from wearing mismatched socks to work. A morning routine that includes listening to the traffic and weather report can help us to choose the best method and route for our commute.

People also omit things. We forget to do certain steps, or do them in the wrong order, so our processes help us to do all of the things we must do, and to do them in the right order. In our morning routine, we tend to bathe, shave, brush teeth, dress, and eat in the same order each day. This consistency helps to ensure that we don't forget to do any of the steps. Just think about what happens on those days when your routine is upset! The results can be embarrassing (if you don't put enough extra thought and effort in to compensating)!

People are imprecise. We don't always focus on important details, so our processes point us to the level of detail and precision that is required of us. Many of us have a process for picking out the clothes we will wear that includes rules about which colors and patterns go with others, and which do not. These guidelines help us to make appropriate choices without spending too much time and effort. Not every error, omission or imprecision can be avoided, though, so our processes also include checks and balances that help us to detect and correct those mistakes before they cause problems. One last look in the mirror before we walk out the door allows us to detect and correct problems before they can ruin our day.

In the same way, the processes that we employ on our projects must be tailored so that they provide the needed support for the people who do the work, while helping them to be as effective and productive as possible. When processes meet peoples' needs, they truly become invisible; and people will often believe they are not following a process at all!

Consistency
While we use processes for many things, we can be inconsistent in how we apply those processes. That inconsistency can result in the benefits of good processes being lost in some cases, or for some people. A process that is followed inconsistently will produce inconsistent results, while one that is followed faithfully every time will produce very predictable results.

Achieving consistency does not necessarily require formality. The example we have been using (a person's morning routine) is almost never formalized, yet most of us follow it faithfully every morning. In fact, achieving consistency is usually pretty easy when the process meets your needs efficiently!

In organizations, though, when many people are involved, process consistency becomes much more challenging. First, it is a rare process that can be tuned to the point that it is invisible to everyone who is affected by it. Each person, if left to their own discretion, may choose to do the activities in a slightly different way. While there are times when those minor differences are of no consequence, more often, those differences can result in problems

Therefore, most processes require someone to pay attention to the consistency with which it is followed. Whether that job is done by peer reviews of work, peer pressure to conform, a "coach" who guides the team members, enforcement by a manger, or double-checks by an auditor, it is generally important to use some mechanism to assure that processes are followed consistently.

Process Documentation
There is tremendous variation in the degree to which processes are documented. In an organization of one person, there is often nothing written down. That one person decided how to do what he or she does, and that is what is done (think of your morning routine). In this one-person situation, however, process documentation is not always absent. It is not unheard of for people to create checklists to help them achieve consistency.

Even in small organizations, there is often little that is written down. The founders of a company figure out how to work with each other, discuss problems that arise, and agree on future actions. Often these agreements are not written down. But as the company grows, the need to document operating procedures, guidelines, and processes grows.

The more people who are involved in the process, the more likely it will be that some amount of process documentation will be helpful in achieving the required consistency. When new staff is added, the documented processes help them to learn the rules and work patterns more quickly, and help them to know what is expected of them.

In large organizations, more process documentation is generally needed so that different groups or functions can dovetail their work effectively to assure that the needs of the organization as a whole are being met.

But it is possible to go overboard in producing process documentation. Some organizations pursuing an ISO 9000 certification or a CMMI Maturity Level Rating produce shelves and shelves of process documents that are aimed at achieving the certification or rating. Unfortunately, when the primary purpose of the processes (to support people in doing the work) is overshadowed by those goals, the resulting processes can be cumbersome. And, of course, if too much process documentation is written, then people won't read it, or will be unable to find in it the guidance they need.

In Short

Processes exist and their primary purpose is to make people efficient and consistent in the way they do things. When processes are cited as problems, it is often because that primary purpose has been obscured in some way. Effective and efficient processes tend to be invisible to those who use them. They mitigate for people's errors, omissions and imprecision, and enable them to apply their creativity, vision and intellect to achieve their goals.  

Tools Support People and Processes

Every project is dependent upon people, processes and tools; they are how the work gets done. These three essential elements are not equal, though. Each has its own strengths and weaknesses, and provides a different value to our projects.

 

People and Their Processes Need Support
People are a critical part of every project, precisely because of the unique abilities they bring to the project. Their creativity, vision and intellect are what enable projects to build complex systems or derive new concepts, but their propensity to error, omission and imprecision put a serious damper on the results that can be achieved.

People make mistakes and those mistakes waste our time and money. They forget things and those omissions force us to go back and rework what has already been done. Also, they are imprecise. What is "good enough" for one person may not be precise enough for another, and is often insufficient for the machines we must interact with.

Processes enable our most precious resource, our people, to work their magic by mitigating for their shortcomings. While people can sometimes efficiently execute processes, this is not always the case. Many processes can have a negative impact on our people's effectiveness, and some processes are so tedious as to become a source of new problems, even as they solve others. In these cases, tools are the key to making our people efficient and effective at executing the processes that support them.

The Nature of Tools
Tools have two primary functions, to leverage or magnify people's effort, making them more efficient, and to replace human effort when a tool can do the job more effectively. Some tools have as their focus one of these things, but most tools combine elements of both.

A compiler is a good example of a tool that primarily leverages human effort. Decades ago, people wrote the actual code that ran on the computer's processor. This was an incredibly time-consuming proposition, as the programmer conceived of the actions the computer needed to take, and decomposed them down into the individual 1's and 0's that were required to make it happen.

Compilers magnify the amount of work programmers can do by allowing them to work at a higher level of abstraction, spending more of their time analyzing the problem at hand and applying their intellect to solving it, while the compiler translates each line of high-level code into dozens (or hundreds) of commands to the computer.

The Internet provides a good example of a tool that actually replaces human effort. When you want to view a certain website, all you need to know is its address (its URL, Universal Resource Locator). There is no need to think about how to get from wherever you are on the Internet to the location of the website you want to visit. After you provide the URL, the computers of the Internet work together to figure out where the pages for that site reside, and how to route your request to that location and its responses back to you.

Eliminating the need for people to understand the details of the Internet, has opened the World Wide Web to use by average people around the world. This is only possible because the Internet does that work invisibly, without any input from the user, other than the URL.

When Do We Need Tools?
Contrary to what many toolmakers would have us believe, we do not need a tool for every job. Many of the tasks we do on a daily basis yield very nicely to people's capabilities and effective processes. But at the same time, we should always be on the lookout for cases where a tool could make people or their processes more efficient. We can leverage our understanding of the two primary functions of tools to help us to identify cases where a tool may be helpful.

Are there time-sinks in our processes? Look carefully at the specific steps and tasks that your people spend their time on. Peoples' strengths revolve around exercising their creativity and intellect. Do your people spend the bulk of their time on creative aspects of the work? Or are there more mundane activities that drain their effort?

When a process demands that people spend significant time on non-creative steps, it is a candidate for a tool. Look for a tool that will magnify people's effort on the mundane activities, allowing them to spend less time on them, and more time on the more challenging aspects of their work. Not only will such a tool make people more efficient, but it will also improve the quality of their work-life, as they spend their efforts on more interesting and challenging work.

Are there processes that don't need human intervention at all? The analysis described above can sometimes yield an important discovery; that there is no aspect of the work that requires a person's unique abilities. In that case, the entire job becomes a candidate for automation. If the appropriate tool can be found, our people's time can be redirected to more important in interesting work.

Tools and Processes
Any time that a tool is adopted, it will necessitate process changes. For example, adoption of a code control tool will require that developers check code out of the repository to work on it, and check it in when they are done. This example is of a relatively minor process change; one that is clearly warranted by the significant benefits that the tool provides.

The process changes that are required by any specific tool is an important consideration. If a tool requires that we make significant changes to our processes, then we need to pause to consider if we should adopt that tool and its required process changes. How will those changes affect our people? Will the new process support the people's work (the key function of a process)? Will the people be able to perform the new process? Do they have the time? Do they possess the requisite skills and knowledge? Does the process support the organization's objectives?

We must always remember that the role of tools is to support people and their processes, making them efficient and effective. Any tool that reduces effectiveness of efficiency is not worth adopting and should be rejected.

Conclusion
Our projects depend on a triad of things in order to run well. We depend on our People (who provide the creativity and intellect), our processes (that compensate for people's weaknesses), and our tools (that make our people and their processes efficient). There is a definite hierarchy in that triad, with people being supreme, and tools in a supporting role for both people and processes.

We need people, processes and tools (just as a three-legged stool needs all three of its legs). Ensuring the success of our projects requires that we always keep these three elements properly balanced. 

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.