"What does your quality assurance group do?" I have asked this question of many executives. Too often they answer, "Quality assurance is responsible for testing our software to ensure it is ready for release." I push, hoping for more, by asking, "Anything else?" Usually, though, the response is little more than, "Well, they manage the defect tracking system. What else would they do?" What more, indeed!
Just as there is much more to configuration management than merely doing good version control, quality assurance goes far beyond mere testing. Indeed, I have seen companies where the quality assurance (QA) group doesn't do testing at all; that is the responsibility of the testing or quality control group. Their QA groups are responsible for all of the other activities that are necessary to really assure quality.
Quality Control versus Quality Assurance
Testing is a quality control (QC) function. What is quality control? QC includes any activity that examines products to determine if they meet their specifications. As this definition applies to software, the specification may be either written specs, or needs as defined by the customer or end users. Not only is testing a QC activity, but so are walkthroughs, reviews, or inspections of work products like requirements, designs, code and documents.
If none of these activities are a QA activity, what do we do in our organization that is QA? Unfortunately, most of us are doing little or no quality assurance. QA includes any activity that focuses on ensuring that the needed levels of quality are achieved. In essence, if QC is about detecting defects, the QA is about avoiding them! QA activities include identifying problems so steps can be taken in the future to avoid those same problems. It includes engineering the processes that are used by the team (analysts, developers, testers, writers) so that high quality products can be built efficiently. This includes ensuring that each team member is performing his or her job consistently and producing consistently good results.
Identifying Improvement Opportunities
Every organization has many opportunities to improve the methods and processes they use. The problem lies with identifying what problems need to be fixed and where it would be most useful to invest in improvement. One of the main functions of quality assurance is to use the organization's own data to identify these opportunities. Your organization maintains a treasure trove of data, just waiting to be mined for valuable information: defect data, development effort data, cost data, and schedule data are can provide useful insights.
The QA analyst searches for patterns of inefficiencies, defects, or other problems. For example, if a certain kind of design flaw comes up consistently during system test, this could indicate the need to improve something earlier in the development process. What needs to be done must be determined by looking at the data from those earlier steps. Maybe a new design methodology would help. Perhaps the designers need to be trained in using the methodology or tools they already have. Maybe the design reviews can be improved or there are no design reviews being done at all (indicating an opportunity to prevent many kinds of design flaws during test). By identifying the high-value problem areas, the QA Analyst helps to ensure that the organization can produce better quality more efficiently on future projects.
Engineering Effective Processes
Even the most brilliant engineers and greatest tools can be made ineffective by poor or inconsistent processes. A big focus of quality assurance is on the processes that are used by those who produce the project's products, those who verify the correctness of those products, and those who manage the work. Writing a process is a relatively easy task. You simply identify all of the things that must be done, determine where the interdependencies are, and reduce this information into the patterns that each person needs to follow. If writing a process is easy, writing one that works well is not. Indeed, any initial process description or procedure must be seen as only the starting point for the real process engineering work.
Only by monitoring a process as it is used to perform real work can we see where its inefficiencies hide. Both the people who must enact the process and the data that they produce provide the information that the Analyst needs to make a merely adequate process truly effective and efficient. Listening to people's criticisms or suggestions about the process can show the Analyst where improvements can be made. The data will clearly show where people's effort is being spent, and where mistakes are generated. The first version of a devise or program that an engineer builds is rarely the final product. These initial items are the basis for the ongoing electronic or program engineering work. In the same way, the heart of process engineering lies not in creating that first version of a process, but in refining it until it meets the needs of the organization and the users of the process.
Ensuring Consistency
The last part of quality assurance is to ensure that people are doing their work consistently. This part of QA is the closest to quality control, but it is still quite different in its focus. Where QC evaluates the product of people's actions, QA evaluates the actions themselves. QC includes reviewing a system design to ensure it satisfies the system requirements and adheres to good engineering principles. QA includes evaluating the design activities. Was the design methodology executed as expected? Does the design specification include all of the required information? Was an effective design review performed? Were all of the appropriate players included in the design activities? Some people see this part of QA as enforcement. If someone is not doing their job the right way, then they are to be rebuked and forced into compliance. While there is a place for enforcement, it should be a minor role at best. When people avoid doing their job in the prescribed way, it is almost always because they believe that the prescribed way is deficient.
The first reaction of a QA analyst to a violation of consistency is to find out why it happens. Why does the person believe he or she knows a better way? What does that person believe to be the best approach? In some cases, that person may have a great idea on which we should capitalize to improve the process. In many cases, people do not understand the value of the steps they are skipping or changing, and they merely need to be educated. Sometimes it is a matter of showing them that they must do things that do not provide value to them personally in order to make the over-all process work well. When it is determined that the process is indeed appropriate, but that the person chooses to compromise it for whatever reason, only then is enforcement appropriate. This is needed because the effectiveness of the over-all process depends on the compliance of everyone involved. Start Doing QA Today
At this point, many of us can see that we do little or no quality assurance in our organizations. But how do we get started? How do we get out of the rat race of build-and-test, build-and-test? How do we begin to take a step back and look at what we do from a QA perspective? QA is an investment in improving our organization. Once we decide to invest, it is simply a matter of freeing up the resources to make the investment, and deciding on first things to do with them.
Start by using your data to find the most troublesome problems, either problems that keep repeating themselves, or those that are most costly. With that sort of problem corrected, you will have the necessary momentum to make QA an integral part of your organization. Quality control helps us to understand the level of quality that we produced on a project. When we supplement QC with quality assurance, we improve the level of quality we will produce on all future projects. That is an investment well worth making!
