Within the test scene there is a vivid discussion about the necessity and use of a certification model for the test experts. The population of testers can be roughly divided into two groups: Firstly, a group that states that it will not do a better job when certified, because the current certifications like ISTQB and ISEB focus on methods and terminology, but fail to look at the practical testing skills of the tester [Bolton, 2008]. Secondly, a group who is pro certification and regards the testing industry as a young yet not fully grown profession that lacks certification models that other professions have been using for ages [Windsor 2007]. Meanwhile, voices are heard that the test profession is accepted in the IT industry and that it has actually grown into a mature profession. The website of the last EuroSTAR conference stated that a mature profession has clearly-defined standards, codes of conduct, and a number of levels of professional competence. Having all those, we might conclude that the testing profession has indeed earned its place among the IT-professions.
Have you ever been in the situation where testing time was compromised? Where someone suggested that a lot of time could be gained by expanding the test team with other resources? Such as users or administrators that happened to be available? I think most of us have, showing that there is still some work that needs to be done. The testing community still has difficulty explaining what the added value of testing is to management, customers and developers [Clermont, 2006]. Many business managers perceive IT as “too little, too late, too costly.” There is a gap between business and IT on both sides of which reign incomprehension and a lack of knowledge about the other discipline [Ommeren, 2006]. This is why business management often have wrong expectations of software development and testing. As a result, testing is frequently pushed into a subordinate role and cannot fully contribute to the development process.
Although the discussion on certification is mainly a discussion between testers, it is a rather interesting one. A good certification model will help a tester to prove he masters his profession. A certified tester is knowledgeable on one or more methods, tools and techniques. But that alone will not solve the problems management has with IT. Rather than awaiting the outcome of the discussion about certification, I think action is required today. We need to align the expectations that business management has on IT with the output it delivers. Testing can help bridge the gap since it leads to better software quality and shorter time to market [Grood, 2008]. But this will only happen when the tester is aware of the business needs and acts in conformance with these in order to make the added value of software testing visible in its full colors (see also textbox 1). That’s what result-driven testing is about.
Result-driven testing
Testing is commonly carried out according to a test methodology. The test methodology ensures that testers and other stakeholders use the same terminology and definitions, and that the work is done in a controlled process flow. The methodology also contains best practices so that the knowledge and experience of others can be used. This contributes to the efficiency and quality of the test project.
Result-driven testing is more than just applying a methodology. The success of a project is not determined by the methodology, but by the way the methodology is applied. Result-driven testing, therefore, encompasses several aspects:
- A result-driven mindset
- Testing knowledge
- A result-driven approach that supports the application of result-driven testing
A number of good methodologies have already been developed and written about. But like any other profession, testing encompasses more than the simple application of a methodology. After all, strict adherence to a specific methodology is no guarantee for success. Success stems from the mindset, enthusiasm, knowledge and skill of the tester. These factors determine whether a methodology is applied successfully and whether testing is result-driven.
A result-driven mindset -Test principles
Since each project is by definition unique, any method that is being used will need to be tuned to the situation. Which parts of the method will be used to their full extent? Which steps that are described are not relevant for my situation? Which remarks are relevant but are going to be used in an altered form? Applying the method means making choices.
What’s important to understand are the elements that influence those choices. For one thing it is of course the knowledge and experience we have. We are inclined to choose the options that worked the last time. We are often a little reluctant to choose the options that we have little knowledge of. Another aspect that has a significant influence on our decisions is our mindset.
The correct application of the methodology depends on the mindset and the expertise of the tester, who has to be able to stress the right things and make the right choices. Ten test principles help result-driven testers put this into practice. The test principles are a short and strong formulation of the tester’s desired mindset, knowledge and working method. They indicate how the tester can create added value and how he can make it visible to the organization.
The test principles enable testers to focus on the anticipated goal without external assistance. By thinking and acting according to the ten test principles, testers assume a result-driven mindset. They make sure that they apply the test methodology in such a way that it achieves optimal added value for the organization.
The ten test principles are not autonomous; they are correlated. Take for example ‘Build trust’. This principle aims to create trust in the product and the test team. Trust in the project will smoothen the go-live decision and is an important motivation for the user to want to work with the new system. This confidence can be created in several ways. One of them is by showing the organization that certain parts of the system have been realized and are of sufficient quality; that certain risks have been mitigated (provide overview). This leads to a stepwise test approach with distinctive test phases. The Go Live decision is also smoothened if all necessary know-how is transferred to the maintenance organization. The principle “Facilitate the entire IT life cycle” has in its origin the understanding that the purpose of a project is not just getting software on the production environment, but support doing business. Building bridges to the ICT Operations department is part of that.
Another way of building trust is providing transparency. By giving insight in the work approach the stakeholders can judge for themselves how well the team masters the test profession and if it takes its responsibilities serious. This is a crucial insight. Otherwise what’s the value of a given release advice to the stakeholders if they do not know how it is established? Insight is also given in the way the activities of test team contribute to the set goals of the business and project. One of them could be the deliverance of a regression test set, thus ensuring reusability, and once more building bridges to the maintenance department and facilitating the IT lifecycle.
The above description is provided as an example of how test principles work in practice. But there is more meaning to be given to them. The reader is therefore challenged to use the test principles and add their own meaning to them too.
Applying the test principles
The principles can be used as a checklist or as driving force behind the tester’s actions.
In the first way, the user regularly checks whether he has given each of the principles enough attention. This will keep him on his toes and focused on providing added value. Too often, certain aspects are forgotten in the heat of the moment. Provided with this self-check the tester has an instrument to evaluate his work.”It was hard work delivering the test scripts in time, but unintentionally we compromised the reusability. Maybe we should put some effort in setting this right, next week?”
Using the test principles as a driving force enables the tester tell his testing story. “Your mind is a black box, it takes skill to explain your testing, and what you are accountable for” [Bach, 2007]. The principles are easily understood by non-testers and help to gain commitment for the testers activities. Imagine the situation were you explain that you plan to deliver a test report by the end of the week. The customer responds a little disappointed. “Err…, I don’t like reports that much. Besides we only have limited time, so I prefer you use the time for executing tests. That is why we did hire you in the first place, remember?” Explain why you think trust in the system is a requirement for acceptance. In order to gain this trust you’ll provide him with an overview and insight. Insight in the quality of the system and the progress of the test process. The principles help communicate (to the stakeholders) why certain activities are important and how they contribute to the anticipated goal.
Either way, a tester who does not apply the principles is doing neither himself nor his customer a favor. It is therefore necessary to regularly check that all of the testers are paying enough attention to all of the test principles. Together, the test principles are the foundation for result-driven testing.
Wanted: result-driven testers
The fast developments in IT make the daily life of the business manager more challenging day by day. The dependency on IT is ever growing. Software can be found in most consumer goods and is increasing in complexity. Most companies earn their money by supplying IT-related services and products, or use IT systems to run their business. The life cycle of software has sharply declined over the past decades. Due to the rapid succession of technological developments and changing markets, systems are constantly adapted. The business manager is dealing with larger uncertainties and has more difficulties to preserve the overview. There is a demand for people who can help the business manager with easy digestible but accurate information on the status of the project and the risk involved with the new or adapted systems.
Software testers can provide this information and have the power to choose the mode in which his manager operates. We can stick to our complicated test-things and leave the worries to our manager. Or we can adopt a result-driven attitude. We’ll take the responsibility to provide and assist the development process and the manager by providing accurate and understandable information. Result-driven testing will lead to happy managers that show appreciation for the test contribution.
If they were not enough arguments for applying the test principles, here’s another:
The figure below shows a recent, random picked job-description for a senior test manager. The description contains many references to the TestGoal test principles. It also shows that as part of the bigger concept, certification can be useful as proof of mastering your test profession. Most of all, it tells us that organizations have a strong need for testers who know how to contribute to the business goals. Result-driven testers are wanted!
Just a job description; there is a demand for testers who practice the test principles.
References
[Grood, 2008] Derk-Jan de Grood, TestGoal - Result driven testing, Springer, ISBN 978-3-540-78828-7
[Bach, 2007] John Bach, Tutorial on session based exploratory testing at Starwest 2007
[Black, 2002] Rex Black, Eurostar, 2002
[Bolton, 2008] Michael Bolton, Presentation at Eurostar 2008
[Clermont, 2006] Surviving in a QA-Organisation, Markus Clermont, Dutch testing day, 2006
[Ommeren, 2006] De wereld op zijn kop, Erik van Ommeren, IT Beheer, issue 9, 2006, SDU
[Windsor 2007] Susan Windsor, Keynote at Eurostar 2007
