Building an Effective Test Organization

Addressing the perceptions
It is very important to address incorrect perceptions about testing. Testing doesn't stop with the finding of defects. In fact, that's where the task begins. A test engineer should get involved at the beginning of the product life cycle in order to understand the product, contribute in each phase of product development, and review and prevent defects. This makes the job of a test engineer very difficult, but also interesting.

Firstly, a test engineer should have good listening, communication and conflict management skills. The test engineer should have highly developed technical skills in the product and testing domains and in software development processes. These points need to be explained to the test engineers in order to emphasise how important testing is in product development.

The following actions, with clear understanding and convictions from the management team, would help in addressing this issue:

• Avoid inducting engineers fresh from university into the testing team.
• Never entertain the notion that people with lesser skill sets can become testers. If you assume that test engineers of any calibre can do justice to testing, it is dangerous and detrimental to the success of the final product itself.
• Always remember the maxim–if a person is not suitable for development, for the same or similar reasons they may not be suitable for testing.
• There is testing in all development and there is development in all testing. Developing code is not only the responsibility of the developers. They also have to do unit testing to ensure the code is correct. Similarly, automation and test suite development are part of testing, which requires very good development skills.
• Rotate your best development engineers into testing and make your whole engineering community understand and appreciate that testing is important.
• Define clear roles and responsibilities. It helps to have clear and precise role definitions for these areas, such as test execution engineer, test strategy engineer and test automation engineer. This will reinforce the notion that testing doesn't mean only test execution where an engineer takes the test cases and keeps on executing them.

Getting the team together
As testing may be the last or penultimate phase of product development, it is important to realise that contribution by a few individuals in the team would not make the product successful every time. Effective teamwork is needed to share skills and work to ensure that the product releases on time. A high degree of optimisation is needed in test teams to absorb the delays in product development. This absorption has to be done without affecting the quality of the product, and that is where creativity and innovation play a major role. Creative teaming is simply pooling ideas from all individuals of the team and implementation by everyone in the team.

The following actions would help in taking the team forward together:

• Train the team on the importance and benefits of working together as one team.
• While you are recognising individuals for achievements, recognise the team.
• Address issues that affect the team working together on a priority basis.
• Provide forums for team members to share concerns, feedback and ideas.
• Always involve the team in making critical decisions such as the schedule and release of the product. In situations where the decisions are taken by the manager or an individual, ask the team whether they have any concerns with the decision and address those issues.
• Provide the team with goals and encourage team members to decide on their own actions to meet those goals.

A set of test engineers can’t be called a team. It is only a sub team. Finding all defects in the product doesn’t ensure a quality product that will be released on schedule. All members of the development, testing and other teams should work together to analyse the impact, reproduce, fix, verify and release the product.

The above actions, while used for testing teams, should also be considered for the larger team, which consists of developers, test engineers and other people who perform various roles that impact a product release. When explaining that the team should work together, it doesn’t mean team members are not independent. Each individual has work assigned, which they complete individually. They then work together with the rest of the team to help ensure that team goals are achieved. One of the biggest problems in teamwork is when the team is waiting for work to be assigned by managers. A manager facilitates the team and defines the team’s goals, but is not there to define low-level details or drive daily/weekly activities. The responsibility of a test manager is to create and nurture an effective self-directed team where team goals are met even in the absence of the manager or some other team members. This also means that support exists for each role or person.

Focus on technology, process and management
When you want the team to grow, it is essential it grows in all dimensions at the same time. Unlike development where you need to know about only one module and/or an interface, the test engineers are expected to know the complete product and its usage. The test team, having only technical knowledge, will not consistently yield good results. Technical knowledge has to be mixed with process understanding/improvement and management (both test management and people management). The following will help achieve this:

• Make sure you and your team learn about new technology (both testing and product domain technology). Promote the transfer of knowledge within the team. Doing this is very difficult as the technology is growing rapidly and test engineers not only need to learn the product domain, but also need to learn the testing technology.
• When a new member joins the team, look at different ways in which they can quickly catch up to reach the average skill level of the team. Provide focused training on product, testing, processes and tools.
• Keep looking at new ways to continuously improve the productivity of the team. Provide metrics and measurements to evaluate progress.
• It is counterproductive to start testing the product that is not ready for testing. Give adequate focus for entry/exit criteria.
• Always keep a list of initiatives that the team should take forward. In test teams, the release often becomes the number one priority, with high stress situations, and it is quite possible for everyone to forget about future technologies and those initiatives.
• Certification of testing is important to raise the quality of testing. It increases the awareness of testing and helps in reducing the risk associated with software development and acquisition.

Many people in the industry believe that new technology always drives the test engineer and not the other way around. So the test engineer is under pressure to learn the technology and test the product at the same time. This scenario has to be changed. A positive scenario would be where the test engineer understands the new technology proactively by being part of the product development team and contributes to ensure that the technology is adopted in the product. Test managers should build a strong image of themselves in the test team, development team and other support teams. This is only possible if the test manager excels in all the dimensions involved (technology, process and management) and keeps improving on it. While important for the test engineers to gain respect/trust among the developers, it is equally important for the test manager to gain credibility within their team, and also in team development. Focusing on technology, process and management will help in achieving this credibility.

Customer perspective
The customer's perspective is imperative in product development and testing. This perspective helps in resolving conflicts that may occur in product development, when making decisions such as whether a feature is needed in the product, what performance level would be accepted and what are the defects to be fixed before the release.

It is always a tough task to build a customer-centric team. Not understanding the customer's requirements and thinking only of technology could adversely affect the product and the customer. 'Technology' is an important aspect of customer satisfaction but if that technology does not provide the solutions that customers want, it may be considered of little use. Any change to the product has to be analysed to find out what value it adds to the customer or what impact that change will have for current and future customers. It is easy to ask why only test engineers need to have the customer focus. Yes, the customer focus is needed for each and every individual in the product development team, irrespective of roles and responsibilities, but the test engineer needs to play a role in initiating, promoting and ensuring that customer perspective and usage are tested.

It is quite possible for developers to get enthused by the technology, as it may be the primary focus of their role. Striking a balance between technology and what the customer wants, or converting the technology into a solution, needs a push in the product team, which is typically expected from a test engineer more than anyone else.

The test engineer needs to ensure adequate test cases exist for each customer requirement and conclude whether they are met. There is no end to testing, however, focusing adequately on customer usage can bring down the risk of releasing an inappropriate product. The focus on testing should be more on the areas that are considered as high critical requirement by the customers. On the contrary, there is no point in spending enormous amounts of effort in an area considered low critical by the customers. The test engineer should be in a position to say this product meets the requirements of X per cent of customers and also suggest various means by which the risk of releasing a product can be minimised.

The following will help achieve customer focus in the test team:

• Encourage testing engineers to interact with customers and include them in regular meetings with customers. Share test practices, processes and test data (whatever you think can be shared) with the customer, if the customer is interested in looking at them and providing feedback.
• Involve them in the support calls with the customer in the post-release phase of the product. This provides them with an understanding of the problems faced by customers and helps in addressing such issues in the future.
• Encourage them to make customer visits. This will help them find out the environments in which the product will be deployed and discover how the product is being used.
• Train and ask your test engineers to put themselves in the customers' shoes. It is not always possible to meet with your entire customer base. The assumption is how do you think the customers will use the product. This will help in the absence of actual data from the customer.
• Encourage test engineers to make decisions based on customer requirements and usage. All decisions taken should be validated with concern to customers. The test engineers need not worry about the number of defects found in the product. They should focus on the impact of those defects on the customer.
• Encourage your test engineers to get the exact setup and usage from the customers and replicate them in your lab environment. Also share the outcome of the testing with the customer so that the customer is aware of what will work and what may have problems even before the customer installs the product.

Summary
The ultimate goal of every organisation is to prevent the defects in the product (eradicate them before they take form), by good processes, rather than finding and fixing them. The testing should be reduced dramatically to "qualify a product", with less focus on finding defects and more focus on "defect prevention". This objective should always remain in the minds of test engineers and test managers for the building of an effective test organization.