Defect Analysis and Managing Testing in Multiple SDLCs: An Interview with David Oddis

David Oddis talks about the importance of having an effective defect analysis process and it being the foundation of process improvement, as well as giving insight on how to manage testing across various SDLCs and the challenges it could present for teams.

Cameron Philipp-Edmonds: Today we have David Oddis. David, thank you so much for joining us today.

David Oddis: Absolutely, glad to be here. Thanks for having me.

Cameron Philipp-Edmonds: Can you start us off by telling us a little bit about yourself?

David Oddis: Absolutely, as Cameron said, my name is David Oddis. I work at the CollegeBoard. I’m from the Baltimore, Maryland area. I have two wonderful boys, Danny and David, and a wonderful wife Janine.

I’ve been in the software testing industry about 18 years now, testing such applications as DNA genotyping and sequencing to voice-recognition software to various eCommerce systems. I love to mentor and teach the testing discipline and I love to study the art of leadership. When I’m not doing those things you can probably find me on the golf course.

Cameron Philipp-Edmonds: Fantastic, do you have a favorite golfer?

David Oddis: Do I have a favorite golfer? Yeah, I like Phil and of course I like Tiger, but I don’t get too caught up with the actual pros.

Cameron Philipp-Edmonds: Can you tell us a little bit about your role on the CollegeBoard?

David Oddis: Sure, I’m the senior director of quality assurance. I run a team of about forty testers and along with those guys we’re responsible for a large portfolio of inward and outward facing applications. I spend the majority of my time probably putting testing strategies together for our chair one level releases, as well as just trying to build environments, healthy and challenging in learning environments, for our teams to be successful both professionally and personally.

Cameron Philipp-Edmonds: Awesome, now you’ve spoken at some of our conferences before and you’re a firm believer in the idea that the foundation for process improvement is defect analysis.

David Oddis: Absolutely.

Cameron Philipp-Edmonds: Why is it important for organizations to have an effective defect analysis process?

David Oddis: Sure, data tells us a lot of things if organizations hope to get better at what they do they need this input. They need these data points. When you have a strong defect analysis process in place it allows organizations to peel the onions to help determine where the opportunities are.

As an example, if you go through a defect analysis process and you see a trend of a lot of defects related to test data, as we dig in and we categorize those defects we may find that, in fact, we have an issue with putting data sets together that mimic production level data or maybe we’re just very simply missing test ideas that cover various data permutations. This will reveal that, and then organizations can then go back and actually do something about that.

Much like if we found a tremendous amount of defects where once we categorize them we saw that these probably should’ve been caught at the unit test level. We now know we need to go do something about our unit testing practices to make it better. At the end of the day, it’s all about getting pointed in the right direction for improvement.

Cameron Philipp-Edmonds: To build upon that a little bit you’re a proponent of an in-depth, cross team defect analysis process that promotes collaboration between the development and QA teams. Why really is that and what causes this particular process to be more effective than the other methods?

David Oddis: I’ve used processes in the past that take on this us-versus-them mentality or this blame game mentality. They just add zero value. When my colleague, Swathi Gabbita, and I sat down to put this defect analysis process together we were clear right out of the gate. It had to be done in a way that allowed true team collaboration. We needed the developers and the QA teams and, quite frankly, the requirement analysts and the PMs, to all work very well together to get to where we want to be.

When we put this process together we really called out very strict guidelines that would encourage not defending, but being open with your thoughts, with your words. That’s how you really get down to an area where you can do something about it.

Our goal was to establish these guidelines and then we would come out and we put these practices out there. We would act as coaches and we would allow the teams to collaborate and challenge each other in a healthy way to actually get down to a point where we could categorize these defects.

Once we started doing that and the more that we started doing that we saw not only the process itself get better and grow, but we saw the team members get better and grow. We saw projects get better, which was our ultimate goal.

Cameron Philipp-Edmonds: Fantastic, so communication’s extremely important to matter what the process is.

David Oddis: Extremely, absolutely.

Cameron Philipp-Edmonds: All right, now to switch gears a little bit. There are multiple SDLCs being implemented across various organizations these days. Based off your opinion, what challenges do you see and what do you recommend to testers when it comes to managing the testing across the various SDLCs?

David Oddis: Yeah, there’s no question. There is an abundance of methodologies out there within the software development industry. Whether we’re dealing with SCRUM or XP, Waterfall, Kanban, or some hybrid concept it of all those, regardless of that there’s still certain pieces of data that testers need. There’s still certain pieces of work that testers perform.

Regardless of whether were doing exploratory or we’re using user storage or use cases or if there’s a bunch of stickies on the wall, we have to be able to come in and assess what needs to be done on these projects and execute it accordingly. As testers, we have to develop testing methodologies that allow us to do that; that allow us to perform the work expected of us at a level higher than expected of us.

The days of formula driven testing they’re becoming extinct. The factory floor mindset for testers it’s becoming extinct. The traditional testing it’s just not allowing us to meet the customer’s needs anymore. I think the testing community really has to go out and continue mining the discipline to look for those key processes, those techniques, those tools. Those mindset gems that allow us to learn and grow and create environments where problem-solving and effective execution strategies contain values needed by the customers.

There’s a lot of good ones out there right now. I mean you’ve got Kennard box context driven approach. You’ve got box rapid software testing approach and there’s numerous others. I’m currently working on one called situational testing, which with the level of empowerment that I’m asking for my testers to have I believe that it puts our testers in a better position to develop the strategies that help meet the project’s needs.

The bottom line is regardless of methodologies testers have a job to do. They have to come in. They have to assess. They have to report on the overall health and status of their applications quickly. I think it’s again important for us to go in and find these gems so that we can grow as a discipline.

Cameron Philipp-Edmonds: Fantastic, to cover this question before you start your explanation, but SDLC stands for software development life cycles. I just want throw that out there in case anyone doesn’t know.

David Oddis: That’s correct.

Cameron Philipp-Edmonds: Now, let’s get on little bit more of a current event here. I understand you're currently in the process of attempting to teach software testing at your local community college. There are a lot of hot topics out there related to an argument that students really aren't prepared for college or life really in general thereafter. I’d like to get your opinion on some of these issues.

Now, there’s several state governments that are pushing to have coding taught in schools. Do you think that’s a good idea?

David Oddis: Absolutely, as far as teaching in the schools, I recently saw a study, it was actually a forecast. I can’t recall who it was from, but it basically stated that computer-related employment’s going to rise by 22% by the year 2020 with the strongest demand in software development. That statistic alone tells us we have to be doing more to prepare and teach our children for their future employment opportunities.

I would also like to advocate the importance of software testing in there as well. Think about this, the University of Cambridge released a study, I think it was in 2013, so I think it was about a year old. It said we spend $312 billion globally, per year, on software defects. That number is incredible.

Imagine what we could do, how we could impact that number if we got better at teaching programming; if we got better at teaching quality; if we got better at advancing the teachings of software testing? How does that number get impacted?

Testing often takes a backseat to development and I think there needs to be a call to action from testers all over the globe to say we need to bring awareness to this. We need to do better at not only teaching and forwarding it more into the software development process. I think it just adds so much more to what we do.

If you think about it this last year for testers it’s been a really great year for us. We had the Target incident. We had the healthcare.gov incident. That showed worldwide the importance of software testing. Then, you take the fact that testing itself it’s becoming more complex. It’s becoming more technical, more thought-provoking. It’s just another call to action that we have to continue growing the discipline, growing the people, changing the mindsets. Testers all over have to start questioning the way that we’ve been doing business for the last thirty years.

Cameron Philipp-Edmonds: Now, I have a question for you to build upon that. Sometimes when people come into the software engineering realm and they come into the curriculum, developers have a little bit more traditionally the sexier role in the terms that they’re creating. How do you get people and your students excited about testing?

David Oddis: That’s a great question. I think it starts with showing them that testing is more than just checking. You can come in; you can really explore much like a scientist sits down and experiments. Testing can be looked at from an experimental standpoint.

We can make it fun, very collaborative. It’s not just off in a vacuum testing based off the requirements. We can come in; we can use critical thinking techniques. We can use interviewing techniques to get down to what we really need to do. We can just make the environment itself fun.

When you put testers in a position where they’re no longer part of a factory floor mindset, they’re really game changers. They’re out there they’re helping to be a part of building something, contributing to some quality project that’s going to better something. It’s going to solve some problem. I think when we can show them that that is in fact what they’re doing I think it just makes the whole experience that much better.

Cameron Philipp-Edmonds: You had a nice little quote in there that software testers are game changers and they’re adding to the betterment of things. I agree with that wholeheartedly.

David Oddis: Absolutely.

Cameron Philipp-Edmonds: Now, with that being said, if programming is being taught to students and it is starting to get into the primary schools and whatnot, how young do you think students should really start to be exposed to programming and testing?

David Oddis: That’s a great question.

Cameron Philipp-Edmonds: To give some background on that there are some people who are proponents of the fact that testing really should just give a child who’s maybe three or four and while he’s learning English, or learning whatever language their learning, to also learn that at the same time.

David Oddis: Yeah, I was just sitting here thinking about my own experience. I can remember in 1982, 1983 my father bringing home a Commodore 64 computer. Never seen anything like this and he started teaching me I think it was basic 2.0 coding at that time. I was getting the old Byte magazines and I would copy code out of that and I would execute it.

As a twelve or thirteen-year-old kid it was mesmerizing. It was very interesting. I had never seen technology like this. I didn’t understand what was going on in the background I just understood I was typing some stuff on this funky looking keyboard and something really cool was happening on this box looking screen.

It was intimidating, but at the same time it was very fun. It was very engaging. It challenged me. To your point about teaching it in elementary, I mean I think that’s a great idea. I mean starting kids young I think it helps overcome that fear. It gradually introduces them to the computer sciences much like we do. Think about it, we do it today with music, math, and science.

We don’t sit them down and teach them how to be a concert pianist, right; we start with middle C and Chopsticks. We advance and we hope they grow into a concert pianist, but we plant the seed. I don’t think there’s many courses that we don’t do that with kids. I think there’s an opportunity to do that and I think it helps eliminate some of the intimidation that goes into that.

Again, it’s a gradual ease and a great opportunity to inspire, to expose kids and students to the world of computer science. Plant the seed, water, nourish, and see how it grows.

Cameron Philipp-Edmonds: Now, the biggest opposition to implementing coding and programming schools is that in order to put it into the curriculum you’re going to have to take something else out. The two strongest theories out there, the two strongest arguments out there is to add programming and testing and coding into the curriculum by replacing either foreign language or cursive. Do you think either of those are viable options?

David Oddis: I think they’re both interesting options. I probably would not be a proponent for the foreign languages. I think we live in a very diverse world and I think it’s very important to experience each other’s cultures. Communication is the key to making many, many things happen. We talked about that earlier. I mean it’s just too key, so I probably wouldn’t be a proponent there.

Cursive, I can see why that one would be on the table. I don’t even know, other than my own testing notes, I don’t know when the last time I read a handwritten document. I think there’s other opportunities too. Could we go back and revisit math and the sciences to see could we incorporate computer science concepts into some of that?

I think to a certain degree there some hand-in-hand that goes on there. I think maybe with a little tweaking there we can even compare these teachings in a manner that allow us to get these very important academics to come together. There’s opportunity there.

Cameron Philipp-Edmonds: Now, there are also some proponents out there that for basic level programming courses because they teach universal fundamentals, just like cursive might, in terms of documentation, conditional if-then logic, flow planning and informational recycling, all those things that programming teach that cursive may not, becoming part of the general ed courses in college too. How do you feel about that? Do you think that it should be gen. ed in college?

David Oddis: Absolutely, I mean I’m probably one of those proponents. There’s no reason why this cannot be a part of the 101s. The world is changing. It’s been changing for some time. I think it’s very important. It’s becoming more technical. It’s becoming more complex. I think it’s important that students be aware of some of the technologies, the concepts, the theories, the thinking that go into making the world as we know it today.

Will it make a difference on retention? Who can say for sure, but it definitely will open minds to the possibilities that are out there. Again, planting that seed and getting better understandings of the computer science areas.

Cameron Philipp-Edmonds: It definitely won’t hurt.

David Oddis: Definitely. Definitely will not

Cameron Philipp-Edmonds: You like to help people. You have a passion for coaching and mentoring testers to further global quality practice. It is there advice that you commonly give out to new entrants into the industry?

David Oddis: I treat every new tester, every mentee differently, but there’s probably some common verbiage that I’ll put around. I definitely tell them to get around good testers. Get a good mentor in their lives. Practice testing, you know the old saying practice thinking. The old saying of practice makes you better. It absolutely applies here.

Get involved with the testing community. Go to some of the better software conferences, talk to testers. Set goals with the notion of what you get by achieving your goals is not as important as what you become by achieving your goals. Then, have fun. Being a software tester it’s an absolute amazing experience. The journey will take you places that you probably have not experienced before. Then, finally I would tell them just never to stop learning. We always have to be learning.

Cameron Philipp-Edmonds: Is there a common problem or misunderstanding that you can identify when you mentor and coach other testers?

David Oddis: It’s a good question. I don’t know if there’s a common overall problem or misunderstanding. I think probably the biggest challenge that I see is when I'm mentoring testers that have already been exposed to the traditional mindset of testing. It takes a lot to relearn. It takes a lot of practice, a lot of discussion to get past a lot of what they’re already thinking which is already embedded in their heads.

There’s a struggle because it goes against a lot of the “how to test software” books that are out there. Yeah, with a little bit of practice, a little bit of time, a little bit of patience we eventually get there.

Cameron Philipp-Edmonds: We’re going to move into a little more of a metaphysical question here. It’s going to get a little deep here. You've been in the testing industry for nearly two decades how would you coach yourself when you first started testing?

David Oddis: How would I coach David Oddis today? I would coach David the same way that I’d coach him today if he was walking in this door for the first time as a tester. I would tell him find a good mentor, find someone who truly understands software testing and I mean software testing, not software checking. I would tell him to practice testing, practice thinking. Think without constraints; get involved with the testing community. Go to conferences; meet other testers, share ideas.

Most importantly, I would prepare him for the fact that it is absolutely important to get back what was so freely given to you. What I mean by that is all of us, regardless of whether we’re testers, developers, project managers, VP’s, CIO’s, we’ve all had somebody in our life help us get to where we are today. It’s important to give that back. For testers all over the globe it’s important to give back to the discipline that’s been so good to us. I would definitely hammer that one home.

Cameron Philipp-Edmonds: Great, I think that works is a perfect conclusion to our interview. Once again, this was the always informative, game changing David Oddis. Thank you so much for joining us today.

David Oddis: Thanks for having me. Thanks to all of the guys over there at SQE. You guys are awesome and I really enjoy everything you do.

Cameron Philipp-Edmonds: All right, that wraps it up. Thank you so much.

David Oddis: Thank you.

David Oddis is College Board's enterprise platforms software quality director, responsible for a large portfolio of applications and databases. He currently concentrates on setting quality management strategy and providing oversight to test teams. David focuses on developing and refining processes and strategies to ensure they align with the business mission. He is a proven evangelist for SQA and enjoys presenting topics on business technology, software testing, coaching, and leadership. His passion is coaching and mentoring other testers to further the global quality practice.