STAREAST 2015 Interview with Rob Sabourin on Exploring Usability Testing for Mobile and Web Technologies

Jennifer Bonine: We are back with our virtual interviews, and I'm fortunate enough to have Rob sitting next to me today. Rob, thanks for being here.

Rob Sabourin: Thank you.

Jennifer Bonine: How are you doing lately? What have you been up to?

Rob Sabourin: Well, I've been traveling the world, teaching and consulting, doing a lot of cool stuff.

Jennifer Bonine: A lot of cool stuff.

Rob Sabourin: Yes, some really cool stuff.

Jennifer Bonine: Where are you seeing ... so for you personally, are you seeing more work in particular regions or areas, or is it pretty much across the globe?

Rob Sabourin: If we do a twelve-month snapshot …

Jennifer Bonine: Yeah, of Rob's life.

Rob Sabourin: Rob's life ... I went around the world four times.

Jennifer Bonine: Four times in twelve months?

Rob Sabourin: I had an emergency mobile testing training crisis in Australia and New Zealand, which I flew from Montreal down to help resolve.

Jennifer Bonine: Wow.

Rob Sabourin: We saved the world. It was great. I had to go to South Africa to help bootstrap the testing community, which is a new thing that's really building up down there in South Africa.

Jennifer Bonine: Interesting.

Rob Sabourin: Totally new. It's really wonderful. I had a chance to participate in some testing, training, and consulting in Malaysia. I also spent two weeks doing …

Jennifer Bonine: I maybe actually saw you in Malaysia, Rob. I think …

Rob Sabourin: I was there.

Jennifer Bonine: We crossed paths, exactly. Literally crossed paths.

Rob Sabourin: There we go. So, Malaysia I was there for two weeks, training, consulting, and conferencing, but a really fun thing I did was I went and I did a deep dive consulting in Bangalore, India, to really look at how people are getting context-driven testing done on outsourced testing projects.

Jennifer Bonine: Really?

Rob Sabourin: And then I did all my other stuff.

Jennifer Bonine: Now that's an interesting topic, right? What I'm hearing, at least here, is some companies or organizations are having challenges with outsourced testing having contextual relevance around some of their products.

Rob Sabourin: Absolutely.

Jennifer Bonine: They're really struggling with how to bridge that gap in their outsource models, and a lot of them have two-, three-year contracts they're locked into. They're not making it work. What's your perspective on that?

Rob Sabourin: Well, I've seen it fail, and I've seen it fail badly, and I've seen it fail really, really, really badly, but I have also seen it succeed brilliantly. The cases that I'm seeing that are really ... I think the game-winning acid mix is called deep cultural immersion. One example is there's a payroll company in New Jersey that is a wholly owned subsidiary in India.

Jennifer Bonine: Really?

Rob Sabourin: And they have a Scrum team that some of the members are in India and some of the members are in New Jersey. They're rotating. There's always a Jersey person in India and an India person in Jersey, and by doing this rotation, they get to understand the new makeup of the culture. They build the relationships, and they come up with a whole slew of informal methods to really try to get efficient communication and, as we say, cultural sensitivity, context sensitivity. I've seen it done, and I think that's done really, really well. Now, that's expensive to do by some accounts, but still, the dollars and cents analysis of it shows after a couple years that the cost of development for them is cheaper per story point than if they did it with a team all in stateside. Of course, that said, there's all sorts of patterns. That's hybrid, where you mix testers and developers. If all the testers are on one side and the developers on the other side, I've seen that work too.

Jennifer Bonine: Really?

Rob Sabourin: It was a great example. A customer, when I was in Australia, a customer of mine has outsourced testing to China, and what they do is they organize the work, so they batch it together, and they use a method called a charter-driven, session-based, exploratory testing, which some of your audience might know about.

Jennifer Bonine: Yup.

Rob Sabourin: What they would do in the morning, so I don't know time zones, but the beginning of the work session, they would have a kick-off, not just for one testing session, but for the set of sessions they do in the day. Then the outsource team would go collaborate, do the work, and at the end of the day, they debrief on a bunch of them. You might have eight or twelve sessions grouped together, and that leads to, in their case, efficiency. It's certainly not as clean as if everybody was co-located, but they get great work done.

Jennifer Bonine: And are you saying …

Rob Sabourin: It's pretty exciting to see it. When it works, it's nice to see.

Jennifer Bonine: It is. Do you think though that that working is the exception or the rule?

Rob Sabourin: Well, for me, it's the exception.

Jennifer Bonine: Right.

Rob Sabourin: For me, it's the exception. Absolutely, there's so many cases where the outsource team is just not aware of what really matters on the project. No matter how hard they try to do the technical work, because they don't know the factors around it, how do they make the trade-offs? Testing is all about …

Jennifer Bonine: Trade-offs.

Rob Sabourin: Trade-offs.

Jennifer Bonine: Absolutely.

Rob Sabourin: Very challenging.

Jennifer Bonine: Well, and you even think about contextual awareness of certain things culturally.

Rob Sabourin: It's a hard one.

Jennifer Bonine: Your cultural awareness and how you grew up. Companies like Petco, right, are designed solely around our pets, and a lot of people in the US and other cultures treat pets as people, right, and in India, pets aren't that way, you know, the same way.

Rob Sabourin: Well, maybe, the exception of certain animals.

Jennifer Bonine: Right, with the exception.

Rob Sabourin: It's definitely ... there's cultural things in every industry, in every business and why I say the deep cultural immersion seems to work is just for factors like that. Frankly, I think because of cultural reasons, sometimes it doesn't make sense to distribute a team. Sometimes it just doesn't make sense. There's other models. There's other things we can do.

Jennifer Bonine: Yeah, and with people who are struggling or that model isn't working for them, I'm sure you have the opportunity to go in and advise them on other alternatives or things they can do.

Rob Sabourin: They don't listen to me very much, but I ...

Jennifer Bonine: You tell them.

Rob Sabourin: It's sort of like I like to understand what's going on and I have opportunities to do it, but some of the ... I failed a lot in trying to influence change. I sort of have seen it work well, and I've seen it work poorly. You go and you say, well, here are the baby steps I think you should follow to make the change. They start along the path, but sometimes they sort of give up too soon. They expect too much in one shot. I attribute this to business decision making that's just sort of saying, hey, we've got to cut this much dollars off of this budget tomorrow. This next project's got to be better. They're not thinking, well, it will take us a few projects …

Jennifer Bonine: To get there.

Rob Sabourin: To get there.

Jennifer Bonine: Yeah, the patience level just isn't there.

Rob Sabourin: A specific example recently in Chicago ... The company's spending all sorts of money on outsource testing to ... This was in India, but it could have been anywhere.

Jennifer Bonine: Could have been anywhere ... in a certain place.

Rob Sabourin: The people doing the testing tested the business rules the way they thought they should be tested. For every rule, they did sort of one positive test to make sure it worked when it should, one negative test to make sure it doesn't work when it shouldn't, which I think is good, but they didn't realize that there's actually rules that interact with each other. They did a lot of good testing from their perspective, but they didn't do the type of testing that the project needed. It took an outside person to point out that. I guess one piece of advice is to even if you're not sure how it's going, maybe get someone from outside to sort of look under the hood once in a while.

Jennifer Bonine: Yeah, and just validate.

Rob Sabourin: Or a tune-up.

Jennifer Bonine: Yeah, check it up. Let's see if it's working, what's not working.

Rob Sabourin: But it's a hard problem, and certainly the economics are still in the advantage of paying labor costs overseas.

Jennifer Bonine: Right, yup.

Rob Sabourin: It's a lot less expensive and the quality of the people are fantastic. They're very, very excellent software engineers, programmers, and testers in the whole world.

Jennifer Bonine: Yeah. It's interesting to see the shift though because I think there was a shift where a lot of companies went full in on that, and now we're seeing some change and some transition. Yeah, all in. We're all going.

Rob Sabourin: It's like someone wrote an article in the Harvard Business Review.

Jennifer Bonine: Right, and everyone went we're doing it.

Rob Sabourin: I think people are learning better how to do it now.

Jennifer Bonine: And modifying that approach, maybe doing hybrids, other things.

Rob Sabourin: Stuff like that, but I still see a lot more failed attempts than-

Jennifer Bonine: Than success. Yup, exactly. What would you say ... You and I have talked at these conference before, we've done these interviews before, for the folks watching who aren't here, what are some of the themes or main things you're hearing that are kind of what people are talking about at this one in particular?

Rob Sabourin: Things I've specifically heard, and I haven't done a big sample, but I've been here for a couple days already, and I met with a lot of delegates. I ran into a lot of people concerned with platform compatibility testing. I thought that was really interesting to see. I think that there's so many different, right now, applications hitting different target environments, web apps, mobile apps, and then web ... different OSes, different platforms, different browsers, mobile ... different devices, different technologies, different versions. It's an explosion of combinatorics that some people are sort of getting revealed to, and it's like a shock.

Jennifer Bonine: Right.

Rob Sabourin: The fun thing is, though, I think our industry knows a lot about how to deal with that. We just have to sort of dig a little bit into our past and realize that we've solved these type of problems with every evolution of technology. We'll be able to do it again. I'd say that's a big concern. That's a big concern I see.

Jennifer Bonine: That makes sense. Anything else you're kind of hearing from the delegates or just in the sessions you've …

Rob Sabourin: How do I do things faster?

Jennifer Bonine: Yeah.

Rob Sabourin: I guess that's not a new thing, but people are at least asking the question more directly now. They're getting more specific about what part of the testing you do faster. How do I know what to test? What not to test and still get value? I like when I hear, especially younger people, possibly their first testing conference coming in and talking about stuff like that. I think that's a very mature thing.

Jennifer Bonine: Yeah, for them to kind of …

Rob Sabourin: It's not all diving after the newest, latest, and greatest technology.

Jennifer Bonine: Exactly, and kind of what are you worried about? In what you're seeing in this profession and in the industry, what are some of the things that give you pause or make you a little concerned?

Rob Sabourin: The things that I think need attention, and not just by me, but by a lot of people in industry is skills, I think. One of the questions I ask you ... ask everybody watching this, can you describe what you do at work without using the word test as a verb? Can you describe testing without just saying I test it. Many people have trouble breaking down their work into the composite activities, and then identify what are the skills to do those activities, and then recognize do we have those skills or not, and then how do we improve them, or develop them, or steer them, or guide them, or coach them. It's easy to say, hey, we need testing coaches or we need testing design methods, but what really are the activities that you do at testing? I think it's a good time now to reflect that on that, especially with the agile transitions we're seeing.

More people, for example, who are traditional programmers who are given testing responsibility, more people who have traditional testing that are given more technical responsibilities that they might be uncomfortable with, and so it breaks down a little bit more into saying let's look at the team and the skills of the team and how we can balance it and then improve it. I really believe that's something that weighs heavily on me. I don't think there's an answer by the way. I really don't. I don't think any of the certification organizations have a skill list to follow. I think some of the speakers have good ideas, but there's room for maybe a concerted effort …

Jennifer Bonine: Around that.

Rob Sabourin: Yeah, maybe SQE or Stark.

Jennifer Bonine: Help solve the problem.

Rob Sabourin: Well, they do by giving us a chance to talk.

Jennifer Bonine: Yeah, absolutely. One of the things, too, if you were to say the difference in skills of what you would see in people in this industry five or seven years ago to someone today, what are you seeing as some of the differences around who's being successful, what that looks like, what those skills look like?

Rob Sabourin: Yeah, well sure. I think gone is the day of the specialist. I think we do have specialists. We'll always need specialists, but teams used to be built up by collecting a bunch of these guys and a bunch of those guys. You're a performance tester, you're a security tester, you're a usability tester, you're all these special roles and right now we're saying, well, I don't know ... someone on the team has got to build some usability testing skills and some performance testing skills and stuff like that. I think that we used to see people who would say, hey, I want to be specialized in this sort of type of testing, and I think it's breaking down a little bit. I don't think that's bad or good. I don't want to judge that, but it's definitely different. It makes it, I think, a challenge for the tool vendors to come up with performance tools that are useful for people who are traditional security testers, for example, or some other type of role.

Jennifer Bonine: Well, exactly, and even having walked through here at the conference if you attend, there's an expo downstairs, which has a lot of those vendors you talk about, the tool vendors. It's interesting just to watch them describe their tools, so instead of saying it's a QA tool, it's a QA development project management tool, right?

Rob Sabourin: Well, we're lucky that the vendors are at least talking about how the tool values ... has value. Not just technically what it does, but yeah, they're really talking about the tools doing a lot of stuff. I don't know if that's good or bad. I think they're being forced into that a little bit.

Jennifer Bonine: Yeah, to say this doesn't ... It's not for this type of person because you're right. The roles are changing and testers are doing some development and developers are doing testing, and there's just more of a mixing of those blends.

Rob Sabourin: I'm certainly going to spend some time looking at the vendors trying to see what direction they're going in. There is a concern with that, of course, is that when you have a highly specialized performance testing tool, and you sort of say let's make a tool that's less specialized, you might lose some of the abilities of the tester or the benefits of the tool. It's interesting. We see this in agile development where they have agile project tracking tools that include bug tracking. Now, traditional bug tracking tools are very powerful, they have a lot of information and metrics, and these agile ones are sort of very light and don't have a lot of the stuff that the traditional tools do. Now, is that good enough? I don't know. I don't want to judge it, but it's very different. It's a tool for everybody, but it's certainly nowhere near ... got the amount of special capabilities that the …

Jennifer Bonine: That they used to.

Rob Sabourin: The traditional …

Jennifer Bonine: And traditionally would have. Now, you brought up briefly certifications as well. A lot of people, traditionally, and I think all if you look across all professions, right, accounting to doctors to everyone, needs certifications to kind of do their job and be proficient and be proclaimed as proficient, so what do you think in terms of there's lots of certifications out there for testing, there's lots of organizations ... I know you said you don't think they've solved the skills problem, but what would you say to someone out there watching, saying, Rob, how important is it that I get certified in Agile or how important is it I get certified by ISTQB, or whoever, name of what company, right? What do you think of that?

Rob Sabourin: Well, first of all, I want to say that I'm a professional engineer, so I want to be totally honest. I'm a professional engineer and I'm happy to be that. I do have a credential, and I have a legal credential, and I had to pass very complex tests to get that and do years of internship, and blood, sweat, and tears entering.

Jennifer Bonine: As a lot of professions, right?

Rob Sabourin: Right, and so I don't think we have such a thing in testing. Therefore, every one of the organizations that is offering certification possibly has value to you, but don't expect that it's going to be like medicine or like nursing or like law or like engineering.

Jennifer Bonine: It's the same right?

Rob Sabourin: It's certainly not at that level of maturity. The profession and the organizations are not really as representative as doctors being accredited. That said though, if you look at training and what they call the syllabus, the educational pathway offered by certification organizations, if you find in that path things that are good for your career, go for it. Take the training, but don't take the training to get a sticker necessarily, take the training because it's got a value to you. Some organizations are also, not that I agree with this, but are saying they would like all their testers to have certain certifications, and so it might be kind of important for your job. It's not necessarily going to make you a better tester, but I've seen it required at a couple of companies. Not too many.

Jennifer Bonine: They're looking for that as an entry point, right, where I want you to have this …

Rob Sabourin: Or they want everybody to theoretically speak the same language or use the same terminology. Is it increasing or decreasing? I don't know, there's certainly a move towards selling certification training, so I cannot ... I look at the training business, and I said there's a lot of demand for it. A lot of times when I'm selling a training course, people ask does it fit any certification and I have to tell them, well no, not really. My courses certainly don't, but people ask. It's on the radar.

Jennifer Bonine: It's on their radar. They're thinking about it. Are you seeing ... in terms of advice you would offer to folks out there watching this who weren't able to attend the conference, who are struggling with where to focus. There's lots of things right now that they can focus on. There's the whole mobile explosion. What do we do with that? There's crowdsourcing and crowdtesting and lots of things that are coming up for topics, what would you give them in terms of advice on where do I focus and where do I refine my career as I'm looking at where am I going?

Rob Sabourin: Let me tell you what I have done, not what I ... I don't know what I'm going to say tomorrow, but what I've done in the past or today, so for people who've asked me that, I usually try to get a little bit of a picture of what they're looking for, but I see two directions that are both very important. One is skills. To understand, especially test design skills. I think people need to hone those skills and be able to apply them without tons and tons of analysis without ... I want you to be able to take those test design skills and design and run your test on the fly as close as you can. Really master test design skill and realize there's variety. There's not just one or two. There's a whole bunch. Learn as many as you can. Treat them like the blades of the Swiss Army Knife. That's one side of the equation.

The other side of the equation is problem solving skills in general. Forget the specific technology, we will always have a specific technology, so go meta on that, and hone up on different problem solving strategies, and I think there's a lot of them out there. You don't have to take a testing course, but you can learn them through organizations, critical thinking. There's technical problem solving, mathematical problem solving, business problem solving. I think that's something that I've been encouraging a lot of people to get into. I think a lot of it applies directly to these really complex testing projects.

Jennifer Bonine: When we're hearing more about the term social tester ... Have you heard that term social tester?

Rob Sabourin: Sort of, yes.

Jennifer Bonine: And the importance of testers having their interpersonal skills and their ability to communicate, work well with the business, translate information ... so it's not just about how technical you are, knowing the book smarts around the actually how to do testing, but how you actually interact and your relationship skills, your ability to communicate well. How important do you think that is?

Rob Sabourin: Well, I guess there's two dimensions. One is soft skills in general I think are important. There's a lot of great ways to learn about them. Unfortunately, many people who are weak in soft skills don't know it. You've got to become a little bit self-aware or have coaches, managers around you who guide you on that. I do think soft skills is awfully important in testing, probably now more than ever. The reason it's important more than ever now I think is collaboration. Collaboration is not just we sing Kumbaya and we play well together. Collaboration is doing deep, hard technical work together to get things done. I've been trying to learn about how to teach collaboration. I don't really know the answer, but I've been collecting dozens and dozens of collaborations, stories, by actually going in, looking at what people do, filming it, studying it very carefully, and starting to catalog these things.

I'm starting to really understand a lot about the depth we have to go to teach it. It's not enough to just say play nice with the program. You really have to be able to collaborate to do technical work with that. It doesn't mean you have to become a programmer, but you got to really be able to work together. That's a multi-directional street. It's not just you. Everyone on the team has that. I think that's collaboration skills and soft skills are very, very important, and I think it's not good enough to just sing Kumbaya. You have to get into deep, technical work.

Jennifer Bonine: Perfect. Thanks, Rob, for our time. It's amazing. It goes so quickly. If people want to get a hold of you. The best way …

Rob Sabourin: You always ask that question.

Jennifer Bonine: I know. I always want them to know how to find you because I maybe didn't ask what they wanted to know.

Rob Sabourin: E-mail's the best. [email protected] is the absolute easiest way.

Jennifer Bonine: Okay, perfect. There you go.

Rob Sabourin: RobSab.

Jennifer Bonine: RobSab.

Rob Sabourin: R-o-b-S-a-b.

Jennifer Bonine: There you go. At gmail.com. See, I had it embedded in my brain.

Rob Sabourin: There you go.

Jennifer Bonine: That's how they'll find you. Thanks, everyone out there watching. We'll be back with more interviews later. Thanks, Rob.

Rob Sabourin: Thank you. Very good.

Rob Sabourin, P. Eng., has more than thirty-four years of management experience leading teams of software development professionals. A well-respected member of the software engineering community, Rob has managed, trained, mentored, and coached hundreds of top professionals in the field. He frequently speaks at conferences and writes on software engineering, SQA, testing, management, and internationalization. Rob wrote I am a Bug!, the popular software testing children's book; works as an adjunct professor of software engineering at McGill University; and serves as the principle consultant (and president/janitor) of AmiBug.Com, Inc. Contact Rob at [email protected].