The Test Lab: STARWEST 2015 Interview with Bart Knaack and Wade Wachs

Jennifer Bonine: All right, we are back with our next virtual interview. I am here with two gentlemen from our Test Lab. You may have heard Mike mention Test Lab earlier in his interview as one of the really neat things here when you're live at the conference. For those folks out there, why don't you guys do a little intro of yourselves and Test Lab, just so people know what I'm talking about, because they obviously see your lab coat. Very impressive. Let them know what that's all about.

Wade Wachs: I'm Wade Wachs. I've been involved in the Test Lab ... Bart and I actually first met at my first STAR conference five or six years ago, and I've been hanging around ever since. Test Lab's a spot to come actually try testing. We sit and talk about it and learn a lot of good things about it. It's a space to actually get your hands dirty and give it a shot—try out the techniques you're learning and get your hands on it.

Jennifer Bonine: That's awesome. Bart, you're a regular.

Bart Knaack: I'm one of the regulars.

Jennifer Bonine: You're a staple.

Bart Knaack: I'm part of the furniture by now.

Jennifer Bonine: Right.

Bart Knaack: I've been doing Test Lab for the last eight to ten years, I guess. One of the big things in the Test Lab is that it's really a place to learn by doing and not by just sitting around and listening to other people tell you stuff.

Jennifer Bonine: Right.

Bart Knaack: It's kind of a tester sandbox; anything goes. Whatever you want to do, we can try to cater for it. We have games, we've got puzzles, we've got robots to test, like this little fella here. We've got some systems on the test, and like Wade already mentioned, we also have Humanitarian Toolbox to test. Maybe, Wade, you could tell more about the people at home that could also participate in that.

Wade Wachs: Sure. One of the main apps that we're testing in the Test Lab this year is Humanitarian Toolbox's crisis check-in app. It's a tool for managing volunteers in major disasters. One of the things that happens a lot of times is, they're running a disaster, they've got a big natural disaster—you have thousands of volunteers that come to help, but those volunteers take management and resources to train; managing that pool of resources, running an app to help that so that we can actually get help to the people that need it. For those that are at home that would be interested in joining along helping in the Test Lab, if you search for "HT Box," Humanitarian Toolbox, on GitHub and go to their crisis check-in app, there's all the details that they would need in the "Read Me." Feel free to log in with your GitHub account and start reporting issues. We'll have prizes available to those either here or remote as well.

Jennifer Bonine: That's awesome, so you guys get to play too.

Wade Wachs: Whoever has the most interesting bugs, we're going to look through those at the end of the conference and ...

Jennifer Bonine: Awesome.

Wade Wachs: ... give that feedback to the developers as well. They really appreciate that.

Jennifer Bonine: That's great. It'd be great if we had one the winners at the end actually be one of our virtual participates.

Bart Knaack: That's if they put in some effort. We're not going to just give away the prize.

Wade Wachs: It's true.

Jennifer Bonine: No, they need to put in the effort, but it would be really cool if that happened. For those of you out there, I think that's a challenge, that you guys get involved and represent the virtual crew out there and do a good job and put in some effort.

Wade Wachs: If you throw any issues, put in that you're virtual so that we can make sure that we know that you're not here at the conference.

Jennifer Bonine: Yeah, so you know that. Yeah, so put that in, that you're a virtual participant. That's great.

Wade Wachs: That'd be great.

Jennifer Bonine: Yeah, because then you can actually get involved in it, even though they're not able to be here and walking around with everyone else.

Bart Knaack: Maybe for next year's test apps we could put in some virtual machines as well so that people can test from home.

Jennifer Bonine: That would be neat.

Bart Knaack: And really participate. Let's see if we can do that. That will be interesting.

Jennifer Bonine: Let's see if you can, Bart. That would be awesome. I think it would be good to get them involved. I don't know if the folks out there know, but there's a quite a few of you that are virtual, so we get hundreds of people that are actually watching us right now in the virtual.

Bart Knaack: Say "Hi!"

Jennifer Bonine: You're famous.

Bart Knaack: I suddenly got nervous, I don't know hwy.

Jennifer Bonine: I know, sorry, I didn't mean to make to you guys nervous.

Bart Knaack: I kind of feel the rock star status. People want to take pictures with you in the Test Lab, and I'm like, "What?" It's cool.

Jennifer Bonine: That's awesome, though. That is awesome.

Bart Knaack: I think it's the stylish coat that does it.

Jennifer Bonine: It is the coat. You look very official. You guys look very official and authoritative. It is, it's a neat thing, I think, that we have here that when you talk about ... when you come to the conferences or you watch, it's a lot of listening to other people talk about stuff or hearing concepts. You guys actually give them an opportunity to apply it.

Bart Knaack: Yeah.

Jennifer Bonine: And see how it works in real life.

Bart Knaack: We've got people in the Test Lab that can answer questions. If we do not have the expertise, we'll try to find it. I think that's cool. I really like the notion, but then again I started it a number of years ago. It's cool, it's cool.

Wade Wachs: There's something to be said, too, about applying things out in your day job. You get back to your day job and there's a lot of political whatever to try and implement new ideas. Test Lab gives you a spot to try them out, quickly, without having to convince all your coworkers that it's the right thing to do.

Jennifer Bonine: Right, exactly.

Wade Wachs: Which I think is great.

Jennifer Bonine: Yeah, no, it's awesome. Do you guys see really good participation? Now that you guys have been around awhile—the Test Lab's not brand-new, it's been here awhile—do people kind of recognize it, especially those alums that have been around before, and they kind of gravitate towards it?

Bart Knaack: It's always a mixture. There's always the few people that are really interested and keen, and there might be newbies, there might be alumni, but it really doesn't matter. There's a number of alumni that we cannot get out of the Test Lab.

Jennifer Bonine: They just want to park there.

Bart Knaack: They just come back in and come back in and come back in. They're like, "Do you have anything new for me to test?" "Yes, there's always something new for you to test." This year we put the puzzles in, a lot of puzzles. That kind of attracts a lot of people. In that sense, I think that's a neat move to do that. Test Labs have been popular and unpopular over the years. I always say, the less people are in the Test Lab, the more people are at the actual tracks. So that's a compliment to the conference management if they have an empty Test Lab.

Jennifer Bonine: Right, that means they're in the sessions.

Bart Knaack: They're in the sessions, and that's good. Either way, I'm always winning.

Jennifer Bonine: It's always a good thing, either way. If people were to come to a conference—obviously not this one—is the Test Lab at all of the TechWell conferences, or just the STAR conferences?

Bart Knaack: I guess by now, it's kind of a thing that they have. Every year, it's not always the same people that do it, because we also have other people that can do it. James Lyndsay can do it, Paul Carvalho can do it. There's other people that can also participate, so Wade and I will not be there every time ...

Wade Wachs: Thank goodness.

Bart Knaack: (laughter)

Jennifer Bonine: You guys don't travel around at all the Test Labs?

Wade Wachs: You don't necessarily always want to come see both of us.

Bart Knaack: There's always a another mixture of people that are running the Test Lab, but the basic idea of the Test Lab will be the same.

Wade Wachs: Yup.

Jennifer Bonine: It's always the same. We mentioned there are prizes at the end for this. What is that based on, or how do you guys judge objectively?

Bart Knaack: Why should we do that objectively?

Jennifer Bonine: Oh, okay. (laughter)

Bart Knaack: It's just purely subjective. We like your face, you'll get the prize.

Jennifer Bonine: If they like you.

Bart Knaack: Basically, we look at the most original bugs that have been found, or maybe someone has found a lot of bugs—that's also a pro, if someone does something out of the box.

Wade Wachs: Sometimes we get to see the effort that people go through to find things, too. There may be a particularly difficult, hard-to-reproduce bug which—especially going in the Test Lab, where people are walking through for ten or fifteen minutes at a time—taking the time to sit down and dig into a particularly difficult issue for an hour and really report it and describe it well. We've seen that rewarded as well.

Jennifer Bonine: What's one of your favorite bugs that you guys have ever seen, or favorite things that you've actually seen?

Bart Knaack: Actually, someone—it happened yesterday—I have a game in there called Scrabble Flash, where you just watch what you can move around and play around with, and someone got it into a state which was erratic; it just didn't do anything anymore, what it should have been doing, and I was like, "How did you do it?" And she was like, "I don't know!" (laughter) It was really a cool bug. She was playing around with it, she was really shoveling it, and at some point she got into this strange state. Even though it's not reproducible, I thought it was a really cool bug because it's like, "Oh my God, what's happening here?" That's a game that's out there in the market.

Jennifer Bonine: Yeah, in the market, so other people can use it. So clearly there's a way to make it so it won’t do anything. Is there a reset on it?

Bart Knaack: Yeah, luckily there is.

Jennifer Bonine: Okay. There's a reset, for those of you out there that have it, if you play around with it and that happens, you can reset. That's the good thing. Hopefully there's a way to get back from it. I bet you guys do see really interesting things and people get engaged in very different ways.

Bart Knaack: Some people really dig in and are just so focused, so concentrated on what they're doing, and it might turn up with very crazy ideas, but as long as you see the passion, as long as you see they're really into it, it's brilliant. It's very good to see.

Jennifer Bonine: Yeah, and just watching them kind of explore in their own way how they want to engage. You guys are there really just to moderate and help?

Wade Wachs: Help and guide and answer questions and keep everything going for people to have a space to do it.

Bart Knaack: We are the stage master, they are the rock stars. That's how it is.

Jennifer Bonine: Those are the people that, at the end, you'll recognize too for all their efforts and things like that. You said puzzles are new this year. This guy's been around a while.

Bart Knaack: Yeah.

Jennifer Bonine: What's this guy, what is his objective that he's supposed to do?

Bart Knaack: All right. That's a good question, because this guy doesn't really have an objective; it has some functions.

Jennifer Bonine: Okay.

Bart Knaack: If you put it on a colored map it will behave based on the pattern is sees underneath, then people have to find out what the pattern is and then describe that. In that sense, it's kind of a reverse-engineering robot. You have to reverse-engineer the program that's in there. Next year, I promise—cross my heart, hope to die—I promise that it will also have an adaptive cruise control built in. We can do adaptive cruise control checking as well; that's going to be cool.

Jennifer Bonine: How is that getting built in? Do you build that in?

Bart Knaack: I'm going to build that in.

Jennifer Bonine: You're going to build that in.

Bart Knaack: Yeah, yeah.

Jennifer Bonine: So on the side, besides doing fabulous Test Lab stuff, you also engineer robots.

Bart Knaack: Yeah. Well, actually, I basically build Lego robots and then modify them.

Jennifer Bonine: Modify them, souped-up Lego robots that we're building in the Test Lab to use. That's awesome. So, next year if you come, you'll get to take advantage of that.

Bart Knaack: Yeah.

Jennifer Bonine: If you're here. Another good reason to be here next year with us. Well, it goes so fast, thank you both for being here with me.

Wade Wachs: Thank you very much.

Bart Knaack: You're welcome.

Jennifer Bonine: Thanks for explaining Test Lab. Like I said, I'll put the challenge back out; I would really love to see one of our virtual participants get involved. Submit those bugs and you could be one of our winners at the end of the contest.

Bart Knaack: That would be great, that would really be great.

Wade Wachs: That would be.

Jennifer Bonine: It would. We'd like to see that. So we challenge you all to do that. We will see you guys after the next session with a few more interviews. Thanks, you guys, for being here.

Wade Wachs: Thanks so much.

Bart Knaack: You're welcome. Take care.

Test advisor at Professional Testing, Bart Knaack has more than twenty years of experience in testing and test management. Bart has contributed to testing projects and processes in roles of test manager, test data management team lead, and process improvement agent. He has coached testers, built up test teams, and developed processes in test, test support, and overall quality management and improvement. Bart is a frequent speaker at conferences, universities, and colleges. In 2009 Bart and James Lyndsay cofounded the TestLab, an exciting part of today’s STAR conferences.