STAREAST 2015 Interview with Dan North on Deliberate Testing in an Agile World

Jennifer Bonine: We are back with our virtual interviews. Now we have Dan North with us. Dan, thanks for joining me.

Dan North: Hello. Thank you for inviting me.

Jennifer Bonine: Glad to have you here. Some of you may have seen Dan yesterday. You had a keynote, you've had some sessions this week, you did a lightning keynote. We've seen a little bit of you.

Dan North: I've been a busy boy over the last couple of days.

Jennifer Bonine: Yeah. They're making you work for your money this week.

Dan North: They really are. I enjoy it, so luckily that works okay.

Jennifer Bonine: Yeah. Why don't you, for those folks who've seen you but maybe don't know your background, give a little bit about your background and what you've done and how you've ended up in the role that you're in now. You're Dan North & Associates, so you are your company.

Dan North: Yeah. Dan North & Associates is a model I got from some other independent consultants. I'm independent.

Jennifer Bonine: Yep.

Dan North: The associates model is, I know lots of people who are way better at things than me, so I'll work with clients and when I need to bring in people who can do certain things, then I reach out to these people and they come in and they help. That's the kind of associates thing. It's more of a network of useful friends rather than a big ...

Jennifer Bonine: It sounds impressive.

Dan North: The whole of Dan North & Associates is basically sitting at this table.

Jennifer Bonine: There you go. See? That's awesome, that's impressive. It sounds great, though. Now, your background, were you always in testing? Did you do development?

Dan North: I'm not in testing. I'm a complete interloper here. I'm fascinated by testing, and I can honestly say some of my best friends are testers.

Jennifer Bonine: Yep.

Dan North: I've been in IT and technology for about twenty-five years as a developer and then as a coach and then getting into organizational things. In 2002, I joined ThoughtWorks, who are an international agile software company, consulting company. I was their tech hire number one in London… woo! They've been going for like ten years or something in Chicago and various other places and they opened a London office. I accidentally ended up working there. A good friend of mine said, “You should work there and sent them my resume.”

Jennifer Bonine: Then you worked there.

Dan North: Then I ended up working there. Most of my career has been a series of flukes.

Jennifer Bonine: Yeah.

Dan North: A series of happy accidents.

Jennifer Bonine: I find that a lot.

Dan North: Yeah. I was there for eight years. That was when I started doing things like this, speaking at conferences and that kind of thing. I was a developer, I was very early adopter of agile methods and I ended up coaching a lot of teams and a lot of developers particularly. That's where the behavior driven development thing came from. I was trained to coach TDD and I kept on getting stuck with the word test, or developers would get stuck. We're saying, “We're going to write a test and then we're going to write some code.” The programmers are saying, “We don't write tests, we've got testers for writing tests.” On the other side, the testers are going, “Don't let them write tests. They're programmers, don't let them write tests.”

I just took the word "test" out of it altogether and started talking about behavior. That's really what you're doing, you're writing an example of behavior and then you implement that behavior and then you write the next example. I was calling it coding by example, so then saying that the behavior drives the development. You take the word test out and suddenly people get it. There's nothing different, they're still doing the same thing.

Jennifer Bonine: Same thing.

Dan North: They're just not hung up on this word test.

Jennifer Bonine: Yeah.

Dan North: That happened. I was there for, as I said, about eight years. End of 2009, I went off and joined a small trading firm for about two and a half years. I joined a very small team. While I was there having been a hand wave-y, conference agile guy for some years now, I found everything I knew about software development turned on its head in a really unsettling way. These guys were writing, I would say, the best quality software I've ever seen any team produce, and they were breaking all the agile rules.

Jennifer Bonine: Really?

Dan North: I was just thinking, “This is wrong and weird. I have to stop them.”

Jennifer Bonine: But it's working.

Dan North: I have to stop them.

Jennifer Bonine: But it's working.

Dan North: It's working, right? Is it a complete fluke, or is there something that they're doing that I can reproduce and share and tell people about? I was in the team, but also studying how we were doing things for about a year. Then I started talking about that. That's where my head's at now. I'm calling it Software, Faster.

Jennifer Bonine: Software, Faster.

Dan North: It's what I think happens after agile methods.

Jennifer Bonine: The evolution.

Dan North: Yeah. All the agile stuff came out in the ‘90s. People are going on brand-new, newly minted Scrum training classes now. This is a 1990s methodology.

Jennifer Bonine: Exactly.

Dan North: It's like twenty-something-years old, right?

Jennifer Bonine: Exactly.

Dan North: XP is nearly twenty years old. All of these methods were designed to take businesses that were working from years down to months, we can release in months, which was impossible. We go from multi-year programs to we can release in months, which was great twenty years ago.

Jennifer Bonine: Exactly.

Dan North: Now I can spin up—like Jason yesterday—I can spin up ten thousand virtual servers instantly.

Jennifer Bonine: Right.

Dan North: Right? A two-week planning horizon doesn't make sense.

Jennifer Bonine: Right.

Dan North: What happens next?

Jennifer Bonine: Right.

Dan North: That's what I'm excited about at the moment.

Jennifer Bonine: Wow.

Dan North: Then I end up here because I know the organizers here, they also do another series of conferences around—they're called Better Software and Agile Development Practices. I spoke at another couple of those. I was invited to talk to testers about testing, which is crazy.

Jennifer Bonine: I know. They let you interlope in.

Dan North: I know. I know. Talk about impostor syndrome.

Jennifer Bonine: Right?

Dan North: I was standing there thinking that these guys are going to tear me apart. They didn't.

Jennifer Bonine: They're all going to know.

Dan North: They were lovely and they were super friendly. I feel safe now. Yeah.

Jennifer Bonine: Yeah. That's why I asked you about it in the beginning. Some of the folks out there may not know, right? They may have thought, oh, Dan's one of us.

Dan North: "He's a tester." No, I'm not. No, I'm not. I'm not. I tell you what, I am neither smart enough nor devious enough to be a tester. Testers are some of the smartest and most devious people I know. I like that.

Jennifer Bonine: Yeah. They're great for that. You talk about…

Dan North: Come here. Break stuff.

Jennifer Bonine: Help me do this.

Dan North: Yeah. Help me break stuff.

Jennifer Bonine: You talk about putting people first. Tell me more about what you mean when you talk about putting people first in your ...

Dan North: I talk about putting people first. I'm a huge fan of systems thinking, systems theory. The idea of systems thinking is essentially everything in a system is connected. You don't look at things in isolation, you look at relationships between things, they're usually more important than the things, or more interesting than the things.

When you start seeing emergent behavior in a system, it's usually because of the structure of that system. If you want different behavior, you need a different shaped system. Most systems are made up of people, have people in them. Pretty much software development is a people thing, business is a people thing. Organizations are people things. They're groups of people. Understanding the social construct of those people and how they interact in the system of work is, I think, one of the biggest levers for being able to affect that system of work.

Jennifer Bonine: Exactly.

Dan North: As an example, I've got a software delivery process and it's backed up at testing. It's always backed up at testing.

Jennifer Bonine: Right.

Dan North: We could shout at the testers louder. That's working for us.

Jennifer Bonine: Yeah. That'll help.

Dan North: That'll help. We could throw more testers at it. We could offshore the testing.

Jennifer Bonine: Right.

Dan North: Let's go to somewhere where people are less expensive and let's get more … or we could look at the system of work and say, we have built a system that presents that. We go and talk to the testers and we say, what do you guys spend a lot of your time doing? We spend a lot of our time doing this stuff. We could move that work upstream and make that a programming activity. As part of programming, I also introduce this testing. That's the thing that happens.

That's going to slow down programming, but it doesn't matter, because that's not where the bottleneck is. It'll increase flow because now there's less testing to do so my testers get through stuff that they need to do faster. The actual work item goes through the pipe faster. Unless you're looking at that holistically, what it looks like you're doing is slowing down programmers.

Jennifer Bonine: Right.

Dan North: Right. When you look at it as a system of human beings and what those human beings' needs and aspirations are and how they work together and how they can collaborate to help each other, suddenly you're seeing it through a whole different lens.

Jennifer Bonine: Yes. Exactly.

Dan North: I find that very exciting.

Jennifer Bonine: Yeah. You can talk about, you have the best processing, you have the best tools, but if you haven't architected it to make the people successful doing it, automatically it falls down at that piece of it. You can have the other pieces but you really need the people working together and understanding the system of how they integrate into that.

Dan North: Absolutely. This is the heart of lean operations. Lean organizations have managers. This idea of it's all self organizing, we don't have managers, lean organizations have managers. The role of the manager is to manage the process, it's to manage the system of work. We hire grownups and we trust them.

Jennifer Bonine: Yep. Absolutely.

Dan North: Right? That's what you do. You hire grownups, you trust them. I don't need someone like … a manager in a lot of organizations is some kind of hybrid of a policeman and a parent.

Jennifer Bonine: Right.

Dan North: Have you tidied your room?

Jennifer Bonine: Right. Exactly. Then issuing punishment.

Dan North: Issuing punishment, exactly. Punishments, incentives, mostly punishments and enforcing and all that kind of stuff.

Jennifer Bonine: Yeah.

Dan North: Whereas Edward Deming's, one of the founders of lean, his famous quote, he said, "People want to do a good job."

Jennifer Bonine: Absolutely. Fundamentally, they want to do a good job.

Dan North: People come to work and want to be able to succeed. Your job as a manager is to create an environment in which they can be successful.

Jennifer Bonine: Absolutely.

Dan North: Nothing else. If you do that, good things will happen.

Jennifer Bonine: Yep.

Dan North: It's exactly the same. No one wants to write crappy software.

Jennifer Bonine: No.

Dan North: No one wants to do poor testing. No one wants to produce massive requirements documents. That's the system of work we've created. If we want different behavior, we need a different system of work.

Jennifer Bonine: Yep. Exactly. No, I completely agree. You also talk about something called deliberate testing in an agile world.

Dan North: Yes. That was my talk yesterday. I use the word deliberate a lot. Dorothy, as she was talking about this in her keynote this morning which I just thought was gorgeous, she's saying automation isn't good or bad. Automated testing isn't good or bad. It's a thing. How you use that thing determines whether it's good or bad. It's all about the context you use that thing in.

Jennifer Bonine: Absolutely.

Dan North: The deliberate part of deliberate testing is, “I've only got so many testers, I've only got so much stuff I can test, what's the most appropriate way to solve that?” There's no, “Do this.” There's no, “This is how you do testing. There you go, we're done now.”

Jennifer Bonine: Yeah. There's the book.

Dan North: Yeah. There's the book, right? It's all good. Then saying, it depends is that consultant shrug thing. You need something a bit more substantial than that. The deliberate thing is to say, “If you understand the various different stake holders who care about this application…” there's a chap called Mark McNeill who does agile UX type stuff in the UK and he has this lovely phrase that I use a lot. He says, “In terms of stakeholders,” he says, “stakeholders are people whose lives you touch.”

Jennifer Bonine: Okay.

Dan North: I just went, “Wow.” People whose lives you touch when you deliver software, so clearly end-users, you touch their lives, right?

Jennifer Bonine: Yep.

Dan North: Then there's the administrators of that software, there's the operations team, you touch their lives. There's the security guys, compliance, audit. There's other developers, other testers. All of these people are going to be impacted by your software.

Jennifer Bonine: Absolutely.

Dan North: Probably end-users more than anything if you're writing internal business systems. You are designing how someone's day is going to go for the eight hours that they are at work. They come in and it's on you as a development team that that person comes into work and goes, “I really like using this system, it really helps me do my job,” or “this sucks and it's going to continue to suck because when I complain no one cares.”

Jennifer Bonine: Yeah. They don't listen.

Dan North: That whole spectrum, that's open to you. Deliberate testing says, “Let's understand who all these stakeholders are, let's understand what their goals are and what their appetite for risk is.” Which parts of it are they going to be most concerned about? Let's use that as a lens to overlay onto the application we're building, both in terms of the components it's made up of and the various flows through it and the user journeys. Now we can have a much more nuanced conversation about what to test, where to test, how much to test, where the diminish in return is, how to make tradeoffs. Now I can be a lot more deliberate and a lot more confident that the kind of testing I'm doing is the most appropriate testing.

There's going to be tradeoffs, right? For all the time I'm testing this, I'm not doing all the other stuff.

Jennifer Bonine: Right.

Dan North: I need to have a pretty good story about why I'm not doing all the other stuff.

Jennifer Bonine: Yeah. Exactly.

Dan North: That's where the deliberate …

Jennifer Bonine: Yeah. I deliberately am saying, these are the areas I want to focus on.

Dan North: These are the most valuable things, this is the way to pay down the most risk, or this is the way to have the best understanding of where this application may be: good, bad, safe, unsafe and in different context. If I'm building a pizza ordering system versus the timing circuit in a pacemaker, I should probably have a fairly different approach to testing.

Jennifer Bonine: Yeah. Exactly.

Dan North: The mental model is still the same. It's where it’s appropriate to test, who are the stakeholders? I don't know, if I'm a tester, I probably don't know compliance, tax law, security. What I'm really good at is asking the questions to find out where those people think I should be doing testing.

Jennifer Bonine: Exactly.

Dan North: It's being able to condense all of that in and turn that into a program of work that says, “I'm confident that I've done the right testing.”

Jennifer Bonine: Exactly. That totally makes sense. Another piece of what you talk about is reducing ignorance. When you talk about reducing ignorance, help us understand the thought process there.

Dan North: It's a very specific type of ignorance. There's a thing called second-order ignorance.

Jennifer Bonine: Yes.

Dan North: Which is, “I don't know what I don't know.”

Jennifer Bonine: Right.

Dan North: The stuff I do know what I don't know, if you look at any kind of software delivery project or schedule or whatever, there's a risk log. There's a risk log and these are all the things that are currently at risk, currently my areas of concern. All of that is called first-order ignorance, that stuff that I know that I don't know. I know there might be surprises. Second-order ignorance are the things that aren't even on that list.

Jennifer Bonine: Yeah. That we don't know.

Dan North: That are going to step out and bite you. Second-order ignorance is what you don't know is that these servers that you ordered are going to be late because what you don't know is the vendor changed account manager. The old account manager left and the new one joined and there's a gap. They didn't know about your order ... You'll find that out just when it's too late.

Jennifer Bonine: Yep.

Dan North: At the same time, you'll also find out that the vendor isn't very reliable, and that the price has gone up and a whole bunch of other things you didn't want to know. That's the stuff that you're second order ignorant about.

Jennifer Bonine: Yes.

Dan North: The deliberate discovery piece and the reducing ignorance is, can we change how we do work so that we're likely to discover that stuff sooner? I can't know it all up front. Some things are just unpredictable, some things will just jump out and bite you. For all the other things, can I structure my work, can I sequence the work so that I'm more likely to surface dragons sooner? I don't need to solve them, I just want to move them from second order ignorance to first order ignorance.

Jennifer Bonine: First. Yep.

Dan North: I want to now know that that thing could jump out and get me. Then I can start to reason about it.

Jennifer Bonine: Right.

Dan North: I think an awful lot of project failures or project delays or cost overruns are when we get near the end and then suddenly we discover all the dragons. Essentially, we front-loaded all the easy stuff because it looked good and it could pay off and we've left all the risky, uncertainty areas there. We need to interact with other people that maybe we don't have access to. We suddenly discover…

Jennifer Bonine: In the end.

Dan North: Too late, to the end. Then suddenly, A) we know we're going to overrun and B) it's completely out of our control.

Jennifer Bonine: Right.

Dan North: It's not things we can do anything about. If you front-load all that, there's an extreme of this, which all happens in lean start up and lean enterprise is when you front load it, you end up discovering that the thing isn't even viable, which is the fading fast idea. We did a little bit of work and we discovered during that little bit of work this is going to cost you four times as much as you thought. If it's still worth doing for four times as much, we'll carry on. If it isn't, it only cost us that much and that much time to learn that.

Jennifer Bonine: Yeah. To figure it out. Absolutely.

Dan North: Which is a pretty good investment.

Jennifer Bonine: Yeah. That's amazing. Good concepts. We are already out of time. It goes so fast.

Dan North: Oh no.

Jennifer Bonine: I know. It goes so fast.

Dan North: It was a absolute pleasure speaking with you anyway.

Jennifer Bonine: You too. If people wanted to get a hold of you, if I didn't ask the questions, they all out there want to hear from you. There are probably lots of other things. How can they do that?

Dan North: I'm on Twitter as @tastapod, T-A-S-T-A-P-O-D, for amusing and very old reasons. I'm at dannorth.net, is my website.

Jennifer Bonine: Dannorth.net?

Dan North: Dannorth.net. Yeah.

Jennifer Bonine: They can find you there?

Dan North: Dannorth.net/blog is my very originally named blog that I occasionally post on.

Jennifer Bonine: Perfect. That's how you find Dan. Dan, thank you so much, it's been great.

Dan North: Thank you.

Jennifer Bonine: This was awesome.

Dan North: Thoroughly enjoyed it.

With more than twenty years of IT experience, Dan North uses his deep technical and organizational knowledge to help CIOs, businesses, and software teams deliver quickly and successfully. Putting people first, Dan finds simple, pragmatic solutions to business and technical problems, often using lean and agile techniques. He originated Behaviour-Driven Development (BDD) and Deliberate Discovery, published feature articles, and contributed to The RSpec Book: Behaviour Driven Development with RSpec, Cucumber, and Friends and 97 Things Every Programmer Should Know: Collective Wisdom from the Experts. Dan is a frequent speaker at technology conferences worldwide and occasionally blogs.