In this interview, Andreas Grabner of Dynatrace explains why you need to pay attention to your users' needs when you're doing your performance testing. He shares his performance testing approaches and explores the top problem patterns that you can learn to spot in your apps.
Jennifer Bonine: All right, we are back with more interviews. Stay tuned, we're having a lot of fun this afternoon with our interviews. I'm very excited Andreas is here.
Andreas Grabner: Hi.
Jennifer Bonine: You spent a lot of time getting here, didn't you? You had a long flight?
Andreas Grabner: I did. But do you actually know that I live on the East Coast now? I'm no longer in Austria.
Jennifer Bonine: I know, well, I knew that, but it's still ...
Andreas Grabner: It's still a long flight. It's still a six-hour flight.
Jennifer Bonine: Five and ... I was just going to say. Yeah, it's five and a half, six hours.
Andreas Grabner: Yeah, it is.
Jennifer Bonine: I mean, do you know, from Minnesota I can get to Europe in that amount of time, almost.
Andreas Grabner: Really?
Jennifer Bonine: Because I'm in the middle, so to get there, it's about six hours.
Andreas Grabner: Yeah? To Europe?
Jennifer Bonine: Yeah.
Andreas Grabner: Okay.
Jennifer Bonine: I can get to Europe then.
Andreas Grabner: Really?
Jennifer Bonine: If I go close.
Andreas Grabner: Yeah?
Jennifer Bonine: Yep, so that's long to me.
Andreas Grabner: It is long.
Jennifer Bonine: That's long.
Andreas Grabner: No, you're right, you're right, no.
Jennifer Bonine: And how are you liking the East Coast now?
Andreas Grabner: The East Coast, I like it a lot, I'm just a little worried about the winter.
Jennifer Bonine: Yeah?
Andreas Grabner: You know, Boston winters.
Jennifer Bonine: Oh.
Andreas Grabner: Two years ago, seven feet of snow.
Jennifer Bonine: Yeah, right, I've heard.
Andreas Grabner: I'm not looking forward to that.
Jennifer Bonine: Record snow.
Andreas Grabner: Yeah, so we'll see how it goes this year.
Jennifer Bonine: So yeah, we'll check in next year, see how you're doing after the snow, see if you still like it.
Andreas Grabner: Yeah.
Jennifer Bonine: But we want to talk about some of the hot topics. You have a session right after this.
Andreas Grabner: I do, yep.
Jennifer Bonine: So it's coming up, so we've just got him for a short amount of time so he can run to his session, but why don't you tell us a little bit about what you're thinking about and what your session's going to be about.
Andreas Grabner: So the session, generally, is about performance. I'm a performance engineer, I'm a performance expert—well, not expert, I don't know, but enthusiast, enthusiast.
Jennifer Bonine: Yes, an enthusiast about performance, which is so important.
Andreas Grabner: It is, and I think especially in the time we live right now, where the way we actually should deliver software, in a very performing way, but we are frustrated if something doesn't work and isn't fast enough.
Jennifer Bonine: Right, yep.
Andreas Grabner: But I think, also the way we actually change to develop, deliver, and run software, it has changed a lot in the last couple of years.
Jennifer Bonine: Yeah.
Andreas Grabner: Which, bring the analogy, a couple of years ago people ... If you think about my phone here, I have a camera but I used to have a Kodak camera, whatever I had.
Jennifer Bonine: Yeah, the big, bulky cameras.
Andreas Grabner: Big, bulky camera. We went on a trip like this, we took pictures, the film wasn't full, so we took it on the next trip.
Jennifer Bonine: Yeah. You could keep ... Yep.
Andreas Grabner: Right? So weeks, months later we finally had the film full, we shipped to wherever did the development.
Jennifer Bonine: Yep.
Andreas Grabner: So months later maybe, we get ...
Jennifer Bonine: Yeah, and you don't even remember what was on it at that point.
Andreas Grabner: Exactly, and then you worry, you've got the pictures and it's like, "Oh my God."
Jennifer Bonine: Right.
Andreas Grabner: "I look dead ugly."
Jennifer Bonine: Yeah.
Andreas Grabner: I would wish I could go back, take the picture, which is better, right?
Jennifer Bonine: Mm-hmm.
Andreas Grabner: That's kind of like waterfall development.
Jennifer Bonine: Right, yeah. Yep, you're going back after the fact, trying to recreate, figure out, but you can't recreate, right? Because you're so far down the line it's like you lost that moment in time to do it.
Andreas Grabner: Exactly, and so it's very late feedback, it's hard to change anything. Now in the modern way, the way we do it, we take the selfie and before I click the button I even know, do I need to ... Which side is better? I think this, I have a strange ear, so we go like this.
Jennifer Bonine: So you like that side better.
Andreas Grabner: So I can make sure that I take a picture that is perfect already.
Jennifer Bonine: Yeah.
Andreas Grabner: Then even on the phone, I can apply filters, I can do coloring.
Jennifer Bonine: Right, we can make changes.
Andreas Grabner: So I can optimize things along the way.
Jennifer Bonine: Yeah.
Andreas Grabner: And then what we can do, we can take the picture, upload it on Instagram, on Twitter, whatever you want to do and the cool thing with that is we get feedback right away.
Jennifer Bonine: Right, so people will say, "Take a better one, that wasn't very good."
Andreas Grabner: So that would be the perfect feedback loop, so here's the thing. I believe we move from the waterfall approach to a more continuous delivery approach, right?
Jennifer Bonine: Yeah.
Andreas Grabner: So we want to deliver features that actually make sense to our users, that they like.
Jennifer Bonine: Yeah.
Andreas Grabner: And we need to understand that everything we push out should have already good quality, right from the start, good performance.
Jennifer Bonine: Yep.
Andreas Grabner: And then we need to take the feedback in and see, do they really like it? Yes or no, how can we optimize it? And I think for testers, because we're the test conference now, what does this mean for testers? For testers it means, I believe, they need to be better integrated into delivery pipeline.
Jennifer Bonine: Yep.
Andreas Grabner: We need to find problems faster and not at the end, just shortly before we deploy into production, especially from a performance perspective. Instead of running hourly long load tests at the end of a big, bulky release, we rather run automated, very short performance checks along the pipeline.
Jennifer Bonine: Along the pipeline.
Andreas Grabner: Yeah, shifting performance left, is what we call it.
Jennifer Bonine: Yeah, and that's so awesome. I've heard that more and more at this conference than ever before, of understanding that both security and performance testing shifts left with everything else.
Andreas Grabner: Yeah, exactly.
Jennifer Bonine: That that has to go the direction that we're going with all the other types of testing we're doing and getting that built in and I think great examples of that are … think about it in terms of your website. So if you're a retailer and you make x number of millions of dollars per hour, per couple of hours, if that site goes down all of a sudden because your developers made a change, you've failed to check what that change did to your performance and now you've got huge volume or stress on that system. You thought you were good, you've certified it a couple of weeks ago, a month ago, for a load on that system and all of a sudden now they made a change, didn't realize it impacted it and you've taken the system down.
Andreas Grabner: Exactly, yeah, so that's why I think if you break everything into smaller pieces but also test it in smaller pieces, the risk is smaller. So I mean, obviously there will always be glitches to go through, but then this also means for testers, I think they should not only think about, we deliver something over into production and we tested it. But I think testers, as well as everybody else in the engineering team, should be interested in how does this stuff really work in production? Do people like that picture, yes or no?
Jennifer Bonine: Right.
Andreas Grabner: Because if they don't like it ...
Jennifer Bonine: They have to know.
Andreas Grabner: They have to know, because if they don't like it for a particular reason that means that will influence the next cycle of the feature development. It may influence the way they test a feature in the future because if testers will actually understand how people are using the software then they can also create better test cases that are more realistic to what real users started to really do.
Jennifer Bonine: Absolutely.
Andreas Grabner: So I think that's why testing is I believe, a very key component but it's changing. It's changing into the left and into the right.
Jennifer Bonine: Yep, yep.
Andreas Grabner: If you want to use the word DevOps or not, I don't care, but I think DevOps is something that is for real. It basically means we all together should have invested interest in the stuff that we built as a company is the right thing for our end users.
Jennifer Bonine: Yep.
Andreas Grabner: And if it's not the right thing we need to optimize it.
Jennifer Bonine: Yep, and what do you think about, for example, expectations of the user? So performance also has something around how long does it take for me to load something, or how long does it take for me to upgrade it, or while I'm doing that are you distracting me enough so that I'm not bored and thinking nothing's happening? Are you also checking and looking for some of those things and understanding the expectation?
Andreas Grabner: Exactly, we call it user experience. Especially perceived performance is something.
Jennifer Bonine: Exactly.
Andreas Grabner: It's not only necessarily what's the total page load time, but also if I'm here, do I think something is happening, do I feel I'm still in the flow?
Jennifer Bonine: Right.
Andreas Grabner: Or am I disrupted, as you said.
Jennifer Bonine: Right.
Andreas Grabner: That's why typically what I see a lot of teams do now, they come up with, I think, a user journey. They figure out what is the perfect user journey from we get their attention, they open up the app, until they do something with us.
Jennifer Bonine: Right.
Andreas Grabner: And what's the perfect path and where are the friction points, so let's make sure the friction points are taken care of, yeah.
Jennifer Bonine: Mm-hmm. No, that makes sense, and having that awareness of what the expectation is, is important as well.
Andreas Grabner: Yeah, exactly.
Jennifer Bonine: Right?
Andreas Grabner: Yeah.
Jennifer Bonine: And knowing ... Because users are a little fickle, our attention span has gotten shorter.
Andreas Grabner: Yeah.
Jennifer Bonine: Because we have access to things pretty readily but we're not willing to wait and are you seeing out there as well that the expectation is if my user experience is bad, if the performance is bad, if I don't like how long it takes to load, I'm just going to get rid of it?
Andreas Grabner: Yeah, of course.
Jennifer Bonine: "I'm going to delete."
Andreas Grabner: It's just, like, five clicks away and you are with the next application that can give you a similar experience or a similar service. That's what it is, yeah.
Jennifer Bonine: Right, exactly.
Andreas Grabner: And what we see, I mean, I work for a monitoring company, and what we see a lot now, monitoring has changed into just being perceived as keeping the lights on in production, but really it's giving valuable input for everybody that has a vested interest in that we built the right thing. So for instance, testers look at production and monitoring data to understand which people on the system, which browsers do they use, how do they navigate through the system, where are they struggling? UX experts are using the same way, you analyze user behavior.
Jennifer Bonine: Yep.
Andreas Grabner: It's a lot of cool things that are happening right now.
Jennifer Bonine: Yeah.
Andreas Grabner: And I believe that the quality engineers and testers have always been at the middle between Dev and Ops, and I think they are a perfect—what's the right word? They are perfectly situated to actually pull everybody together.
Jennifer Bonine: Together.
Andreas Grabner: Exactly.
Jennifer Bonine: It's a great place to be.
Andreas Grabner: It is.
Jennifer Bonine: That's pretty exciting for folks out there.
Andreas Grabner: It is.
Jennifer Bonine: So any big things you're anticipating with the session or things you've maybe already heard, talking to people here, questions they have? Anything that's ...
Andreas Grabner: So in my session that I'm going to have in a couple of minutes, I will be talking about how to actually become more efficient with performance checks, how to become a performance hero. I do a lot of performance analysis every day, so it's easy for me.
Jennifer Bonine: Yeah.
Andreas Grabner: We built a lot of stuff in our products and all the other vendors are doing the same thing, instead of throwing too much data on people that they're overwhelmed with, we want to basically put artificial intelligence, whatever you want to call it, on top of it and then say, here are the things, and if you optimize these three things, this is the benefit. Because in the end, we are all short on time.
Jennifer Bonine: Right.
Andreas Grabner: So we need to figure out what do we need to do to have the best impact.
Jennifer Bonine: Yep.
Andreas Grabner: And so that's what we try to do, and that's something I will teach in the class as well. One of the top things that everybody can do to optimize and analyze performance.
Jennifer Bonine: And so for the folks out there who don't get to attend your session, how can they find you?
Andreas Grabner: They can find me online, I guess. They can use their preferred search engine.
Jennifer Bonine: Okay.
Andreas Grabner: Andreas Grabner or Andy Grabner. They can also search if they want for Dynatrace, I have a YouTube channel.
Jennifer Bonine: You have a YouTube channel?
Andreas Grabner: I have a YouTube channel, yeah.
Jennifer Bonine: Oh my goodness.
Andreas Grabner: It's called the Performance Clinic Channel, and I also have a podcast that is called Pure Performance, so you can find it on Spreaker and iTunes.
Jennifer Bonine: You must hate performance.
Andreas Grabner: I must hate it!
Jennifer Bonine: Yeah, you're totally—you have a podcast, you have a YouTube channel.
Andreas Grabner: Yeah.
Jennifer Bonine: So if you want more information on performance testing, optimizing, becoming ... I love that term, "a performance hero."
Andreas Grabner: Exactly.
Jennifer Bonine: That's very cool.
Andreas Grabner: Yeah.
Jennifer Bonine: If you want more information on that, find him, find his podcast, find all that information.
Andreas Grabner: Yeah.
Jennifer Bonine: So that was great.
Andreas Grabner: Yeah, let's make sure we build stuff that actually makes sense for end-users.
Jennifer Bonine: Yeah, I love it, it's great.
Andreas Grabner: Cool.
Jennifer Bonine: Thanks for being here with us!
Andreas Grabner: You're welcome.
Jennifer Bonine: We've got to get him off to his session.
Andreas Grabner: Yeah, thank you, bye.
Performance enthusiast Andreas Grabner has been a developer, tester, architect, and product evangelist for the past fifteen years for several testing and diagnostics companies including Segue, Borland, Compuware, and Dynatrace. Andreas now helps organizations find performance, scalability, and architectural problems in their applications. Speaking at meetups, user groups, and international conferences, he shares his knowledge and teaches others how to recognize and avoid the formerly-mentioned problems.
