In this interview, TechWell speaks with Adam Auerbach, a technology director for advanced testing and release services at Capital One Financial Corporation. At STARWEST 2015, he presented "Putting Quality First through Continuous Testing."
Jennifer Bonine: All right, we are back with another interview. I have Adam joining us. Adam, thanks for joining us.
Adam Auerbach: Thank you for having me.
Jennifer Bonine: What's kind of nice, Adam, is we've talked to a lot of the folks so far in these interviews, and a lot of them have been more in the consulting side of things or the tools side of things, where they have tools that they own. A lot of our folks watching out there are maybe consultants, but a lot of them work for organizations or companies. Maybe give them background on what you do in your organization that you work with.
Adam Auerbach: Sure. My name's Adam Auerbach. I work at Capital One, and I run the last enterprise testing group. We're called advanced testing and release services. I have all the performance testers, test data management, release engineering, and I run our automation center of excellence.
Jennifer Bonine: Very cool. Why that's kind of neat for the folks out there is you're really one of the first people we've gotten to talk to who is practicing in corporate America, in a group, running an organization, facing some of the challenges that they're probably facing, and I know one of the things that you were talking about is testing, understanding more about DevOps and how that plays into their world and what's going on with that. Maybe, if you could give us your take on the importance of that right now, because I'm sure that's on the top of minds for a lot of folks.
Adam Auerbach: Yeah, absolutely. The DevOps movement is here, the drive towards continuous delivery. As a tester, this is now the time where we have to really embrace automation. We have to be able to keep up with developers. More importantly, we have to have a really good understanding of what is that integration pipeline. What are the different tools? How do we get our tests to be part of those tools? What do we have to do differently from a test design perspective? How do we eliminate some of the constraints so our tests can be run multiple times a day, many times, in order to be able to release more frequently?
That's a lot of what I talked about today was this new world that we're in and the fact that testers have to get out of the manual testing landscape and really level-up from a technical perspective. I gave a bunch of books for people to read and some terms that people need to go and research to get more familiar and get ready for that.
Jennifer Bonine: Absolutely. In terms of what you guys have done, some of the management out there and leadership may be facing the same challenge of, there's a transformation or a transition taking place right around skill sets and what's required and the new landscape as you talk about. How are you managing that within your organization? Because I'm sure, not unlike other people out there, you've got a lot of folks who've been around a long time, maybe have done their roles a long time struggling to make a leap or transition.
Adam Auerbach: Yeah. Capital One's no different. We have done a really great job at providing education for people. If you're a manual tester today and you're going to be embracing ATDE, some of the test automation stuff, we have training classes to put people through. We've trained over five hundred people to date this year. We have AWS classes, we have Java programming classes. Really, wherever it is that you want to take your career, we have that ability. We also bring in technical coaches to provide support for the teams, and if at some point you decide that maybe the way that the landscape is changing, you want to do something different, you can do that too.
There's a lot of opportunities for you to either get the training, or maybe you want to be a ScrumMaster or a release train engineer, something more alongside the business, you have that opportunity, too.
Jennifer Bonine: Mentioning the release train engineers, you guys must be then fairly agile as well in your methodology in what you're trying to use?
Adam Auerbach: Yes, we have over a thousand agile teams.
Jennifer Bonine: Wow, that's amazing. Now, in terms of being able to manage all that work flow and the tools that you guys are using to do some of the reporting and stuff—I know before we started, we talked about it a little bit, but maybe talk a little bit about the integration and some of the tools that you guys have found some success with in leveraging.
Adam Auerbach: Capital One really empowers the teams to make their own local decisions around tools, so almost any tool that you would see on the market for tracking stories and doing testing, we're using them. We have tools that we've built to be able to take all that data and be able to provide reports like traceability and progress. We just rolled out our first 4A open-take source, so we have capitalone.io it's our DevOps dashboard called igea. That basically has widgets that we've built that plug into version one or sonar or Jenkins so you have that real-time dashboard of where you're at from a build perspective and an environment perspective.
Then people are contributing to that from an open source perspective. Some of the standard reporting, we don't do that anymore. If you're a leader, if you want to get a status, you go to an S 2, you participate versus someone producing a report for you.
Jennifer Bonine: Yeah. That's great. It sounds like that's really progressive when you think about large corporations that I talk to all the time, they don't seem to be there yet and it takes a long time for them to evolve and shift. What do you think helped you guys to make such leaps into, not only cutting edge theories around what you should be doing and how to evolve but guiding your teams and allowing them to be self-directed and picking tools, embracing some of these new technologies and things?
Adam Auerbach: It's really been a top-down, bottom-up approach. Leadership is really great and supporting innovation and allowing people to take some risks. They shy away from saying, "This is how you have to do it." They really let things happen but are really supportive when we see something good is happening and how to nurture that. Then, we do a lot of things organically to be able to build that ground swell. We do internal open-spaces to drive adoption for automation. Last week, we did our first software engineering conference. We had 1,000 people from Capital One come to do their own talks, to talk about micro-services, and AWS, and big data. Then we had Gene Kim and Jez Humble there to talk about continuous delivery and DevOps.
Both of those approaches really create this incubator for innovation and ideas and it really happens very ... A lot of what we're doing has happened over the last three years, so again, for an organization of that size, those two things really ...
Jennifer Bonine: That's amazing, because you look at a lot of large organizations and they're struggling with some of the same challenges, but they're not quite ready to move or empower to move to some of those new things. When you think of Capital One and you think of the financial industry in general, you think more slower moving a lot of times. It's really refreshing to hear that you guys are really on the cutting edge with a lot of those concepts.
Adam Auerbach: The mobile apps, the whole digital revolution, you need to be able to keep up. You need to have really good technical talent and we want to be known as a technology company and that's how we can get the best high-quality products to our customers. You can't be slow-moving and slow to adopt. You really have to be out there and part of that movement.
Jennifer Bonine: Where you there before this starts or did you get to see it start where people were saying, "Here's some stuff we need to do and here's how we need to keep up with mobile and the apps and everything that's coming at us"? I think there may be folks out there saying, "I would love to have a situation like that where our senior leadership is supporting it. Where the people ground swell and say we want it and it's working together." How did you guys ... Any suggestions for people out there that may not be in that situation where their senior leadership is risk adverse? There's a lot of industries where people say, "Oh, we're not sure because we can't fail or it can't be wrong."
Adam Auerbach: The biggest thing I'd say is go try it. Find something that is within your control, that you can go and try it and experiment and be able to bring that experiment to leadership and get their support. When we rolled out ATDE, we did some local experiments. We brought that to leadership. Leadership said, "This is a great idea. We're not sure if it's going to scale for everybody" and then we said, "Okay, that's fine. Can you give us somebody on your teams that we could work with?" Then we did further experimentation and then as we start having more success, then we go back to them and then you just get this momentum that starts to build.
You get people on the ground that are excited. Leadership starts to realize this can happen across all our domains.
Jennifer Bonine: Want it to.
Adam Auerbach: Exactly. To me, it's around experimentation and being able to socialize that to be able to get support and trying to find accountability to help drive that because if I'm the only one driving, it doesn't scale for a large enterprise.
Jennifer Bonine: Yeah. How does it work for you guys, just structurally, just so people understand? You said the last centralized testing group ... Do the teams, you said 1,000 agile teams, do you have a centralized group of people but they get distributed out to various product lines, various scrum teams?
Adam Auerbach: Yeah. It's a very hybrid model. For performance testing for example, they are assigned out to all the lines of business and they are on teams and the product owners of those teams are the ones determining the stories, the work for the performance testers for example. As a centralized group, my job is to drive how we do it, to make sure that from a best practices perspective, if we're using Jenkins for example to do the build, then how do we integrate our performance tests into Jenkins so they're able to run.
That's the things ... From a centralized perspective, how do we drive the service for it and then locally, how do they have their autonomy to be able to drive what they're working on.
Jennifer Bonine: Yeah. That sounds like a great model. It's amazing. We're already out of time. It goes so fast. If they want to find you or have more questions, what's the best way for people to find you or reach out to you?
Adam Auerbach: Twitter. I'm on Twitter. @Bugman31.
Jennifer Bonine: Bugman31.
Adam Auerbach: Tweet me.
Jennifer Bonine: They'll be able to find you.
Adam Auerbach: Yes.
Jennifer Bonine: I know we probably just scratched the surface with what some people want to hear or things that they're thinking about, that you've piqued their interest. Perfect. Thank you for joining us.
Adam Auerbach: Thank you.
Jennifer Bonine: Thank you.
Adam Auerbach is the Technology Director for Advanced Testing and Release services for Capital One Financial Corporation, a diversified bank with 65-million customer accounts worldwide and more than 900 branch locations. Adam is responsible for Capital One’s enterprise performance and automated testing departments as well as enterprise release management. Since joining Capital One, he has provided leadership for the agile transformation of their quality assurance group and led the enterprise adoption of DevOps and ATDD. Before joining Capital One, Adam was with Chase and other financial and insurance companies, in various leadership positions focusing on quality and agile practices.
