Getting Started with Security Testing: An Interview with Jeff Payne

Jennifer Bonine: We are back with more virtual interviews. Hopefully you all stay tuned for the next two hours of exciting folks we're going to talk to. Jeff's kicking it off. Jeff, thanks for being here.

Jeff Payne: Thanks for having me.

Jennifer Bonine: For those of you that don't know Jeff, I've talked to him a few times in the past. Some of the things I think would be fun to talk about for the virtual audience is the things you're involved in. Obviously big guru around security, so I always like to talk about security with you.

Jeff Payne: Sure.

Jennifer Bonine: You always know what's going on there, and then the leadership summit.

Jeff Payne: Great.

Jennifer Bonine: For the folks that aren't here Monday through Thursday, Monday–Tuesday is a lot of longer tutorial sessions, which you give some of those. Yesterday and today are the concurrent sessions, which are the shorter sessions. You get little snippets of information just to kind of get you excited about a topic, and you can delve in further later. What starts tomorrow is the leadership summit.

Jeff Payne: Leadership summit.

Jennifer Bonine: For the folks out there that haven't heard of that, or haven't attended, maybe just give them a little insight into the thought process behind creating that leadership summit, what's the format of it because it's a pretty format.

Jeff Payne: It is a cool format. Where it came from was there was a lot of feedback in the STAR conferences that people wanted more around leadership, more around management, how do I deal with myself as a leader, my people that I have to lead and my boss that you should be leading, right?

Jennifer Bonine: Yeah.

Jeff Payne: You need to lead in all directions.

Jennifer Bonine: Exactly.

Jeff Payne: TechWell decided it made sense to start a day just focused on leadership from a quality and testing perspective.

Jennifer Bonine: Exactly.

Jeff Payne: What we do, we do it at STAREAST and STARWEST, is we focus on bringing in a couple of speakers, senior veteran software people, often with a testing background, software development background, who have been leaders to impart their wisdom about leadership and what they found works. That's about half of the day. The other half of the day we spend in small groups where we're tackling different leadership challenges. We ask before we start everybody to give us their thoughts on what keeps you up at night as a leader. What's most important?

I take those topics, I figure out which ones are the most requested topics, so the top of mine ones. We set up tables and everybody goes to the table they want to talk about and they problem solve and brainstorm, and myself and the other speakers facilitate those tables. We come up and leave there with a lot of ideas about how to address those problems. All of that gets written up for the participants and sent out to them later so they have a complete catalog of everything that was discussed.

Jennifer Bonine: I think it's great. What's nice about that is the folks, as you said, attending tonight at the opening reception that they attend, will be putting together those thoughts around topics that they want, submitting them to you. You'll be getting, clustering the ones that are top of mind, and then using those as the table topics.

We've talked about this in the past, the trends on where people's thoughts are going?

Jeff Payne: Yes.

Jennifer Bonine: We tend to see some trending, certain things I think we've said always are there.

Jeff Payne: There's some topics always there, always top of mind.

Jennifer Bonine: There's always some stuff, top of mind. There's some that rotate in and out?

Jeff Payne: Yes.

Jennifer Bonine: Some new ones come in and other ones come out. Any predictions for what some of the new ones will be and what some of the ones you think are still trending as they're going to be there because they're there year after year?

Jeff Payne: I'll start with the ones that are always there. Finding and retaining people is always on the list. How do I keep my people motivated, happy? How do I find the right people? How do I do that in the face of maybe my organization doesn't give the care and feeding to my people that I think they need? How do I do that without a lot of budget and things like that?

Jennifer Bonine: Without a lot of support for it?

Jeff Payne: That's always on the list. In the last five years, agile and dealing with moving to agile is always on the list. How do I educate my people about agile and how do I coach and mentor them to be agile? How do I deal with this move toward agile, is always on the list. Those are two that are always on the list. I bet we'll see and hear something about DevOps this year.

Jennifer Bonine: Really?

Jeff Payne: Yeah, maybe we're starting to move to DevOps. We're trying to understand what that means from a philosophy perspective. Does that change anything about how I manage people, lead people? What does it mean for testers? We might hear about that.

Jennifer Bonine: It hasn't hit the topic yet, huh?

Jeff Payne: No.

Jennifer Bonine: Really, wow.

Jeff Payne: Not yet.

Jennifer Bonine: That's interesting to me. From a conference perspective, we ...

Jeff Payne: It's everywhere.

Jennifer Bonine: It's everywhere. We've had it for a couple of years. It's interesting. I'll be interested to hear if it does hit because it seems to be everywhere. It'll be interesting if now the leaders are starting to go, "We've got to pay attention."

Jeff Payne: "I've got to figure out what to do about this."

Jennifer Bonine: "This is coming." Interesting. Any other ones you think might hit that this year?

Jeff Payne: The only other thing I can think of obviously that's hot right now is Internet of Things. I don't know how much that's percolated up to, do I need to manage different or lead different if our organization is starting to move toward providing those capabilities, or we're testing those capabilities and we're trying to understand our role and what we're responsible for and what's in scope and out of scope, and how do I manage and train people around that. It is something that could pop up.

Jennifer Bonine: I'll be interested to see if that one pops up too. There are things people I think are just starting to think about. That's what's kind of interesting about coming to these conferences is you tend to get some of the leading thoughts as well, right?

Jeff Payne: Right.

Jennifer Bonine: People that are out ahead of it, and it may not be quite mainstream yet. We're talking about it but everyone's going, "That doesn't impact me. It's not touching me yet." One of the things with that is around security and ties into the other area you spend a lot of time focusing on is around security and how do you secure what you have for your organizations, protect your data, protect your information, make sure that you're watching out for those things, especially in regulated industries as well. It's in particularly important to do that.

I was talking to a gentleman. We were talking about IoT and where it's going, one example being in the medical industry. We're seeing today for diabetics, for example, they used to have to prick their fingers all the time to check their blood sugar. Now they've got devices that have been approved where it's a patch that you basically wear on your arm. That patch then can send data and information down to a mobile app. I thought, "Wow, that's spectacular for a diabetic who no longer has to manually check their insulin level." They're automatically getting notified. It's tracked twenty-four by seven. You get more data to trends, which is great. That data is now readily available because it's going through the networks. You now have information out there going down to a mobile app which then you have access to but you want your provider or your doctor to have access to. Who else can get access to it? Great information to have, but now it's out there.

It's probably not super secure information other than they know what your blood sugar is and all that stuff. What I thought the interesting application of that was they said, "What about parents who have children who are diabetics?" The parents get access to that information on their phone. Now parents have availability and readily accessible information at their fingertips. Then you get into can the parents have access and do they have rights to those records? What if the children don't want them? They're eighteen or seventeen or sixteen. Who can have access and what level of security do you need to have around getting that access to people and ensuring they should have it?

Any thoughts on, that's one example, there's many others, with pacemakers, implanted devices, data that comes from those, people's ability to get and access that data, stop and restart implanted devices? Now you're getting a little more ...

Jeff Payne: You're all over. Now it's really safety-critical.

Jennifer Bonine: Now it's safety-critical if they can stop your heart. We're getting a lot of technology out there that exists that's like that. Any thoughts from your side around security, what things to think about as we're getting into that securing the data? Who has access to the data? Getting started in that arena?

Jeff Payne: A couple thoughts. Internet of Things basically just is a supply chain of software and devices relying on other devices, etc. I always look at, and obviously whoever is interacting with the consumer the end customer it obviously the most liable and responsible.

Jennifer Bonine: Absolutely.

Jeff Payne: It all starts with them. What they have to do is really just push back and push down on everything that they buy and use to demand some level of security of what they purchase that goes into their devices. I see it starting at the consumer and being pushed back down from the person selling those devices to the consumer, to everybody in their supply chain that is providing either software or sensors or hardware systems that support that. They're going to have to set and use standards, security standards, and use security testing guidelines to assure that everything they get is secure. Really they're the ones with the most risk.

Jennifer Bonine: That's who you hear about, right?

Jeff Payne: Mm-hmm, yeah.

Jennifer Bonine: If there's a fault or a failure the consumer blames the person closest to them.

Jeff Payne: Ultimately it is they sold the product so it is their responsibility.

Jennifer Bonine: We've seen that in practice with Target Corporation when they had their breach. It wasn't necessarily inside their walls, but it was one of their suppliers who then had an issue which leaked data, which then gets out. They're held accountable financially as well as just from a responsibility standpoint.

Jeff Payne: If you think about so many people use contractors to build their software or outsourcers or others, or they're buying third party components. Almost every single breach can trace back to people outside of the organization that actually was held responsible for it. The problems been around a long time. It's just that Internet of Things is making it real time and safety critical. Now you have to really care about those things.

Jennifer Bonine: What would you recommend for testers out there who say, "We have this group and they're responsible for it so I don't really do much with it." Would you recommend that just about every tester out there at least get some base level of knowledge around awareness of security?

Jeff Payne: Absolutely. I taught a security testing tutorial here. My company Coveros does a lot of helping people learn how to be better security testers. First of all, there's no reason a software tester can't look for some level of security vulnerabilities in their testing. It's not that hard.

Second is, of course if you can find it earlier and fix it it's going to save you a lot of downstream trouble. You don't really want to wait till the end. Pushing it earlier and getting your developers how to understand how to build things to be secure, how to get your testers to learn how to do some security testing is going to significantly decrease your risk. You're not relying on the people that show up at the end to pull it out.

It's hard to pull it out at the end. I always say in the tutorial it's no different than if you wait till the end to do your system testing. If you wait till the end, it's a needle in a haystack to find problems. It's the same in security. You're not going to find all those vulnerabilities if you're coming at it at the end of the lifecycle and you have a fixed amount of time and you're doing some red team or penetration testing.

Jennifer Bonine: Too late.

Jeff Payne: You have to do more.

Jennifer Bonine: Usually those issues, I would assume too, they're not simple ones to fix a lot of times.

Jeff Payne: They're not.

Jennifer Bonine: If you leave it to the end it could, one problem could uncover other problems.

Jeff Payne: Absolutely.

Jennifer Bonine: It could spiral and you're at the end of this lifecycle and now you're all of a sudden holding things up. Finding that stuff early, injecting some of that security testing earlier in the process, is a good thing. We're hearing a lot this conference about, we used to hear shift the testing further up in the lifecycle, shift it left. That includes, and I think things people are now getting more aware of, not just your traditional forms of testing, but security testing, performance testing.

Jeff Payne: Performance, availability, all your nonfunctionals are moving earlier.

Jennifer Bonine: They're moving earlier.

Jeff Payne: They have to.

Jennifer Bonine: They have to, and that's something I think we're seeing is a trend of an awareness which also means testers out there have to have a level of knowledge about some of those things.

Jeff Payne: Yes.

Jennifer Bonine: Not to be overwhelmed by it. A lot of times people go, "Security, that's super specialized. I don't know anything about that."

Jeff Payne: "I don't do that."

Jennifer Bonine: "I don't do that. That's not my deal."

Jeff Payne: The other thing you have to figure out is how can you split the nonfunctional testing into things that you can do earlier but then you don't repeat later. It's one of the things I talk about in our DevOps tutorial. We do a lot of DevOps work too, and we're always trying to push assurance earlier. One of the tricks is don't try to solve the whole problem. Load and performance test. You're not going to be able to finish that until you have a production like environment you can do your final load test on. That doesn't mean you can't measure the performance of the code as you build it and track that. If it's starting to get slow correct it early on instead of waiting till the end to figure that out.

You can do the same thing with security, privacy, other things. You can take a piece of it that relates to the software itself, irrespective of its environment and try to make sure it doesn't get out of control early.

Jennifer Bonine: Break it into those smaller components that you can do earlier on and figure out what that looks like. Would you recommend, or is there any recommendation to for getting an awareness of where all the touch points are, really getting for the testers a map of what are you connected to? Who are all those suppliers? What are all the APIs? Where can my stuff potentially go so that you're making sure you cover all of that?

Jeff Payne: There are some products out there and open source technologies that'll scan your file systems, spider your website. There's, I showed it in the tutorials, carried a tutorial, there's a site called Built With, I think it's builtwith.com, builtwith.com. If you just point a URL it'll dump all the frameworks and things that the website is using. You want to know what all that stuff is. To your point about understanding who your suppliers are, if you're using open source components, which almost everybody is today, there's great tools out there now that'll look at in a DevOps environment, whether the current version of the frameworks or the libraries you're using have known vulnerabilities in them that have been discovered.

A lot of times developers they get something to work with, say, struts or something, and it works. They don't upgrade their version of struts because they're worried it'll break something. That old component might be in your software for years, and meanwhile vulnerabilities are known and found and patches are done and you never update it. You're shipping a product with vulnerable code in it, that not only has vulnerable code in it, but it has a map as to what's vulnerable.

Jennifer Bonine: It's like a roadmap for people.

Jeff Payne: It's public information. Exactly.

Jennifer Bonine: All the people that want to do bad stuff know exactly where to go.

Jeff Payne: You can use these tools like Built With and other things. If you're a hacker that's the first thing they usually do, is recognizance. They try to figure out what you have and what versions of things you use so they can go look. The easiest thing is go see if there are no vulnerabilities. If there are, you're toast.

Jennifer Bonine: You need to ... Do you advocate understanding the mindset of a hacker, where those guys are going, to go there first? Check it out yourself because they're going to go there?

Jeff Payne: Talk to your developers, talk to your testers. There are tools to scan. Understand what languages are used, what platforms you use, what third party components, what scripting engines are in your software. The more that stuff you understand the more you'll understand what it is you're going to have to do from a testing perspective to make sure it was done right.

Jennifer Bonine: We are out of time, but I know we just scratched the surface. People are probably like, "I want to know what all these tools are." If people want to contact you to get more information on the tools, Coveros, getting some help with that security testing, getting started, what is the best way to contact you?

Jeff Payne: You can always go through our website, www.coveros.com. Or just hit me up on Twitter @jeffreyepayne.

Jennifer Bonine: Perfect.

Jeff Payne: It's the easiest way.

Jennifer Bonine: Thanks, Jeff, so much for being here.

Jeff Payne: Thank you.

Jennifer Bonine: I appreciate it.

Jeff Payne: Thank you.

Jeffery Payne is CEO and founder of Coveros, Inc., a software company that builds secure software applications using agile methods. Since its inception in 2008, Coveros has become a market leader in secure agile principles and recognized by Inc. magazine as one of the fastest growing private companies in the country. Prior to founding Coveros, Jeffery was chairman of the board, CEO, and co-founder of Cigital, Inc., a market leader in software security consulting. He has published more than thirty papers on software development and testing, and testified before Congress on issues of national importance, including intellectual property rights, cyber terrorism, and software quality.