Evolve Your Software Architecture for Agile and DevOps: An Interview with Neal Ford

Josiah Renaudin: Welcome back to another TechWell interview. Today I am joined by Neal Ford, a director and software architect at ThoughtWorks, as well as a keynote speaker at this year's Agile, Better Software & DevOps West conference. Neal, thank you so much for joining us today.

Neal Ford: My pleasure.

Josiah Renaudin: Before we actually get into the keynote, can you tell us a bit about your experience in the industry?

Neal Ford: Yes, so most of my professional work now is sort of in the intersection of software architecture and the engineering practices espoused by things like continuous delivery, which includes all the DevOps revolution. I've been a software architect for twenty-ish years or so in various different aspects of the role, and I also work closely with the publisher O'Reilly. I'm sort of the mascot for their software architecture series, kind of subject matter expert that advises them on software architecture titles.

So I've been sort of immersed in the software architecture world for a while, and I'm just finishing up a book on building evolutionary architectures, which is also in that realm.

Josiah Renaudin: And so much of your keynote is about software architecture. It kind of covers the different aspects of it. So for you to kind of kick things off here, why has software architecture traditionally been so difficult to change later on in the process?

Neal Ford: Well, a bunch of reasons go into that, and we touch on a few of those in the book. But one of the reasons was that was never a big motivation for software. That was one of the important kind of "-ilities" that you wanted to support, which is "evolveability." So, when you think about software architecture, you're thinking about implementing both the requirements from a business or domain standpoint but also all of these architectural characteristics, what architects frequently refer to as the "-ilities," like scalability and resiliency and security and all those meta-characteristics of architecture.

And traditionally it's been really hard to build, for example, say, an architecture that you plan to be scalable. It's really hard to evolve that architecture and maintain that scalability. And so, one of the things that we're touching on in both the keynote that I'm going to do and then our book, is how can you build an architectural system in software but make sure that the characteristics you chose that architecture software for don't degrade over time. It turns out that's the same problem as, how can I freely make changes to this architecture over time and make sure that I'm not degrading some aspect the architecture's idea?

Josiah Renaudin: Yeah, and let's dig a little bit more into evolvability, because that's a word that stuck out when I read through your abstract. Can you kind of touch on a few ways you can build evolvability in your architecture, ways that you can build something that allows you to change and advance your architecture over time?

Neal Ford: Sure. That's the essence of what we're talking about with an evolvable architecture, is one that you can make changes to over time without damaging their architecture or degrading it somehow and not adding a lot of technical debt to the architecture as well, as you make changes to it over time. For a long time, the actual kind of tongue-in-cheek definition of software architecture was "the stuff that's hard to change later."

Then over the last four or five years, we've seen a big interest in this microservice style of architecture, and the idea of easy changeability is actually built into the architecture as a first-class concern, and we realized, hey, if you build changeability into the architecture, it's suddenly not so difficult anymore.

And so, that's part of what we talk about in terms of evolvability. How can you build an architecture that doesn't degrade and break over time? And one of the mechanisms that we're using, in fact the primary mechanism we're using ...

And talk around this idea comes from the evolutionary biology and evolution in computing world, which is where the evolutionary architecture part of this comes in, because if you're in evolutionary computing—for example, let's say that you're trying to design an algorithm that's better at image compression. Well so, in genetic computing you have to be able to determine from one generation to the next, is this a better algorithm for image compression? And so, your criteria there is going to be, is the image more compressed using this version of the algorithm than it was the last version of the algorithm? In evolution computing that's known as a fitness function. It is the function by which you evaluate to determine, am I moving closer to the appropriate solution?

And we've co-opted that idea in that architecture to build what we call architectural fitness functions. And so, these are things like metrics or tests, or things like the chaos monkey that Netflix created that continuously gauges the resiliency of their architecture. These are mechanisms that you use and build into your system to make sure that over time, if you built an architecture where scalability is really important, we're going to build a scalability fitness function that evaluates "are we still as scalable as we think we are?" and run that on a continual basis as we make changes to the architecture.

That way if that particular characteristic of the architecture begins to degrade, we can find out right away and start to address it rather than be surprised six months later that, hey, our really scalable architecture suddenly doesn't scale anymore. What's happened in the last six months to do that? We're trying to put more safeguards around that.

Josiah Renaudin: And like the architecture in the software and the testing itself, we've evolved, we've figured out better ways to build things, better ways to structure things. And now that we do have a better understanding of how to create more pliable software architectures, has it become easier to build evolvability into the existing setups instead of when you start from scratch? You can kind of think, all right, this is how things are. We can handle things this way. But when you take an existing structure and you're like, "We're trying to insert some evolvability into this," is that easier than it has been? Is it still really difficult to do?

Neal Ford: Well, totally easier than it has been because we have mechanisms like the deployment pipeline. So we rely on things like the deployment pipeline to be able to execute these fitness functions on an aggressive regular basis every time we make changes to code. So, that kind of infrastructure support's gotten a lot easier. It's a lot easier doing things like DevOps now and being able to build realistic environments automatically and programmatically so that you can realistically test things in parallel different environments to make sure things haven't broken in a continuous deployment kind of scenario. So, it's certainly gotten better from a cool process and awareness standpoint, because we're actually thinking about changing architectures now rather than just kind of throwing up our hands and saying, "Well, we're kind of stuck with the architecture we built, we'll just make the best of it."

And certainly some architectural styles make this a lot easier, like microservices, because it has a really small kind of iterative deployment that holds bounded context in microservices. It's really easy to make smaller changes in that architecture. You can evolve things like layered architecture, which a lot of people have monolithic layered architectures and there are techniques you can use to evolve those. We actually talk about the distinction between adapting an architecture versus evolving it permanently. Adapting it is kind of temporarily modifying it so that you can add and you can build to it versus evolving it, which are more permanent kind of changes.

So, it is possible to evolve existing systems. Certainly not as easy as a system that's built to be able to evolve, but there are avenues for evolvability, particularly because of the really good testing tool support that we've gathered up over the last four or five years.

Josiah Renaudin: Yeah, that makes a lot of sense. And you can't really talk about modern testing of software without bringing up words like agile, DevOps, and continuous integrating. So, have those different ideas sped the industry up so much that you need to be more flexible with your testing? Have these different software practices made it almost impossible to stick to one architecture for an extended period of time? Because for so long it's kind of that, if it ain't broke don't fix it mentality, where as long as it works we're going to keep using it. But now that we're so rapid, we're so iterative, is it really difficult to not continue to update your architecture almost every single year?

Neal Ford: Well, you almost have to, and Forbes a few years ago very famously came out with an article that every company is now a software company. What they mean by that is if you're Delta airlines and your iPad app sucks, it's going to be bad for your stock price. And so, at some point every company has to get some confidence and be able to build and present their software vision out in the world. And so, I think it's gotten more hypercritical now to be a lot more responsive, to be able to build things quickly.

But the other benefit of that is we now have the capabilities like the cloud, here you can build things really quickly and speculatively and see if they're going to resonate with your audience before you invest a lot of time and effort into building your own infrastructure. And elastic scale is one of those things that we've wanted forever in architecture and now with cloud-based systems, we finally do have a reasonable use of elastic scale and that is really nice because that frees up architects to start thinking about business problems more than just these purely plumbing mechanical problems that we had to solve over the last decade or so.

Josiah Renaudin: And you do mention in your abstract that fitness functions build protective scaffolding around central parts to guide the architecture as it evolves. So, how can this guide developers as they're figuring out whether their particular architecture is doing what it's supposed to be doing, what it's actually built to do?

Neal Ford: Well, I'll give you a perfect example of this. This is one of the examples we use in the book is this mechanism that Netflix is creating. This is an example of what we call a holistic continuous fitness function. We have a bunch of categories of these. Holistic means it affects more than one system, so it's a coordination between systems and continual means it runs all the time. And so, what I'm talking about here is the chaos monkey that Netflix related because Netflix realized at one point they had a problem in that they didn't own their deployment infrastructure because it was all done at AWS. And they worried about things like latency and instances disappearing suddenly and all these other kinds of problems that can manifest if they don't have control over it.

So, they created a chaos monkey. And chaos monkey is not some tool that they run on Tuesday afternoon at 3 o'clock, the chaos monkey lives in their infrastructure and their architecture. Which means if you're a service team and you write a service that doesn't handle latency correctly, for example, and you deploy it, the chaos monkeys are going to take it down right away. And so, every developer on that system has to think about, well it has to pass all these fitness functions before it'll even survive in this ecosystem. And that's a great example of using fitness functions as a guideline for developers because developers at Netflix know that you can't be lazy about latency and security and all those things because the tool is just going to kick your service out of the architecture because it doesn't meet those guidelines.

And I think an example of how effective that is, about a month ago Amazon S3, the whole east went down, and everybody that relied on Amazon was affected except Netflix because they created, at one point, the chaos gorilla, which simulates an entire Amazon data center going down. And it worked because they have written their services to be able to withstand a data center going down. A data center went down and Netflix stayed up. That's a great example of using fitness functions both as a verification mechanism as you move forward and deploy things, but also as guidelines for developers to say, "Hey, look. You have to write to the specification or your code's just not going to work."

Josiah Renaudin: And I think there would be riots in the streets if Netflix went down for any extended period of time.

So I think the thing people would love to know is how you can determine whether or not software architecture is worth retrofitting. I mean, you have to take a step back, look at your architecture, and figure out, are there some cases where it's just a better idea to scrap it all? Say, "This is not going to work. Let's start from scratch"? How do you kind of determine whether or not it's worth going forward with what you have, changing certain things, or if it's just, let's throw everything out and start over?

Neal Ford: I do this a lot professionally, so this is kind of where I live right now. The first thing you should do is increase and improve the modularity of your existing system because if you're going to take some sort of monolithic application and try to make it more evolvable, what you're going to have to do is start breaking down the big, huge chunks into smaller chunks. And before you start physically doing that, you can start logically doing that by looking at the component and modular relationship inside your code base and improving the modularity of what you already have.

So there are a slew of tools out in the world. So there's a tool in the job world called X-Ray that lets you analyze the interim model dependency. There's a tool in .Net called In Depend that lets you do the same thing. Most languages have visualization tools that lets you see how coupled things are to one another. And so, going through this exercise of improving the modulars of your existing system does several things. First it shows you how modular your system is and how amenable it's going to be to split into smaller services, which you may then further decompose.

But also it shows how cohesive the actual application is, in terms of actually doing something in a reasonable way. And you may end up in systems, we do sometimes, where everything is simply degraded into a big ball of mud. There's no discernible architectural structure anymore. Maybe they started with a layered architecture, but then at some point reporting needed better performance so they allowed reporting to bypass all the layers and go directly to the database.

There've been a lot of other compromises, and so going through this modularity improvement exercise will show you how well portioned the kind of units of work in your architecture are and then you can make an evaluation, is it worthwhile trying to continue this modularization process and switch from logical to physical modularization or is it better of just rewriting the thing from scratch and just building the stuff that we need?

Josiah Renaudin: All right, fantastic. And I have just one more question for you, Neal, and I do really appreciate you taking the time today. What is the central message you want to leave with your keynote audience? What's the main thing, if they’re considering going to your keynote? What do you want them to take away from what you say?

Neal Ford: Well, the main takeaway is that you can build architectures that evolve. It doesn't take a herculean amount of effort. In fact, you can incrementally start applying these ideas to your existing architecture because one of the things that this keynote will do is make you look at the architectural characteristics and kind of put them on an apples-to-apples basis.

One of the problems that architects have now is that you have these security things that you have to worry about, and you have scalability things, you have performance things, and you don't treat those as the same kind of thing, even though at their heart they are, because at the end of the day you want these tests to be able to verify, is my architecture sound or not? And so, in our mind those are really just fitness functions, and so you can come to my keynote and go back home and start identifying, okay, there are three characteristics to this architecture to protect going forward. Let's start building fitness functions and very incrementally start applying these ideas of evolution in architecture.

So, this is really about, how do you build architectures that maintain their characteristics over time and don't degrade and eventually allow us to be a lot more aggressive about making business-relevant changes to architecture without going through a huge re-engineering effort?

Josiah Renaudin: All right. Fantastic and thank you so much, Neal. I'm looking forward to hearing more about what you have to say about software architectures at your keynote in Vegas.

Neal Ford: My pleasure. Looking forward to it.

Neal Ford is director, software architect, and meme wrangler at ThoughtWorks, a software company and a community of passionate, purpose-led individuals. He thinks disruptively to deliver technology to address the toughest challenges, all while seeking to revolutionize the IT industry and create positive social change. A speaker at hundreds of developer conferences worldwide, Neal is an internationally recognized expert on software development and delivery, particularly where agile engineering techniques and software architecture intersect. Neal has authored magazine articles, seven books (and counting), and dozens of video presentations. Neal’s topics include software architecture, continuous delivery, functional programming, and cutting-edge software innovations. Check out Neal’s website at nealford.com.