From time to time, we tolerate business processes that don’t work as well as we’d like. It’s easy to complain about a process that is slow, clunky, or just plain broken until you try to build one yourself. It’s simply easier said than done.
So, how broken must a process be to spawn an entire social movement? That was the state of application release in 2009 when the DevOps movement emerged. DevOps pioneers realized that the gap between application development and operations was killing their businesses and making IT look incompetent.
DevOps throws out the old silo boundaries and applies much-needed common sense: dDeeply collaborate and communicate; reduce the scope and increase the tempo of service change; automate to eliminate manual errors and wasted time.
During the first two years of DevOps, the pioneers were carving out new, ad-hoc tools to help do this. But as the pragmatic majority prepares to follow the same path, what’s the right way to make this process actually work?
Fortunately, this isn’t the first revolution against inefficient process. The revolution The resolution began with modern manufacturing and the elimination of broken supply chains. However, more recently, the application development world has gone through its own revolution, discarding the lumbering dinosaur of waterfall development for nimble, warm-blooded agile development.
So what can DevOps learn from this recent IT revolution?
Looking to Agile Development Automation for Inspiration
Agile software development kicked off in 2001 with The Agile Manifesto. Agile development isn’t just a better process for creating software; it is a new methodology for automation that’s paired with better processes.
Several such pairings are directly relevant to DevOps.
Automatic Build and Test
Embracing small iterative changes instead of following a static plan is core to agile process. But that’s only possible when you can do a complete build and test cycle of your product in one step and in a short time frame. That’s why agile development teams use build and test automation, such as Jenkins and Cruise Control, to make the rebuild-and-test cycle as short and easy as possible.
Modular Components from Shared Repositories
Few modern software projects are built completely from scratch. Instead, they are built on a variety of open source and licensed components, which is critical to the agile development process, because it enables rapid design change. When a project team suddenly realizes that it needs a beefier database or a more flexible network library, the new components are immediately available from modern public repositories like GitHub and Bitbucket.
Automatic Dependency Management
Of course, picking components off the shelf is only part of the solution. It can be a complex task to assemble a compatible, working set of components, and this process was once a major engineering task in its own right.
Today, development teams let automatic tools such as Maven, Ivy, and Gradle take care of that problem. Simply name an open-source Java component, and Maven can download it and all of its compatible dependencies for your project. This is not just a convenience. Automatic dependency management enables agile development by making it possible to use many interrelated components.
Distributed Version Control
Distributed version control systems such as Git and Mercurial remove agile development’s brakes. In the “bad” old days, centralized version control was a constant headache and productivity impediment. Git and Mercurial aren’t just quicker — they enable another core agile principle, which is to emphasize individual self-direction. Agile teams want engineers working on multiple improvements in parallel, and distributed version control makes it possible to bring all those changes together.
Put these four examples together, and it’s clear why you couldn’t just drop today’s agile methodology on the development tools of the early 1990’s. We needed better automation, not just better ideas.
So how does all that apply to DevOps?
Agile Automation for DevOps
Now that the DevOps pioneers have blazed the trail, it’s time to pave the DevOps highway for the rest of us, and the lessons of agile development are relevant.
Automatic Build and Test
Here’s another way to look at DevOps: At heart, DevOps is about reducing stress. Life is just too short to sweat maintenance windows where one mistake could kill millions in revenue for your company. DevOps emphasizes repeatable automation, and that’s where automatic build and test comes in.
Automatic build and test is clear in a software development context. But what does it mean in the context of maintaining Exchange servers on Windows 2008 or Oracle 11g servers on Linux? That’s where automation is evolving fastest. DevOps needs automation that extends the continuous integration pipeline to continuous deployment.
Simple scripts are too brittle and take too long to build. The right answer, which we’ve seen from a recent spate of automation startups, is model-driven system construction. Automatic build and test becomes straightforward when there’s a machine-readable model for a complete system. Without a model, it’s anyone’s guess whether two deployments will come out the same way.
Modular Components from Shared Repositories
Thanks to the rise of open-source Linux repositories, the Unix world’s situation has improved. But that’s just for software binaries, and only for certain platforms. To really stop reinventing the wheel, Ops needs repositories that span the entire stack — from configuration down to OS patches — for multiple platforms. Modern automation vendors are creating and filling these repositories quickly.
Automatic Dependency Management
Just as automatic dependency management is an enabler (not just an accelerator) for agile software development, DevOps requires dependency management for distributed software systems and system components. Automatic dependency makes it possible to change your base system components without stopping the assembly line just like agile software development.
Traditionally, updating or swapping out system components (such as a middleware library or even an OS security patch) required weeks of manual effort. With the rise of modern, dependency-aware system automation, that task becomes pushbutton easy.
Distributed Version Control
Version control is the last development learning to hit operations. Agile change in any context requires the confidence and safety of multiple individuals trying things out, capturing the results, and combining them into high-quality deployments at the end of a change cycle.
Without flexible version control — or worse, without any version control, as IT operations is accustomed to running — parallel rapid change is simply impossible. Model-driven system construction must enable not only automatic build and test, but also deep version tracking, merging, and rollback for complete systems.
These changes in automation and process pair up identically for DevOps and for agile development:
Process Value Automation
Rapid change cycle Build and test
Recycling of existing data Shared component repositories
Rapid architecture assembly Automatic dependency management
Individual self-direction Distributed version control
Cloud and Mainstream DevOps
How does cloud fit into this story? DevOps solves a nasty problem, but it’s one that many IT departments still manage to live with on an everyday basis.
Cloud is the compelling event that makes the old processes untenable.
Cloud can sound complicated, but every exposition of cloud value comes down to agility. Agility is the key benefit of cloud, and thus it is a direct response to the challenge of DevOps. The rise of cloud is the rise of DevOps.
That’s how DevOps — and the key automation areas of automatic build and test, modular components, dependency management, and distributed version control — will go mainstream. Today, practical use of cloud is a bright differentiator for enterprises that made it work, and so is DevOps. Neither will stay in the land of early adopters for long.
