Defining What’s Really Important in a DevOps Maturity Model

It seems that every person I meet these days is in charge of DevOps for his or her organization. In one company, I counted three developers and two operations guys who were all in charge of DevOps, and their titles ranged from director of DevOps to chief architect. The only problem was that none of them were talking with each other and there was little or no communication or collaboration between teams. Similarly, terms like continuous delivery and deployment seem to be taking on many disparate definitions and perhaps are just idioms for cool tricks and automation. So, how do we go about understanding whether a team is really implementing DevOps correctly with the desired results?

When I do an assessment, I always interview a variety of stakeholders and discuss existing practices with a focus on what is working well and what can be improved. The goal is to create a game plan for moving the team from where it is right now to where it really needs to be.

It is easy to characterize DevOps as being single push-button deploys that can be constantly implemented throughout the day. When I mention that continuous deployment is sometimes not a good idea, my colleagues often look at me as if I have uttered some sacrilegious blasphemy worthy of my being excommunicated from the church of process improvement. But the truth is that there is a lot more to DevOps than just push-button deploys. Many of my colleagues are asking for a DevOps maturity model to help measure where they are now and help indicate where they need to get to as part of the DevOps transformation. I have had mixed feelings about maturity models for many years, starting with the first one that I learned as part of the SEI’s Capability Maturity Model.

Watts Humphrey was one of the thought leaders in the technology industry and published his first Capability Maturity Model in the 1980s. Early maturity models included five levels. In level 1, things were best described as the Wild West, with little or no established and repeatable processes. In level 2, we had basic processes that were repeatable, sometimes even producing consistent results. Level 3 gave us defined processes that were documented and generally improved over time, while level 4 introduced metrics that allowed us to measure and manage processes to show results. The highest level in the CMM was level 5, which indicated continuous process improvement over time. As neat and tidy as this model appears, I never found it very helpful in actually implementing the day-to-day IT controls that needed to be established. Then, what exactly would I want to see in a DevOps maturity model?

The truth is that I am OK with the levels as described but prefer to focus on the capabilities we need to measure. First and foremost, I believe we start with effective communication and collaboration. DevOps needs to begin with including both development and operations, then reaching out to other key groups such as QA, testing, and information security. The next thing I look for is basic automation, with a focus on completely reliable deployments free of human error. This should include automation to build the infrastructure itself, which on my list should be a primary focus from the beginning of the DevOps journey. I also like to see knowledge-sharing across the organization, with a strong focus on effective and ego-less teamwork.

Automation must be the same across all environments, from development test to more controlled environments such as user acceptance testing and production. I am weary of seeing super automation for development test environments but manual, error-prone procedures for the upper environments. It is better to have slightly less fancy automation in the development test environment and focus more on having a consistent and completely reliable approach to production deployments. As I mentioned, deployment automation must be the same across all environments. Obviously, there may be some differences between environments that cannot be avoided, but teams should account for associated risk and plans for addressing any potential issues.

I look for getting operations involved early in the deployment process to learn from developers in what is often called “left-shift,” and operations sharing their own expertise back with developers in what I have begun referring to as “right-shift.” I also want to see very mature environment monitoring, which addresses essential runtime dependencies and effectively gives a heads-up when something bad is about to happen. There also should be a proactive response to incidents when they occur and tight integration with change control to assess and mitigate technical risk.

Continuous integration, continuous delivery, and continuous deployment will always be considered essential to DevOps best practices, but my preferred focus is on creating a secure, trusted application base where deployments are completely verifiable and 100 percent reliable. Equally important is the ability to identify and immediately address any unauthorized changes.

Alignment with product and business are high on my list of DevOps process maturity, along with the realization that DevOps involves much more than just development and operations.

I will continue to work on defining a maturity model for DevOps, and I welcome your thoughts and opinions as we lead organizations to deployment frameworks and methodologies that ensure timely deployment of new features and bug fixes while ensuring reliable and secure systems. The true measure of success is thrilled customers who expect and receive the best service possible.