Agility Throughout the LifeCycle: The Rise of DevOps

Until it is released into the hands of end-users, new developments are, to reference the lean software term, inventory.  A great deal of time, effort and resources have been invested in making changes, but no value is realized for the organization until the development is actually released and used by the end-users.

What is DevOps?


The term DevOps suggests the combining of development and operations together and then applying some of those skills of development to the issues of deployment. It is also about trying to break down many of the barriers that have grown between these two parts of many organizations.

Is It New?


As mentioned at the beginning, we believe that the principles have been around in various forms for many years.

For those of us who do internal software tool development as part of their job (either from scratch or to customize and configure), DevOps has always been a part of the reality that we have lived with day in and day out for at least a decade or two. Just as agile development calls for generalizing specialists across functions, we have had to do the same across the enterprise:

• We have had to be experts not merely (nor even primarily) with tool administration/usage, but also with the organizations business processes. Most, if not all, of the enterprise application architecture.
• We have had to work across not only the software development lifecycle of these tools, but also the deployment lifecycle for installing, upgrading and supporting the tools.
• We have had to do far more than mere administrative support, including new development for any existing internally developed customizations and their configurations.
• We have had to be process experts, application development experts, application deployment exports, tool support and administration experts, etc., for most all the business-activities and their processes surrounding those tools.

Is it Elitist?


One of the dangers is that DevOps is perceived as elitist and as relevant to only a few organizations. See Stephen Nelson-Smith’s article which addresses some of these concerns. Does it mean that all sysadmins must be able to program (or at least script actions)? To some people the answer is yes.  A new breed of sysadmin coders will sweep all before them. I believe that is not strictly necessary, though it can be an advantage. The key principle for me is that the operations team needs to consider how to improve the automation of their tasks, and if they can't automate it easily, to seek help in achieving this automation and the business should allocate necessary resources to doing this.

Is It Just a Fad?


Another danger is that of being perceived as a trendy fad and, thus, something that can safely be ignored by the vast majority!

The underlying principles being espoused under the term DevOps have been around for a long time and successful organizations have been implementing them well for some time.

Let’s look at some of the issues and principles involved.

Global Versus Local Optimizations


Too many companies still have local optimizations rather than optimizations across the complete lifecycle.  Their development may utilize agile methods such as TDD, and seem dramatically improved from previous efforts, and yet releases happen infrequently and are not always successful. It doesn’t matter how good your development is if there is a large stumbling block later in the life cycle.

Improving Communication


There is often a lack of communication between development and operations. Each department is often incentivized very differently.  Development gets rewarded for making changes, operations get rewarded for lack of outages, which often means resisting change. It is important to break down the “Great Walls of Ire” as Brad so memorably put it!

This means improving global communication and, thus, co-ordination. Make sure that new releases are not a surprise to operations.

What Are the Constraints For Dealing With Releases


Even if the software team doing the development and support of the tool(s) has the ability to release & deliver frequently, the organization (enterprise) may not have the capability to assimilate release as frequently

• Some deployments/upgrades may require downtime of business-critical systems or unavailability of business critical repositories and databases, and the business simply can’t tolerate that kind of disruption as frequently or during crunch-times of any particular business-product that uses the tools
• Some software changes go hand-in-hand with some corresponding process-change and/or organizational change to the business, and the customer organizations cant begin utilizing and realizing the value of your latest software/tool changes until they have made the necessary organization and process changes (which means many of us also have to become organizational change agent/experts in addition to process and tool experts)
• Any one of many other existing business-critical systems or IT assets which are needed for the deployment of your tool (or its users) may have critical dependencies or freeze periods making it difficult to deploy new releases of your own tools as frequently as desired.

By improving your communications you become aware of and deal with these issues.

Increasing Automation


This means automation throughout the lifecycle. Agile practices such as continuous integration are now commonplace, but continuous deployment is much less so.

However, some companies are achieving multiple successful deployments of new functionality per day, e.g., John Allspaw’s presentation, "10+ Deploys Per Day: Dev and Ops Cooperation at Flickr".

The key here is having the right mind set to seek out opportunities for automation and then to actually implement them – whether through acquiring tools, or by developing scripts or local tools.

What About ITIL?


Many organizations have reaped large benefits from implementing ITIL practices within their service and support teams. However, the development side of the organization has often not been included in such initiatives. Thus there is a still often a “wall” over which new releases are thrown. Once things are in production, the operations team may very effective at supporting it – but there is still that hurdle to get over.

There is nothing in ITIL that says this is the way it must be. Indeed, ITIL V3 takes a more holistic view of the whole lifecycle starting with service design. The concept of an application service then allows organizations to wrap up application development as part of the overall process.  Adding some useful checks and balances, while also improving communications and co-ordination. The service transition book covers the transition into production.   While looking at how to control it does not mean that it has to be slow or a burden, just well managed

By taking a DevOps mindset, we are not asking operations to give up control – we are working with them to improve the overall process, and typically achieve smoother and faster releases with much less friction. The underlying principles are still the same.

How to Implement DevOps


If you want to implement this, how do you go about it? Can you hire a DevOp and expect all your problems to go away? The key starting point has to be around communication, ensuring that your development and operations departments are working together closely on a daily basis.

Look at what is causing deployment problems and releases to fail. Ensure that deployment considerations are factored in very early in the development cycle. Look at automation possibilities. Make sure you have a well-managed common set of tools including your source repository.

Consider the throughput rate you are achieving: the rate at which changes are actually being deployed.  Also, how long on average that is. Measure inventory, work-in-progress that has not been deployed, and seek to reduce it.

If your software architecture makes frequent deployment difficult, then seek to change it as a high priority (treat this as a major code smell) or item of technical debt. This can be a major issue with legacy code and tools, and may take significant time to fix

Vendors and Tools


There are a number of companies and also open source communities that provide tools and methods:

• Puppet labs puppet
• Opscode’s chef, open source recipes
• Commercial vendors include XebiaLabs and Nolio (also included as part of Serena’s offering)

One of the advantages of the open source offerings is the way it encourages thinking and sharing within the wider community. People can get actively involved and engaged which is a major benefit for all.  This is not to say that the commercial tools are bad, however.

It is also important to realize that a tool will not solve the problems, or necessarily make a huge difference by itself. It is the mindset and practices around the use of that tool that make the most difference.

Summary


This has been, perhaps, a whistle stop tour of some of the issues involved.  Many points have just briefly been touched upon above.

DevOps takes the agile notion of cross-functional self-organizing teams of generalizing specialists across disciplines and lifecycles and applies that directly to the IT and process architecture of the enterprise itself, forcing those of us who are CM/ALM professionals be experts in not only the processes and tools/technology, but also in enterprise application development, deployment and in organizational and process change,

We need to learn and become adept not only at applying CM/ALM to agile projects, but also at applying lean/agile principles and techniques to CM & ALM. That includes figuring out what it means to apply TDD, refactoring and continuous integration, or identification of smells and technical debt for processes and documentation (and even organizations) as well as to the development and deployment of the tools and processes we develop, deploy and support.

The rise of DevOps has many things to commend it.  Many organizations would do well to take the ideas on board. It isn’t just a fad and it is here to stay. Like most things it is not a magic bullet, but if you embrace the key ideas in a pragmatic and sensible manner, you will get real value from them.

Don’t oversell it within your organization, but instead use it as part of your arsenal for improving your process and making your own job more satisfying and effective.

Resources and Further Reading


As well as the links referenced above, see:

• Anti-pattern: The release vehicle - http://www.rendell.org/pebble/software/2010/02/09/1265756844985.html
• Deployment is just a part of dev/ops cooperation, not the whole thing - http://www.kitchensoap.com/2009/12/12/devops-cooperation-doesnt-just-happen-with-deployment/
• 20 DevOps guys you should follow by Matthias Marschall -  http://www.agileweboperations.com/20-devops-guys/
• Conference: http://www.devopsdays.org/
• There is also a recent book "Continuous Delivery" and its corresponding website http://continuousdelivery.com/