Yaniv Yehuda details how DevOps is a natural evolution within the software industry as it drives business value and enables the organization. This article will describe how database management and the database administrators need to be part of any comprehensive DevOps approach.
DevOps is a natural evolution of the software industry, enabling organizations to do more with less resources while differentiating leading and successful companies from their competitors. Delivering products faster with better quality has led to agile practices that are dramatically impacting how organizations deliver software with a faster time-to-market and remarkably better quality. Yet, as agile delivers shorter and smoother development cycles, known as sprints, organizations are also enjoying less risk by deploying smaller development packages with shorter delivery cycles and greater involvement of development in understanding business needs and requirements.
Operations teams, however, are finding themselves challenged to deal with releases on a much more accelerated basis, and this has created many challenges. Releasing software has many challenges as Knight Capital Group discovered recently when the organization suffered a $440 million dollar loss due to old software that was inadvertently reactivated when a new program was installed.
While agile has helped improved development productivity and quality, the next natural step of the software delivery evolution has been to take agile to production, linking development with operations, and giving birth to DevOps. DevOps is a set of practices and principles intended to help development and operations work more effectively together. The challenge is how to effectively implement DevOps in a demanding business setting.
Implementing DevOps
The main goal of operations is to maintain stable and healthy applications, while the main goal of development is to rapidly produce new features to meet business and customer needs. Change is implicitly the greatest enemy of stability, and understanding the conflict between development and operations is one of main goals of DevOps.
Bridging the DevOps conflict, or taking down the wall that has grown between these silos, is partly a cultural change and partly a technological change. Better communication, better collaboration, and more productive relationships between development and operations will surely contribute to a healthier and more productive organization, but just communicating alone is not enough to enable organizations to succeed in today’s challenging business environment.
Technology and work-related concepts must support this transformation, or we will be left with a lot of good will, and not much in terms of successful process improvements. We need to increase our overall efficiency and deal with risk resulting from business end users thirst and demand for frequent changes, robust features and new product releases.
Our experience has been that the key factors for successful DevOps are:
- Adopting agile concepts over waterfall concepts, thus enabling smaller, more focused and iterative blocks of development, that result in quicker time to market, as well as smaller changes, reducing risk and also reducing the potential to operations, and, ultimately, the customer.
- Better collaboration and coordination, either supported by employees who's goals are to gather information and make sure everyone is in sync and become the pivot point that drives change adoption; or by collaboration tools, chats, portals, and conferences to ensure wider understanding and familiarity or everyone on both “Dev” and “Ops” in the changes about to be implemented, what needs to be considered by both parties, and, well, a good discussion of the risks and concerns of everyone.
- Using automation based on accessible information is the last piece. This deals with frequent changes as one cannot rely on people's ability to remember everything they do, what might it influence, what would they need to take into account in case of scope of work changes, etc.
These reduce risk in many ways. Before DevOps, many organizations found that while getting ready to deploy changes, one feature, or piece of the project, could be dropped due lack of stability, or any other reason, and other pieces should still need to function and be deployed correctly with all of their dependencies. The DevOps approach helps to ensure that everyone understands the required changes and that essential changes are not accidently missed.
DevOps also teaches us to manage our environment better and to ensure that we deploy to a production-like environment early in the process. Too often, we find that the target environment for deployment is unstable and actually changed due to a hot fix, which originates from another team working in parallel without properly communicating with the other stakeholders. We also learn that changes might need to be deployed to multiple sites, each one with its own issues so automation is a must-have. Also, detailed information for each change including often overlooked changes to the database are essential.
Above all, automation ensures repeatability and reduces manual work and the potential for mistakes, thereby driving more reliable and less error-prone deployments. One common challenge faced by DevOps teams is trying to tame the many specialized technologies including robust database technologies that have their complexities and challenges.
A DevOps and Automation Gap: The Database
To practice DevOps one needs to be able to practice deployment automation. Otherwise deployments and releases will require manual steps and processes, which are not always accurately repeatable, prone to human errors, and cannot be handled with high frequency.
In order to properly deploy an application, you need to deploy everything dependent with it. As simple as the previous sentence sounds, it's true meaning is sometimes more complicated. Deploying updates to a client-based application, for example, will revolve mostly around putting files in their proper places and registering a few with some additional commands—quite simple.
Deploying updates to a web-server application will be a bit trickier with more parameters to deal with, but not conceptually different.
Dealing with the databases, however, is another thing entirely. The database, unlike other software components and code or compiled code, is not a collection of files. It is not something you can just copy from your development to testing and to production, obviously, because the database is a container of our most valued asset—our data. It holds all application content, customer transaction, etc. Business data must be preserved.
Unlike Java, C#, or HTML files or compiled code, a database shouldn’t be copied; it should only be altered at its target environment to mimic exactly the changes that we need to promote from our source environment (moving from development to testing to production, etc.). In order to promote database changes, a transition code needs to be developed; scripts to handle database schema structure (table structure), database code (procedures, functions, etc.), and content used by the application (metadata, lookup content, or parameters tables).
In future articles, I will discuss strategies for automating the complex changes to the database within the framework of agile software methodology and, especially, DevOps.
Summary
DevOps is a natural evolution of the software industry; it's not a revolution.
As business needs are the most significant driver of change, we are now doing more with less and delivering robust features faster. The database and the DBAs need to be first-class stakeholders in the DevOps revolution!
