DevSecOps: Incorporate Security into DevOps to Reduce Software Risk

By now, most organizations have heard of DevOps, and many have begun to adopt DevOps practices as a key enabler of software delivery. Organizations that employ an agile approach find DevOps practices a natural extension, and DevOps truly enables agile practices to flourish.

Organizations typically start with implementing continuous integration, test-driven development, and test automation early on. Agile delivery teams embrace small, iterative development and increased code quality, and with these practices in place, continuous delivery and continuous monitoring practices start taking hold. Teams focus on faster delivery with less human interaction, and successful organizations build more collaboration between development and operations teams as they start working toward a shared goal.

While all these practices provide strategic benefits like breaking down the traditional silos between development, test, and operations, it is unfortunate that many organizations stop there. In my experience, most organizations fail to integrate their security programs into their development efforts, resulting in lengthy security compliance activities and lots of vulnerabilities identified late in the delivery lifecycle. Does that sound like DevOps to you?

These issues are why the concept of DevSecOps is making such a large impact in the security community. DevSecOps is a growing movement to incorporate security into DevOps practices in order to ensure flaws and weaknesses are exposed early on through monitoring, assessment, and analysis, so remediation can be implemented far earlier than traditional efforts. By failing fast with security testing, organizations reduce risk of a security incident and decrease the cost of rework.

Exploring a DevSecOps Workflow

Here’s an example of DevSecOps in action:

1. Developers create the code and tests, which are managed by a version control system like Git.
2. Changes are committed to the Git.
3. Jenkins pulls the code from the repository and builds and runs unit tests, as well as static code analysis to identify code quality bugs and security defects.
4. An infrastructure-as-code tool, like Chef, provisions an environment, deploys the application, and applies security configurations to the system.
5. Jenkins runs a test automation suite against the newly deployed application, including UI, back-end, integration, API, and security tests.
6. If the application successfully passes all tests, the application is deployed to production using the same infrastructure-as-code tool used in the previous environments.
7. The production environment is continuously monitored by tools like New Relic and Splunk to detect active cyber security threats.

DevSecOps provides a number of benefits among development, security, and operations. It eliminates silos, promotes collaboration and teamwork, and identifies vulnerabilities early while still providing better, faster delivery. DevSecOps also contributes business value through dollars and resources saved, improved operations, diminished security threats, reduction of rework, and increased quality through automated testing, as well as the delivery of projects and products early and often, with less cycle time to the customer.

In short, we can spend more time adding customer value to our software and less time and money fixing costly security vulnerabilities identified late in the delivery process or in production.

DevSecOps Means More Than Automation

When people think of DevSecOps, the first thing that comes to mind is automation. A strong DevSecOps environment should employ tools that automate continuous integration, delivery, testing, deployment, and monitoring.

While automation is certainly important, it’s just as important (if not more) to build the mindset that everyone is responsible for security, with the goal of safely distributing security decisions to those with the highest level of context without sacrificing the quality, privacy, and safety required by the system. This type of model typically relies on “shifting security left,” or engaging security earlier in the software development and operations processes.

Changing an organization’s mindset and culture doesn’t happen overnight, so don’t expect integrating security into DevOps to be any different. It requires hard work, training, coaching across the entire organization, and patience.

While there is no one right way to change organizational culture, a few critical components are necessary for DevSecOps to take hold:

• Building a knowledge base: Raising a developer’s security knowledge plays enormous dividends. Ongoing training and learning activities ensure not only that developers know how to be responsible for security, but also that they continue to stay on top of cyber security best practices.
• Promoting openness: Openness in communication encourages collaboration and continuous improvement between development and security. Transparency of information using metrics and dashboards can be an effective mechanism for communicating what’s really going on in a quantitative way. By building trust and cooperation through openness, organizations can ensure security does not become reactionary.
• Creating cyber security champions: The lack of highly qualified security professionals can make the transition from DevOps to DevSecOps difficult. Savvy organizations identify individuals who understand security within traditional dev and ops groups and trust these individuals to coach DevSecOps teams and act as the security conscience of the team during the transition.

Implementing the DevSecOps Process

The primary goal of DevSecOps is to ensure security and operations team members are engaged and collaborating with development from the very beginning of a project or product development. In addition to cultural shifts, it demands a toolchain of technologies to facilitate collaborative change. It requires pushing past departmental lines for more effective planning, design, and release of secure products.

As organizations continue to build upon automated delivery, they find there are opportunities to test for issues beyond typical bugs: potential security flaws, design defects, and code weaknesses. Imagine being able to identify and fix flaws earlier in delivery process, before they are exposed to the public.

Implementing DevSecOps in an organization requires building a single group of engineers (developers, admins, testers, and security engineers) that has end-to-end responsibility of the application, from requirements to deployment to monitoring and back to implementing new changes. This process forms a set of stages that can be carried out in a continuous loop until the desired product is achieved.

The diagram below shows the steps in a DevSecOps lifecycle, as well as some standard tools used in the toolchain.

Plan. All projects require planning. DevSecOps projects must plan user stories with more than just features descriptions. They should include functional and nonfunctional requirements (like security and performance), acceptance test criteria, UI and UX designs, and threat models. Security begins at planning before a single line of code is developed.

Develop. Generally, it is much less expensive to develop secure software than to correct security issues after the software package has been completed. Development teams should start by assessing the maturity of the practices, gaining sufficient resources to provide necessary guidance (like the OWASP secure development guide), and implementing code reviews of software design and implementation.

Build. Automated build tools do more than compile code. A tool like Gradle can be used to conduct test-driven development, enforce standards for release artifact generation, and ensure design and implementations comply with team coding standards and security best practices through static code analysis.

Test. Test automation in a DevSecOps environment is much more than UI-focused Selenium tests. Strong security testing practices should include unit, front-end, back-end, API, database, and passive security testing. Passive security testing can be completed with little to no effort by utilizing a robust testing framework, likeSelenified, with a security scanner in proxy mode.

Secure. Traditional security testing doesn’t go away in DevSecOps organizations; we just anticipate identifying far fewer issues late in the development process. When we identify vulnerabilities with security scanning, we often have a greater context of the issues and can more confidently determine if the vulnerability is a potential exploitation or a false positive.

Deploy. Using an infrastructure-as-code tool, like Chef, automated provisioning and deployments can be utilized to expedite delivery of software and ensure more consistency in the development process. It also can be used to audit properties and configurations across the IT infrastructure, as well as to enforce secure configurations for all systems and services.

Operate. Routine maintenance and upgrades are an important component of any operations team. Zero-day vulnerabilities need to be patched rapidly to reduce exposure time. DevSecOps teams leverage infrastructure-as-code tools so that updates can be applied to the entire organization’s infrastructure rapidly and consistently, with no human error.

Monitor. Implementing a strong continuous monitoring program makes it possible to obtain real-time evidence of how your system is performing and exploitations that may be taking place against your system or its data. This allows organizations to review whether controls and systems function as intended on an ongoing basis.

Scale. Virtualization and the cloud is an important piece of any modern IT infrastructure. The ability to scale infrastructure to the changing demands of its userbase, or even being able to completely replace a compromised environment in minutes, are now real-world problems that we can’t successfully solve with traditional data-center operations.

Adapt. Continuous improvement is a hallmark of any strong agile practice. DevSecOps practices also must continuously improve and adapt as issues (whether usability, security, or performance) are identified. This informs decision-making, planning, and how teams improve the overall software development lifecycle.

Measuring DevSecOps Success

When utilizing DevSecOps practices, success should be measured quantitatively by the efficiency of continuous development, quality of code, threat detection, and release cycles. Establishing key metrics to determine success—and tracking an organization’s progress against those metrics—is important.

Here are some frequently used DevSecOps metrics:

• Deployment frequency
• Lead time
• Test coverage
• Detection of threats, security defects, and flaws
• Mean time to repair
• Mean time to recovery

DevSecOps success relies on gradual changes to various concepts within your organization. Existing frameworks can be enhanced or replaced with new practices. With the rise of DevSecOps, we get to truly define how operations, engineering, and security can be brought together to achieve unparalleled success.