Environments Management—A Key Component for Release Management

The demand for software environments has increased as more organizations use agile software development practices and are required to provide a fast turnaround of deliverable IT projects. In a rapid IT delivery cycle, software test environments provide the platform on which applications are validated through various test phases, including the system test, regression test, and user acceptance test, among others.

However, the complexity of the test applications landscape, the number of applications under scope, and the integration architecture and technology variants can all increase the challenges of managing software test environments—including maintaining complex hardware and software deployments, documenting and defining a standard provisioning process, and providing general housekeeping practices.

Depending on the complexity of test phases and the deployment and infrastructure architectures, it can be cumbersome and time consuming for you to properly manage test environments. However, test environments that are not defined and managed can potentially derail your overall release process as frequent changes to the software environments can delay the overall release cycle and test timelines.

The Nuts and Bolts of Software Environments
Environments are deployment platforms on which applications are deployed and configured. They are made available during various phases of the deployment lifecycle, including build, install, configure,and verify. A software environment is a collection of hosted servers and application software that provides a platform for execution and testing of applications. Typically an environment consists of:

• One or more physical or virtual servers
• Hosted applications, including deployment, installations, and releases on web servers; application servers; database servers; legacy applications; etc.
• Deployment environment configurations
• Network Infrastructures, such as firewalls, SAN Storage, network switch, etc.

The release team provides the services for deployment on the environments and makes it available during various stages of release and test cycles.

Relationship Between Environments and Deployments
Although environments and deployments are similar in that they are both related to environments, they do have their differences. Deployment management primarily focuses on planning, executing, and managing deployments, whereas environment management centers on identifying and sandboxing a set of resources on which you can perform deployments. Only after environments have been identified and scoped should you initiate a deployment.

Although environments management is a part of the release management process, it also involves the following: environments planning, environments configuration, environments verification, and environments communication.

Figure 1: A release management workflow chart detailing environments management synced up with release management as software progresses from development to live production.

Environments are planned and created based on the release and test strategy formulated by stakeholders during the IT development lifecycle. Typically, software development progresses from the initial design and development phase to the QA/testing phase to the eventual live deployment of the product. Software test environments often differ from each other by their purpose and line of business supported. The “customer-facing software environment,” which is the live production environment where the user is actually using the application is just one of many different software environments.

When a product’s development is in its initial stages, the release and planning strategies require development environments in order to allow testing across multiple phases before the software is deployed. As test teams begin to test the applications, other environments are created and managed by the release team based on the requirements of the test and release cycles.Typically, release environments fall in the following categories:

• Development Environments are environments created for the development teams to validate builds and releases.

• System Test Environments are environments that facilitate integration tests during the system test phase. A system test is the initial testing phase during the QA cycle. Typically in these environments, the QA team will be conducting black box testing to validate various system and functional capabilities. From the environment perspective, a system test may not have the hardware and software configurations similar to what may be applied to live.

• User Acceptance Test Environments are created for end-user testing in order to validate the system by meeting the agreed upon functionalities. These environments are used during the later stages or end phase of testing prior to the application being accepted by the customer during the live release. The deployment architecture and environment configurations are similar to live.

• Preproduction Environments emulate live production. They are used as staging environments before an application is deployed to live and also as test environments in which you can validate live issues. These environments have hardware that mirrors the hardware used in the production stage. Application deployment and configuration would be the same as it would be on live. For example, if clustering application servers are used on live, the same should be provided in these environments.

• Performance Test Environments and Capacity Testing Environments are release environments created and configured to facilitate testing of system performance—responsiveness, stability, etc.—during the performance testing phase. Typically load testing, stress testing, and soak testing are performed in these environments. 

• Live Environments are environments where applications are deployed on succesful completion through various QA test phases and on final acceptance from the user. Actual users of the system will use the applications deployed on live environments, hence these environments are considered critical for the business.

Environments Management Process
Because software releases are always deployed on environments, the environments management process is linked to release management. Software release management emphasizes planning and managing the release for successful delivery, but in order for it to succeed, test environments are needed to track a team's progress. The environments management process helps in identifying, scoping, planning, and allocating the right environments for release. Environments management is an iterative process, involving the following steps detailed in the next sections.

Environments Planning
This is the planning phase of environments management. In this phase, a release manager interacts with stakeholders—such as QA managers, development managers, and technology architects (infrastructure/software)—to understand the requirements of the environments. The release manager must define and manage all of the indentification, scoping, and planning pertaining to the environments.

Regarding identification, the release manager must define the software environments in the context of the software project or product being tested and released. The release manager should also be working with the test manager to come to an agreement regarding the scope and identification of the test environments. The release manager must Identify environment requirements in order to align with the release cycles, QA test phase, and training requirements of projects. The release manager is responsible for managing the progression of software release, deployment, and configuration through various test phases and live. In order to deploy, configure, and manage an application, it is essential that the release manager knows what environments are required.

When a release manager performs scoping, he identifies the hardware and software resources required for environments. In this phase, the release team should be receiving overview of all the different interfaces, applications, hardware, and network requirements needed for the provisioning of release to environments. Typically, a release team interacts with the development, infrastructure, architects, and test team in order to get an overview of the environments requirements.The release manager must also plan the creation and allocation of environments to be in line with the environments requirements. It is a standard practice for a release manager to create and update environment plans. An environment plan can be a simple Excel spread sheet containing information, such as the type of environment (system test, UAT, etc.) used, the allocated hardware and its configuration, and dates. The environment plan can also be as detailed as a project plan.

Environments Configuration
Environment may differ based on configurations. Configurations are changes done on a environment to change the behavior of an environment, and they need to be managed in a controlled manner. Environment configurations are changes introduced to applications within an environment that affect the run-time functionality of applications. Configurations can be as simple as database connectivity or as complex as changing network configurations across a suite of servers in order to enable business functionality via an application. Whether it's hardware configuration or software configuration, configurations can change the run-time behavior of a test environment and its usability, and it is for this reason that you should never apply a development server configuration in a system test or live environment.

CI Identification
Configuration items (CI) are changes or configurations that need to be identified and managed on environments. The identification of CIs is one of the critical functionalities in environments configuration. This identification process involves gathering the attributes of CI, including the CI's name, version, location, and value, as well as its behavior across multiple environments. CIs are configurations that can change the run-time behavior of a system, an example of which is the information about a database connection, such as driver details or the initial connection pool size. In order to identify a CI, the release team needs to define a process to do so. Such a process involves reviewing and gathering details on CIs introduced in the system via the release process. This information should be gathered either via the documentation that comes with the release or by gathering the information from the teams introducing the CIs.

It can be a nightmare when development configurations have been applied to live. For example, if a database server name is not identified as a CI and the same configuration has been applied to live, there could be a situation in which information is available on development servers during live. It would take a major effort to resolve such an instance.

CI Management
CI management is a process for managing the identified CI. In order to manage the CIs, you could perform activities such as creating a CI repository or defining the change control mechanism for managing changes to the CIs.

Creation of CI Repository
Identified configuration items used by the environment's release process need to be persisted (stored and documented. Persisting configuration items ensures that the release team maintains a unique set of CIs that can be used during the release cycle.

The Definitive Media Library (DML) is the ITIL v3 term used to outline the creation of a repository for CIs, and it is the centralized location for managing CIs that are then used for various releases. The structure of the DML—its design and contents—is based on factors including the configuration, release cycles, environments, and other specific information pertaining to the products used. The creation and management of DML is important from an environment management perspective, because environments will be configured based on the validated CIs maintained in the DML.Control of CIs
Something that I have come cross regularly is that CIs have been changed or modified regularly on the environments after the release and deployment stages. It is very hard and time consuming to debug environment problems when CIs are not controlled. If project teams dramatically alter CIs in environments without the knowledge of the release team or without a change process, it can be difficult for the release team to identify why the run-time behavior of the application varies across environments. Someone mistakenly uninstalling software or changing some configuration in an environment can cause the application to behave differently than originally expected. Having a configuration item repository helps in resolving such issues, because one can refresh or restore an environment based on the CIs from the repository.

As project teams occasionally introduce on the fly changes to an environment in order to improve functionality for the QA team, new undocumented changes are being implemented. The release team needs to resolve the same CI issues across multiple environments. If a change to a configuration item is not managed in the CI repository, the change can go unnoticed until a major issue occurs later. The release team, which is the key contributor to a CI repository, should own and define the control and updates of CIs.

Environments Verification
The environments verification is a process where the release manager regularly validates environments used across the broad spectrum of projects. It’s quite common that environments are allocated during the planning phase to projects, and over a period of time, the environments may not be required. Depending on the environments requirements across multiple projects, there could be a demand for a multiple number of environments in order to facilitate either testing or training. When either the projects have been completed or the training has finished, it is essential for the release manager to review if the environments allocated are still being used. The environments verification process provides general housekeeping activities to implement in the environments, and it regularly validates the usage of resources. These housekeeping activities could involve backing up servers, stopping the servers, uninstalling applications, upgrading servers, and clearing out installed files and applications in order to free up space.

The process provides several methods for collecting metrics and providing feedback to management, including:

• Visibility and information to feed project plan and delivery commitments, such as explaining what is available and when it is available.

•  Data and metrics to calculate the cost—associated with the hardware, software, and people—and return on investment as part of the individual project and organization as a whole.
• Measurements for utilization, productivity, and costs associated with the resources. 
• Informational data pertaining to what the team has and what the team needs or does not need in the future.

Environments Communication
Environments communication is a process where the release manager communicates information relating to the environments. In order to provide good communication, the release manager should have answers to the following questions: What needs to be communicated? Who needs to know? How should we communicate?It’s common that multiple environments share resources, including physical or virtual servers, application servers, and database servers. Shared resources can affect the functionality of the environments, as changes to the external interfaces and networks affect the behavior and testing capabilities of the multiple environments. Information pertaining to changes needs to be actively discussed. This information includes the current environments being used, the users of the environment, the date of release, the applications configured, and the hardware and network changes introduced. Environment communication could be a medium for informing teams about infrastructure changes that could affect the environments used by teams. For example, there could be two project teams (Project A and Project B) working on application servers hosted from one physical server. However, due to the requirements of Project A, the server may need to restart, which could lead to downtime with its associated risks for Project B. Hence, it is important that a release manager communicates information to various stakeholders of the environments. The prime stakeholder for the environments' communication may include the QA team, project team, release team, operational support team, architects, and other business stakeholders.

Release managers also need to introduce a communication medium where all the stakeholders of environments have information pertaining to resources used within the environments. Some common mediums through which information on environments is provided include: environments dashboard, environments usage Wiki, environments catalogue, and Excel spread sheets.

Conclusion
Environment issues in non-production environments can cause lost time on IT projects. Organizations should look at efficiently implementing an environments management process in order to counter this issue. Effective environment management improves the quality, availability, and efficiency of the environments in order to meet milestones, as well as ultimately reducing both the time to market and costs.

About the Author
Subhendu Mohapatra is a senior consultant with Bearing Point, Ireland. Subhendu has been working with BearingPoint providing consultancy and helping a wide variety of clients efficiently plan release and environments management activities critical to sustaining and delivering IT services. BearingPoint is one of the leading IT consulting organization specializing in providing IT management consultancy.