Ineffective operating system level security can jeopardize your entire SCM operations and the hundreds of millions of dollars of software assets you are trying to control. It is important to match SCM to your company’s security model and use OS security to exercise control over application runtime environments, which CCM tools typically do not control. Here are some important guidelines for using operating system security to provide robust SCM for Windows and *NIX platforms.
Understand Your Company’s Security Model
Your company’s security model, likely stops at securing hardware, operating system (OS) and database management system (DBMS) security. By operating system I am referring to the base operating system such as Windows 2000 or AIX and the mechanisms these OS’s use to secure file systems. I won’t belabor web security here because a web server is just an application on top of network (hardware), DBMS and OS security.
Hardware and OS security are relatively mature fields and your company’s security policies are likely to be widely understood and implemented. SCM, on the other hand, is often not as well understood throughout the organization and its implementation given a later start. This is to be expected since SCM is in many respects an extension, or specialized form of, security that includes additional change controls and reporting.
While OS security attempts to secure file and memory objects, and database security database objects, SCM attempts to secure and control changes to business objects that (should) map reasonable well to the file, memory and database objects. Since SCM usually involves getting into the nitty-gritty of files and database objects, you should understand the ins and outs of your company’s policies and conform to them to avoid run-in’s with you security organization.
Make Sure Security Is Implemented
Secondly, make sure your company’s security model is actually implemented to the detail you need for the applications you have or intend to have under SCM. If you are tasked with implementing SCM for a particular mission critical application and it is not locked down according to your company’s guidelines, then guess what? You’re going to be implementing OS level security also! Otherwise, you can’t trust the integrity of the file and database objects that the enterprise objects are based on. And then what’s the point of managing objects that you can’t trust the integrity of?
Many Enterprise-oriented SCM tools today require effective OS and DBMS level security measures, not surprisingly. They are no different than any other application in this regard, including web servers. Without these measures in place the control functions of your SCM tool can be bypassed or otherwise put at risk.
You may find that your company’s security policies are not detailed enough or otherwise inadequate to protect your enterprise objects. You may also find the policies are too strict and inhibit highly beneficial SCM controls or audit functions. Work closely with your security organization to make modifications with the understanding that you are working toward a common goal.
Securing Application QA Test and Production Runtime Environments
Unlike the mainframe, few distributed platforms CCM tools attempt to manage application runtime environments. This very important area is the machine and location on the machine where the application actually executes. To implement effective SCM, you must resort to the design and implementation of OS-level security measures in addition to mastering one or more software tools designed for SCM functions.
Aside from preventing malicious changes or disruptions to the application, OS-level security measures can act as a change control mechanism, limiting access to those who with the appropriate role to change them. This in turn has two consequences: greater segregation and clarity of roles and responsibilities, and reduction of change volume.
The benefits of locking down the production runtime environment to a limited few charged with production control responsibilities has long been known. On the distributed platforms you have little choice but to employ OS level security, including managing the passwords to the machines themselves. This concept should also be extended to critical QA test environments.
A classic example of the utility of OS security acting as change control mechanism is in the identification and resolution of defects in QA testing. Development code is first handed off for install into the test environment. The QA tester may then notify a developer of a defect identified in testing. The developer, with an understanding of the issue and an eye for a possible resolution, applies a fix to the testing environment in-situ. The reasoning is that this is a very efficient way to test the defect resolution since a nice test environment is already set up. This may involve compiling an executable in a development area with source code in the development environment and transferring the executable to the test environment.
The problem is now that it is too easy to make changes to the test environment, and those changes are not made in a controlled manner. The test environment effectively becomes a development environment too, and with only a few corrections made, the integrity of both the development and testing configurations come into question. This disrupts what is a normally a linear process of develop-and-then-test creating something more murky. It happens that the code is then released directly to production from the QA test environment, and no one can remember what changes were made where after the next round of Unreal Tournament.
By limiting access to the test environment to the testers, much greater accountability is placed on the testers for the integrity of that environment. Greater segregation and clarity of the activities in the test environment is achieved and the tester won’t have to point a finger at the developer as being responsible for supposedly “fixing” the defect that led to a successful QA test but failed in production. The testers are equally responsible. This tighter access requires a robust turnover mechanism so that the handoff of changes from development to QA is repeatable and efficient. This is where we look for CCM tools to provide the functionality.
Other Things to SecureUnlike the mainframe, few distributed platforms CCM tools attempt to manage application runtime environments. This very important area is the machine and location on the machine where the application actually executes. To implement effective SCM, you must resort to the design and implementation of OS-level security measures in addition to mastering one or more software tools designed for SCM functions.Aside from preventing malicious changes or disruptions to the application, OS-level security measures can act as a change control mechanism, limiting access to those who with the appropriate role to change them. This in turn has two consequences: greater segregation and clarity of roles and responsibilities, and reduction of change volume.The benefits of locking down the production runtime environment to a limited few charged with production control responsibilities has long been known. On the distributed platforms you have little choice but to employ OS level security, including managing the passwords to the machines themselves. This concept should also be extended to critical QA test environments.A classic example of the utility of OS security acting as change control mechanism is in the identification and resolution of defects in QA testing. Development code is first handed off for install into the test environment. The QA tester may then notify a developer of a defect identified in testing. The developer, with an understanding of the issue and an eye for a possible resolution, applies a fix to the testing environment in-situ. The reasoning is that this is a very efficient way to test the defect resolution since a nice test environment is already set up. This may involve compiling an executable in a development area with source code in the development environment and transferring the executable to the test environment.The problem is now that it is too easy to make changes to the test environment, and those changes are not made in a controlled manner. The test environment effectively becomes a development environment too, and with only a few corrections made, the integrity of both the development and testing configurations come into question. This disrupts what is a normally a linear process of develop-and-then-test creating something more murky. It happens that the code is then released directly to production from the QA test environment, and no one can remember what changes were made where after the next round of Unreal Tournament.By limiting access to the test environment to the testers, much greater accountability is placed on the testers for the integrity of that environment. Greater segregation and clarity of the activities in the test environment is achieved and the tester won’t have to point a finger at the developer as being responsible for supposedly “fixing” the defect that led to a successful QA test but failed in production. The testers are equally responsible. This tighter access requires a robust turnover mechanism so that the handoff of changes from development to QA is repeatable and efficient. This is where we look for CCM tools to provide the functionality.
It can’t be emphasized enough that the OS must be secured prior to utilizing application SCM. I see examples, about once per year per machine in secure environments, where the system administrators (those immune to OS security measures) mistakenly change the file owners or permissions, remove links or just plain delete something they shouldn’t have. The command line nature of *NIX administration and the muddy line between application and OS on Windows means mistakes will happen every now and then. The point is that if this happens in secure environments, then imagine the situation in more unsecured environments.
A more surreptitious problem occurs with development tool runtime environments. These runtime environments refer to a set of executables (not file locations as in the production and QA test environments). They are closely associated with the operating system, so that an update to the operating system can alter the runtime environment and crash the application. This is as important in Windows environments as *NIX, the difference being that typically only a system administrator has access to make these changes on *NIX.
A real-life instance of this occurred due to an innocent update made to an Oracle client by the system administrator on a UNIX machine. This happened on a development server, and the developers, from their point of view, inexplicably started getting compile errors building their C++ application. Unbeknownst to all, the update to the Oracle client included an operating system patch that upgraded the C/C++ runtime executables and libraries. The system administrator had installed the update as the root user, which bypasses OS file security. The operation of the application was put at great risk since this was the only build server for this marketing application. The developers would have been unable to build a production fix if it were necessary until they backed out the Oracle client and the OS and C/C++ runtime patch. This shows the need to lock down the development tool runtime environments and involve operating system administrators in SCM. In this case, there was an approval-based change system in place, but the impact was not fully assessed ahead of time.
Summary
As they say, HTH. Hope this helps. Here is a summary of what we’ve covered:
- Understand your company’s security model down to the details at the file level. You’ll need this to implement SCM in a manner compliant with your company’s security model.
- Make sure hardware, OS and DBMS security mechanisms are in place prior to attempting further controls for SCM. Work with your security organization to make policy changes if necessary.
- Be sure to secure application runtime environments for production and critical QA testing for better segregation of roles and responsibilities and controlling and limiting changes.
- Work out procedures with “Those With Infinite Access” (the system administrators), such as an approval based change control mechanism for operating system/runtime environment changes.
CCM tools do not usually attempt to manage OS security and runtime environments, nor should they be expected to because of the complexity and the rapidly evolving nature of Windows security functions. And so, OS security is a critical part of achieving effective SCM on the distributed platforms. Thanks to Asad Shaikh of ABN AMRO, N.A. for helpful discussion.
Sean Blanton is Director of Consulting for Catalyst Systems Corporation. Sean has been with Catalyst for 6 years as a distributed platforms change and configuration management consultant and as a developer, trainer and product contributor for Openmake. He has a Ph.D. in Physics from the University of Chicago and a B.S. from Columbia University.
