Deployment: Configuration Management and Vulnerability Management (CMVM)

The overall goal of the Configuration Management and Vulnerability Management practice is change management. The SSG and application owners must ensure their ability to track authorized changes to applications and to detect unauthorized changes and activity. Application owners must enforce adherence to corporate policy.

Patching and updating applications, version control, defect tracking and remediation, incident handling.
  Objective Activity Level
CMVM1.1 know what to do when something bad happens create or interface with incident response 1
CMVM1.2 use ops data to change dev behavior identify software defects found in operations monitoring and feed them back to development
CMVM2.1 be able to fix apps when they are under direct attack have emergency codebase response 2
CMVM2.2 use ops data to change dev behavior track software bugs found during ops through the fix process
CMVM2.3 know where the code is develop an operations inventory of applications
CMVM3.1 learn from operational experience fix all occurrences of software bugs found in operations (T: code review) 3
CMVM3.2 use ops data to change dev behavior enhance the SSDL to prevent software bugs found in operations
CMVM3.3 ensure processes are in place to minimize software incident impact simulate software crisis
CMVM3.4 engage external researchers in vulnerability discovery operate a bug bounty program

CMVM Level 1: Use operations monitoring data to drive developer behavior. The SSG must support incident response. The SSG must use operations data to suggest changes in the SSDL and developer behavior.


Create or interface with incident response. The SSG is prepared to respond to an incident. The group either creates its own incident response capability or interfaces with the organization's existing incident response team. A regular meeting between the SSG and the incident response team can keep information flowing in both directions. In many cases, software security initiatives have evolved from incident response teams who began to realize that software vulnerabilities were the bane of their existence.


Identify software defects found in operations monitoring and feed them back to development. Defects identified through operations monitoring are fed back to development and used to change developer behavior. The contents of production logs can be revealing (or can reveal the need for improved logging). In some cases, providing a way to enter incident triage data into an existing bug tracking system (many times making use of a special security flag) seems to work. The idea is to close the information loop and make sure security problems get fixed. In the best of cases, processes in the SSDL can be improved based on operational data.


CMVM Level 2: Ensure that emergency response is available during application attack. Managers and the SSG must support emergency response to ongoing application attacks. Managers and the SSG must maintain a code inventory. The SSG uses operations data to direct evolution in the SSDL and in developer behavior.


Have emergency codebase response. The organization can make quick code changes when an application is under attack. A rapid-response team works in conjunction with the application owners and the SSG to study the code and the attack, find a resolution, and push a patch into production. Often times, the emergency response team is the development team itself. Fire drills do not count; a well-defined process is required.


Track software bugs found during ops through the fix process. Defects found in operations are fed back to development, entered into established defect management systems, and tracked through the fix process. This capability could come in the form of a two-way bridge between the bug finders and the bug fixers. Make sure the loop is closed completely. Setting a security flag in the bug tracking system can help facilitate tracking.


Develop operations inventory of applications. The organization has a map of its software deployments. If a piece of code needs to be changed, operations can reliably identify all of the places where the change needs to be installed. Sometimes common components shared between multiple projects are noted so that when an error occurs in one application, other applications that share the same components can be fixed as well.


CMVM Level 3: Create a tight loop between operations and development. The SSG must ensure the SSDL both addresses code deficiencies found in operations and includes enhancements that eliminate associated root causes.


Fix all occurrences of software bugs found in operations. The organization fixes all instances of each software bug found during operations and not just the small number of instances that have triggered bug reports. This requires the ability to reexamine the entire codebase when new kinds of bugs come to light. (See [CR3.3 Build capability for eradicating specific bugs from entire codebase].) One way to approach this is to create a rule set that generalizes a deployed bug into something that can be scanned for using automated code review.


Enhance the SSDL to prevent software bugs found in operations. Experience from operations leads to changes in the SSDL. The SSDL is strengthened to prevent the reintroduction of bugs found during operations. To make this process systematic, the incident response post mortem could include a "feedback to SSDL" step. This works best when root cause analysis pinpoints where in the SDLC an error may have been introduced or slipped by uncaught. An ad hoc approach is not sufficient.


Simulate software crisis. The SSG simulates high-impact software security crises to ensure software incident response capabilities minimize damage. Simulations could test for the ability to identify and mitigate specific threats or, in other cases, could begin with the assumption that a critical system or service is already compromised and evaluate the organization's ability to respond. When simulations model successful attacks, an important question to consider is the time period required to clean things up. Regardless, simulations must focus on security-relevant software failure and not natural disasters or other types of emergency response drills. If the data center is burning to the ground, the SSG won't be among the first responders.


Operate a bug bounty program. The organization solicits vulnerability reports from external researchers and pays a bounty for each verified and accepted vulnerability received. Payouts typically follow a sliding scale linked to multiple factors, such as vulnerability type (e.g., remote code execution is worth $10,000 versus CSRF is worth $750), exploitability (demonstrable exploits command much higher payouts), or specific services and software versions (widely-deployed or critical services warrant higher payouts). Ad hoc or short-duration activities, such as capture-the-flag contests, do not count. [This is a new activity that will be reported on in BSIMM6.]