Governance: Strategy and Metrics (SM)

The overall goals for the Strategy and Metrics practice are transparency of expectations and accountability for results. Executive management must clarify organizational expectations for the SSDL so that everyone understands the importance of the initiative. In addition, executive management must set specific objectives for all SSDL stakeholders and ensure that specific individuals are made accountable for meeting those objectives.

Planning, assigning roles and responsibilities, identifying software security goals, determining budgets, identifying metrics and gates.
  Objective Activity Level
SM1.1 make the plan explicit publish process (roles, responsibilities, plan), evolve as necessary 1
SM1.2 build support throughout organization create evangelism role and perform internal marketing
SM1.3 secure executive buy-in educate executives
SM1.4 establish SSDL gates (but do not enforce) identify gate locations, gather necessary artifacts
SM1.6 make clear who's taking the risk require security sign-off
SM2.1 foster transparency (or competition) publish data about software security internally 2
SM2.2 change behavior enforce gates with measurements and track exceptions
SM2.3 create broad base of support create or grow a satellite
SM2.5 define success identify metrics and use them to drive budgets
SM3.1 know where all apps in your inventory stand use an internal tracking application with portfolio view 3
SM3.2 create external support run an external marketing program

SM Level 1: Attain a common understanding of direction and strategy. Managers must ensure that everyone associated with creating, deploying, operating, and maintaining software understands the written organizational software security objectives. Leaders must also ensure that the organization as a whole understands the strategy for achieving these objectives. A common strategic understanding is essential for effective and efficient program execution.


Publish process (roles, responsibilities, plan), evolve as necessary. The process for addressing software security is broadcast to all participants so that everyone knows the plan. Goals, roles, responsibilities, and activities are explicitly defined. Most organizations pick and choose from a published methodology such as the Microsoft SDL or the Cigital Touchpoints and then tailor the methodology to their needs. An SSDL process evolves as the organization matures and as the security landscape changes. In many cases, the methodology is published only internally and is controlled by the SSG. The SSDL does not need to be publically promoted outside of the firm to count.


Create evangelism role and perform internal marketing. In order to build support for software security throughout the organization, someone in the SSG plays an evangelism role. This internal marketing function helps keep the organization current on the magnitude of the software security problem and the elements of its solution. Evangelists might give talks for internal groups, extend invitations to outside speakers, author white papers for internal consumption, or create a collection of papers, books, and other resources on an internal web site and promote its use. Ad hoc conversations between the SSG and executives or an SSG where "everyone is an evangelist" does not achieve the desired results. A canonical example of such an evangelist is Michael Howard's role at Microsoft.


Educate executives. Executives learn about the consequences of inadequate software security and the negative business impact that poor security can have. They also learn what other organizations are doing to attain software security. By understanding both the problem and its proper resolution, executives come to support the software security initiative as a risk management necessity. In its most dangerous form, such education arrives courtesy of malicious hackers or public data exposure incidents. Preferably, the SSG demonstrates a worst-case scenario in a controlled environment with the permission of all involved (e.g., actually showing working exploits and their business impact). In some cases, presentation to the Board can help garner resources for an ongoing software security initiative. Bringing in an outside guru is often helpful when seeking to bolster executive attention.


Identify gate locations, gather necessary artifacts. The software security process will involve release gates/checkpoints/milestones at one or more points in the SDLC or, more likely, the SDLCs. The first two steps toward establishing release gates are: 1) to identify gate locations that are compatible with existing development practices and 2) to begin gathering the input necessary for making a go/no go decision. Importantly at this stage, the gates are not enforced. For example, the SSG can collect security testing results for each project prior to release, but stop short of passing judgment on what constitutes sufficient testing or acceptable test results. The idea of identifying gates first and only enforcing them later is extremely helpful in moving development toward software security without major pain. Socialize the gates, and only turn them on once most projects already know how to succeed. This gradual approach serves to motivate good behavior without requiring it.


Require security sign-off. The organization has an initiative-wide process for accepting security risk and documenting accountability. A risk acceptor signs off on the state of all software prior to release. For example, the sign-off policy might require the head of the business unit to sign off on critical vulnerabilities that have not been mitigated or SSDL steps that have been skipped. Informal risk acceptance alone does not count as security sign off, as the act of accepting risk is more effective when it is formalized (e.g., with a signature, form submission, or similar) and captured for future reference.


SM Level 2: Align behavior with strategy and verify adherence. Managers must explicitly identify individuals responsible for software security risk management accountability. These individuals, are in turn responsible for ensuring successful performance of SSDL activities. SSDL managers must ensure quick identification and modification of any SSDL behavior resulting in unacceptable risk. To reduce unacceptable risk, managers must identify and encourage the growth of a software security satellite (see the Roles section above).


Publish data about software security internally. The SSG publishes data internally on the state of software security within the organization. The information might come as a dashboard with metrics for executives and software development management. Sometimes, publication is not shared with everyone in a firm, but rather with the relevant executives only. Publishing information up to executives who then do something about it and drive change in the organization suffices. In other cases, open book management and publishing data to all stakeholders helps everyone know what's going on, with the philosophy that sunlight is the best disinfectant. If the organization's culture promotes internal competition between groups, this information adds a security dimension to the game.


Enforce gates with measurements and track exceptions. SDLC security gates are now enforced: in order to pass a gate, a project must either meet an established measure or obtain a waiver. Even recalcitrant project teams must now play along. The SSG tracks exceptions. A gate could require a project to undergo code review and remediate any critical findings before release. In some cases, gates are directly associated with controls required by regulations, contractual agreements, and other business obligations and exceptions are tracked as required by statutory or regulatory drivers. In other cases, gate measures yield key performance indicators that are used to govern the process.


Create or grow a satellite. The satellite begins as a collection of people scattered across the organization who show an above-average level of security interest or skill. Identifying this group is a step towards creating a social network that speeds the adoption of security into software development. One way to begin is to track the people who stand out during introductory training courses. (See [T2.7 Identify satellite through training].) Another way is to ask for volunteers. In a more top down approach, initial satellite membership is assigned to ensure complete coverage of all development/product groups. Ongoing membership should be based on actual performance. A strong satellite is a good sign of a mature software security initiative.


Identify metrics and use them to drive budgets. The SSG and its management choose the metrics that define and measure software security initiative progress. These metrics will drive the initiative's budget and allocation of resources so simple counts and statistics won't suffice. Metrics also allow the SSG to explain its goals and its progress in quantitative terms. One such metric could be security defect density. A reduction in security defect density could be used to show a decreasing cost of remediation over time. The key here is to tie technical results to business objectives in a clear and obvious fashion in order to justify funding. Since the concept of security is already tenuous to business people, making this explicit tie can be very helpful.


SM Level 3: Practice risk-based portfolio management. Application owners and the SSG must inform management of the risk associated with each application in the portfolio. The SSG must advertise its activities externally to create support for its approach and enable ecosystem security.


Use an internal tracking application with portfolio view. The SSG uses a centralized tracking application to chart the progress of every piece of software in its purview. The application records the security activities scheduled, in progress, and completed. It incorporates results from activities such as architecture analysis, code review, and security testing. The SSG uses the tracking application to generate portfolio reports for many of the metrics it uses. A combined inventory and risk posture view is fundamental. In many cases, these data are published at least among executives. Depending on the culture, this can cause interesting effects through internal competition. As an initiative matures, and activities become more distributed, the SSG uses the centralized reporting system to keep track of all of the moving parts.


Run an external marketing program. The SSG markets the software security initiative outside the firm to build external support. Software security grows beyond being a risk reduction exercise and becomes a competitive advantage or market differentiator. The SSG might write papers or books. It might have a public blog. It might participate in external conferences or trade shows. In some cases, a complete SSDL methodology can be published and promoted externally. Sharing details externally and inviting critique can bring new perspectives into the firm.