Governance: Strategy and Metrics (SM)
The overall goals for the Strategy and Metrics practice are transparency of expectations and accountability for results. Executive management must clarify organizational expectations for the SSDL so that everyone understands the importance of the initiative. In addition, executive management must set specific objectives for all SSDL stakeholders and ensure that specific individuals are made accountable for meeting those objectives.
SM Level 1: Attain a common understanding of direction and strategy. Managers must ensure that everyone associated with creating, deploying, operating, and maintaining software understands the written organizational software security objectives. Leaders must also ensure that the organization as a whole understands the strategy for achieving these objectives. A common strategic understanding is essential for effective and efficient program execution.
SM1.1
Publish process (roles, responsibilities, plan), evolve as necessary. The process for addressing software security is broadcast to all participants so that everyone knows the plan. Goals, roles, responsibilities, and activities are explicitly defined. Many organizations begin with a published methodology such as OWASP SAMM Microsoft SDL, or the Cigital Touchpoints and then tailor the methodology to their needs. An SSDL process evolves as the organization matures and as the security landscape changes. In many cases, the methodology is published only internally and is controlled by the SSG. That is, the SSDL does not need to be publically promoted outside of the firm to count.
SM1.2
Create evangelism role/internal marketing. In order to build support for software security throughout the organization, the SSG plays an evangelism role (regardless of whether it also plays an audit role). This internal marketing function helps the organization understand the magnitude of the software security problem and the elements of its solution. The SSG might give talks for internal groups, extend invitations to outside speakers, author white papers for internal consumption, or create a collection of papers, books, and other resources on an internal Web site and promote its use. In some cases, an internal software security evangelist is identified and tasked with these activities.
SM1.3
Educate executives. Executives learn about the consequences of inadequate software security and the negative business impact that poor security can have. They also learn what other organizations are doing to attain software security. By understanding both the downside and its proper resolution, executives come to support the software security initiative as a risk management necessity. In its most dangerous form, downside education arrives courtesy of malicious hackers or public data exposure incidents. Preferably, the SSG demonstrates a worst-case scenario in a controlled environment with the permission of all involved. In some cases, presentation to the Board can help to garner resources for an ongoing software security initiative. Bringing in an outside guru or victim is often helpful.
SM1.4
Identify gate locations, gather necessary artifacts. The software security process will involve release gates/checkpoints/milestones at one or more points in the software development lifecycle (SDLC) or SDLCs. The first two steps toward establishing these release gates are to identify gate locations that are compatible with existing development practices and to begin gathering the input necessary for making a go/no go decision. Importantly at this stage, the gates are not enforced. For example, the SSG can collect security testing results for each project prior to release, but stop short of passing judgment on what constitutes sufficient testing or acceptable test results. The idea of identifying gates first and only enforcing them later is extremely helpful in moving development toward software security without major pain. Socialize the gates and only turn them on once most projects already know how to succeed.
SM1.6
Require security sign-off. The organization has a process for security risk acceptance and accountability. The risk acceptor signs off on the state of the software prior to release. For example, the sign-off policy might require the head of the business unit to sign off on critical vulnerabilities that have not been mitigated or SSDL steps that have been skipped. Risk acceptance alone does not count as security sign-off, as the act of accepting risk is more effective when it is formalized.
SM Level 2: Align behavior with strategy and verify behavior. Managers must explicitly identify those individuals responsible for software security risk management accountability, who are in turn responsible for ensuring successful performance of SSDL activities. SSDL managers must ensure quick identification and modification of any SSDL behavior resulting in unacceptable risk. To reduce unacceptable risk, managers must identify and encourage the growth of a software security satellite (see the Roles section above).
SM2.1
Publish data about software security internally. The SSG publishes data internally on the state of software security within the organization with the philosophy that sunlight is the best disinfectant. If the organization’s culture promotes internal competition between groups, this information adds a security dimension to the game. The information might come as a dashboard with metrics for executives and software development management. In some cases, publication is not shared with everyone in a firm, but rather with the relevant executives. Publishing information up to executives who then do something about it and drive change in the organization suffices. In other cases, open book management and publishing data to all stakeholders helps everyone know what’s going on.
SM2.2
Enforce gates with measures and track exceptions. Gates are now enforced: in order to pass a gate, a project must either meet an established measure or obtain a waiver. Even recalcitrant project teams must now play along. The SSG tracks exceptions. A gate could require a project to undergo code review (and remediate any critical findings) before release. In some cases, gates are directly associated with controls required by regulations, contractual agreements, and other business obligations and exceptions are tracked as required by statutory or regulatory drivers.
SM2.3
Create or grow social network/satellite system. The satellite begins as a collection of people scattered across the organization who show an above-average level of security interest or skill. Identifying this group is a step towards creating a social network that speeds the adoption of security into software development. One way to begin is to track the people who stand out during introductory training courses. (See [T1.4] Identify satellite through training.) Another way is to ask for volunteers. In some cases, initial satellite membership is assigned and complete coverage of all development/product groups is ensured. A strong satellite is a good sign of a mature software security initiative.
SM2.5
Identify metrics and drive initiative budgets with them. The SSG and its management choose the metrics that define software security initiative progress. These metrics will drive the initiative’s budget and allocation of resources. Metrics also allow the SSG to explain its goals in quantitative terms. One such metric could be security defect density. A reduction in security defect density could be used to show a decreasing cost of remediation over time. The idea here is to tie technical results to business objectives in a clear and obvious fashion in order to justify funding. Since the concept of security is already tenuous to business people, making this explicit tie can be very helpful.
SM Level 3: Practice risk-based portfolio management. Application owners and the SSG must inform management of the risk associated with each application in the portfolio. The SSG must advertise its activities externally to create support for its approach and enable ecosystem security.
SM3.1
Use internal tracking application with portfolio view. The SSG uses a tracking application to chart the progress of every piece of software in its purview. The application records the security activities scheduled, in progress, and completed. It incorporates results from activities such as architecture analysis, code review, and security testing. The SSG uses the tracking application to generate portfolio reports for many of the metrics it uses. A combined inventory and risk posture view is fundamental. In many cases, these data are published at least among executives. Depending on the culture, this can cause interesting effects. As activities become more distributed, the SSG uses a centralized reporting system to keep track of all of the moving parts.
SM3.2
Run external marketing program. The SSG markets the software security initiative outside the organization to build external support for the software security initiative. Software security grows beyond being a risk reduction exercise and becomes a competitive advantage. The SSG might write papers or books. It might have a blog. Members could give talks at conferences or trade shows. In some cases, a complete SSDL methodology can be published and promoted externally.