Common Weakness Enumeration

Coverage Claims Representation (CCR)

NOTICE: As of 4/16/2024, the CWE Compatibility Program has been discontinued. The product listings included in this section have been moved to "archive" status.

MOVING FORWARD: Please follow these CWE Compatibility Requirements to consider your product or service "CWE Compatible."

Archived:

With the advancing maturity and increasing adoption of the Common Weakness Enumeration (CWE™), growing numbers of vendors of software analysis tools and services express their findings of weaknesses in code, design, and architecture using CWE identifiers. This common language for expressing weaknesses has eliminated much of the ambiguity and confusion surrounding exactly what the tool or service has found. At the same time, different vendors take different approaches as to how they look for weaknesses and what weaknesses they look for. This draft of a CWE CCR is a means for software analysis vendors to convey to their customers exactly which CWE-identified weaknesses they claim to be able to locate in software. The word claim is emphasized since neither the CCR itself nor the CWE Compatibility Program verify or otherwise vet these statements of coverage.

A claim of CWE coverage made using the CWE Coverage Claims Representation (CCR) is an eXtensible Markup Language (XML) document containing information about:

• who is making the claim,
• what product or service the claim refers to,
• when the claim was made,
• where the tool or service claims to be able to find weaknesses, i.e., what programming languages and/or binary formats are analyzed, and
• the specific CWE identifiers for which coverage is claimed and details of that coverage.

This overview is intended to introduce the concept of CCR and some of its likely uses. A detailed description of the CCR can be found in the draft CCR XML Schema: CWE_Coverage_Claims_Schema_v0.3.xsd.

The main goal of the CCR is to facilitate the communication of unambiguous statements of the intention of a tool or service to discover specific, CWE-identified weaknesses in software. These statements of claim are intended to allow the providers of software analysis tools and services and the consumers of those tools and services to share a single, unambiguous understanding of the scope of software weakness analysis. CCR wants users of tools and services to be aware and informed of the coverage of the tools and services they make use of in analyzing their software, and when specific classes of weaknesses or individual weaknesses are of specific concern, they can make sure their tools and services are at least trying to find them. Having a mis-match between an organization's focus and the capabilities of their tools and services is not something to be discovered after using and depending on them, but rather is something that should be addressed in the initial discussions and exploration of bringing those capabilities to bear for the organization.

It is anticipated that the CCR will also foster innovation in the technology of software analysis tools and services by allowing vendors to clearly state their intentions with respect to weakness discovery and understand more clearly when there is a need for targeting additional weaknesses to address their customer's concerns. Currently, a tool that does a very deep analysis on a small subset of the entire set of CWE-defined weaknesses may be judged as inadequate by potential customers since, by definition, it fails to discover a broad set of weaknesses. However, with the CCR, the tool provider could supply a CCR document for that tool, clearly setting expectations as to the set of weaknesses that the tool attempts to discover. Tool consumers could then evaluate tools based on what specific CWE-identified weaknesses those tools claim to discover and how that coverage fits within their needs, rather than comparing it to the entire set of CWE-defined weaknesses.

A number of use-cases have inspired the creation of the CCR. They include:

'Tool chest' selection

Organizations that depend on software are increasingly using a variety of methods to identify those CWE-defined weaknesses that represent the greatest risks to their missions and vital information. Whether using a CWE Top 25 List, an OWASP Top Ten List or a set of CWE's identified as part of the Common Weakness Risk Analysis Framework (CWRAF) with the Common Weakness Scoring System (CWSS), it becomes essential to have a clear understanding as to exactly which CWE-defined weaknesses are being looked for by software analysis tools or services. The CCR enables analysis consumers to select tools and/or services for inclusion in their software analysis 'tool chest' based on the vendors claimed coverage of CWE's.

Contracting for code analysis services

Recognizing that there are many CWE's that cannot currently be automatically detected by tools, organizations will often engage firms that offer services to analyze their software using a combination of tools and human experts. The CCR allows a clear understanding by both parties as to what CWE-defined weaknesses are covered by the analysis process. For example, an organization seeking code analysis services for software developed in-house might request a CCR document from each vendor submitting a response to an RFP for code review services.

Tool comparison

Organizations or efforts that wish to evaluate software analysis tools and produce comparisons based on that analysis have a need for clear, unambiguous statements of those weaknesses that the tool claims to be able to find. This then allows fair comparisons of performance between tools that claim to discover the same weaknesses or types of weaknesses without penalizing tools that do not make claims for those weaknesses.

Figure 1 depicts the structure of a CCR document. Note that each CCR document can contain multiple sets of coverage claims (not depicted). Each set of coverage claims is tied to a specific version of the CWE dictionary and specifies the vendor, tool name, tool version, language type (e.g., source, byte code or binary), language (e.g., C, Java, x86 .exe, etc.) and the date of the claim. Following this is the set of CWE-specific claims.

Each claim contains the CWE identifier, the CWE weakness name (for readability and ease of use), and the 'Match Accuracy' attribute which specifies the extent of the claim. Each claim for a specific CWE ID can contain zero or more 'rules' that is used to specify any vendor-specific rule identifiers, numbers or names that correspond to the associated CWE ID. For example, if a tool has a category of weaknesses it finds using a rule or pattern called 'BLIND-SQL', that information could be provided in the rules section of the claim to clarify the relationship of the vendor-specific rule and the CWE referenced. Rules can be identified by names or identifiers and are informational fields only they have no defined or required values. Finally, each claim and rule can have associated free-form text comments.

The Match Accuracy element defines the specific type of claim being asserted and must be one of the following values:

Exact - The CWE entry exactly covers the same weakness/weaknesses as the given rule set.

CWE-more-abstract - The CWE entry covers more concepts than the given rule set, but there are not any more precise matches available. For example, a rule set might detect resource consumption for a resource that is not specifically covered by CWE.

CWE-more-specific - The CWE entry is more specific than the weakness reported by the given rule set, but the entry's parent(s) are not appropriate matches. This might indicate a difference in perspective between CWE and the capability providing the coverage mapping. It could also include a single rule that covers multiple CWE entries (which might imply that there would be multiple claims for a single rule/rule set).

CWE-partial - The CWE entry is only a partial match with the weakness reported by the given rule set, but the entry is the closest available match.

Not-covered - The CWE entry is not covered by any rule set. The provider is not required to include information about uncovered CWEs. The intention of this assertion is to provide a means for tool vendors to explain why their tool does not claim to discover a certain CWE-defined weakness, if they so choose.

No-CWE-available - There is no CWE entry available that closely matches the weakness reported by the given rule set, but the provider believes that a CWE entry should exist for the reported weakness. The associated CWE_ID should be 0.

Not-CWE-applicable - The rule/rule set is not applicable to CWE, i.e., it is not necessarily about a weakness. This could include rule sets related to coding style conformance, informational messages about the scan, etc. The associated CWE_ID should be -1. The provider is not required to include information about non-applicable rules.

Unknown - The match accuracy is unknown. Typically this would be used by a third party who is creating a coverage claim and does not have insight into the technology.

No-claim - The creator of the CCR document is asserting no claim with respect to this CWE.

The CCR is defined such that any CWE that exists in the specified version of CWE that is not the subject of a specific claim in the document is treated as if the CWE was included with a Match Accuracy of 'No-claim.' Therefore, the creator of a CCR document need only define elements for those CWE's where a claim is being asserted.

The following is a depiction of portions of the CCR XML Schema v0.3:

The following is an example of a CCR document for a fictional static source code analysis tool:

A draft of the CWE CCR XML Schema, along with a sample claims example document are available for public review and comment at: CWE_Coverage_Claims_Schema_v0.3.xsd and claims-example.xml. All interested parties are encouraged to review the specification and provide feedback to: cwe@mitre.org.