CWE

Common Weakness Enumeration

A Community-Developed List of Software Weakness Types

CWE/SANS Top 25 Most Dangerous Software Errors
Home > CWE/SANS Top 25 > 2010 CWE/SANS Top 25 - Final Draft Changes and Discussion  
ID

2010 CWE/SANS Top 25 - Final Draft Changes and Discussion

Changes and Discussion Items

Date: December 13, 2010

  • Published 1.07 of Top 25 - Updated details to be consistent with the release of CWE 1.11, primarily with mitigations and some name changes.

Date: September 27, 2010

  • Published 1.06 of Top 25 - Updated details to be consistent with the release of CWE 1.10, primarily with mitigations.

Date: June 29, 2010

  • Published 1.05 of Top 25 - Modified official name of the list, replacing "Programming" with "Software"

Date: June 21, 2010

  • Published 1.04 of Top 25 - Updated details to be consistent with the release of CWE 1.9, primarily with mitigations.

Date: April 5, 2010

  • Published 1.03 of Top 25 - Updated details to be consistent with the release of CWE 1.8.1, primarily with improved mitigations and additional CAPEC mappings.
  • Removed duplicate entry in Focus Profile summary

Date: February 25, 2010

  • Top 25 document, version 1.02
  • Fixed "Bypass security" impact in CWE-306; added pointers to SANS Application Security blog.

Date: February 17, 2010

  • Top 25 document, version 1.01
  • Added links to Microsoft SDL
  • Added page numbers to PDF

Date: February 17, 2010

  • Initial public version of 2010 Top 25 document (1.0)

Date: February 13, 2010

CWE Top 25 - Voting Results

All,

As mentioned before, a number of Top 25 voters did not like the restriction of 4 Critical votes for importance, and 4 Widespread votes for prevalence.

Even if this restriction did not exist, it appears that the impact on the Top 25 would have been minimal.

A number of voters initially supplied me with ballots that exceeded these restrictions. I took these ballots, combined them with other voters' ballots that satisfied the restrictions, and generated the voting results to see what happened.

The comparison is attached. Only one item would have fallen off the Top 25. The top 5 items did not even change positions, and neither did the bottom 5 of the original 41. Only about 25% of the entries rose or fell more than 2 places.

Since about half of the voters supplied ballots after reminders about the restrictions, the data set for unrestricted voting is incomplete. So, I don't think it's appropriate to make a separate profile using this data set just to provide alternate rankings. So I've decided to drop it. However, I don't think this is a big loss, since the comparison suggests there was a good chance that the difference wouldn't be that significant anyway. I will chalk this up to "lessons learned" to consider for next year, and document it accordingly in the methodology portion of the document.

- Steve

Date: February 3, 2010

CWE Top 25 - Summary of Activities

All,

Here is a summary of recent activities and what's coming ahead.

  1. As a reminder, the public release of the Top 25 has been postponed until Tuesday, February 16.
  2. Voting for the Nominee List is closed. We received ballots from 28 different organizations! We're thrilled with the response.
    Within a couple of days, I will contact each voter and send their ballots back to them for review, with their own score and ranking, along with the final scores/rankings.
  3. I have decided on which metric will be used for the general list. Basically, the rationale is to keep it simple. For each vote for a CWE, a sub-score will be assigned based on:
    (Importance X Prevalence)
    where each factor gets a rating from 1 to 4, i.e., no "extra credit" for the forced limits on Widespread and Critical ratings. The final score is a summation of individual ratings per CWE.
    Betsy Nichols is evaluating the quality of the data (e.g. distributions and factor independence) but at this point, we have what we have.
  4. Many voters wanted to use more than 4 Widespread or Critical ratings, so we will account for this in a separate "focus profile" (formerly referred to as a "technology profile"). I will reuse the original ballots that I received from people; if your first ballot followed the restriction, feel free to re-submit an updated ballot that assigns as many "Widespread" and "Critical" ratings as you feel is appropriate.
  5. The following Focus Profiles are likely to make it into the final version. These will be posted to the discussion list, and we'd love to have active participation and review for any of these.
    • Unrestricted Widespread/Critical ratings: previously discussed.
    • Educational Emphasis: which weaknesses should be emphasized in an educational setting; uses alternate metrics and 3 factors.
    • Automated vs. Manual Analysis: which weaknesses are detected by automated techniques vs. manual.
    • Language-independent and Language-specific issues: may contain a small breakdown of each item by its prevalence in major languages.
    • Priorization by Importance: an alternate view of the voting data in which the metric emphasizes Importance, intended for software consumers or those just getting started in security.
    • Design vs. Implementation: checklist view of which weaknesses are fixed in design, implementation, or both.

    Other considerations include:

    • Established developer ratings: using ONLY submitted votes from software vendors who participated, create a separate ranking - showing what vendors with established secure development practices care about.
    • Web-based and Unmanaged code may be subsumed by the language-focused/language-independent profiles.
  6. I plan to post the first full draft of the Top 25 document on Thursday evening, February 4. Individual focus profiles will also be posted for separate review.

For those who can participate, we will be looking for people to review the draft document (any section you wish), contribute to individual focus profiles, or evaluate the detailed writeups of the individual entries.

Thanks again to everybody who's been helping us out!

- Steve and Bob

Date: January 26, 2010

CWE Top 25 - Progress Update and Upcoming Activities

All,

Here is an update on the CWE Top 25.

  1. We have received at least 15 voting ballots for the Nominee List, and we may get 20 or more. This was more than we could hope for, so thank you for all your efforts!
    The voting deadline has been extended two more days, to Thursday, January 26 - a week before the publication of the official Top 25.
  2. Many voters did not like the restriction in which Critical importance and Widespread prevalence were only allowed a maximum of four votes each. This will be accounted for in the scoring metric by giving "extra credit" to these ratings, and it will be a lesson learned for next year.
  3. Now that the data collection for voting is almost complete, it is time to discuss some scoring methods to use. Shortly after this email, I will propose various metrics and discuss the pros and cons. I will be working with Betsy Nichols in the next couple days to analyze the utility of the proposed metrics and whether they pass the appropriate statistical smell tests.
    A final decision on the metric to use will be made on Friday. This will then be used to rank, and most likely select, the official entries for the General List.
  4. The various comments will be more closely reviewed and synthesized into a summary of the final general list. One of the most common themes is that many of the assessments of importance varied heavily on context, such as the type of application or environment. My hope is that some of this context will be elucidated in tech profiles. At the very least, they become important inputs for MITRE's work on a Common Weakness Scoring System (CWSS).
  5. I have been working on the technology profiles concept and have a general sense of which profiles would be nice to include in the first public release. Some of these ideas will be proposed in the next day. I think the approach will be to recruit contributors to individual tech profiles, who may review the results and offer any commentary. Pascal Meunier has spent a good deal of effort in formulating an educational profile, complete with an alternate quantitative ranking with three different factors.
  6. The original draft for the most effective mitigations has been improved, though it needs a few more edits. I will have a proposal within a couple days.
  7. We are beginning to solicit supporting quotes, as well as the experiences from anybody who used the 2009 Top 25 list and saw benefit from it. If you're interested, contact Bob Martin (ramartin@mitre.org) and Steve Christey (coley@mitre.org).
  8. Review of mitigations for individual CWE entries, as well as the associated commentary, will begin no later than Monday, February 1.
  9. The publication of the Top 25 will be Thursday, February 4.

Thanks again to everyone for their continued support of this effort!

Steve and Bob

Date: January 11, 2010

Scoring and Ranking

All,

I spoke with Elizabeth Nichols about different ways to quantitatively rank or score weaknesses on the Top 25. While there were a number of ideas covered, most of them would be too time-consuming, subject to error because of small sample size, etc. Next year, I hope to take some more time to explore some of the ideas.

However, for this year, I believe we have hit upon a basic approach that will have some flexibility and provide additional lessons learned to help us improve techniques in the future.

  1. The Keep/Remove/Add votes that I've received so far have been used to create a draft list of weaknesses. Additional opinions will be solicited for the next few days.
  2. A "Nominee List" will be gathered from the recommendations, adjusting for abstraction and general feedback.
  3. A polling period will take place for a week. Each contributor will be polled to assess each weakness based on the criteria we select - such as prevalence and severity - using a selection of fixed, discrete values (e.g., High, Medium, Low). The use of discrete values will simplify a number of problems I'm facing because of the wide variety of prevalence data I'm using.
  4. We can then devise scoring functions with appropriate weights, then use the resulting scores to rank the entries. I intend to propose some type of scoring function, and Elizabeth will do some data-munging magic to see if she can infer appropriate weights from the submitted data itself. This would be an interesting exercise in and of itself, but if we both come up with roughly-equivalent answers, then it will be some kind of validation.

These items are included in the updated schedule that I just sent.

- Steve

Proposal: Use "Importance" instead of "Severity"

For scoring and ranking items on the master list, I suggest that we use these two factors:

  • Prevalence
  • Importance

Given the multitude of threat profiles and perspectives, and some of the discussion on this list, using a factor of "Severity" poses some difficulties. First, as evidenced by the CWE-209 discussion, the perception of severity can vary widely. Several people have said that even though it doesn't cause that much damage by itself, it is so important for defense-in-depth that it should still be included on the list. Second, alternate perspectives such as educational priority may not consider severity to be the most important factor. There is also the desire to quantify results and come up with appropriate rankings.

I propose that we use a new factor, instead of severity: "importance."

This will allow individuals to vote on how important a weakness is, using their own criteria that reflects their daily work. Top 25 contributors can effectively encode what is most important to them - whether it's severity, frequency of exploit, need for education, etc. - in a way that is specific to their own needs (and their customers' needs).

Using some kind of mechanism that combines Prevalence and Importance (to be determined), we can then assign individual scores to each CWE entry, then rank the Top 25 entries according to those scores.

The result will be a generalized recommendation that merges diverse perspectives to come up with something approaching a consensus. Each individual technology profile could still define its own criteria, scores, and ranking.

Importance

For each prospective CWE on the Top 25, I propose using the following inputs for importance.

Criterion: "If this weakness appears in software, what priority do you use when making recommendations to your consumer base? (e.g. to fix, mitigate, educate)."
  • Critical - this weakness is more important than any other weakness, or it is in the top 4. It should be addressed as quickly as possible, and might require dedicating resources that would normally be assigned to other tasks. (Example: a buffer overflow might receive a Critical rating in unmanaged code because of the possibility of code execution.)
  • High - this weakness should be addressed as quickly as possible, but it is less important than the most critical weaknesses. (Example: in some threat models, an error message information leak may be given high importance because it can simplify many other attacks.)
  • Medium - this weakness should be addressed, but only after High and Critical level weaknesses have been addressed.
  • Low - it is not urgent to address the weakness, or it is not important at all.

Prevalence

For each prospective CWE on the Top 25, I propose using the following inputs for prevalence.

I welcome any suggestions on the percentage ranges that I chose.

Criterion: For each "project" (whether a software package, pen test, educational effort, etc.), how often does this weakness occur or otherwise pose a problem?
  • Widespread - 70% or more of the projects; or, when discovered, there are frequently multiple instances of the problem within the same project
  • High - 50% to 70% of projects
  • Medium - 10% to 49% of projects
  • Low - Less than 10% of projects
Nominee List, Draft 1

Draft 1 - Status Summary (January 11, 2010)

top25CWE-311 (NEW) Failure to Encrypt Sensitive Data
top25CWE-672 (NEW) Operation on a Resource after Expiration or Release
top25CWE-772 (NEW) Missing Release of Resource after Effective Lifetime
top25CWE-754 (NEW) Improper Check for Exceptional Conditions
top25CWE-89Improper Sanitization of Special Elements used in an SQL Command ('SQL Injection')
top25CWE-79Failure to Preserve Web Page Structure ('Cross-site Scripting')
top25CWE-78Improper Sanitization of Special Elements used in an OS Command ('OS Command Injection')
top25CWE-73External Control of File Name or Path
top25CWE-352Cross-Site Request Forgery (CSRF)
top25CWE-362Race Condition
top25CWE-209Information Exposure Through an Error Message
top25CWE-665Improper Initialization
top25CWE-285Improper Access Control (Authorization)
top25CWE-327Use of a Broken or Risky Cryptographic Algorithm
top25CWE-259Hard-Coded Password
top25CWE-732Incorrect Permission Assignment for Critical Resource
top25CWE-330Use of Insufficiently Random Values
removedCWE-404Improper Resource Shutdown or Release
removedCWE-319Cleartext Transmission of Sensitive Information
removedCWE-426Untrusted Search Path
removalCWE-494Download of Code Without Integrity Check
addedCWE-311 (NEW) Failure to Encrypt Sensitive Data
addedCWE-672 (NEW) Operation on a Resource after Expiration or Release
addedCWE-772 (NEW) Missing Release of Resource after Effective Lifetime
addedCWE-754 (NEW) Improper Check for Exceptional Conditions
additionCWE-131Incorrect Calculation of Buffer Size
additionCWE-129Improper Validation of Array Index
additionCWE-120Buffer Copy without Checking Size of Input ('Classic Buffer Overflow')
additionCWE-134Uncontrolled Format String
additionCWE-749Exposed Dangerous Method or Function
additionCWE-302Authentication Bypass by Assumed-Immutable Data
additionCWE-306No Authentication for Critical Function
additionCWE-307Failure to Restrict Excessive Authentication Attempts
additionCWE-434Unrestricted File Upload
abstractionCWE-119Failure to Constrain Operations within the Bounds of a Memory Buffer
abstractionCWE-682Incorrect Calculation
abstractionCWE-77Improper Sanitization of Special Elements used in a Command ('Command Injection')
abstractionCWE-642External Control of Critical State Data
abstractionCWE-94Failure to Control Generation of Code ('Code Injection')
mitigationCWE-116Improper Encoding or Escaping of Output
mitigationCWE-20Improper Input Validation
mitigationCWE-602Client-Side Enforcement of Server-Side Security
mitigationCWE-250Execution with Unnecessary Privileges

Draft 1 - Status Notes

Status: top25 (weaknesses on this draft of the Top 25)

There are 17 entries with this status.

CWE-311 (NEW) Failure to Encrypt Sensitive Data

ABSTRACTION CHANGE: Added to replace its child, CWE-319 cleartext transmission; this parent also covers the failure to encrypt data in storage.

CWE-672 (NEW) Operation on a Resource after Expiration or Release

ABSTRACTION CHANGE: Added as a more-specific child of 404, along with its sibling, CWE-772 Missing Release of Resource after Effective Lifetime. This covers frees, double-frees, etc.

CWE-772 (NEW) Missing Release of Resource after Effective Lifetime

ABSTRACTION CHANGE: Added as a more-specific child of 404, along with its sibling, CWE-672, Incorrect Operation on a Resource after Expiration or Release. CWE-772 is more specific and can handle memory leaks, file descriptor exhaustion, etc.

CWE-754 (NEW) Improper Check for Exceptional Conditions

ABSTRACTION CHANGE: This could effectively capture CWE-252 (unchecked return value), which was on the cusp in 2009.

CWE-89Improper Sanitization of Special Elements used in an SQL Command ('SQL Injection')

Possibly remove SQL and OS command injection and replace with CWE-77 - Command Injection.

SQL injection was in more than 5% of all CVEs in 2008-2009, and the the number 1 issue in 2008.

CWE-79Failure to Preserve Web Page Structure ('Cross-site Scripting')

In 5% or more of all CVEs in 2008-2009.

CWE-78 (DEBATE) Improper Sanitization of Special Elements used in an OS Command ('OS Command Injection')

Possibly remove SQL and OS command injection and replace with CWE-77 - Command Injection.

CWE-73External Control of File Name or Path

Could possibly be removed in draft 1 and merged into its parent, CWE-642. However, 642 is probably too high level, and this weakness is already an abstraction of CWEs that people are nominating like file inclusion, symlink following, and path traversal.

Path traversal was more than 5% of all CVEs in 2008-2009.

CWE-352Cross-Site Request Forgery (CSRF)
CWE-362 (DEBATE) Race Condition

Some people were unsure about keeping this.

CWE-209 (DEBATE) Information Exposure Through an Error Message

Actively debated, largely for severity.

CWE-665 (DEBATE) Improper Initialization

Some people were unsure about keeping this. Possibly narrow it down to CWE-457: Use of Uninitialized Variable, but maybe "use of uninitialized data" is actually more important (a new CWE might need to be created).

CWE-285 (DEBATE) Improper Access Control (Authorization)

Perhaps too high-level? But this doesn't have many good children.

CWE-327Use of a Broken or Risky Cryptographic Algorithm
CWE-259 (DEBATE) Hard-Coded Password

If another authentication issue is accepted, then should this be removed? If a mitigation of "assume reverse engineering can work" is adopted, this might be a good place.

CWE-732Incorrect Permission Assignment for Critical Resource
CWE-330 (DEBATE) Use of Insufficiently Random Values

This is a root-cause type of problem. Should it be moved to mitigations?

Status: removed (weaknesses that were removed in this version)

There are 3 entries with this status.

CWE-404Improper Resource Shutdown or Release

This was replaced with CWE-772 (Missing Release of Resource after Effective Lifetime) and CWE-672 (Use of a Resource after Expiration or Release). This would cover memory leaks, file descriptor leaks, use-after-free, double-frees, etc.

Some have suggested replacing this with lower-level entries such as memory leak (CWE-401). Memory leak's parent is CWE-772 Missing Release of Resource after Effective Lifetime, which could then cover other variants such as file descriptor and DB connection exhaustion. However, some issues such as double-frees would be missed; those might be rolled into CWE-672 Use of a Resource after Expiration or Release.

CWE-319Cleartext Transmission of Sensitive Information

Replaced with its parent CWE-311, Failure to Encrypt Sensitive Data, which would cover missing encryption in storage as well as transmission.

CWE-426Untrusted Search Path

Suggested for removal due to limited prevalence. Could also be rolled up under its parent CWE-642, external control of critical state data, which is also on the list.

Status: removal (weaknesses being considered for removal)

There are 1 entries with this status.

CWE-494 (DEBATE) Download of Code Without Integrity Check

Suggested for removal by people who say that most software doesn't have an ability to download its own updates, so the prevalence of CWE-494 is very low. There is some CVE data, but its prevalence is also low.

Status: added (weaknesses that were added in this version)

There are 4 entries with this status.

CWE-311 (NEW) Failure to Encrypt Sensitive Data

ABSTRACTION CHANGE: Added to replace its child, CWE-319 cleartext transmission; this parent also covers the failure to encrypt data in storage.

CWE-672 (NEW) Operation on a Resource after Expiration or Release

ABSTRACTION CHANGE: Added as a more-specific child of 404, along with its sibling, CWE-772 Missing Release of Resource after Effective Lifetime. This covers frees, double-frees, etc.

CWE-772 (NEW) Missing Release of Resource after Effective Lifetime

ABSTRACTION CHANGE: Added as a more-specific child of 404, along with its sibling, CWE-672, Incorrect Operation on a Resource after Expiration or Release. CWE-772 is more specific and can handle memory leaks, file descriptor exhaustion, etc.

CWE-754 (NEW) Improper Check for Exceptional Conditions

ABSTRACTION CHANGE: This could effectively capture CWE-252 (unchecked return value), which was on the cusp in 2009.

Status: addition (weaknesses being considered for addition)

There are 9 entries with this status.

CWE-131Incorrect Calculation of Buffer Size

Could replace its general parent, CWE 119.

CWE-129Improper Validation of Array Index

could replace its general parent, CWE 119.

CWE-120Buffer Copy without Checking Size of Input ('Classic Buffer Overflow')

The most basic overflow-related error. Could replace its general parent, CWE 119.

CWE-134Uncontrolled Format String

Could be considered if CWE-119 is removed.

CWE-749Exposed Dangerous Method or Function

Level 3 issue from CVE (0.4% through 1.4%)

CWE-302Authentication Bypass by Assumed-Immutable Data

This is one of 3 main children of CWE-287: Improper Authentication, which is not being proposed because it is too general.

Extremely common auth problem in CVE, e.g. cookie saying "administrator=true".

CWE-306No Authentication for Critical Function

This is one of 3 main children of CWE-287: Improper Authentication, which is not being proposed because it is too general.

CWE-307Failure to Restrict Excessive Authentication Attempts

This is one of 3 main children of CWE-287: Improper Authentication, which is not being proposed because it is too general.

CWE-434Unrestricted File Upload

Easily in CVE's top 15 for 2008. Has overlap with CWE-73 and/or CWE-642, whose abstraction problems have not been fully addressed yet.

Status: abstraction (weaknesses being considered for abstraction changes)

There are 5 entries with this status.

CWE-119 (DEBATE) Failure to Constrain Operations within the Bounds of a Memory Buffer

With some extra room on the list, it might be better to use lower-level CWEs instead of this one.

This covers more than 5% of all CVEs in 2008-2009. "Memory corruption", a separate category in CVE, was between 1.5 and 4.9% of CVEs in 2008-2009.

This could be covered by CWE-120 (classic overflow) and CWE-131 (incorrect calculation of buffer size).

CWE-682Incorrect Calculation

If CWE-131 "incorrect calculation of buffer size" is accepted in favor of the more generic parent CWE-119, then would this be neeeded? Are there any other areas, besides memory management, where this weakness particularly critical? (perhaps financial calculations)

Alternately, could replace this with 2 lower-level entries - integer overflows and CWE-681: Incorrect Conversion between Numeric Types.

CWE-77Improper Sanitization of Special Elements used in a Command ('Command Injection')

SQL injection could be thought of as a command-injection issue. If CWE-77 is adopted, then we could remove SQL injection and OS command injection, as well as CWE-94 "code injection", and just use this one.

We would still need to deal with XSS.

CWE-642 (DEBATE) External Control of Critical State Data

CWE-73 (external control of pathname) is a child that's currently also on the Top 25 list. Since CWE-73 covers path traversal, symlinks, RFI, etc., removing 73 in place of 642 might be too abstract. However, CWE-642 applies to other types of problems too (e.g. cookie manipulation and other web-heavy problems), so it's uncertain what to do here.

CWE-94 (DEBATE) Failure to Control Generation of Code ('Code Injection')

Many have treated this weakness as a high-level parent to SQL injection, XSS, and OS command injection - so there's a bit of overlap here. However, the general intention for this entry is for internally-generated code that is internally executed, *not* code or commands that are sent to downstream components. Maybe this should be removed entirely due to that confusion, but placed on a watch-list since CWEs like 95 and 96 are rising in popularity.

Status: mitigation (weaknesses that have been moved to mitigations)

There are 4 entries with this status.

CWE-116 (DEBATE) Improper Encoding or Escaping of Output

Moved to mitigations section.

CWE-20 (DEBATE) Improper Input Validation

Moved to mitigations section.

CWE-602 (DEBATE) Client-Side Enforcement of Server-Side Security

Moved to mitigations section.

CWE-250 (DEBATE) Execution with Unnecessary Privileges

Moved to mitigations section.

Status: member (weaknesses that are members of the current list and the previous draft)

There are 13 entries with this status.

CWE-89Improper Sanitization of Special Elements used in an SQL Command ('SQL Injection')

Possibly remove SQL and OS command injection and replace with CWE-77 - Command Injection.

SQL injection was in more than 5% of all CVEs in 2008-2009, and the the number 1 issue in 2008.

CWE-79Failure to Preserve Web Page Structure ('Cross-site Scripting')

In 5% or more of all CVEs in 2008-2009.

CWE-78 (DEBATE) Improper Sanitization of Special Elements used in an OS Command ('OS Command Injection')

Possibly remove SQL and OS command injection and replace with CWE-77 - Command Injection.

CWE-73External Control of File Name or Path

Could possibly be removed in draft 1 and merged into its parent, CWE-642. However, 642 is probably too high level, and this weakness is already an abstraction of CWEs that people are nominating like file inclusion, symlink following, and path traversal.

Path traversal was more than 5% of all CVEs in 2008-2009.

CWE-352Cross-Site Request Forgery (CSRF)
CWE-362 (DEBATE) Race Condition

Some people were unsure about keeping this.

CWE-209 (DEBATE) Information Exposure Through an Error Message

Actively debated, largely for severity.

CWE-665 (DEBATE) Improper Initialization

Some people were unsure about keeping this. Possibly narrow it down to CWE-457: Use of Uninitialized Variable, but maybe "use of uninitialized data" is actually more important (a new CWE might need to be created).

CWE-285 (DEBATE) Improper Access Control (Authorization)

Perhaps too high-level? But this doesn't have many good children.

CWE-327Use of a Broken or Risky Cryptographic Algorithm
CWE-259 (DEBATE) Hard-Coded Password

If another authentication issue is accepted, then should this be removed? If a mitigation of "assume reverse engineering can work" is adopted, this might be a good place.

CWE-732Incorrect Permission Assignment for Critical Resource
CWE-330 (DEBATE) Use of Insufficiently Random Values

This is a root-cause type of problem. Should it be moved to mitigations?

Summary of Recent Top 25 Discussions

Date: January 5, 2010

Here is a high-level summary of the discussions and feedback that we have received so far. I apologize for over-simplifying the extensive feedback we have already received.

  • Technology Profiles
  • Structure of Top 25 Document
  • Threat Models
  • Overlap / Abstraction Issues Between Entries
  • Prioritization and Ranking
  • Section on the Most Effective Mitigations
  • Suggested Additions / Removals

Technology Profiles

The opinions on the use of technology profiles were largely positive.

Several new profiles were recommended - on the list and privately - including:

  • education
  • client-side software (and server-side software)
  • managed code
  • mobile devices

There were some concerns about the originally-proposed profiles: web-based, unmanaged, and embedded. There is overlap between these profiles; for example, embedded code might provide web-based functionality that is written in unmanaged code.

It is not clear whether this type of overlap is problematic.

Possible process:

  • identify a small number of important technology profiles to be ready for release in the final document (early February).
  • each profile defines and uses its own criteria for prioritizing items.
  • other profiles can be added to the Top 25, or published separately, well after initial release.

Next steps:

  • MITRE updates the example document to reflect the use of these profiles.
  • MITRE and Top 25 contributors determine whether profile overlap is acceptable, and if so, what actions (if any) need to be taken to minimize or clarify the overlap
  • Top 25 contributors may request (or be requested by MITRE) to further concentrate on refining one or more of these profiles, including the prioritization criteria. (Later sections in this summary will cover some specifics.)

Structure of Top 25 Document

Assuming the use of technology profiles, there was no support for assigning 25 items to each profile, then combining them to create a master list that would most likely consist of more than 25 items.

There has been solid support for building a single master list - 25 items - then providing separate appendices for each technology profile. Each profile could use a profile-specific ranking of items from the master list, and a supplementary group of on-the-cusp (watch list) entries could be used to identify issues that are critical for that technology profile, even if they are not important enough to make the master list.

Next steps:

  • MITRE will post an updated example document that reflects this structure.

Threat Models

The 2009 Top 25 assumed a simple threat model: a skilled and determined attacker for whom integrity and confidentiality was more important than denial of service.

Since the 2010 Top 25 will use multiple profiles, it may make more sense to allow each profile to identify its own threat model - assuming that the profile's prioritization criteria want to consider a threat model. (For example, an educational profile might choose to prioritize weaknesses based on ease-of-understanding by students instead of considering the real-world threats posed by those weaknesses.)

Given the use of multiple profiles this year, and the fact that last year's threat model was not always applicable, the pursuit of more detailed threat models will probably be abandoned for this year, except possibly for the master list.

Next steps:

  • TBD; partially covered by profile-specific actions in the previous section.

Overlap / Abstraction Issues Between Entries

2009 entries such as CWE-20 (improper input validation) had a significant amount of overlap with other items on the list, such as XSS and SQL injection. Some entries were at a very high level, such as improper authorization, while others were at a lower level. In some cases, the Top 25 would include both a low-level and its parent (such as the entries related to external control of filenames and critical data).

Entries such as these could be addressed by one or more of the following:

  1. Using lower-level CWE entries (possibly creating them)
  2. Resolving / normalizing cases when both an entry and its parent/grandparent were included in the 2009 list.
  3. De-prioritizing the entries related to "root cause," possibly changing them to a cusp/watch-list role.
  4. Moving some entries, such as CWE-20, to a separate section that identifies the most important mitigations.

Next steps:

  • MITRE will suggest abstraction changes and build a straw-man

Prioritization and Ranking

While the topic was not raised explicitly, there seems to be support for prioritizing the items on this year's list.

There were not any objections to continuing last year's emphasis on prevalence of occurrence and severity to determing inclusion on the Top 25. Some people advocated using a third factor that captures the frequency of real-world attacks that exploit the issues.

There has also been solid support for using quantitative methods to rank items, which also would ultimately determine which items make it onto the Top 25 list.

MITRE has received several submissions, privately and to the discussion list, which contain real-world weakness or vulnerability data that might be difficult to combine. For example, some submissions use low-level CWEs, and others use high-level ones. CVE vulnerability data is also available; a summary of 2008/2009 results was posted. However, it has been noted that CVE is subject to unpredictable fluctuations, such as the spike in symbolic link issues in 2008 due to a single researcher's efforts.

As mentioned previously, there is a possibility that each technology profile might use its own prioritization scheme.

Next steps:

  • During the first week of January, MITRE will work with Elizabeth Nichols of PlexLogic to determine if there are suitable approaches for data collection or polling that will support quantitative analysis.
  • MITRE will investigate the use of attack frequency as another factor in ranking.
  • Supporters of individual profiles can begin discussion of criteria for profile-specific ranking.

Section on the Most Effective Mitigations

There was solid support for creating a separate section that covers important mitigations - or "safe practices" - that could eliminate or reduce the severity of many items on the Top 25. Such mitigations might include running with low privileges, input validation, using well-vetted security features instead of inventing new ones, etc.

One benefit of this approach would be to reduce the apparent overlap of many issues on the 2009 Top 25.

This is being informally referred to as "Top 10 Mitigations," but there is not a clear need to limit (or expand) the list to 10 items.

Next steps:

  • MITRE will identify entries from the 2009 Top 25 that might make more sense to relocate to the mitigation section
  • MITRE will create a sample section containing sample mitigations, and solicit input on those suggestions.
  • Top 25 contributors can nominate the most effective mitigations

Suggested Additions / Removals

Contributors were given last year's 2009 list, and asked whether the items should be kept or removed. Contributors were also given a list of "on the cusp" from last year, as well as other recommendations, then asked whether those entries should be added.

The most controversial item has been CWE-209 (error message information leak), which was on the 2009 Top 25. Supporters state that even if it is not severe on its own, it is often an important part of real-world attacks, so for that reason it should be considered. Dissenters acknowledge the role in attacks but believe that the Top 25 should have a greater emphasis on the critical weaknesses for which CWE-209 is an intermediate step.

There were some suggestions for error handling and/or error checking, for which several entries were nominated.

As the discussion of addition/removal evolved, it became apparent that the 2009 Top 25 does not appropriately clarify what its primary purpose is, which would then affect which items should be added or removed.

Next steps:

  • MITRE will process and summarize the specific feedback that has already been received
  • MITRE and Top 25 contributors will clarify the purpose of the Top 25 (e.g., a general educational/awareness tool, or a utility to help focus the development of software assurance requirements).
  • Top 25 contributors will continue to offer their feedback/suggestions for more additions and removals

More information is available — Please select a different filter.
Page Last Updated: January 12, 2017