FLOSS Best Practices Criteria (Gold Badge)

This is the set of best practices for Free/Libre and Open Source Software (FLOSS) projects to achieve the Open Source Security Foundation (OpenSSF) Best Practices gold badge. You can show this list with just the criteria or with additional information. The full set of criteria are also available.

See criteria discussion for more information about these criteria.

Gold

Basics

Prerequisites

Project oversight

  • The project MUST have a "bus factor" of 2 or more. {Met URL} [bus_factor]
  • The project MUST have at least two unassociated significant contributors. {Met URL} [contributors_unassociated]
    Details:
    Contributors are associated if they are paid to work by the same organization (as an employee or contractor) and the organization stands to benefit from the project's results. Financial grants do not count as being from the same organization if they pass through other organizations (e.g., science grants paid to different organizations from a common government or NGO source do not cause contributors to be associated). Someone is a significant contributor if they have made non-trivial contributions to the project in the past year. Examples of good indicators of a significant contributor are: written at least 1,000 lines of code, contributed 50 commits, or contributed at least 20 pages of documentation.
    Rationale:
    This reduces the risk of non-support if a single organization stops supporting the project as FLOSS. It also reduces the risk of malicious code insertion, since there is more independence between contributors. This covers the case where "two people work for company X, but only one is paid to work on this project" (because the non-paid person could still have many of the same incentives). It also covers the case where "two people got paid working for Red Cross for a day, but Red Cross doesn't use the project".

Other

  • The project MUST include a copyright statement in each source file, identifying the copyright holder (e.g., the [project name] contributors). {Met justification} [copyright_per_file]
    Details:
    This MAY be done by including the following inside a comment near the beginning of each file: "Copyright the [project name] contributors.". See "Copyright Notices in Open Source Software Projects" by Steve Winslow.
    Rationale:
    This isn't legally required in most jurisdictions, per the Berne Convention. For example, copyright notices have not been required in the US since 1979. On the other hand, this is not hard to add. Ben Balter's "Copyright notices for open source projects" provides some good arguments for why it *should* be included: "First, someone may want to use your work in ways not allowed by your license; notices help them determine who to ask for permission. Explicit notices can help you prove that you and your collaborators really are the copyright holders. They can serve to put a potential infringer on notice by providing an informal sniff test to counter the 'Oh yeah, well I didn’t know it was copyrighted' defense. For some users the copyright notice may suggest higher quality, as they expect that good software will include a notice... Git can track these things, but people may receive software outside of git or where the git history has not been retained." In addition, we have been informed by the Linux Foundation's SPDX community that having this information is extremely valuable for relicensing and for checking to determine if a copyrighted work is derived from another. While version control systems do track versioning within a project, when files are copied between projects this information is often lost. Having the copyright notice information helps those researching sources, e.g., if they wish to try to relicense something.
  • The project MUST include a license statement in each source file. This MAY be done by including the following inside a comment near the beginning of each file: SPDX-License-Identifier: [SPDX license expression for project]. {Met justification} [license_per_file]
    Details:
    This MAY also be done by including a statement in natural language identifying the license. The project MAY also include a stable URL pointing to the license text, or the full license text. Note that the criterion license_location requires the project license be in a standard location. See this SPDX tutorial for more information about SPDX license expressions. Note the relationship with copyright_per_file, whose content would typically precede the license information.
    Rationale:
    Files are sometimes individually copied from one project into another. Per-file license information increases the likelihood that the original license will be honored. SPDX provides a simple standard way to identify common licenses, without having to embed the full license text in each file; since this makes the criterion easier to do, we specifically mention it. Technically, the text after "SPDX-License-Identifier" is a SPDX license expression, not an identifier, but the tag "SPDX-License-Identifier" is what is used for backwards-compatibility.

Change Control

Public version-controlled source repository

  • The project's source repository MUST use a common distributed version control software (e.g., git or mercurial). {Met justification} [repo_distributed]
  • The project MUST clearly identify small tasks that can be performed by new or casual contributors. {Met URL} [small_tasks]
    Details:
    This identification is typically done by marking selected issues in an issue tracker with one or more tags the project uses for the purpose, e.g., up-for-grabs, first-timers-only, "Small fix", microtask, or IdealFirstBug. These new tasks need not involve adding functionality; they can be improving documentation, adding test cases, or anything else that aids the project and helps the contributor understand more about the project.
    Rationale:
    Identified small tasks make it easier for new potential contributors to become involved in a project, and projects with more contributors have an increased likelihood of continuing. Alluxio uses SMALLFIX and OWASP ZAP uses IdealFirstBug. This is related to criterion installation_development_quick.
  • The project MUST require two-factor authentication (2FA) for developers for changing a central repository or accessing sensitive data (such as private vulnerability reports). This 2FA mechanism MAY use mechanisms without cryptographic mechanisms such as SMS, though that is not recommended. {Met justification} [require_2FA]
    Rationale:
    2FA is used by Node.js and the Linux kernel projects. See "Linux Kernel Git Repositories Add 2-Factor Authentication" by Kontin Ryabitsev and "Linux Foundation Protects Kernel Git Repositories With 2FA" by Eduard Kovacs.
  • The project's two-factor authentication (2FA) SHOULD use cryptographic mechanisms to prevent impersonation. Short Message Service (SMS) based 2FA, by itself, does NOT meet this criterion, since it is not encrypted. {Met justification} [secure_2FA]
    Details:
    A 2FA mechanism that meets this criterion would be a Time-based One-Time Password (TOTP) application that automatically generates an authentication code that changes after a certain period of time. Note that GitHub supports TOTP.
    Rationale:
    SMS is easier and lower cost for many people, but it also provides much weaker security. It has been argued that SMS isn't really 2FA at all; we permit it, because it's better than nothing, but we don't recommend it because of its weaknesses. So Hey You Should Stop Using Texts for Two-Factor Authentication

Quality

Coding standards

  • The project MUST document its code review requirements, including how code review is conducted, what must be checked, and what is required to be acceptable. {N/A justification} {Met URL} [code_review_standards]
    Details:
    See also two_person_review and contribution_requirements.
    Rationale:
    Code review is a cornerstone of quality and secure coding practices. Projects often seek new contributors but lack training and documentation to increase the number of reviewers. An increase in code reviewers lowers maintainer workload while aiding in meeting the badge requirement two_person_review. See issue 699 from GeorgLink.
  • The project MUST have at least 50% of all proposed modifications reviewed before release by a person other than the author, to determine if it is a worthwhile modification and free of known issues which would argue against its inclusion {Met justification} [two_person_review]
    Rationale:
    Review can counter many problems. The percentage here could be changed; 100% would be great but untenable for many projects. We have selected 50%, because anything less than 50% would mean that most changes could go unreviewed. See, for example, the Linux Kernel's "Reviewer's statement of oversight". Note that the set of criteria allow people within the same organization to review each others' work; it is better to require different organizations to review each others' work, but in many situations that is not practical.

Working build system

  • The project MUST have a reproducible build. If no building occurs (e.g., scripting languages where the source code is used directly instead of being compiled), select "not applicable" (N/A). {N/A justification} {Met URL} [build_reproducible]
    Details:
    A reproducible build means that multiple parties can independently redo the process of generating information from source files and get exactly the same bit-for-bit result. In some cases, this can be resolved by forcing some sort order. JavaScript developers may consider using npm shrinkwrap and webpack OccurenceOrderPlugin. GCC and clang users may find the -frandom-seed option useful. The build environment (including the toolset) can often be defined for external parties by specifying the cryptographic hash of a specific container or virtual machine that they can use for rebuilding. The reproducible builds project has documentation on how to do this.
    Rationale:
    If a project needs to be built but there is no working build system, then potential co-developers will not be able to easily contribute and many security analysis tools will be ineffective. Reproduceable builds counter malicious attacks that generate malicious executables, by making it easy to recreate the executable to determine if the result is correct. By itself, reproducible builds do not counter malicious compilers, but they can be extended to counter malicious compilers using processes such as diverse double-compiling (DDC).

Automated test suite

  • A test suite MUST be invocable in a standard way for that language. {Met URL} [test_invocation]
  • The project MUST implement continuous integration, where new or changed code is frequently integrated into a central code repository and automated tests are run on the result. {Met URL} [test_continuous_integration]
    Details:
    In most cases this means that each developer who works full-time on the project integrates at least daily.
    Rationale:
    See Martin Fowler There has been some shift in the meaning of the term continuous integration. Historically the term continuous integration focused on the first part - the frequent integration - and not on its testing. However, over time the emphasis has shifted to include the notion of running automated tests as soon as the code is integrated. This criterion is merely SUGGESTED at passing level. A subset of this criterion is required for passing+1; see automated_integration_testing. Here, we require both the continuous check-in and its testing.
  • The project MUST have FLOSS automated test suite(s) that provide at least 90% statement coverage if there is at least one FLOSS tool that can measure this criterion in the selected language. {N/A justification} {Met justification} [test_statement_coverage90]
    Rationale:
    This increases the statement coverage requirement from the previous badge level, thus requiring even more thorough testing (by this measure).
  • The project MUST have FLOSS automated test suite(s) that provide at least 80% branch coverage if there is at least one FLOSS tool that can measure this criterion in the selected language. {N/A justification} {Met justification} [test_branch_coverage80]
    Rationale:
    This adds another test coverage requirement, again requiring more thorough testing. A program with many one-armed "if" statements could achieve 100% statement coverage but only 50% branch coverage (if the tests only checked the "true" branches). Branch coverage is probably the second most common test coverage measure (after statement coverage), and is often added when a stricter measure of tests is used. Branch coverage is widely (but not universally) implemented.

Security

Use basic good cryptographic practices

  • The software produced by the project MUST support secure protocols for all of its network communications, such as SSHv2 or later, TLS1.2 or later (HTTPS), IPsec, SFTP, and SNMPv3. Insecure protocols such as FTP, HTTP, telnet, SSLv3 or earlier, and SSHv1 MUST be disabled by default, and only enabled if the user specifically configures it. If the software produced by the project does not support network communications, select "not applicable" (N/A). {N/A allowed} {Met justification} [crypto_used_network]
  • The software produced by the project MUST, if it supports or uses TLS, support at least TLS version 1.2. Note that the predecessor of TLS was called SSL. If the software does not use TLS, select "not applicable" (N/A). {N/A allowed} {Met justification} [crypto_tls12]

Secured delivery against man-in-the-middle (MITM) attacks

  • The project website, repository (if accessible via the web), and download site (if separate) MUST include key hardening headers with nonpermissive values. {Met URL} [hardened_site]
    Details:
    Note that GitHub and GitLab are known to meet this. Sites such as https://securityheaders.com/ can quickly check this. The key hardening headers are: Content Security Policy (CSP), HTTP Strict Transport Security (HSTS), X-Content-Type-Options (as "nosniff"), and X-Frame-Options. Fully static web sites with no ability to log in via the web pages could omit some hardening headers with less risk, but there's no reliable way to detect such sites, so we require these headers even if they are fully static sites.

Other security issues

  • The project MUST have performed a security review within the last 5 years. This review MUST consider the security requirements and security boundary. {Met justification} [security_review]
    Details:
    This MAY be done by the project members and/or an independent evaluation. This evaluation MAY be supported by static and dynamic analysis tools, but there also must be human review to identify problems (particularly in design) that tools cannot detect.
    Rationale:
    Security review is important, because security problems often come from subtle interactions of components. Reviewing the system as a whole can help find these problems. Ideally this would be independent, but that often requires a lot of money, and we would rather have some review than none at all. We do not require a specific level of review; this is difficult to quantify given the different environments, requirements, and sizes of various projects. Kevin Wall noted, "If passing+2 is going to be the highest back level, I'd also like to see some sort of mandatory code inspection (possibly SAST assisted), and when applicable, some sort of DAST (for APIs, probably just fuzzing), where failed tests would have to be added to the regression test suite." It's difficult to get agreement on the details of what a security review must include, but we believe that the stated criteria would be agreed on.
  • Hardening mechanisms MUST be used in the software produced by the project so that software defects are less likely to result in security vulnerabilities. {N/A justification} {Met URL} [hardening]

Analysis

Dynamic code analysis

  • The project MUST apply at least one dynamic analysis tool to any proposed major production release of the software produced by the project before its release. {N/A justification} {Met justification} [dynamic_analysis]
    Rationale:
    Static source code analysis and dynamic analysis tend to find different kinds of defects (including defects that lead to vulnerabilities), so combining them is more likely to be effective.
  • The project SHOULD include many run-time assertions in the software it produces and check those assertions during dynamic analysis. {N/A justification} {Met justification} [dynamic_analysis_enable_assertions]