DeepSpeed

If breaking changes are made, they are documented in the release notes for which PRs have caused these changes, and the PRs themselves contain the needed steps to upgrade or links to those if they are dependent on other packages.

https://github.com/microsoft/DeepSpeed/issues

No new vulnerability reports to GitHub, those submitted to Huntr and processed via Microsoft are all credited.

Vulnerability reporting is covered under the SECURITY.md file here: https://github.com/microsoft/DeepSpeed/blob/master/SECURITY.md#security

The specific coding styles that are used are identified in the config files here: https://github.com/microsoft/DeepSpeed/blob/master/.pylintrc

This is automatically enforced in the "formatting" checks that use pre-commit and a number of tools, yapf, clang-format, etc. This is listed here: https://github.com/microsoft/DeepSpeed/blob/master/.pre-commit-config.yaml

Flags are only appended, for example, LD_FLAGS are only ever appended to here: https://github.com/search?q=repo%3Amicrosoft%2FDeepSpeed%20LDFLAGS&type=code

This is covered by the python build system, there are no additional verbose/debugging information that isn't preserved.

This is covered by the JIT compilation, these directories are not built if not specified.

Identical builds from the same commit/command/environment is guaranteed based on the build flow, there is no randomness/etc.

This is covered by python, just pip uninstall deepspeed even if installed from PyPI or built from source.

This project meets the python standard conventions for build locations and artifacts.

This is covered by the python packages and dependencies, just pip install deepspeed or it can easily be built quickly.

These are listed in the requirements files in the requirements folder: https://github.com/microsoft/DeepSpeed/tree/master/requirements

The only potential issue is that torch is required to be installed before deepspeed and there is no computer processable way to confirm this.

This is currently done via GitHub alerting us as well as collaboration with those projects.

The external dependencies can easily be updated by creating a PR and modifying the requirements.txt (or equivalent file). The standard components from python are also used, but where not possible, the other python packages can easily be updated and re-built with via the requirements file.

The only non-FLOSS software that is depended upon is CUDA but that is only a requirement on Nvidia GPUs. The python packages don't rely on any deprecated or obsolete functions.

Tests and requirements including code review by at least one other person are required for the master branch. This requires formatting, python checks, as well as unit tests in a number of different files:

test yml files located here: https://github.com/microsoft/DeepSpeed/tree/master/.github/workflows

Regression tests are added and normal CI tests are run nightly as well (with bugs/issues opening new GitHub issues to be easier to track)

Sample nightly CI: https://github.com/microsoft/DeepSpeed/blob/master/.github/workflows/nv-nightly.yml Other nightly tests: https://github.com/microsoft/DeepSpeed/blob/master/.github/workflows/nv-torch-latest-v100.yml#L14C1-L14C24

Sample opening bug if regression tests fail: https://github.com/microsoft/DeepSpeed/blob/c7f58c899f6f099a35d968bdad973f24b842c8c6/.github/workflows/nv-sd.yml#L67C1-L73C64

pytest coverage is used, .coveragerc is here: https://github.com/microsoft/DeepSpeed/blob/c7f58c899f6f099a35d968bdad973f24b842c8c6/tests/.coveragerc#L3

Coverage is output via the GitHub Actions UI

This is covered in the New Feature Contribution Guidelines listed here: https://github.com/microsoft/DeepSpeed/blob/master/CONTRIBUTING.md#new-feature-contribution-guidelines

This is also covered in the New Feature Contribution Guidelines section 2: Implementation and verification: https://github.com/microsoft/DeepSpeed/blob/master/CONTRIBUTING.md#step-2-implementation-and-verification

New errors must be addressed and old errors are slowly being resolved.

This is done to the best of our knowledge, defaults are sane, and the software has been checked by Microsoft's Open Source Security team for secure design decisions.

The software doesn't support cryptographic algorithms on its own, there is no need for a user to specify an algorithm.

The project does not interact with these credentials, only relies on other software like HuggingFace Hub for example.

The project does not output its own security software so there is no need for these protocols.

The software does not output any code that would need to rely on TLS.

The software does not use/require TLS, but the website does support TLS 1.2.

Software output by DeepSpeed does not require TLS.

All releases are tagged and pushed to git, and the .whl builds are pushed to PyPI, but the tags themselves are not cryptographically signed by anything other than what is provided by GitHub.

This is not applicable due to the nature of the project and what it does.

This is covered by input parsing and validation of inputs, and the fact that the software wraps torch.

PyFlake/Flake8/MyPy are all run in the pre-commit formatting/run on all PRs: https://github.com/microsoft/DeepSpeed/blob/master/.pre-commit-config.yaml

We do ship C++ binaries and these should soon be covered as above, we haven't added this coverage yet thought. Though these are cuda style kernels and cannot be fuzzed independently of the main DeepSpeed program.