Python download checksum in MD5 #1227

Thank you for the suggestion and it is something that is worth doing. It will require changes to both the web site and our release management process and both are volunteer-led efforts. More importantly, for each release file we already provide a GPG signature file (via the SIG link after each file) which provides more robust verification than a simple checksum.

This is a great suggestion, or at least do what nodejs does where they place the SHA256 checksum in a text file at the same location as the download, so that checksum validation is more scriptable. Example.... https://nodejs.org/download/release/v16.20.0

More recently, release artifacts have been signed with Sigstore, for example:

More info: https://www.python.org/download/sigstore/

@sethmlarson Do you think it's worth switching MD5 for something like SHA-256, or are people better off using Sigstore instead? Should we drop MD5?

Checking the MD5 is easy, and as long as it's provided, there will be people using that.
If the MD5 can't be relied upon, it will just give people a false sense of security.
Removing it will encourage people to use other more reliable methods (which should be documented clearly).

@ezio-melotti, @hugovk What do you all think about the future of this issue? Should we close it, or point to sigstore, or something else?

I would like to provide checksums that aren't MD5 (in addition to making the Sigstore verification steps more visible). Back-filling the correct values is in progress (cc @woodruffw) since we need to make sure the artifacts are the correct ones first (ie by verifying GPG sigs)

ISTM that two things could be done:

• replace MD5 checksums with SHA-256 ones
• add a paragraph that explains briefly how to verify the downloads and/or links to a different page with more exhaustive instructions

For cross-checking/backfilling purposes: https://github.com/woodruffw/cpython-release-tracker has a dump of every hosted version of CPython's release assets, including SHA256 hashes (which are computed when generating that repo).

Right now, each release in the CMS has an MD5 sum field, which is filled in by the release automation when uploading each file, but not a SHA-256.

I suggest we start by:

1. add a SHA-256 field to the CMS
2. update the release scripts to update the SHA-256 field
3. then we can start displaying them on download pages
4. later, we can deal with backfilling

Questions:

• If a release has SHA-256 checksums, do we only display those in the table and not MD5?
• Do we need to make the MD5 checksums available for such releases somehow?
• If only SHA-256, shall we stop calculating and storing them for new releases?
• (Compare: we stopped doing GPG when adding SigStore)

Thanks to @JacobCoffee for the PRs to the release automation and the website, we now have SHA-256 checkums!

Starting from 3.15.0a4:

• https://www.python.org/downloads/release/python-3150a4/
• https://www.python.org/downloads/release/python-3150a5/

We still calculate and upload MD5 checksums to the database, but they're not shown in the table.

For older releases with only MD5 checksums and no SHA-256, we still show MD5:

• https://www.python.org/downloads/release/python-3150a1/

These checksums are shown last in the table, to prioritise Sigstore and SBOM.

There's some ideas at #2867 (comment) and #2867 (comment) about improving the display of the long SHA-256 checksums in the table.

Is there anything left to do here?

A few things:

• Improve display
• Stop generating/uploading MD5?
• Possibly backfill SHA-256 for some releases?

And so far, SHA-256 has only been made available for 3.15 alphas, and we have 3.13 and 3.14 coming next week, which have more visibility and might get us some workflow feedback.

can be another issue if desired