Pin Python 3.9.16 on Cygwin CI by EliahKagan · Pull Request #1814 · gitpython-developers/GitPython (original) (raw)

Conversation

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})

This uses Python 3.8.16 (provided by the Cygwin package python38 at version 3.8.16-1), to work around the problem that pip has begun to block on some PyPI package downloads when Python 3.9.18 (provided by the Cygwin package python39 at version 3.9.18-1) is used.

I also tried a bunch of other stuff, which is listed below and can be examined in full detail, with all individual diffs and most CI results, at #2.

Try not installing/upgrading wheel for Cygwin CI

This is for a recent problem where "pip install -U" in the virtual environment in a Cygwin test job seems to block indefinitely on downloading the wheel package itself (not other packages' wheels).

Try not upgrading/installing pip/setuptools either on Cygwin
Try installing pytho39-wheel Cygwin package

Maybe this will overcome the next blockage, which is the codecov PyPI package, downloading a .tar.gz file.

Try upgrading wheel, but after upgrading pip
Try always running pip on Cygwin as "python -m pip"
Try using a venv on Cygwin
Use "python -v -m pip" to see some of what's going on
Undo venv; use "python -m pip -vvv" to see what's going on
Undo all debugging changes except passing "-vvv"
Try with "--no-cache-dir"
Try with different tmp dir for pip runs
Try with python39=3.9.16-1
Try not upgrading setuptools
Try not installing Cygwin python39-pip package
Run pip freeze effectively

This doesn't fix the bigger issue, it just addresses something from the last commit.

Try not installing python39-virtualenv either
Try giving IPv4 for files.pythonhosted.org in hosts file
Try downloading wheel with wget

This is not a usable solution, but it is useful for troubleshooting.

Try with python39-pip=23.0.1-1

And don't upgrade it or other PyPI packages.

Pin pip with pip (Cygwin package doesn't pin)

This tries with an older pip, but if the problem is the build rather than the version, then it would also help.

Stop pinning; keep skipping -U for PyPA; instrument with -vvv

This won't fix it but is diagnostic, to reveal the URL for the coverage package, so I can see what happens when that is installed more manually.

Try installing coverage[toml] separately
Unset -vvv to see the bigger picture more easily
Try killing pip after a timeout and rerunning it
Use SIGKILL
Increase timeout from 70 to 120 seconds per try
Give each try a little more time than the last

Since it has to verify previous work.

Tweak (re)try parameters
Try Python 3.8

The latest currently packaged version of Python 3.9 for Cygwin is 3.9.18 (provided by the Cygwin package python39 at version 3.9.18-1). That version, at least as we are using it, has a problem where pip has begun to block on some PyPI package downloads.

In 73ebcfa (#2), I worked around this problem by downgrading the minor version of Python to 3.8. But it is better to use 3.9 if we can, since it is currently the latest minor version of Python in the Cygwin repositories, and also because (relating to that) it is used more often, and thus probably used more often with GitPython, than 3.8.

This upgrades Python on Cygwin but not all the way. It upgrades it to the latest (or latest currently available) patch version of 3.9 packaged for Cygwin of those that strictly precede 3.9.18 where the problem occurs. That version is 3.9.16, provided by the Cygwin package python39 at version 3.9.16-1.

This version may eventually no longer be available for download from Cygwin's repositories, so hopefully a real solution or better workaround will be found by then, or perhaps a future update to the package itself will fix the problem.

I also tried some more other stuff since finding 3.8 to work in 73ebcfa. Changes since then are listed below. They can be examined in full detail, with individual diffs and CI results, at #3.

Try Python 3.9 with other details the same

Changing it to Python 3.8 worked, but I want to check that it was actually the use of Python 3.8, rather than other seemingly small changes made to support using Python 3.8, that made the difference.

Revert "Try Python 3.9 with other details the same"

This reverts commit b55cbfb.

Try 3.9 again, with both python39=3.9.16-1 python39-pip=23.0.1-1
Back to 3.8; try another GitHub Action

Python 3.8 worked with cygwin-install-action, but I want to make the change to setup-cygwin by itself first before trying it with 3.9, in case I am using setup-cygwin incorrectly.

Try 3.9 with this setup-cygwin action
Try pinning with setup-cygwin
Try not pinning, but no -U for PyPA, with setup-cygwin

Pinning and skipping -U for PyPA packages worked. Let's see if it was really pinning that made the difference.

Try pinning just python39-pip

Pinning works, and merely omitting the -U for PyPA package doesn't. Examining the output of runs that used install-cygwin-action and attemped pinning Cygwin package versions shows newer versions were installed, whereas pinning is really happening with setup-cygwin.

This tries pinning just the Cygwin package for pip, rather than for Python 3.9. I don't expect this to work.

Try pinning just python39=3.9.16-1

And not pip, but this does not add back the -U for PyPA yet.

Add back -U for PyPA packages
Try pinning python39=3.9.16-1 with old action/everything

This is extremely unlikely to work, I just want to check.

Try just setup-cygwin and pinning python39=3.9.16-1

That is, this puts back all the other stuff the way it was on the main branch when the breakage occurred, besides changing from cygwin-install-action to setup-cygwin to make pinning work and using it to get version 3.9.16-1 of the Cygwin python39 package.

EliahKagan changed the title~~Use Python 3.9.16 instead of 3.9.18 on Cygwin CI~~ Pin Python 3.9.16 on Cygwin CI

Jan 28, 2024

This was referenced

Jan 28, 2024

lettuce-bot bot referenced this pull request in lettuce-financial/github-bot-signed-commit

Feb 15, 2024

renovate bot referenced this pull request in allenporter/flux-local

Feb 16, 2024

EliahKagan added a commit to EliahKagan/GitPython that referenced this pull request

Jan 4, 2025

We pinned Python 3.9.16 on Cygwin CI in gitpython-developers#1814 (by requiring 3.9.16-1 as the exact version of the python39 Cygwin package, along with other supporting changes). We did this to solve a problem where Python 3.9.18-1, which contained a bug that broke GitPython CI (and various other software), would be selected.

Version 3.9.18-1 was marked back to being a "test" package shortly after the bug was reported, and was subsequently removed altogether from the Cygwin repositories. Because the affected package version effectively no longer exists, and because this issue is known and a non-"test" version still affected by it is very unlikely to be released in the future, this pinning has been decisively unnecessary for some time, though still not harmful.

This commit undoes the pinning, so that the python39 package can be installed at a higher version if one becomes available. This serves two purposes.

There is work under way in porting Python 3.12 to Cygwin. To test this with GitPython (either while it is in development or later), it will be useful to turn the Cygwin test job into a matrix job definition, generating two jobs, one for Python 3.9 and one for Python 3.12. Since 3.12 will probably not benefit from pinning, dropping pinning simplifies this.
If the port of Python 3.12 to Cygwin is successful, it might lead to a solution to the but that currently keeps 3.9.18 from being made available for Cygwin. In that case, another 3.9.18-* Cygwin package would be released, which we would want to use.

Although this is uncertain, the change is a simplification, so I think it is reasonable to do now.

Note that the pinning being undone here only affects the distinction between different 3.9.* versions. python39 and python312 are different Cygwin packages altogether, with correspondingly different python39-* and python312-* associated packages; this is not unpinning Python 3.9 in a way that would cause Python 3.12 to be selected instead of it.

EliahKagan added a commit to EliahKagan/GitPython that referenced this pull request

Jan 4, 2025

This commit undoes the pinning, so that the python39 package can be installed at a higher version if one becomes available. This serves two purposes.

There is work under way in porting Python 3.12 to Cygwin. To test this with GitPython (either while it is in development or later), it will be useful to turn the Cygwin test job into a matrix job definition, generating two jobs, one for Python 3.9 and one for Python 3.12. Since 3.12 will probably not benefit from pinning, dropping pinning simplifies this.
If the port of Python 3.12 to Cygwin is successful, it might lead to a solution to the but that currently keeps 3.9.18 from being made available for Cygwin. In that case, another 3.9.18-* Cygwin package would be released, which we would want to use.

Although this is uncertain, the change is a simplification, so I think it is reasonable to do now.