Test macOS on CI by EliahKagan · Pull Request #1752 · gitpython-developers/GitPython (original) (raw)

This expands the CI test matrix in pythonpackage.yml so it tests on macOS in as well as Ubuntu and Windows. To make this work, macOS has to be allowed more extra_time in test_blocking_lock_file. Previously all non-Windows systems were given the same amount of small extra time, while Windows--both native Windows and Cygwin--were given more. Now macOS is given more--even more than Windows, as it occasionally needs that.

Although macOS (at least on this CI) is slower than Windows (both native Windows and Cygwin) in that specific part of that test (which is timed for an assertion), it is faster than native Windows overall (and thus of course much faster than Cygwin). It looks like this would still hold up even if we did not install WSL on native Windows, albeit by a slightly smaller margin. This accords with what I usually observe for Python on GitHub Actions CI: Ubuntu tends to be faster than macOS which tends to be faster than Windows.

Currently the macos-latest GitHub Actions runners are macOS 12: macos-latest is a another name for macos-12 at this time, though it will refer to a later version eventually. It seems macOS 13 (the macos-13 runners) are faster by a small amount. This seems to improve performance on that one tricky part of test_blocking_lock_file negligibly and I would not prefer it for that reason. But it also makes the whole run a little faster. That makes the macOS runners available for another CI test job sooner, and may make CI complete sooner on some PRs/branches, at least those with many pushes. So I've used macos-13, even though it means the runner should be revisited and macos-13 probably changed to macos-latest once it is macOS 13 or later.

There are faster macOS runners, in terms of the virtual machine itself rather than the software on it. I did not use those, because I don't believe they are available for unpaid usage, even from public projects. I would also be somewhat disinclined to use them for GitPython, except in temporary and experimental cases. If tests are especially slow on the basic runners, I think that is a valuable source of information, especially if it happens as a regression.

Although the whole runs are faster on macOS than Windows, I think there may be fewer concurrent macOS runners available per organization. If so, it might turn out that having this many test jobs is more inconvenient for macOS than Windows. Furthermore, Ubuntu and macOS are more similar to each other than either is to Windows; GitPython currently seems to do no explicit special-casing for macOS, while there is plenty for Windows; and there are a number of known Windows-specific issues for GitPython. Therefore, the case for testing all six Python versions GitPython supports on macOS is, in my opinion, much weaker than on Windows. However, the first point in the "Which versions should we test?" section of #1745 does apply: the runs are pretty fast.

So I've proposed that all versions be tested, at least initially. You can let me know if you'd like me to add another commit to test on fewer. That can also be done separately and later if experience shows it's warranted.