CI: Force newer S3FS by lithomas1 · Pull Request #48291 · pandas-dev/pandas (original) (raw)

Is there a reason to pin down s3fs?

mamba is pulling really old versions of s3fs by chance(I think it's happening because old s3fs are not pinning down aiobotocore, allowing for mamba to pull down the newest aiobotocore versions which are not actually compatible with s3fs). Really old s3fs working with the newest aiobotocore is more luck than anything else.

This PR is just forcing mamba not to do this by setting a lower bound to a s3fs that pins down aiobotocore to a version it can support. The minimum version is still getting tested in the min version build so I don't think we're losing coverage.

Could we enforce botocore>=1.23.0 instead? Looks like the six changes occured there: boto/botocore@5d4ecca

I haven't tested this, but I'm pretty sure aiobotocore pins/controls the botocore version(IIUC it messes around with botocore internals so they have to pin down to a specific botocore version), so doing that would probably just result in an unsolvable environment.

(As for why we care about aiobotocore, its a dependency of s3fs)

I agree with you on the point of bumping s3fs though(I think we should probably do it ASAP for 2.0).