[Python-Dev] (Licensing question) backport of shutil.copyfile() functionality (original) (raw)

Gregory P. Smith greg at krypto.org
Tue Mar 12 20:32:35 EDT 2019

On Tue, Mar 12, 2019 at 2:55 PM Giampaolo Rodola' <g.rodola at gmail.com> wrote:

On Tue, Mar 12, 2019 at 3:01 AM Glenn Linderman <v+python at g.nevcal.com> wrote: On 3/11/2019 4:35 PM, Giampaolo Rodola' wrote:

Hello, some time ago I contributed a couple of patches to speedup shutil.copy*() functions: https://bugs.python.org/issue33671 https://bugs.python.org/issue33695 I would like to backport both functionalities so that they can be used on Python 2.7 and <3.8 and put it on PYPI. In order to do so I will basically have to copy some parts of shutil module (copytree() function + the unit-tests I added in BPO-33671 and a couple of other things). Are there constraints regarding this in terms of license? Am I supposed to use GPL? (I was thinking about using MIT) Note: in this package called "zerocopy" I will probably want to expose other functionalities such as tee(), splice() and CopyFileEx and TransmitFile on Windows, so it's probably gonna be half a backport and half a brand new project. Thanks.

Thanks for the contributions. I don't know about the licensing. I wonder if you should make two packages, though... one just exactly a backport of the shutil speedups, and the second containing the new functionalities. That was my initial thought as well (a "backports.shutilcopy" module targeting copy* functions only), but (especially after playing with this today) I think I have something a bit more ambitious in mind. I'm currently experimenting with different things which could be baked in a third-party lib and possibly contributed back to Python later on:

If you might want some of this contributed back to Python later on, you should not use the GPL.

  1. on OSX we could use f/copyfile() syscall to copy file attrs/metadata;

    that may be useful to speedup shutil.copystat() and shutil.copymode() 2) copytree() on OSX could take advantage of f/copyfile() + COPYFILERECURSIVE (which is probably too platform-specific for inclusion) 3) on Linux we could use copyfilerange() as a replacement for os.sendfile() in shutil.copyfile() (it's supposed to be faster) 4) on Linux ioctl() + FICLONE could be used to implement CoW (copy on write) instantaneous copies, and could be added as shutil.cowcopyfile() (haven't look into Windows yet) 5) I was thinking about backporting socket.socket.sendfile() as well, which uses os.sendfile() on POSIX but not TransmitFile on Windows (asyncio does though) 6) another idea (but I'm not sure if it's possible, as I still have to dig into that) is a new socket recvfile() function boosted up by tee() / splice() on Linux, which maybe could be contributed back as socket.socket.recvfile()

    -- Giampaolo - http://grodola.blogspot.com

Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/greg%40krypto.org -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20190312/1e2721bd/attachment.html>

More information about the Python-Dev mailing list