WIP: [2.7] bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705) by matrixise · Pull Request #9938 · python/cpython (original) (raw)
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Conversation12 Commits3 Checks0 Files changed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})
[2.7] bpo-24568: Fix read/write greater than 2 GiB on macOS (GH-1705) [2.7] bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705)
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should also patch:
- Modules/_io/fileio.c: fileio_readinto(), fileio_readall(), fileio_read(), fileio_write()
- Modules/posixmodule.c: posix_read(), posix_write()
You can add the #define at the end of Include/pyport.h.
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.
Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.
@vstinner In fact, I should mark this PR as "DO-NOT-MERGE" because I didn't finish it yesterday :/
[2.7] bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705) WIP: [2.7] bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705)
WIP: [2.7] bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705) [2.7] bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705)
| @@ -6805,6 +6805,9 @@ posix_read(PyObject *self, PyObject *args) |
|---|
| return posix_error(); |
| } |
| Py_BEGIN_ALLOW_THREADS |
| if (size > _PY_READ_MAX) { |
| size = _PY_READ_MAX; |
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hum. It seems inefficient to allocate size bytes and only later pass _PY_READ_MAX. This truncation should be done before PyString_FromStringAndSize().
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we update also Py_UniversalNewlineFread()?
Shouldn't we update also Py_UniversalNewlineFread()?
It's implemented with fread() which uses size_t for the length parameter. I hope that the libc on macOS implements fread() properly for read larger than 2 GiB. If it's not the case, it's would be a bug in the libc, no? :-)
[2.7] bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705) WIP: [2.7] bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705)
ok, I will continue this PR, try to synthesize your comments and try to fix this PR. thanks
On macOS, fix reading from and writing into a file with a size larger than 2 GiB.
Need to finish this PR, I have macOS, will try to port the patch to Python 2.7.
Thank you for your patience and I am really sorry for the big delay 👎
Need to finish this PR, I have macOS, will try to port the patch to Python 2.7. Thank you for your patience and I am really sorry for the big delay -1
I don't understand the purpose of such comment. It sends me an unsolicited notification (email).
@matrixise EOL snuck up on you before this got finished, so I'm closing it for you :)