| msg300043 - (view) |
Author: Lin Tian (tianlynn) * |
Date: 2017-08-10 04:48 |
| utf8_toUtf8(const ENCODING *UNUSED_P(enc), const char **fromP, const char *fromLim, char **toP, const char *toLim) { char *to; const char *from; const char *fromLimInitial = fromLim; /* Avoid copying partial characters. */ align_limit_to_full_utf8_characters(*fromP, &fromLim); for (to = *toP, from = *fromP; (from < fromLim) && (to < toLim); from++, to++) *to = *from; *fromP = from; *toP = to; if (fromLim < fromLimInitial) return XML_CONVERT_INPUT_INCOMPLETE; else if ((to == toLim) && (from < fromLim)) // <===== Bug is here. In case (to == toLim), it's possible that // from is still pointing to partial character. For example, // a character with 3 bytes (A, B, C) and form is pointing to C. // It means only A and B is copied to output buffer. Next // scanning will start with C which could be considered as invalid // byte and got dropped. After this, only "AB" is kept in memory // and thus it will lead to invalid continuation byte. return XML_CONVERT_OUTPUT_EXHAUSTED; else return XML_CONVERT_COMPLETED; } |
|
|
| msg300044 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) *  |
Date: 2017-08-10 04:58 |
| This is not our code. Please use corresponding bug tracker [1] if you have found a bug in Expat. But I think there is not a bug here. [1] https://github.com/libexpat/libexpat/issues |
|
|
| msg300268 - (view) |
Author: Lin Tian (tianlynn) * |
Date: 2017-08-14 22:44 |
| Reactive this issue as to let you know that libexpat has confirmed and fixed the bug and they are interested in porting the fix to python. Reactive this in case you want to know what's going on and make a decision accordingly. (Sorry, I'm not very familiar with process here) |
|
|
| msg300309 - (view) |
Author: (sping) * |
Date: 2017-08-15 18:38 |
| For the record, the upstream fix is commit https://github.com/libexpat/libexpat/commit/74a7090a6eb92c27b7010287a4082de6b357fa42 and it will be part of Expat 2.2.4. |
|
|
| msg300605 - (view) |
Author: (sping) * |
Date: 2017-08-20 18:28 |
| Expat 2.2.4 with a fix has been released now: https://github.com/libexpat/libexpat/releases |
|
|
| msg301011 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2017-08-30 05:25 |
| I confirm the regression (see for reproducer). Victor, do you mind to update expat to 2.2.4? This issue can be classified as a security issue, since a regression was added in security update. |
|
|
| msg301012 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2017-08-30 05:41 |
| Corresponding Expat issue: https://github.com/libexpat/libexpat/issues/115. |
|
|
| msg301084 - (view) |
Author: Benjamin Peterson (benjamin.peterson) * |
Date: 2017-09-01 04:58 |
| We can put expat 2.2.4 in 2.7.14 final. |
|
|
| msg301270 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-09-04 21:42 |
| I produced attached PR 3315 using attached cpython_rebuild_expat_dir.sh + revert Modules/expat/expat_external.h change to keep #include "pyexpatns.h". |
|
|
| msg301287 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-09-04 23:58 |
| New changeset 759e30ec47048cb9835c62aaeac48748c8151390 by Victor Stinner in branch 'master': bpo-31170: Update libexpat from 2.2.3 to 2.2.4 (#3315) https://github.com/python/cpython/commit/759e30ec47048cb9835c62aaeac48748c8151390 |
|
|
| msg301399 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-09-05 23:30 |
| New changeset e5f2f8038540f9f06478f842f8f7313df4d2e59b by Victor Stinner in branch '3.6': bpo-31170: Update libexpat from 2.2.3 to 2.2.4 (#3315) (#3350) https://github.com/python/cpython/commit/e5f2f8038540f9f06478f842f8f7313df4d2e59b |
|
|
| msg301422 - (view) |
Author: Ned Deily (ned.deily) * |
Date: 2017-09-06 00:57 |
| New changeset 297516ea509c72d8ebed3a9b3ce200f023aca0b7 by Ned Deily (Victor Stinner) in branch '3.3': [3.3] bpo-30947, bpo-31170: Update expat from 2.2.1 to 2.2.4 (#3352) https://github.com/python/cpython/commit/297516ea509c72d8ebed3a9b3ce200f023aca0b7 |
|
|
| msg301424 - (view) |
Author: Benjamin Peterson (benjamin.peterson) * |
Date: 2017-09-06 01:07 |
| New changeset c00d5087cd308cc8be41e0afd8df27726185347f by Benjamin Peterson (Victor Stinner) in branch '2.7': bpo-31170: Update libexpat from 2.2.3 to 2.2.4 (#3315) https://github.com/python/cpython/commit/c00d5087cd308cc8be41e0afd8df27726185347f |
|
|
| msg302159 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-09-14 09:30 |
| I wrote an non-regression test for the Python master branch using the test case attached to https://github.com/libexpat/libexpat/issues/115: PR 3570. |
|
|
| msg302483 - (view) |
Author: Ned Deily (ned.deily) * |
Date: 2017-09-18 19:43 |
| I believe all the branches except 3.5 and 3.4 have been updated so adjusting the Versions field accordingly. All yours, Larry! |
|
|
| msg302833 - (view) |
Author: Larry Hastings (larry) * |
Date: 2017-09-24 08:04 |
| New changeset 86a713cb0c110b6798ca7f9e630fc511ee0a4028 by larryhastings (Victor Stinner) in branch '3.4': [3.4][Security] bpo-30947, bpo-31170: Update expat from 2.2.1 to 2.2.4 (#3353) https://github.com/python/cpython/commit/86a713cb0c110b6798ca7f9e630fc511ee0a4028 |
|
|
| msg302898 - (view) |
Author: Larry Hastings (larry) * |
Date: 2017-09-25 00:58 |
| New changeset f2492bb6aae061aea47e21fc7e56b7ab9bfdf543 by larryhastings (Victor Stinner) in branch '3.5': [3.5][Security] bpo-30947, bpo-31170: Update expat from 2.2.1 to 2.2.4 (#3354) https://github.com/python/cpython/commit/f2492bb6aae061aea47e21fc7e56b7ab9bfdf543 |
|
|
| msg302925 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-09-25 08:27 |
| New changeset e6d9fcbb8d0c325e57df08ae8781aafedb71eca2 by Victor Stinner in branch 'master': bpo-31170: Write unit test for Expat 2.2.4 UTF-8 bug (#3570) https://github.com/python/cpython/commit/e6d9fcbb8d0c325e57df08ae8781aafedb71eca2 |
|
|
| msg302927 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-09-25 08:43 |
| New changeset 5f5da728aec9c4f74cc771fbf30037b64a447514 by Victor Stinner in branch '2.7': bpo-31170: Write unit test for Expat 2.2.4 UTF-8 bug (#3570) (#3745) https://github.com/python/cpython/commit/5f5da728aec9c4f74cc771fbf30037b64a447514 |
|
|
| msg302930 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-09-25 09:00 |
| New changeset ad051cbce1360ad3055a048506c09bc2a5442474 by Victor Stinner in branch '3.6': bpo-31170: Write unit test for Expat 2.2.4 UTF-8 bug (#3570) (#3746) https://github.com/python/cpython/commit/ad051cbce1360ad3055a048506c09bc2a5442474 |
|
|
| msg302931 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-09-25 09:02 |
| libexpat has been upgraded to version 2.2.4 in Python 2.7, 3.3, 3.4, 3.5, 3.6 and master. I added an unit test to Python 2.7, 3.6 and master. I'm not sure about backporting the new unit test to Python 3.3, 3.4 and 3.5. I close the issue. |
|
|
| msg302975 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-09-25 19:48 |
| Oh, it seems like Steve Dower found an issue on Windows: PR 3751. I reopen the issue. |
|
|
| msg303037 - (view) |
Author: Larry Hastings (larry) * |
Date: 2017-09-26 13:00 |
| New changeset 44c1b62939a6192776dc9d093546154044cb2ecb by larryhastings (Steve Dower) in branch '3.5': [3.5] bpo-31170: Fix inclusion of expat in Windows build projects. (#3751) https://github.com/python/cpython/commit/44c1b62939a6192776dc9d093546154044cb2ecb |
|
|
| msg307254 - (view) |
Author: Larry Hastings (larry) * |
Date: 2017-11-29 18:50 |
| New changeset 8b11e8de7aedacfbbcc8c780f3c4097396f1d1a3 by larryhastings (Victor Stinner) in branch '3.4': [3.4] bpo-31170: Fix inclusion of expat in Windows build projects (#3785) https://github.com/python/cpython/commit/8b11e8de7aedacfbbcc8c780f3c4097396f1d1a3 |
|
|
| msg309469 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2018-01-04 14:45 |
| Can this issue be closed now? |
|
|
| msg309471 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2018-01-04 15:52 |
| Ah yes, it can be closed. I was waiting 3.4 and 3.5 fixes to be fixed, which is now the case. |
|
|