Issue 17647: subprocess.communicate() should preserve colored output on Windows (original) (raw)

Issue17647

Created on 2013-04-06 21:19 by Caitlin.Potter, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
unit-tests.png Caitlin.Potter,2013-04-06 21:19 Screenshot demonstrating problem
Messages (12)
msg186168 - (view) Author: Caitlin Potter (Caitlin.Potter) Date: 2013-04-06 21:19
In migrating from GNU autoconf/automake build systems to a python-based build system (Waf), I've been slightly annoyed that coloured text output from unit test programs is lost on the windows platform (the gtest framework uses ::SetConsoleTextAttribute on windows) Ideally, the coloured output from the test sets would be preserved so that problems could be easily identified and stand out (See attached image for demonstration of problem) This might be scoffed at as a minor problem because nobody uses SetConsoleTextAttribute anyways, and even if they do it would only affect the windows platform. But just the same, preserving coloured output on windows should be doable. I'd be happy to work on a patch for this myself, but I'm new to the python tree and am not completely sure where to find what I'm looking for. I think an if mswindows: clause wherever stdout.read() is would probably work, but I'm not sure where that would be.
msg186234 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2013-04-07 17:59
I don't see how this is a subprocess problem, or could be fixed in subprocess. IIUC, SetConsoleTextAttribute() only has an effect if the output is connected to a console. But that is not the case if you redirect the output to a pipe (which is presumably what Waf does so it can capture the output).
msg186235 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-04-07 18:21
Oh, good, I thought that was probably the case but I don't know Windows enough to have been sure. Caitlin: if you can prove sbt and me wrong by writing a patch that works (without breaking anything else :), I think it would be considered. Certainly on unix if you write ANSI color codes to stdout and the reader doesn't strip them, they will be preserved and can be redisplayed, so being able to do something similar on Windows would be nice. I'm pretty sure there isn't any place stdout.read() is called, though. The stdout output from the originating process will be being read by a different process via stdin.read(). And as sbt pointed out, neither one of those will be a console.
msg186236 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2013-04-07 18:39
On 07/04/2013 7:21pm, R. David Murray wrote: > Certainly on unix if you write ANSI color codes to stdout and the reader > doesn't strip them, they will be preserved and can be redisplayed, so > being able to do something similar on Windows would be nice. Although sensible unix programs capable of producing coloured text refuse to do so (by default) if output is a pipe rather than a tty.
msg186241 - (view) Author: Caitlin Potter (Caitlin.Potter) Date: 2013-04-07 20:02
I'm not entirely positive that it would be doable, but looking at the subprocess code, it looks like we do have an open handle to the windows stdout buffer, including buffer attributes, so it should be possible to translate coloured attributes into ANSI codes, Whether this would or would not break anything else is a different story, of course. Obviously there are times where you wouldn't want ANSI color codes in the output, and I'm not sure how you could differentiate between a pipe to a terminal buffer, or a pipe to a file. Somewhat relevant, but perhaps not so much: The google test framework does actually test isatty() before using ANSI escape characters, however I've tested this same test program in a development environment, with colours preserved, which was a bit curious (the test I've put together is available at http://github.com/caitp/waftest, however it will break on a lot of systems, depending on the way pthread needs to be used) On Sun, Apr 7, 2013 at 2:39 PM, Richard Oudkerk <report@bugs.python.org>wrote: > > Richard Oudkerk added the comment: > > On 07/04/2013 7:21pm, R. David Murray wrote: > > Certainly on unix if you write ANSI color codes to stdout and the reader > > doesn't strip them, they will be preserved and can be redisplayed, so > > being able to do something similar on Windows would be nice. > > Although sensible unix programs capable of producing coloured text > refuse to do so (by default) if output is a pipe rather than a tty. > > ---------- > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue17647> > _______________________________________ >
msg186242 - (view) Author: Caitlin Potter (Caitlin.Potter) Date: 2013-04-07 20:05
> however I've tested this same test program in a > development environment, *unix* development environment (xterm, ubuntu 12.04), rather.
msg186244 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2013-04-07 20:38
On 07/04/2013 9:02pm, Caitlin Potter wrote: > I'm not entirely positive that it would be doable, but looking at the > subprocess code, it looks like we do have an open handle to the windows > stdout buffer, including buffer attributes, so it should be possible to > translate coloured attributes into ANSI codes, The handle for stdout is just the readable end of a pipe. It is not a console, GetConsoleScreenBufferInfo() will not work with it, and there are no coloured attributes associated with it.
msg186245 - (view) Author: Caitlin Potter (Caitlin.Potter) Date: 2013-04-07 20:50
Then perhaps nothing can be done from the python side of things, that's too bad. On Sun, Apr 7, 2013 at 4:38 PM, Richard Oudkerk <report@bugs.python.org>wrote: > > Richard Oudkerk added the comment: > > On 07/04/2013 9:02pm, Caitlin Potter wrote: > > I'm not entirely positive that it would be doable, but looking at the > > subprocess code, it looks like we do have an open handle to the windows > > stdout buffer, including buffer attributes, so it should be possible to > > translate coloured attributes into ANSI codes, > > The handle for stdout is just the readable end of a pipe. It is not a > console, GetConsoleScreenBufferInfo() will not work with it, and there > are no coloured attributes associated with it. > > ---------- > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue17647> > _______________________________________ >
msg186657 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-04-12 18:08
We seem to agree that this is an OS+application issue, not a Python issue. I think the red FAILEDs would be nice for unittest (a possible separate issue).
msg186658 - (view) Author: Caitlin Potter (Caitlin.Potter) Date: 2013-04-12 18:14
A suggestion to work around this from #waf on freenode: http://codepad.org/1Y8K9e2m So it is probably not a big deal and can be wrapped up. But still it would be nice if Windows had native support for ANSI colours. On Fri, Apr 12, 2013 at 2:08 PM, Terry J. Reedy <report@bugs.python.org>wrote: > > Terry J. Reedy added the comment: > > We seem to agree that this is an OS+application issue, not a Python issue. > > I think the red FAILEDs would be nice for unittest (a possible separate > issue). > > ---------- > nosy: +terry.reedy > resolution: -> invalid > status: open -> closed > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue17647> > _______________________________________ >
msg186662 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-04-12 18:42
Caitlin, when you reply by email, please snip the previous post as I have here, as it is already displayed directly above the reply on the website.
msg186667 - (view) Author: Caitlin Potter (Caitlin.Potter) Date: 2013-04-12 19:03
Sorry Terry, gmail hides it by default, didn't notice.
History
Date User Action Args
2022-04-11 14:57:43 admin set github: 61847
2013-04-12 19:03:08 Caitlin.Potter set messages: +
2013-04-12 18:42:45 terry.reedy set messages: +
2013-04-12 18:14:46 Caitlin.Potter set messages: +
2013-04-12 18:08:52 terry.reedy set status: open -> closednosy: + terry.reedymessages: + resolution: not a bug
2013-04-07 20:50:59 Caitlin.Potter set messages: +
2013-04-07 20:38:59 sbt set messages: +
2013-04-07 20:05:46 Caitlin.Potter set messages: +
2013-04-07 20:02:58 Caitlin.Potter set messages: +
2013-04-07 18:39:22 sbt set messages: +
2013-04-07 18:21:18 r.david.murray set nosy: + r.david.murraymessages: +
2013-04-07 17:59:46 sbt set nosy: + sbtmessages: +
2013-04-06 21:19:39 Caitlin.Potter create