Within a do_GET hander for a BaseHTTPServer.BaseHTTPRequestHandler, trying to write unicode data causes a UnicodeEncodeError exception. It should be possible to send Unicode data to the browser. The enclosed Python file demonstrates the issue.
The diagnostic printed is: File "C:\Apps\Python25\lib\socket.py", line 255, in write data = str(data) # XXX Should really reject non-string non-buffers The comment indicates the developer was aware of the bug. See also similar bug in writelines(), near line 267.
Due to its nature it is impossible to transmit unicode over the wire. Unicode must always be encoded to bytes before it can be stored on the hard disk or transmitted. Typically it's UTF-8 but in your case it depends on the client's browser and the Request header. The simple BaseHTTPServer isn't clever enough to encode your unicode data on the fly. You have to do it yourself.
As implemented it's not even possible to send UTF-8, because the "data = str(data)" line only accepts seven bit ASCII with the default encoding. Since there's no easy way to change the encoding "str()" uses, some other mechanism should be available to do the encoding (as implied by the "XXX" comment).
Yes, it's possible to send UTF-8 data: >>> data = u"testdata umlaut öäü".encode("utf-8") >>> data 'testdata umlaut \xc3\xb6\xc3\xa4\xc3\xbc' >>> type(data) <type 'str'> >>> data == str(data) True >>> data is str(data) True You have to encode your unicode string to a byte string. u''.encode(encoding) always returns a string. str() on a string doesn't alter a string. As you can clearly see it's a NOOP (no operation).
This issue has been migrated to GitHub: https://github.com/python/cpython/issues/45751 