Implement TranscodingStream, a streaming equivalent of Encoding.Convert by GrabYourPitchforks · Pull Request #35145 · dotnet/runtime (original) (raw)

Resolves #30260.

The general idea is that if you have an inner Stream which represents machine-readable text under one encoding (say, shift-jis), this allows creating a wrapping Stream which represents that same machine-readable text under a different encoding (say, UTF-8). The primary scenario for this is as an adapter where you have System.Text.Json (as UTF-8) working on the outer Stream but where the client needs the response payload in a different encoding.

Usage (showing writing)

Encoding responseEncoding = GetResponseEncoding(); Stream responseStream = OpenResponseStream();

if (responseEncoding is UTF8Encoding) { // write directly to the response await JsonSerializer.SerializeAsync(responseStream, ...); } else { // insert our adapter Stream transcodingStream = Encoding.CreateTranscodingStream( innerStream: responseStream, innerStreamEncoding: responseEncoding, outerStreamEncoding: Encoding.UTF8 /* this is what System.Text.Json emits /, leaveOpen: true / if needed */); await JsonSerializer.SerializeAsync(transcodingStream, ...); await transcodingStream.DisposeAsync(); }

Note: When writing, the call to Stream.Close / Stream.Dispose / Stream.DisposeAsync is the thing that flushes the underlying Encoder / Decoder instances. This shouldn't normally matter as long as the caller isn't trying to write ill-formed data to the outer Stream. But if the caller does emit an incomplete multi-byte sequence (say, the UTF-8 bytes [ E0 BF ]) at the end of the stream, the fallback mechanism won't be invoked until the outer stream is disposed.