Compilation fails when stdin is read multiple times (original) (raw)

December 3, 2024, 4:51pm 1

If a compilation requires multiple jobs that receive input from stdin, then each job will attempt to read from stdin. This can cause the first job to consume the input from stdin while the other jobs get an empty buffer if they are not in the same process. For example:

echo "int main() { return 0; }" | clang -x c++ - -arch arm64 -arch x86_64

This will first produce an intermediary object file with an empty x86 slice since the input was consumed while generating the arm slice. Then it results in a linker error due to failing to find the main symbol in the empty x86 slice.

I’ve came up with a few ideas, and was wondering if anyone had any thoughts or other ideas:

• As for detecting a situation when stdin is read multiple times, I’m not sure the best way to go about doing this. The key issue is really when a non-seekable file is read multiple times. If a non-seekable file corresponds to an “other” file then one conservative approach could be looking through each Jobs’ InputInfos to determine if an “other” file or - is an input in multiple jobs. I’m not sure if there are cases when an input for some job might not actually be read, or if there could be cases where - refers to something seekable.
• In my testing it didn’t seem like llvm::sys::fs::equivalent was returning true for different forms of stdin (such as "/dev/stdin" and llvm::sys::fs::getStdinHandle) so I’m unsure if this method could be used to detect if the same non-seekable file is an input for multiple jobs.
• Supposing we can detect when stdin might be read multiple times, I thought it would probably be best to emit an error. But if the detection is not precise, then perhaps a warning or leaving this unfixed is preferred? The driver could also dump stdin to a temp file and redirect each job’s stdin from there, but this extra copying seems to go against the spirit of the driver design.
• See here for a rough example commit.

I know this is likely an uncommon usecase, so I appreciate the input