[llvm-dev] RFC: Add a preprocessor to yaml2obj (and other YAML tools) (original) (raw)

Fangrui Song via llvm-dev llvm-dev at lists.llvm.org
Mon Feb 3 14:04:25 PST 2020

I am adding -D k=v to yaml2obj, similar to clang -D. This makes it easy to generate {32-bit,64-bit} x {big-endian,little-endian} tests.

--- !ELF FileHeader: Class: ELFCLASS[[BITS]] Data: ELFDATA2[[ENCODE]] Type: ET_DYN Machine: EM_X86_64

RUN: yaml2obj -D BITS=32 -D ENCODE=LSB %s -o %t.32le

RUN: yaml2obj -D BITS=32 -D ENCODE=MSB %s -o %t.32le

RUN: yaml2obj -D BITS=64 -D ENCODE=LSB %s -o %t.64le

RUN: yaml2obj -D BITS=64 -D ENCODE=MSB %s -o %t.64be

See https://reviews.llvm.org/D73828 for examples how -D simplifies tests.

Do people think it may be useful in other YAML tools? If yes, I'll move the yaml2obj implementation (https://reviews.llvm.org/D73821 ) to include/llvm/Support/YAMLTraits.h llvm::yaml::Input so that other YAML tools can use the feature.

Do people prefer a different syntax? I think [[PATTERN]] is nice because it is what FileCheck -DFILE=... uses:

CHECK: ... [[FILE]]

FileCheck only preprocesses patterns in CHECK lines. D73821 preprocesses both comment lines (which include CHECK lines) and non-comment lines (which include YAML). It is not a problem that the YAML preprocessor also processes CHECK lines, because tokens on a comment line will be ignored.

If -D UNDEF= is not specified, should [[UNDEF]] in the source be considered an error? I think it is fine not to treat it as an error because there can be legitimate use cases of unterminated [[, for example, [[ in a string literal. YAML parsing is complex. I don't expect the preprocessor to be smart enough to recognize string literals. (llvm/lib/Support/YAMLParser.cpp does not seem to provide raw strings of spaces and comments. Hooking a preprocessor into the scanner does not seem to be simple.)

Do people know other preprocessing features which may be useful?

More information about the llvm-dev mailing list