[llvm-dev] Multiple documents in one test file (original) (raw)

David Blaikie via llvm-dev llvm-dev at lists.llvm.org
Mon Jul 13 18:27:36 PDT 2020

(+Richard - it's handy to include folks from previous discussions explicitly so everyone can more easily keep track of the conversation)

On Mon, Jul 13, 2020 at 6:17 PM Fangrui Song via llvm-dev < llvm-dev at lists.llvm.org> wrote:

Sometimes it is convenient if we can specify multiple independent tests in one file. To give an example, let's discuss test/MC/ELF/debug-md5.s and test/MC/ELF/debug-md5-err.s (.file directive in the assembler).

a) An invalid .file makes the whole file invalid. Because errors lead to a non-zero exit code, We have to use RUN: not llvm-mc %s for the whole file. This often lead to users placing good (RUN: llvm-mc %s) and bad tests (RUN:_ _not llvm-mc %s) separately. For some features, having both good and bad tests in one file may improve readability. b) .debugline is a global resource. Whenever we add a (valid) .file, we contribute an entry to the global resource. If we want to test some characteristics when includedirectories[0] is A, and other characteristics when includedirectories[0] is B, we have to use another test file. The arguments apply to many other types of tests (opt on .ll, llc on .ll and .mir, clang on .c, yaml2obj on .yaml, etc). I have a patch teaching llvm-mc about an option to split input: https://reviews.llvm.org/D83725 (30+ lines) In a comment, Richard Smith mentioned whether we can add a separate extractor utility: _ _# RUN: extract bb %s | llvm-mc - 2>&1 | FileCheck %s --check-prefix=BB_ _or_ _# RUN: extract bb %s -o %t.bb_ _# RUN: llvm-mc %t.bb 2>&1 | FileCheck %t.bb_ _

Could make "extract" work a bit like "tee" so it can still be one line:

RUN: extract bb %s -o %t.bb | llvm-mc - 2>&1 | FileCheck %t.bb

(could even make it a bit shorter for convenience - 'ex' or something)

The advantage is its versatility. The downside is somewhat verbose syntax.

Some questoms: 1. What do people think of the two approaches? An in-utility option vs a standalone utility. 2. For llvm-mc, if we go with an option, is there a better name than --doc-id? David Blaikie proposed --asm-id (This is my personal preference, trading 30+ lines in a utility for simpler syntax) 3. If we add a standalone utility, how shall we name it? (Note that llvm-extract exists, but people can probably distinguish 'extract' from llvm-extract

I think some of the truly internal utilities are named without the llvm prefix - isn't stuff like "not" actually implemented as a local tool? hmm, guess not, maybe that's a built-in inside lit. Only risk I can think of with the name is the auto-name expansion of lit replacing any token 'ex' with the full path to the tool, so you might have to be careful about not using that character sequence as a standalone argument on a RUN line - but that seems OK.

LLVM Developers mailing list llvm-dev at lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200713/f6e7dc12/attachment.html>

More information about the llvm-dev mailing list