[llvm-dev] [RFC] Compiled regression tests. (original) (raw)

David Greene via llvm-dev llvm-dev at lists.llvm.org
Wed Jul 1 11:26:42 PDT 2020

Michael Kruse via llvm-dev <llvm-dev at lists.llvm.org> writes:

Unfortunately, most tests we have do not even use placeholders for metadata nodes, including those generated by updatetestchecks.py. I looked into improving the script in this regard, but its implementation is function-centric, making it difficult to add module-level placeholders.

I actually have an implementation that does this for the update scripts. It's on my queue to upstream. In addition I have a number of enhancements that allow the update scripts to generate more focused, robust tests. The first of these is D68230 which is currently stuck and I'd appreciate some more eyes on it.

As a result, I estimate that I had to invest about twice the time to update/fix tests than writing the code changes themselves. Due to my experience, I find updating FileCheck tests very frustrating. I'd prefer not to test whether specific metadata nodes are present, but whether the LLVM API to query them (such as isAnnotatedParallel) returns the expected result.

I think both could be useful. With my update script enhancements I was able to successfully match specific metadata nodes and check them against metadata definitions at the Module level, though I'll admit my use-case is probably simpler than yours.

            -David

More information about the llvm-dev mailing list