Rework turtle-like test suites by aucampia · Pull Request #1986 · RDFLib/rdflib (original) (raw)

This patch reworks the turtle-like test suites (N-Quads, N-Triples,
Turtle, TriG). The changes includes:

• A new DAWG Manifest processor:
  • Incorporates pytest expected failure support in test parameter generation.
  • Supports URI mapping so that local filesystem URIs can be mapped to the
    correct remote base URIs and still work correctly with local files.
  • Supports custom entry types so that suite specific handling can be
    implemented separately from the core manifest processor. This will
    be used for SPARQL test suite and JSON test suite processing.
• Updated EARL reporter:
  • Support for the new DAWG Manifest processor.
  • Writing of EARL reports by default if a report prefix is
    specified in the DAWG Manifest processor.
  • Reports output is sorted so that it can be compared with text based
    diff tools.
  • Fixed reporting of xfail tests. These were reported as skipped before,
    now they are reported as failed.

The testing is also more accurate, and now tests which were reported as
passing before are now correctly reported as failing. With one exception
all these failures are related to leniency in the parser, as our parser
accepts input that should raise an error when parsing.

Other changes:

• Moved IRI related test utilities to a separate module.
• Added generated reports for turtle-like formats to git with -HEAD
  suffix.

Subsequent changes will use this new processor for SPARQL and RDF/XML,
the draft for these can be seen in #1969.

Summary of changes

Checklist

• Checked that there aren't other open pull requests for
  the same change.
• Checked that all tests and type checking passes.
• Considered granting push permissions to the PR branch,
  so maintainers can fix minor issues and keep your PR up to date.