Conformance tests

The algorithm is validated against two test suites: the W3C R2RML test suite and the RML test suite (PostgreSQL subset). Both are included as git submodules. Forward mapping for both suites is performed with RMLMapper v8.0.1.

Setup

Initialize the submodule:

git submodule update --init --recursive

Running the test suite

There are two ways to run the conformance tests: from the terminal via pytest, or through a web dashboard that provides richer feedback for debugging.

Terminal

Pytest manages the PostgreSQL containers automatically, so no manual Docker setup is needed beyond having Docker running:

uv run pytest -v

To run a single test case from either suite:

uv run pytest tests/test_conformance.py::test_r2rml_conformance[R2RMLTC0001a] -v

uv run pytest tests/test_conformance.py::test_rml_conformance[RMLTC0001a] -v

To generate an HTML coverage report:

uv run pytest --cov --cov-report=html -v

Web dashboard

The dashboard runs through Docker Compose and lets you run individual test cases or the full suite. For each test case it shows the generated SPARQL queries, the reconstructed SQL, and a side-by-side comparison of the original and inverted database content, which is useful when diagnosing why a particular inversion fails.

docker compose up

The interface is available at http://localhost:5000. Results are saved to test_results/ as JSON and Markdown reports.

W3C R2RML test suite

The R2RML test suite contains 62 test cases, broken down as follows:

Category	Count
Successfully inverted	22
Not supported	13
Non-invertible	18
Forward mapping failed	9

Not supported (13)

These test cases use SQL queries as logical sources (rr:sqlQuery), which the algorithm does not handle.

Non-invertible (18)

Each of these falls into one of the known limitation categories:

Reason	Count
Partial mappings (unmapped columns)	8
Non-unique subject templates (duplicate rows lost)	3
IRI column term type (ambiguous base IRI resolution)	3
Combined causes (unmapped tables/columns and duplicates)	2
Constant-only mapping	1
NULL values in subject template	1

Forward mapping failed (9)

In nine test cases the forward mapping produces no RDF output, so inversion cannot be attempted.

RML test suite

The RML test suite contains 60 PostgreSQL test cases.

Category	Count
Successfully inverted	12
Not supported	9
Non-invertible	27
Forward mapping failed	12

Not supported (9)

All nine use SQL queries as logical sources (rml:query), which the algorithm does not handle, same as the R2RML counterpart (rr:sqlQuery).

Non-invertible (27)

Reason	Count
Partial mappings (unmapped tables)	12
Partial mappings (unmapped columns)	11
IRI column term type (ambiguous base IRI resolution)	3
Constant-only mapping	1

11 of the 12 unmapped-tables cases are inflated by a case-sensitivity bug: the table name in the mapping (e.g. Patient) is compared literally against the PostgreSQL catalog name (patient), which lowercases unquoted identifiers. The mismatch causes these tables to appear unmapped even though they have a triples map. With a case-insensitive comparison these 11 would fall into the unmapped-columns category instead, since each of them maps only a subset of the table’s columns. The single genuine unmapped-table case is RMLTC0012a, where the Lives table has no triples map at all.

The 11 unmapped-columns cases are structurally similar: the table itself is reconstructed, yet one or more columns have no corresponding predicate-object map and are therefore lost during inversion.

Forward mapping failed (12)

In twelve test cases the forward mapping produces no RDF output, so inversion cannot be attempted.