KROWN

This benchmark runs 58 scenarios generated by KROWN pinned at 824438c. The catalog is read from its RMLMapper configurations for RawData, Mappings, NamedGraph, JoinsRelation, and JoinsMultiple. It uses the scales documented in the KROWN 0.9.0 results (DOI: 10.5281/zenodo.10973892). The 1-1 relation and one-join-condition scales are absent from those configuration files, so the adapter adds those two cases explicitly and records the overlay in the result provenance.

The forward phase is executed by KROWN’s Executor: it discovers the generated metadata, starts KROWN’s PostgreSQL resource, loads the source data, invokes the selected engine, collects system metrics, preserves the generated RDF, and applies KROWN’s 15-second cooldown. The adapter pins the container image of that engine without modifying the KROWN submodule.

KROWN does not provide an inversion resource. For the backward phase, the adapter loads the same generated CSV files into separate source and destination schemas, invokes the selected inversion engine, and performs round-trip validation. This phase uses KROWN’s Collector and Stats classes directly, including the same 15-second cooldown. Local code also produces the JSON summaries and plots.

TM denotes a Triples Map; POM, a Predicate-Object Map; NG, a Named Graph Map; and SM, a Subject Map.

Scenario	Parameter values
Raw data: rows	10K, 100K, 1M, 10M
Raw data: columns	1, 10, 20, 30
Raw data: cell size	500, 1K, 5K, 10K
Mappings: TMs + 5 POMs	1, 10, 20, 30 TMs
Mappings: 20 TMs + POMs	1, 3, 5, 10 POMs
Mappings: NG in SM	1, 5, 10, 15 NGs
Mappings: NG in POM	1, 5, 10, 15 NGs
Mappings: NG in SM/POM	1/1, 5/5, 10/10, 15/15 NGs
Joins: 1-N relations	1-1, 1-5, 1-10, 1-15
Joins: N-1 relations	1-1, 5-1, 10-1, 15-1
Joins: N-M relations	3-3, 3-5, 5-3, 10-5, 5-10
Joins: join conditions	1, 5, 10, 15

Duplicate and empty-value scenarios, including JoinsDuplicate, are excluded: RDF does not preserve duplicate rows, and empty values in subject templates can suppress rows.

Validation

RawData exposes every source column, so its reconstructed tables must match the originals and return FULL.

The other suites omit source information. A recoverable scenario passes with PARTIAL only when its reconstructed columns are ordered subsets of the originals, contain no values absent from the corresponding source rows, and reproduce the same RDF dataset when mapped again.

A column that the reconstruction leaves entirely NULL counts as not reconstructed, the same as a column the engine omits. Engines differ in how they report what they could not recover, and this keeps the comparison on what was recovered. KROWN source tables hold a value in every cell, so an entirely NULL column always means the engine recovered nothing for it.

Mapping scenarios with more TMs than POMs and dynamic named-graph scenarios whose graph maps reference a column no predicate-object map exposes return AMBIGUOUS. Their surplus subject templates and graph maps differ only in the referenced column, so the graph carries those values without saying which column produced them. Such columns are left out of the reconstruction, which stays sound but can no longer rebuild the graph: the round trip is not attempted, since the mapping reads columns the reconstruction does not provide. Everything else the scenario exposes is reconstructed and measured, so AMBIGUOUS scenarios report throughput like any other.

The joins scenarios return AMBIGUOUS for the same reason. Their join condition equates a child column with a parent column and no term map emits either, so the graph records which rows are related without recording the value they are related on.

NON_INVERTIBLE is reserved for mappings that leave no recoverable column at all. No KROWN scenario is expected to return it.

Reported timings exclude validation. Results include source size, RDF statement counts, mapping structure, throughput, validation losses, summary statistics, and 95% confidence intervals.

Forward engines

Engine	Implementation	Image
rmlmapper	RMLMapper v8.1.0, built locally from this repository	kgconstruct/rmlmapper:v8.1.0
souffle	the Datalog resource of KROWN_Extended, which compiles the mapping into a Soufflé program	alloka/souffle:v1.0.0
morphkgc	Morph-KGC v2.2.0, pulled from Docker Hub	kgconstruct/morph-kgc:v2.2.0

Validating an inversion involves two materializations. The forward phase maps the source schema and produces the original dataset; the inversion writes its reconstruction into the destination schema; the validator then maps that destination schema and compares the second dataset with the first. A reconstruction that recovered everything the mapping reads yields the same dataset twice. Since nothing guarantees that two engines write the same statement the same way, the destination schema is mapped by the engine that produced the original dataset.

Inversion engines

KROWN_INVERSION_ENGINE selects which implementation performs the backward phase. Both receive the same mapping and the same RDF graph produced by the forward phase, write into the same destination schema, and are checked by the same validator. They differ in how they report a column they could not recover: kgi leaves it out of the destination table, souffle recreates the source schema and leaves the column NULL. The validator treats the two the same way, as described under validation.

Engine	Implementation
kgi	the local implementation, which generates SPARQL queries from the mapping
souffle	the Datalog approach of KROWN_Extended

Measurement modes

Set KROWN_MODE to one of these values:

Mode	Measured work	Work outside measurement
forward	KROWN Executor mapping step	generation and local validation
backward	inversion under KROWN Collector	generation, one KROWN forward run used to prepare the RDF input, local database setup, and validation
roundtrip	KROWN Executor mapping step and the paired inversion measurement	generation, local database setup, and validation

The reported stage durations come from KROWN’s metrics.csv: they are the difference between the first and last sample for that step, matching the duration calculation in KROWN Stats. The forward and backward phases keep separate metric files because only the forward phase can be represented by an unmodified KROWN scenario. In roundtrip mode, the runner pairs each forward iteration with the corresponding backward iteration and reports inversion overhead as inversion time / forward time × 100.

The official KROWN collector runs on the Linux host with a default sampling interval of 0.1 seconds. Its CPU, RAM, swap, disk, and network values describe the whole host, not an isolated benchmark process. Run the benchmark without unrelated workloads when comparing machines or revisions.

Legacy results

The checkpoint collected on 16 July 2026 used the earlier local runner and one iteration per scenario. The raw results and aggregated statistics remain available for historical comparison. Their timing and resource fields are not directly comparable with runs produced by the aligned adapter.

Outcome	Scenarios
FULL	8
PARTIAL	13
NON_INVERTIBLE	6
OUT_OF_MEMORY	11
TIMEOUT	3
FAILED	17

Of the 31 failed scenarios, 15 failed during forward mapping: 11 ran out of memory, three timed out, and one ended after a Java Virtual Machine crash. The official RMLMapper/PostgreSQL results also report resource limits for the same configurations: out-of-memory failures for 10 million rows and value sizes of 5,000 and 10,000 characters, and timeouts for named-graph scenarios and join-condition cases with 5, 10, or 15 conditions. See the KROWN record.

The other 16 failures are RDF round-trip mismatches after forward mapping completed. They remain recorded as failures pending investigation.

Raw data

Mappings

Named graphs

Joins

Run

Run all suites with the default five iterations in roundtrip mode:

make benchmark-krown

I sets the number of iterations per scenario and must be odd and at least 3. Select a measurement mode and suites with KROWN_MODE and KROWN_SUITES:

make benchmark-krown I=3 KROWN_MODE=forward KROWN_SUITES=mappings,named-graphs

KROWN_SUITES accepts raw, mappings, named-graphs, and joins, and defaults to all. The 10M-row raw-data case and the 1M-row named-graph cases require the most time and disk space.

To rerun one scenario, copy its generated name from the benchmark output:

make benchmark-krown I=3 KROWN_MODE=backward KROWN_SCENARIO=namedgraph_0SM-NG_5POM-NG_1TM_1POM_True

KROWN_SCENARIO requires an exact name. When combined with KROWN_SUITES, the selected scenario must belong to one of those suites. Results, statistics, and plots contain only the selected scenario.

A run that stops before the end writes its measured scenarios to a partial results file. KROWN_RESUME continues that session from the scenario it stopped on, taking the session directory that holds the file:

make benchmark-krown I=3 KROWN_RESUME=benchmarks/krown/results/krown_1785061943_roundtrip_rmlmapper_kgi

KROWN_FORWARD_ENGINE and KROWN_INVERSION_ENGINE select the two engines independently, and the Makefile builds only the images the selection needs:

make benchmark-krown I=3 KROWN_FORWARD_ENGINE=souffle KROWN_INVERSION_ENGINE=souffle

Change the system sampling interval with KROWN_INTERVAL, in seconds:

make benchmark-krown KROWN_INTERVAL=0.5

The runner generates one scenario at a time and removes it after its iterations. Each run creates a session directory named after the timestamp, the mode, and the two engines, under benchmarks/krown/results/. Forward artifacts are stored under each scenario’s forward/results/ directory. In backward mode, the unmeasured forward artifacts are kept under preparation/results/. Backward metrics are stored under backward/results/. The session also contains raw and aggregated JSON files, timing plots, and KROWN’s stats.csv, summary.csv, and aggregated.csv files for each measured phase.