Back to blog

Mapping legacy data to an ontology without losing your mind.

Legacy data migration works when mappings become versioned, testable artifacts that preserve identity, expose ambiguity, and prove their output.

A legacy column name tells you how somebody stored data. It does not tell you what that data means.

That distinction is the difference between a migration you can correct and a conversion script you become afraid to rerun. The engineering rule I trust is to treat the mapping as a versioned, testable artifact. The ontology is the destination contract. The mapping and its evidence are the migration product.

Consider a legacy field called approvedBy. A sharper ontology distinguishes technicallyApprovedBy, riskAcceptedBy, and legallySignedBy. The tempting migration is a rename. The honest migration starts by admitting that the old field does not contain enough meaning to justify one.

Preserve the source before interpreting it

I would start with a stable snapshot and profile it before assigning ontology terms. Which values are null? Are identifiers unique? Do codes have a documented domain? Which joins recover context, and which merely look plausible?

Then preserve the source identity. For a row such as review_id=42, approved_by=17, approval_type=NULL, the target needs deterministic identities for review 42 and person 17. It also needs a trail back to the original keys and source value.

This does not mean exposing poor source identifiers as the public vocabulary. The target ontology should own stable IRIs and sharper concepts. The old keys belong in the reconciliation and provenance trail so a rerun produces the same identities and a correction remains explainable.

That extends the rule from Schema evolution in production ontologies: old data must remain intelligible while meaning gets sharper. Normalization without traceability destroys the evidence needed to tell whether the new graph is better or simply different.

State what the mapping knows

The mapping should say how source records become resources, properties, and values. It should also say where that translation stops.

The W3C R2RML Recommendation makes these decisions explicit for relational data: logical tables select source rows, subject maps generate resource identities, and predicate-object maps generate statements. Even code-to-IRI translation has to be specified because the target IRIs are not present in the database.

R2RML is not mandatory. For a small, stable export, a short transformation program can be easier to maintain. What matters is that the identity policy and semantic choices are visible, versioned, and executable. A spreadsheet of mapping intentions is documentation. It is not the migration.

For approvedBy, the mapping can emit a specific approval relation when reliable context supplies the type. When the type is absent, it should preserve the original assertion and produce an ambiguity result. It should not guess that every approval was technical because that is the most common value in a sample.

Quarantine meaning, not mess

Not every imperfect value deserves human review. Whitespace, known date formats, and casing rules can be normalized deterministically. Ambiguity about identity or meaning is different.

The SKOS mapping properties make this distinction concrete. Exact, close, broader, narrower, and related matches are different claims. owl:sameAs is stronger still: it says two individuals are the same resource. Treating all resemblance as identity manufactures certainty the source never contained.

An exception queue should therefore retain the source record, proposed mapping, reason for uncertainty, and mapping version. That turns an awkward row into a reviewable decision. It also prevents one unresolved record from blocking harmless normalization elsewhere.

Prove the graph before publishing it

A mapping is not finished when it emits triples. It is finished when representative source records remain traceable, expected queries still answer, validation results are reviewable, and rerunning the same snapshot produces the same identities.

The W3C R2RML test cases pair database inputs with expected RDF outputs. That is the right regression pattern. Keep small input-to-output fixtures beside the mapping. For review 42, the expected graph should contain stable review and person identities plus source provenance, no invented approval subtype, and a reviewable ambiguity result.

Then validate the generated graph. SHACL 1.2 Core produces a conformance result and detailed validation findings, which makes it useful as a publish gate for identifiers, datatypes, classes, and required relationships. Counts and important queries catch a different class of loss. PROV-O supplies a vocabulary for connecting the source snapshot, mapping run, generated graph, and responsible agents.

There is overhead here. Versioned mappings, fixtures, provenance, and an exception queue are more machinery than a one-off script. For a disposable export, that machinery can be excessive. But if the target graph will be rerun, corrected, or trusted by another system, the script was never really one-off.

The sane migration is the one that can show its work.