Structure Beats Magic ← All principles
Universal principles · Integration

Cleaned and structured

The middle layer is where loose records become something you can trust. Each record gets a shape, a type and a place, and a rule checks it before anything is built on top of it.

Line drawing of mixed shapes entering a gate with a checklist, and leaving as neat rows in a table; a gold spark marks the check.

What it is

Integration turns raw records into consistent ones. Dates get one format. A person mentioned in an email, a photo and a calendar entry becomes one person, not three. Every unit gets frontmatter that says what it is, and a rule checks that it holds.

Why it matters

An AI, a dashboard and a search box all fail the same way on messy input: they give a confident answer built on the wrong record. Integration is where you catch that, once, instead of in every answer afterwards.

Integration is where the structure becomes queryable. My files are the source of truth; a DuckDB database is rebuilt from them whenever they change. The database is a view I can throw away, because the files and the rules that read them are what count.

What it looks like

At home. One folder per unit, with a note of the same name that carries its properties. Photos get a sidecar with date, place and people. A rebuild script reads all of it into tables, and a check refuses records that break the agreed shape, such as a date that does not parse.

At work. This is the integration layer of a data warehouse: business keys, conformed definitions and data-quality rules that run before anything reaches a report.

The usual mistake

A fix made by hand in the final tool stays there. If you fix a date in the app that shows it, the fix lives only there. Fix it in the source or in a rule, and every view downstream gets it too.

Where it stops

Every rule you add is one more thing to maintain, and a rule that is too strict blocks good records. Start with the few checks that catch real errors, and add more when a mistake shows you where.

Go further