How the corpus is built: and what we won't let into it.

What's in here: the honest funnel

Three numbers, all true, all live. Refreshed nightly from the inspection database.

Defects observed

242,618

Individual defect findings extracted from reports: every crack, every conducive condition, every switchboard note. Severity-tagged 1–5.

Types indexed

30,388

Canonical defect vocabulary produced by LLM clustering of the raw observations. The A–Z index lists every one.

Curated articles

90

Inspector-authored entries with severity rationale, suburb back-references and remediation guidance. Growing weekly.

The gap between the three numbers is the point: 242,618 raw observations collapse into 30,388 canonical types, of which 90 currently have full inspector commentary. The rest render from dedup of the raw prose, readable but not yet inspector-voiced. We prioritise curation by observation frequency and by whether the defect is already linked from one of our 225 suburb pages.

Three intake streams

Every entry traces back to a typed source so you can weight it. Source type is published on every entry profile.

• Real: observations distilled from anonymised SURE inspection reports across WA and NSW (2016 onward). These carry actual instance counts and suburb back-references.
• Synthetic: LLM-generated variants seeded by real cases, used to broaden symptom and remedy coverage. Always reviewed by a qualified inspector before publishing.
• Standard: entries derived from clauses of the National Construction Code, AS 4349.1, AS 4349.3 and related Australian Standards (where licensing permits). Citations are explicit.

k-anonymity ≥ 5, no PII, no addresses

Reports are stripped of client names, addresses, lot identifiers and inspector signatures at ingest. Aggregations (frequency, era distribution) only publish where at least 5 independent properties contribute to the bucket. Single-property anecdotes are never quoted verbatim.

One defect, twelve dimensions

Each defect type is structured into a single schema covering category, subcategory, description, causes, symptoms, remedies, severity range, era relevance, material relevance, source type, optional standard citation and a quality score. The schema is versioned in src/schemas/defect-type.ts and validated on every ingest.

Collapsing raw prose into canonical lines

Inspection reports are written on the job, often in shorthand. The same defect might be logged as "Ground movement", "Ground movement.", "Differential settlement" or "Building movement" across hundreds of reports. Before we ship a defect page, we collapse those variants in two passes.

1. Normalisation: lowercase, collapse whitespace, strip trailing punctuation. Merges obvious variants by exact normalised key; keeps the longest original text as the representative.
2. Embedding clustering (fast-follow): each remaining line is embedded with Gemini text-embedding-004; lines within cosine distance 0.15 of a cluster medoid are treated as the same canonical item. Cluster size is published on the entry so you can see how many times each line appeared independently.

The dedup pipeline is idempotent and runs as a materialised view on Postgres; we keep the top 5 canonical causes, symptoms and remedies per defect type, rebuilt nightly.

Vectors first, graph second

Every defect description, cause, symptom and remedy is chunked, embedded with Gemini text-embedding-004, and stored in pgvector. The search bar on the index page hits the live Go service on k3s, which fans out:

1. Cosine similarity over chunk embeddings (top-k candidates).
2. Graph expansion through FalkorDB to pull related causes, symptoms, remedies and standards.
3. Score blending (vector + graph proximity) before returning ranked results.

Standards are quoted, not paraphrased

Where a standard underpins a defect, the entry exposes the standard code (e.g. AS 4349.1) and clause where licensing allows it. Standards content is never republished in full . We link to the source. NCC references follow ABCB usage guidelines.

A score, not a vibe

Every entry carries a 0–1 quality score combining: completeness of fields, agreement between independent inspector observations, presence of supporting standards, and recency. Entries below 0.6 stay in draft and never publish. Threshold and rubric are versioned.

CC BY-NC 4.0 with attribution

The DefectIndex corpus is published under CC BY-NC 4.0. Use it freely for research, journalism and consumer education with attribution to SURE Building Inspections DefectIndex. Commercial reuse requires a licence agreement. Get in touch.

What the NCC and AS rights look like

Some entries reference the National Construction Code, which is licensed by the Australian Building Codes Board under CC BY-ND 4.0, © Commonwealth of Australia and States and Territories of Australia. We cite NCC clause numbers as factual references; we never reproduce, paraphrase, or summarise NCC clause text. For the authoritative wording, read the Code at ncc.abcb.gov.au.

Entries also reference Australian Standards (e.g. AS 3660, AS 3740, AS 1288, AS/NZS 3000, AS 1670.6, AS 3786, AS 1926, AS/NZS 3500) published by Standards Australia. These are cited by number only; their text is copyrighted and licensed separately. The DefectIndex contains no reproduction of AS clause text. Purchase the authoritative standards through Standards Australia or access via an eligible library subscription.

If you believe any entry crosses from citation into derivation, email hello@surebuildinginspections.au and we will review and correct immediately.