standard tier · 72 briefs · 1,368 runs · results db2e24e

CityFormGen

Generative and optimisation methods for middle housing, compared under matched compute

RESEARCH GENERATIVE-DESIGN BENCHMARK - NOT CONSTRUCTION DOCUMENTATION OR CODE CERTIFICATION.

How far does repair have to move a proposal?

Deterministic repair takes any set of rectangles (overlapping, outside the lot, missing rooms) and projects it onto the benchmark's slicing-tree representation, then refines it. Light repair tunes only proportions; full repair may also rewire the plan. Both get the same 300-evaluation budget per proposal.

The LLM condition has not run for this release. No model credentials were configured, so H3 and H4 are reported as not run. The pipeline, prompt and frozen-record format are built and tested; the two sources below — random sketches and heuristic partis — are the controls the LLM would be compared against.

Ablation

SourceRepairFeasibleRoom displacement, mIoU with proposalAdjacency edit distanceTopological edits
Heuristic partifull repair34.7%
23.6%–45.8%		1.77
1.49–2.08		0.46
0.41–0.51		0.13
0.10–0.16		0.0
0.0–0.0
Heuristic partilight repair34.7%
25.0%–45.8%		1.28
1.05–1.56		0.51
0.47–0.55		0.03
0.02–0.04		0.0
0.0–0.0
Heuristic partino repair4.2%
0.0%–9.7%		–
–––		–
–––		–
–––		0.0
0.0–0.0
Random sketchfull repair10.2%
5.6%–15.3%		6.36
5.90–6.83		0.07
0.06–0.08		0.70
0.68–0.73		0.0
0.0–0.0
Random sketchlight repair0.5%
0.0%–1.4%		4.74
4.23–5.30		0.12
0.10–0.13		0.64
0.60–0.67		0.0
0.0–0.0
Random sketchno repair0.0%
0.0%–0.0%		–
–––		–
–––		–
–––		0.0
0.0–0.0

Mean over briefs (seeds averaged), 95% bootstrap interval beneath. Adjacency edit distance is the Jaccard distance between the door-capable neighbour sets before and after repair; for a proposal, rooms within 0.75 m count as neighbours.

Heuristic parti: light repair − no repair
30.6% (20.8%–41.7%)
Heuristic parti: full repair − no repair
30.6% (20.8%–41.7%)
Random sketch: light repair − no repair
0.5% (0.0%–1.4%)
Random sketch: full repair − no repair
10.2% (5.6%–15.3%)

Before and after

Regenerated deterministically from the ablation's proposals. Left: the proposal as submitted. Right: after full repair.

Random sketch, brief B0000: repaired to feasibility (stage: topology search)

adu_pair plan · B0000-sketch-0Plan drawing of a adu pair on a 16.5 by 19.9 metre lot, 2 storeys, 2 dwellings, 13 rooms. Dwelling U0 (2BR) has 6 rooms totalling 58 square metres. Dwelling U1 (1BR) has 5 rooms totalling 48 square metres. Building footprint 62 square metres; open space 270 square metres.Common18.3 m²Ent3.3 m²Liv21.7 m²Kit8.0 m²Bath3.8 m²Bed 113.0 m²Bed8.5 m²GROUND FLOOR5 mCommon18.3 m²EntLiv17.9 m²Kit7.6 m²Bath5.0 m²Bed 114.8 m²FIRST FLOOR5 m
proposalpenalty 67.07
adu_pair plan · B0000-sketch-0~repPlan drawing of a adu pair on a 16.5 by 19.9 metre lot, 2 storeys, 2 dwellings, 13 rooms. Dwelling U0 (2BR) has 6 rooms totalling 99 square metres. Dwelling U1 (1BR) has 5 rooms totalling 99 square metres. Building footprint 110 square metres; open space 224 square metres. This design satisfies every benchmark hard constraint. Objective scores, higher is better: density 0.89, daylight 0.64, privacy 0.87, circulation 0.62, carbon 0.79, cost 0.64.Liv32.7 m²Kit10.7 m²Bath5.6 m²Bed 129.6 m²Bed14.2 m²GROUND FLOOR5 mEnt9.5 m²Liv11.1 m²Kit15.5 m²Bath43.5 m²Bed 119.6 m²FIRST FLOOR5 m
after full repairmoved 3.28 m · IoU 0.14 · adjacency edit 0.48✓ feasible

Random sketch, brief B0001: repair did not reach feasibility

courtyard plan · B0001-sketch-0Plan drawing of a courtyard on a 19.2 by 24.8 metre lot, 2 storeys, 4 dwellings, 24 rooms. Dwelling U0 (1BR) has 5 rooms totalling 48 square metres. Dwelling U1 (1BR) has 5 rooms totalling 53 square metres. Dwelling U2 (2BR) has 6 rooms totalling 57 square metres. Dwelling U3 (2BR) has 6 rooms totalling 61 square metres. Building footprint 84 square metres; open space 394 square metres.Common24.2 m²Ent3.3 m²Liv17.8 m²Kit7.4 m²Bath4.5 m²Bed 114.6 m²Ent3.6 m²Liv21.2 m²Kit8.2 m²Bath4.5 m²Bed 115.6 m²GROUND FLOOR5 mCommon24.2 m²Ent3.5 m²Liv17.1 m²Kit7.2 m²Bath5.1 m²Bed 114.8 m²Bed9.3 m²Ent3.8 m²Liv19.9 m²Kit8.5 m²Bath4.8 m²Bed 113.9 m²Bed10.1 m²FIRST FLOOR5 m
proposalpenalty 181.91
courtyard plan · B0001-sketch-0~repPlan drawing of a courtyard on a 19.2 by 24.8 metre lot, 2 storeys, 4 dwellings, 24 rooms. Dwelling U0 (1BR) has 5 rooms totalling 86 square metres. Dwelling U1 (1BR) has 5 rooms totalling 84 square metres. Dwelling U2 (2BR) has 6 rooms totalling 84 square metres. Dwelling U3 (2BR) has 6 rooms totalling 86 square metres. Building footprint 218 square metres; open space 261 square metres. This design is infeasible; failing categories: bad proportion, insufficient area, no street access, open space failure. Objective scores, higher is better: density 0.72, daylight 0.41, privacy 0.76, circulation 0.54, carbon 0.79, cost 0.55.Common47.4 m²Ent7.2 m²Liv14.3 m²Kit23.1 m²Bath17.9 m²Bed 123.6 m²EntLiv11.9 m²Kit37.0 m²Bath9.0 m²Bed 121.5 m²GROUND FLOOR5 mCommon47.4 m²Ent17.4 m²Liv19.7 m²Kit12.6 m²Bath8.6 m²Bed 111.7 m²Bed14.2 m²Ent9.0 m²Liv10.9 m²Kit29.4 m²Bath5.9 m²Bed 114.7 m²Bed16.2 m²FIRST FLOOR5 m
after full repairmoved 9.14 m · IoU 0.04 · adjacency edit 0.78✕ infeasible

Heuristic parti, brief B0001: repaired to feasibility (stage: ratio refine)

courtyard plan · B0001-hprop-0Plan drawing of a courtyard on a 19.2 by 24.8 metre lot, 2 storeys, 4 dwellings, 24 rooms. Dwelling U0 (1BR) has 5 rooms totalling 96 square metres. Dwelling U1 (1BR) has 5 rooms totalling 96 square metres. Dwelling U2 (2BR) has 6 rooms totalling 96 square metres. Dwelling U3 (2BR) has 6 rooms totalling 96 square metres. Building footprint 216 square metres; open space 262 square metres.Common24.2 m²Liv37.9 m²Kit15.8 m²Bath8.8 m²Bed 127.3 m²Liv37.9 m²Kit15.8 m²Bath8.8 m²Bed 127.3 m²GROUND FLOOR5 mCommon24.2 m²Ent5.7 m²Liv32.3 m²Kit12.9 m²Bath7.4 m²Bed 121.7 m²Bed16.2 m²Ent5.7 m²Liv32.3 m²Kit13.1 m²Bath7.2 m²Bed 121.7 m²Bed16.2 m²FIRST FLOOR5 m
proposalpenalty 4.80
courtyard plan · B0001-hprop-0~repPlan drawing of a courtyard on a 19.2 by 24.8 metre lot, 2 storeys, 4 dwellings, 24 rooms. Dwelling U0 (1BR) has 5 rooms totalling 82 square metres. Dwelling U1 (1BR) has 5 rooms totalling 82 square metres. Dwelling U2 (2BR) has 6 rooms totalling 82 square metres. Dwelling U3 (2BR) has 6 rooms totalling 82 square metres. Building footprint 183 square metres; open space 292 square metres. This design satisfies every benchmark hard constraint. Objective scores, higher is better: density 0.52, daylight 0.62, privacy 0.76, circulation 0.66, carbon 0.73, cost 0.53.CommonLiv30.3 m²Kit7.0 m²Bath6.3 m²Bed 135.4 m²Ent4.2 m²Liv30.3 m²Kit6.6 m²Bath5.9 m²Bed 135.4 m²GROUND FLOOR5 mCommonLiv27.7 m²Kit11.1 m²Bath6.4 m²Bed 114.7 m²Bed17.7 m²Liv27.7 m²Kit11.3 m²Bath6.2 m²Bed 114.7 m²Bed17.7 m²FIRST FLOOR5 m
after full repairmoved 1.29 m · IoU 0.52 · adjacency edit 0.00✓ feasible

Heuristic parti, brief B0000: repair did not reach feasibility

adu_pair plan · B0000-hprop-0Plan drawing of a adu pair on a 16.5 by 19.9 metre lot, 2 storeys, 2 dwellings, 13 rooms. Dwelling U0 (2BR) has 6 rooms totalling 100 square metres. Dwelling U1 (1BR) has 5 rooms totalling 100 square metres. Building footprint 118 square metres; open space 211 square metres.Common18.3 m²Ent5.9 m²Liv33.4 m²Kit13.5 m²Bath7.6 m²Bed 122.8 m²Bed16.9 m²GROUND FLOOR5 mCommon18.3 m²Ent6.5 m²Liv39.7 m²Kit16.4 m²Bath9.2 m²Bed 128.4 m²FIRST FLOOR5 m
proposalpenalty 8.00
adu_pair plan · B0000-hprop-0~repPlan drawing of a adu pair on a 16.5 by 19.9 metre lot, 2 storeys, 2 dwellings, 13 rooms. Dwelling U0 (2BR) has 6 rooms totalling 63 square metres. Dwelling U1 (1BR) has 5 rooms totalling 63 square metres. Building footprint 75 square metres; open space 254 square metres. This design is infeasible; failing categories: bad proportion. Objective scores, higher is better: density 0.43, daylight 0.58, privacy 0.77, circulation 0.60, carbon 0.71, cost 0.39.Common11.5 m²Liv26.2 m²Kit6.2 m²Bath6.6 m²Bed 112.5 m²Bed9.4 m²GROUND FLOOR5 mCommon11.5 m²Ent3.4 m²Liv23.9 m²Kit8.0 m²Bath3.3 m²Bed 124.6 m²FIRST FLOOR5 m
after full repairmoved 1.03 m · IoU 0.47 · adjacency edit 0.00✕ infeasible