RESEARCH GENERATIVE-DESIGN BENCHMARK - NOT CONSTRUCTION DOCUMENTATION OR CODE CERTIFICATION.

Seventy-two lots, seven methods, one checker.

Every method gets the same seeded middle-housing briefs, the same 17 benchmark constraints and the same six proxy objectives, under a matched compute budget. What they find is compared as a trade-off surface, never as a single score.

Plan drawing of a adu pair on a 16.5 by 19.9 metre lot, 2 storeys, 2 dwellings, 13 rooms. Dwelling U0 (2BR) has 6 rooms totalling 101 square metres. Dwelling U1 (1BR) has 5 rooms totalling 101 square metres. Building footprint 116 square metres; open space 218 square metres. This design satisfies every benchmark hard constraint. Objective scores, higher is better: density 0.97, daylight 0.65, privacy 0.93, circulation 0.56, carbon 0.80, cost 0.67. — **B0000** · knee of the pooled frontRandom sketch + repair✓ all 17 constraints met

The brief with the most methods finding a feasible design (seven). Inspect this plan · see the whole brief.

62/72

briefs where at least one method found a feasible design

57%

highest solve rate: Heuristic-seeded NSGA-II

0.084

highest mean hypervolume: Heuristic

1,368

runs: 72 briefs × 3 seeds × 7 methods, 1,500 evaluations each

Pre-registered hypotheses

H0Optimisation beats random search SupportedNSGA-II adds 0.061 hypervolume over random search and simulated annealing 0.040, paired over briefs.
H1Constraint programming is the most feasible Not supportedCP-SAT solves 45% of runs; Heuristic-seeded NSGA-II solves 57%. CP-SAT timed out on 97 of 216 runs.
H2Evolutionary search finds broader fronts SupportedNSGA-II beats simulated annealing on hypervolume (+0.022) and spread (+0.017).
H3LLM proposals: better adjacency, worse geometry Not runNot run: no LLM credentials were configured for this release.
H4Repair recovers LLM feasibility at a cost Not runNot run: no LLM credentials were configured for this release.
H5No method dominates SupportedThe best method differs across the six objectives and runtime (4 different winners).

Statistics, intervals and tests

No method leads everywhere

Solve rate by typology: the share of runs that found at least one design meeting every constraint. The method that is best on duplexes is not the method that is best on courtyards.

Mean over briefs of the per-brief solve rate across seeds.
method	adu pair	courtyard	duplex	fourplex	rowhouse	triplex
Heuristic	0.33	0.67	1.00	0.50	0.42	0.25
NSGA-II	0.78	0.19	0.97	0.03	0.69	0.53
Random sketch + repair	0.69	0.17	0.89	0.06	0.50	0.36
CP-SAT	0.92	0.00	1.00	0.17	0.06	0.58
Heuristic-seeded NSGA-II	0.69	0.75	0.89	0.22	0.33	0.56
Simulated annealing	0.72	0.11	1.00	0.00	0.28	0.42
Random search	0.33	0.00	0.72	0.00	0.00	0.00

Read the benchmark

Briefs72 seeded lots across six typologies and three difficulty bands
Pareto explorerevery feasible design, any two objectives, nondominated sets ringed
Algorithmsmatched-compute comparison with confidence intervals and convergence
Repairhow far deterministic repair has to move a proposal to make it feasible
Failureswhich of the 17 constraints stop designs, and how often
Carbon proxywhat the embodied-carbon number counts, and what it does not
Daylight proxya geometric availability score, not a simulation
Generation labrun the deterministic planner in your browser (demo, not results)
Methodsrepresentation, constraints, objectives, protocol, amendments