Proof-of-method run of the board-frontier theory against the public
lumins/settlers-of-catan-gamesdataset (50 four-player games on Kaggle, mirrored on GitHub). Three rounds of analysis. Round 1 (board archetypes): aggregate winner-vs-loser bias toward the city engine confirmed; conditional refinement only partially supported. Round 2 (scorecard correlations): the trade-efficiency paradox replicates cleanly — tradeNet vs. VP r = −0.31 at N=200, matching Roman’s 47k finding; port leverage shows bad-ports-hurt-good-ports-don’t-help (12.5% vs. 25% baseline). Round 3 (production-luck decomposition): the metric I called “production luck” turns out to be ~80% expansion success and only ~20% RNG. Same board, same dice — players diverge by 2-3× in actual production based on how they expand. Catan is “expansion success on top of position,” not “dice variance on top of position.” Confirms Roman’s “play matters more than the start” qualitatively and mechanistically.
Links: Catan — 47,000 Games of Empirical Findings, Gaming, Bilateral Trade Valuation, Monopoly Theory, Frontier Trade Theory, Subgraph Investment Optimization
Data: lumins/settlers-of-catan-games on Kaggle — 50 four-player games (200 player-rows), starting placements (settlement positions with adjacent tile numbers and resources), dice-roll histograms, port adjacencies, and resource-flow stats. Mirrored at DevinRS/Settlers_of_Catan_Analysis. Local copy: raw/datasets/catan-50games/catanstats.csv.
Analysis scripts (saved tools, the “code over tokens” rule applied):
raw/datasets/catan-50games/analyze.js — board-archetype classifier, prediction-1 testraw/datasets/catan-50games/scorecard.js — per-game scorecard + cross-game correlations, port-leverage analysis (round 2)raw/datasets/catan-50games/decompose-luck.js — RNG-vs-expansion decomposition of productionLuck (round 3)The four predictions from the board-frontier theory are:
What this dataset has: starting placements with full pip-and-resource info, port adjacencies, dice histograms, final VPs, snake position. What it lacks: per-turn moves, trades, robber placements, opponent perception.
For each player, computed pip-weighted exposure per resource (pip-weight for each of the 6 hex adjacencies = 1+2+3+4+5 for number tokens 2/12, 3/11, 4/10, 5/9, 6/8; desert = 0). City engine = pip(wheat + ore + sheep); road engine = pip(wood + brick). Per game, aggregated all 4 players’ exposures into a board-level resource distribution proxy. Classified each board:
Note: this is a proxy — we never see the full 19-tile board, only the 24 hex-adjacencies the placements collectively touched. The aggregate skews toward what was picked, not what was available, which is endogenous. A better dataset would record the full hex layout. The proxy still distinguishes city-rich from road-rich boards usefully because pip-weighted exposure is roughly proportional to the player-perceived value of each resource on that board.
| Metric | Winners | Losers | Δ |
|---|---|---|---|
| Mean cityShare | 0.63 | 0.61 | +0.016 |
Roman’s 47k showed “winners had slightly higher ore/wheat/sheep exposure.” The 50-game dataset shows the same direction at a small magnitude. With N=50, this is “consistent with” not “proven,” but the proof-of-method works. The features are computable; the direction is right.
| Board archetype | Games | Winner cityShare | Loser cityShare | Δ (winner − loser) |
|---|---|---|---|---|
| city-rich | 22 | 0.69 | 0.68 | +0.014 ✅ direction right |
| balanced | 18 | 0.59 | 0.59 | +0.003 (≈ 0, as predicted) |
| road-rich | 10 | 0.55 | 0.50 | +0.046 ❌ direction wrong |
City-rich boards: winners do lean more city-engine than losers, but the gap is tiny (+0.014). Predicted direction confirmed but the effect is small enough to be sample noise.
Balanced boards: near-zero gap, as theory predicts.
Road-rich boards: winners still have HIGHER city-engine exposure than losers (Δ = +0.046, opposite of theory’s prediction). Three readings:
Reading B is most consistent with what we know: the city engine has a structural advantage (cities double production; dev cards add hidden VPs) that operates across configuration. The board-frontier theory may need refinement: archetype effect exists but is layered on top of a universal city-engine bias, not a clean either-or.
| Position | n | Win rate |
|---|---|---|
| P1 | 50 | 16.0% |
| P2 | 50 | 36.0% |
| P3 | 50 | 28.0% |
| P4 | 50 | 20.0% |
Roman’s 47k: position win rates ≈ 24.9% / ~25% / >25% / ~25%, essentially balanced.
This 50: P2 dominates (36%) and P1 underperforms (16%) by a huge margin. At N=50 per position, the standard error on a win rate ≈ √(0.25 × 0.75 / 50) ≈ 6 percentage points, so the 20-point spread between P1 and P2 is multiple standard errors — unlikely to be pure noise.
The most likely explanation: this dataset’s games may not use standard snake-order placement. The 50-game dataset comes from one group’s home games (per the Medium write-up by the original author and the duddhawork.com analysis), and home games often use simplified placement rules (everyone places in straight turn order, or some other house variant). Without knowing this group’s rules, we can’t draw conclusions about snake-order balance from this dataset.
This is a methodological finding about the dataset rather than a finding about Catan. Worth recording: Roman’s “turn order is balanced” claim requires the snake-order placement rule; in home games with non-snake rules, position 1 may be substantially disadvantaged.
| Archetype | P1 | P2 | P3 | P4 | P1 − P4 range |
|---|---|---|---|---|---|
| city-rich (n=22) | 22.7% | 31.8% | 22.7% | 22.7% | 0.000 |
| balanced (n=18) | 11.1% | 33.3% | 38.9% | 16.7% | −0.056 |
| road-rich (n=10) | 10.0% | 50.0% | 20.0% | 20.0% | −0.100 |
Theory predicted: variance widens on archetype-skewed boards. The data shows variance does widen as boards move from city-rich (flat) to road-rich (P1 underperforms by 10 points). But:
Verdict: not interpretable at this scale. Right dataset is needed.
| Classifier | Within-game accuracy |
|---|---|
| Random pick among 4 players | 25.0% |
| Placement-only (max totalPip) | 48.0% |
| City-engine-only (max cityEnginePip) | 42.0% |
| Archetype-aware (city/road by board) | 42.0% |
The archetype-aware classifier did not beat the placement-only classifier — actually performed slightly worse. At face value this looks like a refutation of Prediction 3.
But the comparison to Roman’s 27% is the wrong comparison, and this finding clarifies why. Roman’s classifier was across-game outcome prediction — given the opening features of one player in one game, predict whether that game was a win. My classifier is within-game ranking — given four players in one game, pick the one most likely to win. The within-game problem is dramatically easier because all four players see the same board and the same dice; the relative pip exposure between them is highly predictive of the relative outcome.
A fair test of Prediction 3 would require running Roman’s analysis design on this dataset: train on a feature set excluding board archetype, then add board archetype as a feature, and see whether held-out across-game accuracy lifts. With N=50 across-game prediction is statistically underpowered to detect a 3-percentage-point lift (which is what Roman would expect to see at 27% → 30%), so this test is unreliable on this data regardless.
The intra-game classifier finding does tell us something useful: between players on the same board, total pip exposure dominates archetype alignment. That’s consistent with reading B above — the universal city-engine bias plus raw production rate together overwhelm the conditional archetype signal at this N.
Built scorecard.js to do deeper per-game analysis: pip exposure per resource, expected vs. actual production, trade net, robber events (in/out), and port-leverage. Aggregated cross-game correlations across all 200 player-rows. The most important finding falls out of the trade-net column.
| Metric | r vs. VP | r vs. win |
|---|---|---|
| Production | +0.655 | +0.392 |
| Production luck (actual − expected) | +0.649 | +0.445 |
| Production ratio | +0.578 | +0.407 |
| pipTotal (starting position) | +0.295 | +0.219 |
| Robber net (used − hit) | +0.105 | +0.185 |
| City pip-share | +0.125 | +0.104 |
| Trade net (gain − loss) | −0.313 | −0.173 |
The trade-efficiency paradox is replicated at N=50. Players whose trade-net was positive (received more than they gave) won less often and had fewer VPs. The correlation is significant at this N (p < 0.001 for the VP coefficient). Roman’s 8% / 25% / overpay-for-position finding at N=47k shows up here as r = −0.31 at N=200. Across two completely independent datasets, the direction is the same: trade winners are not game winners.
The strongest single predictor of winning is production above starting-pip expectation (r = 0.445). Originally framed as “production luck,” but the decomposition below shows the bulk of this metric is expansion success (building more settlements + upgrading to cities mid-game), not dice variance. Players who expanded well from their opening had high productionLuck and won. Starting pips alone correlate only r = 0.22 with winning; what you DO with the starting pips correlates twice as strongly.
After the round-2 results, Chris flagged a real issue: the dataset has no record of mid-game settlements/city upgrades. The “production luck” metric (actual − expected from starting pips) attributes everything that exceeded starting-pip expectation to “luck,” but a player who expanded from 2 settlements to 5 settlements + 4 cities would 3-4× their pip exposure during the game — that’s strategy, not RNG.
Ran a decomposition (decompose-luck.js) to isolate the two components.
Test 1: does board-level RNG luck lift the team’s collective production luck? If yes, productionLuck is RNG-driven. If no, it’s player-level (expansion).
Both are weak. A truly RNG-driven metric would show r near 1.0 because lucky-roll boards would lift every player. The signal is mostly elsewhere.
Test 2: within-game variance. Same board, same dice, four players — how much do they diverge?
The 81% / 18% split is the smoking gun. If RNG drove productionLuck, players on the same board would over- or under-perform together (low within-game variance). Instead, they diverge dramatically. Same board ≠ same outcome — by a long way. That’s expansion success.
Test 3: residual after removing board effect. Per-player “personal luck” = productionLuck − game’s mean productionLuck.
If the win signal lived in board-level RNG, removing the board mean would tank the correlation. It barely moves (0.649 → 0.592). Almost all the win-predictive power is in which player on each board expanded better, not which board got lucky.
Test 4: scale of the production multiplier.
The typical player produces 46% more than starting pips predict. The high end (2.60×) is a player who built a lot of extra settlements and upgraded several to cities. The low end (0.66×) is a player who got dogpiled / robbered into stagnation. This is the structural variance the within-game variance test surfaced, viewed per-player.
The narrative “Catan is dice variance on top of position” was wrong. The correct narrative from this data:
Catan is expansion success on top of position. Starting placement matters (r = 0.22 with winning) but is mostly a foundation. The far larger contribution to winning is what you do with the starting position during the game — how aggressively you expand into adjacent tiles, how reliably you upgrade settlements to cities, how well you maintain access to building resources. The “play matters more than the start” frame Roman’s 27% classifier suggested is now mechanistically grounded: at least 80% of the production-above-expectation lives in the player-level expansion-success component, not in RNG.
This aligns with Chris’s intuition that Catan is play-dominant, not dice-dominant. It also sharpens what “the play” actually means: roads-to-new-vertices, settlements-on-new-tiles, city-upgrades-on-existing-tiles. The metric of skill we’d want from per-turn data is expansion rate and city-conversion rate — exactly what the board-frontier theory was reaching for. The 50-game dataset can’t see it directly, but the decomposition above shows the SHAPE of it: most of the win-correlation lives in the player’s per-game divergence from their tablemates, not in the board’s collective luck.
| Port type | Count | Win rate | Mean VP |
|---|---|---|---|
| Leveraged (matching pip ≥ 5) | 16 | 25.0% | 6.8 |
| Unleveraged (matching pip < 5) | 8 | 12.5% | 5.9 |
| Baseline (random player) | — | 25.0% | 6.5 |
A leveraged 2:1 port (with high matching production) yields baseline win rate. An unleveraged 2:1 port yields half baseline. Reading: ports don’t help above baseline, but bad ports hurt. A 2:1 port replaces what would otherwise be a third tile-adjacency. If the matching resource isn’t strong, you’ve sacrificed a productive vertex for a useless trade option. The port is a don’t-waste-it play, not a win-condition play — Roman’s 47k analysis didn’t surface this because she didn’t decompose ports by matching production strength.
This is a sharper version of the capability-value-over-face-value lens: the face value of a 2:1 port is “trade efficiency”; the capability value is “trade efficiency conditional on producing enough of the resource to use it.” Holders with no matching production are paying for an unused capability. This is the Catan specimen of Capability Without Leverage (failure mode B: leverage gated on matching production).
The scorecard format (one game = one card) is the right reading unit for spotting patterns the aggregate stats blur. Example findings from the top-3 most-extreme-board games:
The scorecard scaffolding is what’s actually valuable here — when we get access to a turn-by-turn dataset (Roman’s 47k or a successor), the same script extends naturally with per-turn columns. This is the proof-of-method for what an empirical Catan analyzer should look like.
The dataset’s game-aggregate-only shape means several questions stayed out of reach:
At N=50, several theoretically-interesting things are unmeasurable:
These are the questions where the Monopoly project is the better proving ground — full simulator access, controllable conditions, unlimited N. The Catan board-frontier theory inherits the validated form from the Monopoly time-axis form once that’s mature.
Three directions, ordered by ROI: