Market Efficiency and Human Limits

The market always gives society exactly what they want within the constraints of what is possible — but humans may not be smart enough to live in a free market.

Links: Economics, Value and Profit, US Spending Per Student, K-12 Instruction vs. Administration, The Limits of Utopia, Scope Confusion, Civilizational Cycles, Accounting Identities as Domain-Matching — the meta-pattern (GDP-as-welfare-metric across tariffs and immigration), The GDP-Identity Tariff Fallacy — the symmetric form on tariffs


The Core Thesis

The market always gives society exactly what they want within the constraints of what is possible.

This is not a defense of outcomes. It’s a structural claim about the mechanism. The market is a distributed computation engine that solves supply and demand simultaneously through the price signal. It processes information as fast as reality allows. What economists call “market failures” are mostly just information that hasn’t propagated yet — and any intervention assumes the intervener has better information than the distributed market, which is almost always wrong (Hayek’s knowledge problem).

What the Market Actually Responds To

The market responds to effective demand — want backed by purchasing power. Dollar-weighted demand is not a flaw; it’s the signal. The weighting is the information. Those who generate more value for others accumulate more purchasing power, which gives their preferences more weight in future allocation. This is a feature of the mechanism, not a bug.

The teacher example: We say we want better teachers. We pay for nicer clothes. The market believes the credit card, not the survey. Revealed preference is the only honest measure of what people actually value. The uncomfortable truth is that the market is right to believe the credit card — that’s the revealed truth about what people value at the margin.

But: a person who puts their own money into private school is sending the correct market signal — public education isn’t delivering value proportional to cost. And then they rightly get angry at the ~$20k per student the government spends on their behalf in a system they’ve already rejected with their wallet. They can’t opt out. They can’t redirect. The market signal is being drowned out by compulsory revenue collection. Full enrollment in public schools isn’t demand — it’s a captive audience.

Economists Must Look at Both Sides

A persistent failure in economic analysis is evaluating only supply or demand, never both simultaneously. The market never makes this error — price is the intersection of both sides. But humans analyzing the market constantly isolate one variable and tell a story about it.

The immigration example: Most economists say immigration is good because GDP goes up. This only looks at the supply side. Immigrant workers also put demand on the economy — they buy things, use services, occupy housing. More demand and more supply is not automatically better.

The fix is simple: GDP per capita — the economist’s own definition of standard of living. If an immigrant contributes more than this measure, they are increasing the standard of living (GDP/cap goes up). If they don’t, they are decreasing it. The market, absent coercive distortions, would naturally select for the first type — employers would recruit productive workers because it’s profitable. Immigration policy that substitutes political criteria for economic ones is itself an anti-market force.

The GDP/capita solvency formula

The formal version that makes the supply-demand symmetry explicit:

Immigrant's net welfare contribution = (marginal production) − (marginal consumption)
                                     ≈ (their productive output) − (their per-capita share of resource demand)

The three cases:

Marginal production vs. GDP/capita Effect on aggregate GDP Effect on per-capita GDP Effect on existing residents
Above the line (high-skilled, integrated, working-age, immediate productive contribution) Up Up Better off on average
At the line (matching median resident profile) Up Flat Unchanged on average
Below the line (low-skilled without integration, non-working dependents, language/credential barriers, service-heavy household composition) Up Down Worse off on average

Aggregate GDP rises in all three cases. Per-capita GDP — the actual standard-of-living measure — rises only in the first case, holds in the second, and falls in the third. The mainstream economist case (“immigration is good because GDP rises”) collapses these three by using only the aggregate metric. It treats the second and third cases as equivalent to the first, which they aren’t.

This is the symmetric form of the GDP-identity tariff fallacy: same identity-misreading, applied to the supply-side of immigration instead of to the −M term of the trade identity. Both arguments use aggregate GDP as the welfare metric when per-capita or distributional measures would tell a different story. See accounting-identities-as-domain-matching.md §”GDP-as-welfare-metric” for the meta-pattern; gdp-identity-tariff-fallacy.md for the tariff-side version.

Distributional layer

The GDP/capita test handles the aggregate welfare question. The distributional layer is separate and also matters: immigration that increases per-capita aggregate may still hurt specific deciles (bottom-decile workers facing wage competition; lower-income residents facing housing competition; service-strained communities) while benefiting other deciles (high-end employers, asset-holders benefiting from housing-price appreciation, consumers of immigrant-provided services). The aggregate-vs-distribution distinction the vault has been building (see Lyn Alden trade-deficit analysis) applies here cleanly: a policy can raise aggregate per-capita welfare and concentrate the gains in specific deciles while concentrating the costs on others. Both observations can be true; treating either alone as the complete answer is the framing mistake.

The combined test: does the policy improve per-capita aggregate welfare, AND is the distribution of gains and costs defensible? Either alone isn’t enough. The mainstream “immigration is good for GDP” claim is silent on both questions.

The pattern repeats everywhere:

The market is a simultaneous equation solver. Humans are narrative thinkers. We do one side at a time and often stop before we get to the other.

Where the Market Fails

Two categories, both real:

1. Coercive Forces Outside the Market

Government price controls, subsidies, tariffs, monopoly grants, regulatory capture. These distort the price signal the market needs to function. The market can’t give society what it wants if the information mechanism is being overridden. This is straightforward and widely acknowledged (if not widely acted upon).

2. Human Information Processing Limits

It is possible humans are not smart enough to live in a free market.

The market is a perfect machine that requires an imperfect operator. The theory works — but it requires participants who can:

Humans often can’t. This is the 99-problems / Chesterton’s fence issue. Modern society buries so much complexity that consumers can’t make good choices. They end up relying on outside forces — media, brands, political tribes, experts — to do their thinking. And those outside forces have their own incentives, which are often anti-market.

The failure cascade:

  1. Modern complexity overwhelms human information processing capacity
  2. Humans outsource their thinking to institutions and intermediaries
  3. Those intermediaries have their own incentive structures
  4. Those incentives are often misaligned with market efficiency
  5. The outsourced decisions corrupt the inputs to the market
  6. The market faithfully processes corrupted inputs
  7. Outcomes look like “market failure” but are actually input failure

Even the professional class trained to understand markets — economists — fails this test. They analyze one side of the ledger and call it analysis. If the experts can’t process both supply and demand simultaneously, the average consumer has no chance.

The market doesn’t fail. The participants fail the market. And the gap between what the market requires of us and what we’re capable of — that gap is where every problem lives.

The Government-Guaranteed Revenue Distortion

Whenever government guarantees revenue to an industry — through subsidies, loan backstops, mandatory participation, or tax-funded allocation — the same distortion plays out. The mechanism is general. Education, healthcare, housing, and retirement are all instances of the same law.

The Three Steps

  1. Government guarantees revenue to an industry — subsidy, loan backstop, compulsory participation, per-capita allocation. The money arrives whether or not the recipient delivers value.
  2. The industry optimizes for capturing the guaranteed revenue — not for delivering value to the end consumer. The consumer isn’t the one paying, or can’t opt out, so their preferences stop mattering at the margin.
  3. Costs rise, quality decouples from price — because the feedback loop between “this isn’t worth what I’m paying” and “I’ll stop paying” has been severed. The normal market correction — consumers walking away — is structurally impossible.

That’s the whole law. Every industry that receives guaranteed government revenue follows this path. The specifics differ; the mechanism doesn’t.

Case Study 1: Education

Education is the cleanest demonstration because we can see the same mechanism at two levels — K-12 and higher education — with decades of data.

K-12: The Voucher Half-Step

School vouchers fix one problem and leave a bigger one untouched:

The three steps at K-12:

  1. Guaranteed revenue: Compulsory taxation funds ~$20k per pupil. You pay whether you use it or not.
  2. Industry captures it: Schools optimize for enrollment counts and compliance, not value delivered. Per-pupil allocation sets an artificial price disconnected from outcomes.
  3. Costs rise, quality decouples: Spending up 3-4x in real terms since 1960. Test scores flat. Benefits spending up 81% while salaries up 8%. The money goes where the consumer can’t see it.

Vouchers redirect the money but don’t change the amount or the compulsion. You still end up with costs rising faster than outcomes — just with more competition around the margins.

Higher Education: Government-Backed Student Loans

Federal student loans are the college-level version, and we have 30+ years of data:

  1. Guaranteed revenue: Federally backed loans that can’t be discharged in bankruptcy. Schools get paid regardless of whether the degree delivers value.
  2. Industry captures it: Tuition rises to absorb whatever loans make available. Some schools became, functionally, loan procurement operations — marketing degrees to people who saw the upside (credential, career change) but couldn’t process the downside (debt load, opportunity cost, degree devaluation from oversupply). One-sided analysis by the participants, exactly as predicted.
  3. Costs rise, quality decouples: Degree programs multiply. Enrollment is pushed on everyone regardless of fit. People who should have been in vocational tracks at 16 are $80k in debt at 24 with a degree the market doesn’t value.

The Death of Vocational Tracking

In the 1980s, public schools still guided students toward vocational training as early as high school. This was closer to a market outcome — it recognized that people have different comparative advantages and matched them accordingly.

That system was dismantled by a narrative: “everyone deserves college.” It sounded moral but ignored supply and demand. The trades now face severe labor shortages because we told an entire generation that plumbing is beneath them. Meanwhile a master plumber out-earns most liberal arts graduates and carries no debt.

The pattern: a well-intentioned narrative overrode the market signal, backed by government money that made the bad decision feel free at the point of purchase. By the time the bill came due, the decision was irreversible.

TODO: Add data and references — tuition inflation vs. CPI, vocational enrollment trends 1970s–present, trade salary comparisons, student loan default rates, comparative international models (German apprenticeship system).

Case Study 2: Forced Savings (Social Security)

Social Security is the purest instance of the three steps because there’s not even a pretense of consumer choice.

  1. Guaranteed revenue: Mandatory payroll tax. You cannot opt out, redirect the funds, or choose your provider. The “industry” receiving the guaranteed revenue is the government itself.
  2. The operator captures it: Politicians optimize Social Security for electoral incentives — expanding benefits, resisting reform — not for delivering returns to the contributor. There is no competitor. There is no exit.
  3. Costs rise, returns decouple: The same dollars invested in an index fund would produce dramatically higher returns. But the contributor can’t make that comparison at the point of purchase because there is no purchase — there’s a payroll deduction. The feedback loop isn’t just severed; it was never wired.

Social Security proves the law generalizes beyond industries that receive government money to any system where government intermediates between the earner and the outcome. The mechanism is identical: guaranteed revenue, capture by the operator, and the death of the price signal.

TODO: Add data — Social Security ROI vs. index fund returns by cohort, trust fund depletion projections, international comparisons (Singapore CPF, Chile AFP).

The Circular Dependency

Removing these distortions through democratic means requires an electorate that understands markets well enough to vote for market solutions. But the current education system — the one shaped by the distortions — doesn’t produce that electorate.

This may be the deepest version of the human-limits problem: you can’t get to free markets from here through the democratic process, because the voters have been shaped by the very distortions you’re trying to remove.

The market will likely find solutions anyway — it always does, working around the distortions. But the path and the timeline are open questions.

What to Do About It

The tempting answer is intervention — regulation, mandates, expert panels. But this runs headfirst into Hayek’s knowledge problem: the interveners know less than the distributed market, and they have their own incentive distortions.

The better answer, though it falls flat in practice: be a better consumer. Learn how to negotiate. The market rewards informed participants. The difficulty is that this advice requires exactly the information processing capacity that most people lack.

This is an open problem. The mechanism is sound. The inputs are corrupted. And the corruption is partially structural (complexity) and partially self-inflicted (outsourced thinking).

Tags

economics, free-markets, education, externalities