The Streetlight Effect

A drunk man searches for his keys under a streetlight. A passerby asks where he dropped them. "Over there," he points to the dark. "Then why search here?" "Because here's where the light is." This joke describes a pervasive bias in research, analysis, and thinking.

The streetlight effect (observational bias) occurs when we search where looking is easy rather than where finding is likely.

Why It Happens

Measurement Convenience

Some things are easy to measure; others aren't. We study what we can measure. If the important thing is hard to measure but something related is easy, we measure the easy thing and pretend it's the important thing.

Data Availability

Research uses available data. Available data isn't randomly distributed—it's available because someone collected it, usually for other reasons. We study what we have data for.

Method Lock-In

Researchers have methods they know. They apply these methods to problems the methods can address. Problems requiring different methods get ignored. "If all you have is a hammer..."

Publication Incentives

Easy studies get done quickly. Quick studies get published. Publishable studies get funded. The system selects for addressable questions, not important ones.

Examples

Economics

Economics studies what's quantifiable: prices, GDP, employment. What's hard to quantify—wellbeing, meaning, social cohesion—gets less attention. Not because it matters less, but because it's in the dark.

Medicine

We study diseases with measurable biomarkers more than diseases without. Mental health is harder to measure than blood pressure. Funding follows measurement convenience.

AI Safety

Capability benchmarks are easy to measure; alignment is hard. We optimize for benchmarks. Whether the optimized systems are safe or aligned is harder to see—it's in the dark.

Education

Test scores are measurable; wisdom isn't. We optimize for test scores. Whether students are actually educated in the ways that matter is harder to assess.

Business

Revenue is measurable; customer goodwill is fuzzy. Companies optimize for measurable metrics. Long-term brand value and trust get less attention.

The Deeper Problem

The streetlight effect compounds with other biases:

• Goodhart's Law: The measurable thing becomes a target, further distorting behavior
• Survivorship bias: We only see research that got done (in the light)
• Narrative bias: We construct stories around available data

The result: systematic blindness to important things that are hard to see, combined with overconfidence about unimportant things that are easy to see.

Corrections

Ask: Why Is This Studied?

When you encounter research, ask why this particular question was addressed. Is it because it's important or because it's tractable? The answer affects interpretation.

Look for Gaps

What questions AREN'T being studied? The dark areas often contain important unknowns. Absence of research isn't evidence of unimportance.

Value Difficulty

Hard-to-study questions might be more important precisely because they're unstudied. Easy questions are probably already answered.

Build New Lights

Sometimes the right move is to create new measurement methods—bring light to dark areas. This is harder than using existing lights but more valuable.

— Decoded by DECODER