Mind · Reasoning
LudicFallacy
Mistaking an artificial, rule-bound model of a situation for the real situation, and its yardstick for the real goal
Definition
Mechanism
A situation is studied through a simplified, well-defined version of it: a game of chance with known odds, a lab growth medium with one nutrient, a benchmark with one loss function, a contest with standard targets. Conclusions drawn inside that artificial setting are then applied to the real world as if the setting's rules, distributions and success measure were the real ones. Because the artificial version has been built for comparability, it rewards whatever ranks well on its single yardstick, which can be the opposite of what succeeds in the wild (yeast mutations that break unneeded pathways look beneficial in the lab and are harmful in nature).
Invariants
- The artificial setting fixes the rules, distributions and measure of success in advance
- The real situation has more targets, more varied conditions or unknown distributions than the model
- Success in the model is not evidence of success in reality without checking the difference
Risks
Failure modes
Optimising for a benchmark or lab fitness measure that does not match real viability
Treating known-odds probability as a description of open-ended risk
Relationships
Root pattern: builds on nothing in this library.
Used by
References
A problem in which meta-accuracy trades off against meta-precision, so that solving it requires changing shape
Reference