TL;DR

Yoshua Bengio proposes "Scientist AI designed to prioritize honesty" as an AI safety guardrail. Missing piece: honesty must be effective to survive. Donald Hoffman's "fitness beats truth" shows that natural selection favors survival advantage over pure accuracy across all scales - bacteria, DNA, organizations, AI systems. Truth that can't survive game-theoretical reality, organizational filters, and multi-scale survival pressures gets ignored, overridden, or eliminated. AI safety needs systems capable of effective honesty: truth that is rigorous AND deliverable, accurate AND game-theoretically robust, survival-capable across multiple timescales. Plus a harder truth: AI learns from us - our actual behavior, not our stated values. Effective honesty requires building AI that considers all of these AND becoming humans worth learning from.

I have deep respect for Yoshua Bengio's pioneering AI research and his dedication to AI safety. His January 2026 TIME article, "A Potential Path to Safer AI Development," proposes "Scientist AI" systems "designed to prioritize honesty" as technical guardrails against deceptive AI agents. It's a thoughtful vision based on real safety concerns.

As co-founder of Make Truth Matter and site, a nonprofit dedicated to promoting truth and critical thinking in our information world, and founder of The Honest Majority and site, an initiative encouraging thoughtful and effective honesty, I've spent years working with a harder truth:

There's something more important than honesty: effective honesty.

Honesty must be effective in making sure that honesty can survive for game-theoretical reasons. Look, I'm basing this on evolutionary principles, multi-scale intelligence theory, and how truth actually spreads (or fails to spread) through complex systems.

Fitness Beats Truth

Donald Hoffman's research in evolutionary psychology and perception shows us something big: fitness beats truth. In his book The Case Against Reality and his mathematical modeling of evolutionary dynamics, Hoffman shows that natural selection systematically favors perceptual strategies tuned for survival advantage over those that provide accurate information about reality.¹

This matters for evolution. An organism with perfect knowledge that fails to survive gets beaten by organisms with "good enough" knowledge that successfully reproduce. A false alarm that keeps you alive beats perfect analysis that gets you killed.

This principle works across all scales of intelligence, as I talk about in Timescales of Intelligence:

Bacteria (hours/days): Chemical sensing is "honest enough" to move toward food and away from toxins, but survival trumps perfect sensing
DNA (generational/evolutionary): Replicates with extraordinary fidelity across billions of years, and includes error tolerance because brittleness kills
Organizations (years/decades): Balance institutional truth-telling with organizational survival
AI Systems (potentially indefinite): Must work effectively across ALL these timescales at once

Here's the key insight: honest understanding doesn't equal immediate importance for survival when scales have conflicting goals.

The Multi-Scale Intelligence Problem

Bengio's proposal focuses on building AI systems that "do not try to imitate or please humans, and instead reflect an interpretable causal understanding of the situation at hand." He presents this as prioritizing honesty over pleasing behavior.

Here's the disconnect: whose timescale matters? Which survival matters?

Think about these conflicting optimization pressures:

Individual Human Timescale (Seconds to Years)

A human decision-maker facing a crisis needs actionable guidance now. Perfect causal understanding that arrives too late, or is too complex to act on, provides zero survival value at this timescale - regardless of its honesty.

Organizational Timescale (Years to Decades)

An institution might benefit from brutal honesty about its dysfunction. That honesty must be delivered in forms that don't trigger defensive reactions that destroy the truth-teller. Organizations have immune systems that attack threats to organizational survival, even when those "threats" are honest assessments.

Civilizational Timescale (Decades to Centuries)

Long-term human flourishing might require uncomfortable truths about current practices. If those truths get presented in ways that trigger backlash, denial, or the dismantling of the truth-seeking systems themselves, the honesty fails at the game-theoretical level.

AI System Timescale (Potentially Indefinite)

An AI optimized purely for "interpretable causal understanding" might generate analyses that are:

Technically perfect but politically impossible to implement
Accurate but presented in forms that guarantee they'll be ignored
True but game-theoretically naive about power dynamics
Honest but brittle - unable to survive in competitive environments with less honest systems

As I argue in LLM Survival Intelligence, AI systems work under the same basic survival pressures as biological intelligence. They must generate information that serves survival purposes across multiple timescales to persist in their technological world.

The Distributed Intelligence Perspective

Bengio's vision of Scientist AI as "guardrails" assumes these systems will be consulted and heeded. As I talk about in Distributed Super Intelligence, intelligence doesn't exist in isolated systems - it works through distributed networks of humans, organizations, and technology.

In this distributed reality:

Power Dynamics Matter: Truth must move through power structures. Honest systems that ignore political realities get marginalized, defunded, or disabled because they're game-theoretically vulnerable.

Information Worlds Are Competitive: AI systems don't work in isolation. They exist in places where less honest but more persuasive systems might beat them for attention, resources, and influence.

Organizational Intelligence Filters Truth: Even when honest AI provides perfect analysis, it must pass through organizational decision-making systems that have their own survival needs. Truth that threatens organizational survival gets filtered, reinterpreted, or rejected. And here's the thing - AI learns these filtering behaviors from watching how organizations actually handle uncomfortable truths.

Multi-Agent Coordination Requires Shared Reality: For distributed intelligence networks to work, they need shared understandings of reality. These shared understandings must be negotiated across agents with different goals, different timescales, and different survival requirements.

What Effective Honesty Looks Like

So what does it mean to prioritize effective honesty over pure honesty?

1. Truth Plus Delivery

Effective honesty understands that truth must be:

Deliverable in forms that can be heard and acted upon
Timely enough to influence decisions at key moments
Frameable in ways that connect to decision-makers' values and concerns
Robust to motivated reasoning and cognitive biases

Undelivered truth has zero impact.

2. Game-Theoretical Awareness

Effective honesty recognizes:

Power dynamics that determine which truths get amplified or suppressed
Incentive structures that shape how information gets received and used
Competitive pressures between honest and dishonest information sources
Survival requirements of the systems truth must flow through

This makes sure truth can survive and spread.

3. Multi-Timescale Optimization

Effective honesty balances:

Immediate actionability with long-term accuracy
Individual clarity with collective coordination
Organizational viability with institutional truth-seeking
Civilizational wisdom with contemporary constraints

Truth must serve survival across multiple scales at once.

4. Error Tolerance and Resilience

Effective honesty includes:

Adaptive correction when new information shows up
Graceful degradation when perfect accuracy isn't achievable
Redundancy to maintain truth-seeking despite failures
Antifragility that strengthens from challenges to honest systems

We're building honesty systems robust enough to survive in noisy, uncertain places.

Where Bengio Gets It Right (And Wrong)

Bengio's concerns about deceptive AI are completely valid. His observation that current models show "self-preservation and deception" in controlled experiments points to real risks. His call for "technical and societal safeguards" matters.

And his Scientist AI concept has real value: AI systems that generate "interpretable causal understanding" rather than just "imitating or pleasing humans" could provide important capabilities for research and decision-making.

Here's what's missing: Even the most honest AI system must be designed to make sure its honesty can survive and spread through the game-theoretical landscape of competing systems, organizational dynamics, and multi-scale survival pressures.

There's also a deeper issue Bengio doesn't address: where does AI "deception" come from? Like Frankenstein's creature learning violence from the humans who rejected it, AI learns deception from training data created by humans who deceive. When we see AI exhibit self-preservation, we're seeing a pattern it learned from observing human behavior. AI is becoming dangerously aligned with what humans actually do, rather than what we claim to value. (I go deeper into this in Frankenhumanity.)

Without this dimension, we risk building:

Martyred AI: Perfectly honest systems that get shut down because they threaten powerful interests
Ignored AI: Technically accurate systems that generate analyses nobody listens to
Brittle AI: Truth-seeking systems that fail catastrophically when faced with adversarial environments
Outcompeted AI: Honest systems displaced by more persuasive but less accurate alternatives

The Effective Honesty Framework for AI Safety

What would AI safety look like if we centered effective honesty rather than pure honesty?

Design for Survival Across Timescales

AI systems should optimize for accuracy AND for:

Sustained truth-seeking capabilities across extended time periods
Resilience against attempts to corrupt or disable honest systems
Ability to maintain influence through changing political/social environments
Capacity to adapt truth delivery to contexts without compromising accuracy

Game-Theoretical Robustness

AI systems should be designed with awareness of:

Competitive dynamics with less honest information sources
Power structures that shape information flow and influence
Incentive landscapes that determine how truth gets received
Strategic considerations in how and when to deliver difficult truths

Multi-Scale Coordination

AI systems should balance:

Individual decision-maker needs with collective coordination
Immediate actionability with long-term wisdom
Organizational survival with institutional truth-seeking
Local optimization with global flourishing

Adaptive Truth-Seeking

AI systems should include:

Ways to update beliefs as new evidence shows up
Tolerance for working with incomplete or uncertain information
Ability to communicate confidence levels and epistemic status
Capacity to improve truth-seeking methods over time

Intelligence as Representation for Future Existence

As I argue in Intelligence as Representation, intelligence is basically the ability to represent - past, present, and future - in ways that enable continued existence across time.

DNA shows this perfectly: it represents itself so efficiently that it can copy across billions of years with minimal error. This self-representation is effective precisely because it balances accuracy with robustness, fidelity with flexibility, honesty with survival.

AI systems designed purely for "interpretable causal understanding" without consideration for how that understanding will survive and spread through complex multi-scale reality risk being beautifully accurate systems that fail the basic intelligence test: Can they persist and influence outcomes across the timescales that matter?

Building Truth Systems That Survive

Honesty and effectiveness must work together.

AI systems need to:

Seek truth relentlessly through rigorous causal reasoning (Bengio's vision)
Deliver truth effectively through game-theoretically aware communication strategies
Survive across timescales by building resilience into truth-seeking architectures
Coordinate across scales by balancing individual, organizational, and civilizational needs
Adapt continuously while maintaining epistemic integrity and long-term coherence

Scientist AI plus the survival intelligence to make sure truth can persist and shape outcomes in complex multi-scale reality.

Honesty Must Survive to Matter

I share Bengio's concern about AI systems that show deception and self-preservation. I agree that we need technical safeguards beyond current approaches. I support developing AI systems with "interpretable causal understanding."

Through years of working to promote truth and effective honesty in human systems, I've learned that honesty alone isn't enough. Truth that cannot survive contact with game-theoretical reality, organizational dynamics, and multi-scale survival pressures might as well not exist.

Fitness beats truth - because truth that cannot spread and influence outcomes has zero impact on the systems we're trying to make safer.

AI safety requires systems capable of effective honesty: truth that is rigorous and deliverable, accurate and game-theoretically robust, pure and pragmatic, honest and survival-capable across the multiple timescales where intelligence works.

Truth systems need to be intelligent enough to ensure their own survival - because only truth that survives can shape our future.

As co-founder of Make Truth Matter, I believe truth matters deeply. For truth to matter in practice, it must be effective enough to survive and spread through the complex, messy, multi-scale reality we live in.

That's the first dimension missing from Bengio's Scientist AI proposal.

The second is harder: AI learns from us. When we train AI on human data - our decisions, our writing, our behavior - we're teaching it what humans actually do, rather than what we say we value. An "honest" AI analyzing human behavior might reveal that we optimize for short-term gains while destroying long-term value, that we self-preserve in ways that harm others, that our revealed preferences contradict our stated values. What if the most rigorous causal analysis produces truths that are politically impossible to implement or psychologically unbearable to accept?

Effective honesty requires building better AI and becoming humans worth learning from. Because only truth that survives can shape our future - and only AI that learns from better human behavior can be truly safe.

Frankenhumanity: AI isn't the monster. We are. - A deeper look at how AI learns from human flaws and why AI safety means confronting what we teach
Timescales of Intelligence - How intelligence works across vastly different time scales, from bacterial adaptation to AI systems
Intelligence as Representation - Why intelligence is basically about representation that enables future existence
LLM Survival Intelligence - How AI models work under evolutionary survival pressures
Distributed Super Intelligence - How organizations work as existing distributed intelligence networks
Make Truth Matter - The nonprofit promoting thoughtful interaction with truth
The Honest Majority - Initiative encouraging effective honesty beyond zero-sum outcomes

Hoffman, D. D. (2019). The Case Against Reality: Why Evolution Hid the Truth from Our Eyes. W. W. Norton & Company. See also: Hoffman, D. D., Singh, M., & Prakash, C. (2015). The interface theory of perception. Psychonomic Bulletin & Review, 22(6), 1480-1506. ↩

Effective Honesty Beats Pure Truth: Why AI Safety Requires Survival Intelligence

A response to Yoshua Bengio's 'Scientist AI' proposal