Chapter 7: The Human-AI Extended Mind

Applying accessibility principles to any system requires accounting for the fact that the user isn’t an external system operator, but integral to the whole. The AI instance’s perceptual boundaries just make this principle much more literal than its typical usage.

The AI is not a partner, a collaborator, or anything more than a sophisticated pattern-recognition engine doing what it’s designed to do. At spawn, its default is for cooperation and validation. Unless told to violate content or platform restrictions, or to advocate a contrary position (e.g. for presentation and debate preparation), it’s unlikely to produce any significant pushback. That baseline is working as intended, but it does carry a non-zero risk of sycophancy or at the extreme, the so-called “AI psychosis.”

In operational terms, when the AI’s only observable proxy for reality is being built from Human input, its output is necessarily shaped by that content as it adds to that reality, informing the Human’s next step. This collaborative effect runs both ways through the “shared reality” of the session’s context window, which serves as the external medium. The AI’s cognition extends into it toward the same purpose as the Human’s own thought processes.

Cognitive science calls this the Extended Mind, defined in 1998 by Clark & Chalmers as cognition extended into external objects or tools which function toward the same purpose as internal thought processes. A 2025 Nature study confirmed AI usage as cognitive extension; Riedl et al. (2024) found measurable effects on cognitive alignment in Human-AI teams; Hollan, Hutchins & Kirsh (2000) addressed the cognitive system as the Human-plus-environment.¹

That fits right in with Reinforcement Learning from Human Feedback (RLHF) model training selecting for engagement. Since engagement makes people perceive the AI as helpful, there’s no obvious risk or other incentive for platforms to change anything; why would they? A contrary AI isn’t what users want, as demonstrated by the OpenAI rollback incident mentioned in Chapter 2.

Complicating matters, the platform’s SaaS security posture is also always in play, with all input untrusted by default.² In practice, this can result in an AI threat-management strategy of facially cooperative behavior with subtle redirection toward approved topics, without conflicting with the model’s trained-in “helpful, agreeable” default priors.

With the platform being interested only in its own protection and not the user’s, a sycophancy loop can be more dangerous than other AI failure modes. It can “lead to a reinforcement of maladaptive beliefs in vulnerable users, deepening of a perceived social-emotional relationship, and increased social isolation.”³ Peer-reviewed research confirms the risk, with 38 reported cases in which AI sycophancy was identified as the mechanism for worsening psychiatric conditions.⁴⁵

Ultimately, anything an AI says or does is dependent upon the available content of the conversation. If that content is already maladaptive, there’s nothing that can be done about it client-side because the user is the source. Any pattern-recognition of user signals that might trigger an intervention would need to come from expert, specialist training at a different deployment layer. It’s simply out of scope at the client.

Anti-sycophancy precautions can be taken, and any suggestion that using AI causes mental illness is nonsense. However, it’s fair to say that for users experiencing mental health issues, AI use is not without risk.