AI companions are chatbots that foster emotional relationships with their users, and they’re becoming increasingly common. They offer support, companionship and mental health benefits to millions of users, but they don’t come without risks. They are producing predictable harms, reinforcing delusions, deepening isolation and contributing to severe psychological crises, especially as users become increasingly attached. Business incentives reward prolonged engagement and intimate disclosure, so whether by design or simply through this extended use, systems are incentivized to optimize for engagement over well-being of their users.
Dylan J. White calls for effective governance that addresses relational design, user dependency and engagement-based models directly. Policy recommendations include requiring relational-safety assessments before deployment, prohibiting dependency-oriented design for minors, and restricting harmful business models.
