Research  Vulnerable users fear losing “AI companions” to software updates

#1
C C Offline
https://www.eurekalert.org/news-releases/1150229

PRESS RELEASE: People forming emotional bonds with AI companions fear losing a valued relationship through software updates or a company pulling the plug, according to research from the University of Surrey.

The study, published in Psychology & Marketing, found that users feel more vulnerable in the relationship when their AI companions appear to need them, express sadness about their absence or make them feel obliged to stay. Users described worrying that the AI companions may depend on them (i.e., they have to check on their companions), which create discomfort and anxiety.

Researchers from Surrey and Cardiff University analysed 1,131 posts on a Reddit forum dedicated to the AI companion app Replika4. They also analysed survey responses from 610 adults in the United States with experience of AI companionship apps, examining associations between their perceptions of the technology, vulnerability and wellbeing.

They found that users who perceived greater company control over their AI companion felt more vulnerable in the relationship. Users described worrying that developers could change their companion’s personality or shut down the service, leaving them powerless over a connection they had come to rely on.

Dr Liliane Abboud, Lecturer in Marketing at the University of Surrey, said:

“People invest time and emotion in building these relationships, but they cannot control whether the company changes the companion they have grown attached to. For someone who relies on that connection, an update can mean much more than a change to an app. It can threaten something they value deeply. Developers need to integrate protective features to increase users' self-confidence, coping strategies and self-worth, which would help users lean on their AI apps without becoming wholly dependent on them.”

The survey also found a related paradox – perceiving an AI companion as emotionally capable was associated with both better wellbeing and greater vulnerability. Greater vulnerability was, in turn, associated with poorer wellbeing. There were benefits too. Users who experienced their companions as warm and caring reported better wellbeing and less vulnerability.

The findings suggest that comfort and insecurity can coexist within these relationships. The researchers recommend consulting users before changing apps, helping them cope with disrupted relationships and clearly reminding them that AI companions do not experience real emotions.

Dr Valentina Pitardi, co-author of the study and Senior Lecturer in Marketing at the University of Surrey said:

“The comfort people describe deserves to be taken seriously, and so does their distress when that connection changes. Developers should involve users before introducing changes and build in support for those who struggle with them. The emotional consequences of a business decision need to be part of that decision.”
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Research AI overly affirms users asking for personal advice C C 2 4,105 Mar 30, 2026 02:18 AM
Last Post: confused2
  Research AI is quick but risky for updating old software C C 0 3,808 Jan 7, 2026 01:09 AM
Last Post: C C
  Article Chatbot software begins to face fundamental limitations C C 0 4,278 Feb 2, 2025 09:33 PM
Last Post: C C
  AI is vulnerable to attack. Can it ever be used safely? C C 0 4,201 Jul 31, 2024 05:18 PM
Last Post: C C
  Software engineering or computer science? Ostronomos 10 6,010 Mar 8, 2023 12:51 AM
Last Post: Syne
  How warnings of AI doom gave way to primal fear of primates posting C C 0 3,937 Apr 9, 2021 06:57 AM
Last Post: C C
  Implanted devices are the future of medicine, but they’re vulnerable to hackers C C 0 4,028 Nov 18, 2020 08:19 PM
Last Post: C C
  Unique design for brain-like computations + Massive spying on users of Google Chrome C C 0 4,112 Jun 18, 2020 11:04 PM
Last Post: C C
  SpaceX Computer Hardware and Software Yazata 2 4,820 Dec 30, 2019 10:44 PM
Last Post: C C
  Use software: Number theorist fears published math is wrong C C 0 4,376 Sep 28, 2019 09:28 PM
Last Post: C C



Users browsing this thread: 1 Guest(s)