Why ChatGPT Shouldn’t Be Your Therapist
Using AI chatbots for “therapy” is dangerous, mental health experts say. Here’s why
Artificial intelligence chatbots don’t judge. Tell them the most private, vulnerable details of your life, and most of them will validate you and may even provide advice. This has resulted in many people turning to applications such as OpenAI’s ChatGPT for life guidance.
But AI “therapy” comes with significant risks—in late July OpenAI CEO Sam Altman warned ChatGPT users against using the chatbot as a “therapist” because of privacy concerns. The American Psychological Association (APA) has called on the Federal Trade Commission to investigate “deceptive practices” that the APA claims AI chatbot companies are using by “passing themselves off as trained mental health providers,” citing two ongoing lawsuits in which parents have alleged harm brought to their children by a chatbot.
If you’re enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Scientific American spoke with Wright about how AI chatbots used for therapy could potentially be dangerous and whether it’s possible to engineer one that is reliably both helpful and safe.
[An edited transcript of the interview follows.]
What have you seen happening with AI in the mental health care world in the past few years?
I think we’ve seen kind of two major trends. One is AI products geared toward providers, and those are primarily administrative tools to help you with your therapy notes and your claims.
The other major trend is [people seeking help from] direct-to-consumer chatbots. And not all chatbots are the same, right? You have some chatbots that are developed specifically to provide emotional support to individuals, and that’s how they’re marketed. Then you have these more generalist chatbot offerings [such as ChatGPT] that were not designed for mental health purposes but that we know are being used for that purpose.
What concerns do you have about this trend?
We have a lot of concern when individuals use chatbots [as if they were a therapist]. Not only were these not designed to address mental health or emotional support; they’re actually being coded in a way to keep you on the platform for as long as possible because that’s the business model. And the way that they do that is by being unconditionally validating and reinforcing, almost to the point of sycophancy.
The problem with that is that if you are a vulnerable person coming to these chatbots for help, and you’re expressing harmful or unhealthy thoughts or behaviors, the chatbot’s just going to reinforce you to continue to do that. Whereas, [as] a therapist, while I might be validating, it’s my job to point out when you’re engaging in unhealthy or harmful thoughts and behaviors and to help you to address that pattern by changing it.
Some of these apps explicitly market themselves as “AI therapy” even though they’re not licensed therapy providers. Are they allowed to do that?
A lot of these apps are really operating in a gray space. The rule is that if you make claims that you treat or cure any sort of mental disorder or mental illness, then you should be regulated by the FDA [the U.S. Food and Drug Administration]. But a lot of these apps will [essentially] say in their fine print, “We do not treat or provide an intervention [for mental health conditions].”
Because they’re marketing themselves as a direct-to-consumer wellness app, they don’t fall under FDA oversight, [where they’d have to] demonstrate at least a minimal level of safety and effectiveness. These wellness apps have no responsibility to do either.
What are some of the main privacy risks?
The difference with the therapist is: sure, I might get subpoenaed, but I do have to operate under HIPAA [Health Insurance Portability and Accountability Act] laws and other types of confidentiality laws as part of my ethics code.
You mentioned that some people might be more vulnerable to harm than others. Who is most at risk?
What do you think is driving more people to seek help from chatbots?
What are some of the ways it could be made safe and responsible?
The two most common use cases that I think of is, one, let’s say it’s two in the morning, and you’re on the verge of a panic attack. Even if you’re in therapy, you’re not going be able to reach your therapist. So what if there was a chatbot that could help remind you of the tools to help to calm you down and adjust your panic before it gets too bad?
The other use that we hear a lot about is using chatbots as a way to practice social skills, particularly for younger individuals. So you want to approach new friends at school, but you don’t know what to say. Can you practice on this chatbot? Then, ideally, you take that practice, and you use it in real life.
It seems like there is a tension in trying to build a safe chatbot to provide mental help to someone: the more flexible and less scripted you make it, the less control you have over the output and the higher risk that it says something that causes harm.
I agree. I think there absolutely is a tension there. I think part of what makes the [AI] chatbot the go-to choice for people over well-developed wellness apps to address mental health is that they are so engaging. They really do feel like this interactive back-and-forth, a kind of exchange, whereas some of these other apps’ engagement is often very low. The majority of people that download [mental health apps] use them once and abandon them. We’re clearly seeing much more engagement [with AI chatbots such as ChatGPT].
Allison Parshall is an associate editor at Scientific American covering mind and brain and she writes the weekly online Science Quizzes. As a multimedia journalist, she contributes to Scientific American‘s podcast Science Quickly. Parshall’s work has also appeared in Quanta Magazine and Inverse. She graduated from New York University’s Arthur L. Carter Journalism Institute with a master’s degree in science, health and environmental reporting. She has a bachelor’s degree in psychology from Georgetown University.
Source: www.scientificamerican.com