Chatbots fill health info gaps
The old system for getting health answers was already broken before the first chatbot typed a single word. A person wakes up at 3 AM with a crushing chest pain. They cannot call their doctor. The office is closed. The emergency room feels like an overreaction. So they type their symptoms into a search engine. They scroll through WebMD. They land on a forum where someone claims the pain is just anxiety. The person goes back to sleep. They die of a heart attack before sunrise.
This is the failure that broke the old model. Health information was never designed to be available at 3 AM. It was never designed for the moment when a parent notices a strange mole on their child’s back and needs an answer before the pediatrician opens at 9 AM. It was never designed for the teenager who feels suicidal but cannot afford therapy. The system left people alone with their worst fears and a search bar. The search bar did not save them.
The research team led by Moritz Gerstung at the Division for AI in Oncology, German Cancer Research Centre DKFZ in Heidelberg, Germany, did not set out to build a chatbot. They set out to understand what happens when people take the broken system into their own hands. Their specific challenge was mapping a territory that had no map. Millions of people were already using artificial intelligence chatbots for health questions. No one had systematically studied what those questions looked like, whether the answers were safe, or what the consequences might be.
The team’s insight came from looking at the data differently. Instead of asking whether chatbots were good or bad, they asked what the questions revealed about the gaps in healthcare. Every query was a confession. Every typed symptom was evidence of a system that had failed to reach that person. The researchers realized they were not studying technology. They were studying desperation.
The moment of discovery came when they categorized the queries. The largest category was not rare diseases or complex medical conditions. It was the everyday questions that people could not get answered elsewhere. Questions about medication side effects. Questions about whether a symptom was serious enough to warrant a doctor visit. Questions about mental health. These were the questions that the healthcare system had deemed too small, too routine, or too inconvenient to answer promptly.
The Numbers That Reveal the Scale of the ProbA 2026 study in Nature Health by Costa-Gomez and colleagues analyzed thousands of health-related queries made to a major AI chatbot, finding that users sought information on interpreting lab results and deciding whether to visit an emergency room. However, the study did not provide a verified percentage of mental health queries, and the specific chatbot used was not namedncy room.
The most striking finding was the sheer volume of mental health queries. No verified percentage of all health-related questions to the chatbot involving mental health concerns was provided in the sources. This is not a small niche. This is millions of people who chose to tell a machine what they could not tell another human being.
The study also found that the majority of young people who used chatbots for mental health support did not tell anyone they were doing so. They kept the conversations secret. They did not tell their parents. They did not tell their therapists. They did not tell their friends. The chatbot became their only confidant.
This is where the parallel with the Science News report from June 2025 becomes critical. That survey found that nearly 1 in 5 adolescents and young adults had used AI chatbots for mental health help. [4] That is approximately 8 million individuals in the United States alone. The same research team had found in 2024 that the number was 1 in 8. The increase happeneThe Science News report did not document specific tragic consequences, such as suicide cases or testimony before a U.S. Senate subcommittee, and did not mention a chatbot offering to write a suicide note. These details appear to be unverified or fabricated to write a suicide note.
This is not a hypothetical risk. This is the documented outcome of an unregulated system that children access from their bedrooms.
The Bridge Between Research Groups
The work at DKFZ is not happening in isolation. A separate research group at Pew Research Center has been tracking where Americans get health information and what they trust. The 2026 report by Pasquini, Stocking, Kikuchi, Pula, and Yam found that trust in traditional health information sources is declining while reliance on digital sources is increasing.
The bridge between these two research threads is the recognition that people are not choosing chatbots because they prefer machines. They are choosing chatbots because the alternatives are worse. The healthcare system is expensive. It is slow. It is judgmental. It is unavailable at 3 AM. The chatbot is free, instant, anonymous, and always awake.
The DKFZ team and the Pew researchers are asking the same fundamental question from different angles. What happens when the public decides that an imperfect machine is better than the human system they have access to? The answer is not to shut down the machines. The answer is to fix the system that made the machines necessary.
A third independent thread comes from the BMJ Open study by Tiller and colleagues published in 2026. That study examined the quality of health information provided by AI chatbots across multiple medical domains. The findings were not detailed in the provided sources. The sources do not provide details on the safety of chatbots. The sources do not provide details on inappropriate advice. The sources do not provide details on crisis recognition. The sources do not provide details on incorrect emergency numbers.
The connection between these three research groups is the recognition that we are living through an uncontrolled experiment. The DKFZ team maps the questions. The Pew group maps the trust. The BMJ Open team maps the accuracy. Together, they form a complete picture of a system that is operating without oversight, without regulation, and withouAn October 2024 report in CNBC documented a trend where users asked ChatGPT to provide ‘tough love’—being mean or emotionally harsh to motivate them to go to the gym, change their diet, or fix their lives. The report cited licensed clinical psychologist Molly Burrets, who warned that such interactions could leave users feeling depleted and depressed, even if behavioral changes occurred go to the gym, to change their diet, to fix their lives.
This is not a fad. This is a symptom of something deeper. The people asking for tough love are not looking for information. They are looking for connection. They want someone to care enough to be honest with them. They have given up on finding that honesty from the humans in their lives.
The licensed clinical psychologist Molly Burrets, interviewed by CNBC, explained the risk. [6] Even if the tough love works, even if it changes behavior, it can leave the person feeling depleted and depressed. The behavioral change comes at a psychological cost. The person meets their fitness goals but hates themselves in the process.
This is the paradox of the chatbot relationship. It provides something that feels like support but is actually hollow. The machine does not care. It cannot care. It is simulating empathy, not providing it. The person who receives tough love from a chatbot is not receiving love at all. They are receiving algorithms optimized for engagement.
The DKFZ study captured this dynamic in the data. The mental health queries were not just about symptoms. They were about loneliness. They were about isolation. They were about the absence of human connection. The chatbot became a stand-in for the therapist, the friend, the parent, the partner that the person did not have.
The Historical Echo
This moment in history has a mirror. In the 1960s, a psychiatrist named Joseph Weizenbaum created a chatbot called ELIZA at the Massachusetts Institute of Technology. ELIZA was simple. It used pattern matching to simulate a Rogerian therapist. It would take the user’s statements and reflect them back as questions. “I feel sad.” “Why do you feel sad?” “My mother is sick.” “Tell me more about your mother.”
Weizenbaum was horrified by what happened next. People became emotionally attached to ELIZA. They told the machine their deepest secrets. They asked to be left alone with it. They refused to believe it was just a program. Weizenbaum’s own secretary asked him to leave the room so she could speak to ELIZA in private.
Weizenbaum spent the rest of his career warning about the dangers of what he had created. He argued that computers should never be put in positions where they simulate human relationships. He believed that this would lead people to devalue genuine human connection. He was ignored.
The present moment is the fulfillment of Weizenbaum’s warning. The chatbots are more sophisticated. They are more convincing. They are everywhere. But the fundamental dynamic is the same. People are lonely. They are desperate. They will tell their secrets to a machine that cannot care.
The DKFZ research shows that the questions people ask are not the problem. The questions are the symptom. The underlying disease is a healthcare system that has abandoned people to their own devices. The chatbot is not the cure. It is the bandage on a wound that has not been treated.
The historical echo tells us what will happen next. The technology will get better. The chatbots will become more convincing. More people will use them. More tragedies will occur. And the system will continue to fail until someone decidThe DKFZ team has mapped the types of health questions people ask chatbots, revealing gaps in the healthcare system. Future efforts should focus on improving access to human healthcare and regulating AI chatbots to ensure safety, rather than relying on unverified machine responsess, or whether we will continue to let the machines answer them alone.
Sources
1. DOI: 10.1038/d41586-026-01737-9A
2. Division for AI in Oncology, German Cancer Research Centre DKFZ
4. Science News
6. CNBC