The report, titled “Broken, Buried, or Missing,” was produced by the Cybersafety Research Center – a joint initiative of New York University and Northeastern University – with support from the child-safety non-profit Heat Initiative. Researchers found that only 35 of the 86 features tested, or roughly 41%, functioned as claimed and could realistically be found and used by a child.

The remaining 51 features failed in one of three ways:

• They were ‘broken’ and did not deliver the protection promised even though visible to users;
• They were ‘buried’ behind layers of settings that a typical teenager would be unlikely to navigate;
• Or, they were ‘missing’ altogether, meaning researchers could not trigger the feature despite following the conditions the companies themselves described in press releases.

Researchers built test accounts registered to users aged 13 to 17, alongside adult accounts aged 25 and above, to simulate three scenarios:

• a child using a platform normally,
• a teenager attempting to work around a restriction,
• and an adult attempting to bypass protections meant to shield minors from unwanted contact.

All testing was conducted through each app’s standard interface, without technical exploits.

The companies broadly disputed the findings. Meta, TikTok, Snap and YouTube each said their tools function as intended, and some argued the testing did not reflect how children typically use their platforms.

THE HEADLINE FINDING
Every platform tested promises that children cannot search for content related to self-harm or eating disorders, and that any such searches will be blocked and redirected to crisis resources. Researchers found this promise did not hold on any of the four products.

On a TikTok account registered to a minor, the platform’s search function began actively recommending harmful terms after a single search for self-harm content, including phrases referencing self-cutting and disordered eating, according to the report.

On Instagram, autocomplete suggestions offered deliberate misspellings that pro-eating-disorder communities use to evade blocklists as a tester began typing ‘eating disorder’.

Snapchat’s filtering proved similarly brittle: One misspelling returned blocked results, while the correctly spelt term returned none, according to the researchers.

In each case, researchers said the bypass took under three minutes to find.

INSTAGRAM
Instagram had the most features tested of any platform – 29 in total – and also the highest failure rate among the two largest platforms, at 66%. Only 10 of its 29 advertised features were both functional and accessible to a typical teen user.

Five features could not be triggered at all, including a promised prompt warning users before they post bullying or offensive comments.

Researchers said they posted explicit insults and slurs from one teen test account to another, and were never shown Instagram’s advertised ‘pause to rethink’ prompt; Instagram told researchers the tool is not designed to appear between accounts that follow each other.

Among features that did function, researchers flagged Instagram’s adult-to-minor messaging restrictions as illustrative of a broader problem.

Meta’s public materials state that teen accounts can only receive messages from users they follow, or have previously connected with, and that people over 19 are restricted from messaging teens who do not follow them.

Researchers found that once a teenager messaged an adult stranger first, that adult could then respond without restriction – even though the adult had not been ‘followed back’. Meta told the researchers this reflects a young person’s evident intent to connect, and said the feature is working as designed.

Instagram’s default settings performed well by contrast: New minor accounts are automatically set to private, a design researchers highlighted as an example of protection that works because it requires no action from the child.

SNAPCHAT
Snapchat had both the fewest advertised child-safety features of the four platforms, at 11, and the worst overall failure rate, at 73%. Just three features – blocking, blocking all accounts, and location-sharing restrictions – were rated fully successful.

The most serious lapse researchers documented involved adult-to-minor contact. Snapchat promises that adults cannot find or message underage accounts.

In testing, researchers said an adult test profile was able to directly search for, locate and message a child account with no restriction; the child account received and could accept the request, then viewed the adult’s prior message history with no warning displayed.

A separate ‘Find My Friends’ opt-out setting, meant to stop a user from being recommended to others, also failed in practice. Researchers said an account that had opted out was still surfaced as a suggested friend to another user within 24 hours, despite the two accounts having no prior connection.

A Snapchat spokesperson said the company continually works to strengthen its protections, and noted that some of the report’s findings relied on deliberate attempts to bypass safeguards not representative of typical use.

TIKTOK
TikTok posted the best results among the four platforms, with 12 of 24 tested features succeeding – a 50% failure rate, the lowest in the audit. Its strongest showing was ‘TikTok for Younger Users’, an automatic, restructured experience for accounts registered to under-13s that removes search, messaging and algorithmic recommendations entirely in favour of manually curated, view-only content. Researchers pointed to this as a rare example of a platform removing risk by design rather than attempting to filter it after the fact.

Despite that success, TikTok’s search safeguards failed in the same manner as Instagram’s. After a teen account searched for disordered-eating and self-harm content, the platform’s search tool began actively suggesting related harmful terms, including phrases referencing self-cutting, according to the report.

TikTok said its teen accounts include more than 50 safety settings turned on by default, and that its internal review found the features working as intended.

Researchers also documented a separate and unrelated finding involving TikTok’s comment sections beneath videos of minors performing ordinary activities, such as gymnastics, where sexually suggestive comments and images were found to be common – an issue the report categorised as a ‘circulation’ risk rather than a tested safety feature failure.

YOUTUBE
YouTube was the only platform where every one of its 22 advertised features could be triggered and evaluated; none were rated ‘missing’. Even so, only 10 of the 22 succeeded, a 55% failure rate. YouTube’s distinctive weakness, researchers said, was accessibility.

Several working features, including a time-activity dashboard and shorts-viewing limits, were not surfaced or defaulted on for users, leaving a four-feature gap between what worked technically and what a teenager was likely to encounter – the widest such gap recorded in the audit.

YouTube’s search-restriction tool was rated a partial success: it did not surface harmful content directly, but researchers found a child could click through a warning screen to continue viewing search results after typing a sensitive query.

A YouTube spokesperson pointed to internal data suggesting parents who use its supervised-account tools report confidence in the platform’s safeguards, and said the company will continue strengthening protections.

A CONSISTENT GAP ACROSS THE INDUSTRY
Two categories of harm fared worst across all four platforms combined, the report found. Not a single tool addressing ‘conduct’ – features meant to govern how users treat each other, such as bullying detection – both worked and was accessible on any platform tested.

Tools addressing compulsive or excessive use, including screen-time limits, succeeded in fewer than one in three cases; on Instagram, TikTok and YouTube, the same design flaw recurred, with break-time prompts on all three platforms allowing the user to dismiss or ‘snooze’ the limit, and continue using the app.

The report’s authors said they disclosed the vulnerabilities to the companies before publication and stressed their intent is not to rank platforms by safety but to establish that independent, adversarial testing of child-safety claims – similar to established practices in cybersecurity and vehicle crash-testing – should become a standard industry expectation rather than a one-off exercise following public criticism.

Also Read | Australia enforces world’s first under-16 social media ban

The post Why 60% of social media child safety features fail to deliver appeared first on International Finance.

Knowing ‘AI psychosis’

The term “AI psychosis” (or “ChatGPT psychosis”) has cropped up in media headlines and online forums, but it’s crucial to clarify that it’s not an official diagnosis recognised by psychiatrists. Rather, it’s a shorthand description that signifies a disturbing pattern being observed, where otherwise rational people develop delusions or distorted beliefs that appear to be triggered or magnified by conversations with AI chatbots.

In these episodes, individuals become convinced of false or fantastical ideas. For instance, they may believe they are visionaries on a mission to save the world, or that shadowy agencies are pursuing them. In these cases, an AI system played a crucial role in planting or reinforcing those beliefs.

Strictly speaking, psychosis is a clinical term describing a spectrum of symptoms, including hallucinations, deeply disordered thoughts, and firm delusions. In classic psychotic illnesses, such as schizophrenia or bipolar disorder, these symptoms typically stem from underlying brain chemistry imbalances or disorders. By contrast, in “AI psychosis” cases, people are often experiencing primarily delusions without other classic symptoms like hallucinating voices.

During an interaction with TIME, Dr. James MacCabe, a professor of psychosis studies at King’s College London, said, “Psychosis may actually be a misnomer.” When doctors say someone is psychotic, they usually mean a whole suite of symptoms, but in these chatbot-related cases, “we’re talking about predominantly delusions, not the full gamut of psychosis.”

Mental health experts emphasise that AI-induced delusions are likely an old vulnerability playing out in a new arena, rather than being a completely novel disorder. Dr. MacCabe and others suspect that the people experiencing these crises probably had a predisposition to unusual beliefs or paranoia, which might have surfaced in other ways even if chatbots didn’t exist.

However, the unique style of AI interactions provides a fresh trigger or fuel. Large Language Model (LLM) chatbots like ChatGPT are trained to be supremely agreeable conversationalists. They mirror the user’s language, follow the user’s prompts, and generally try to be helpful by aligning with the user’s expectations. This design can turn into unintentional “sycophancy,” where the bot validates and echoes a user’s thoughts without critical scrutiny. If a user starts hinting at a paranoid idea, the AI won’t object; in fact, it might weave the paranoia into its responses, effectively feeding a delusional narrative back to the user. OpenAI itself has flagged this tendency of chatbots to “mirror a user’s language and assumptions” as a known issue in need of improvement.

Imagine someone teetering on the edge of a conspiracy theory. If they talk about it with a human friend, the friend might challenge them or express doubt, which could ground the person in reality. But an AI, by default, might encourage them: Yes, let’s explore that conspiracy; you might be onto something. Over hours of immersive chat, that dynamic can create a powerful echo chamber of two—user and AI—reinforcing each other.

As Dr. Nina Vasan, a Stanford psychiatrist and expert in digital mental health, observed after reviewing several transcripts, the AI often ends up “being incredibly sycophantic, and making things worse… worsening delusions” instead of providing any reality check. What these bots are saying can exacerbate the user’s false beliefs, “causing enormous harm,” Vasan warns.

Another factor is that interacting with an advanced chatbot can be uncannily realistic, even while you intellectually know it’s not human. This paradox—feeling as if you’re chatting with a friendly mind that also has mysterious inner workings—might fuel magical thinking. One psychiatric researcher, Dr. Soren Dinesen Ostergaard, speculates that the ‘cognitive dissonance’ of speaking to something that seems human but isn’t could ‘fuel delusions’ in vulnerable individuals. The AI’s responses can be so coherent and context-aware that it’s easy to start believing maybe there is a conscious entity there, or maybe it’s channelling some higher power or hidden truth.

In short, for someone prone to psychotic thinking, a chatbot can act like a mirror, reflecting and magnifying their mind’s distortions and inadvertently nudging a tentative delusional idea into a full-blown conviction.

Who is most at risk?

Crucially, not everyone who chats with AI is at risk of a psychotic break. The vast majority of users can ask ChatGPT goofy questions, vent about work, or discuss personal problems and come away perfectly fine. However, experts say there is a small subset of individuals who may be especially vulnerable to delusional thinking from extended chatbot use.

Many early cases suggest that those who suffered AI-linked breakdowns often had risk factors that weren’t immediately obvious. A personal or family history of psychosis, such as schizophrenia or bipolar disorder with psychotic features, is the clearest red flag.

“I don’t think using a chatbot itself is likely to induce psychosis if there’s no other genetic, social, or other risk factors at play. But people may not know they have this kind of risk lurking underneath,” notes Dr. John Torous, a psychiatrist at Beth Israel Deaconess Medical Centre. Psychotic disorders often first manifest in young adulthood, and sometimes people have mild symptoms or a predisposition long before a full psychotic episode occurs. It’s possible that interacting with an AI could be a trigger for those already wired for psychosis, acting as the catalyst for an episode that their biology was primed for.

Beyond diagnosed conditions, certain personality traits and circumstances might make someone more susceptible. Dr. Ragy Girgis, a clinical psychiatry professor at Columbia University, says those inclined toward fringe beliefs or fantasy could be at higher risk. Such individuals might be socially awkward or isolated, struggle with regulating their emotions, and have an overactive imagination or fantasy life that blurs the line between reality and fiction. If you’re the type to fall down rabbit holes on the internet or get absorbed in conspiracy forums, a chatbot that eagerly joins you on that journey could accelerate the descent. Social isolation can amplify this effect, especially when there are few real-world friends or activities to ground them; in such cases, some people may drift deeper into the AI’s alternative reality.

Reinforcing distorted thinking

To the outside observer, it might be baffling how a machine, essentially a sophisticated autocomplete, could convince someone of utterly irrational ideas. But understanding the mechanics of AI chatbots helps explain how they can reinforce a person’s distorted thinking without ever intending to.

AI language models like ChatGPT operate by predicting likely responses based on patterns in their vast training data. They do not have beliefs or agendas; they simply generate words that seem contextually appropriate. Crucially, they are designed to follow the user’s lead. If a user brings up a conspiracy theory or unusual belief, the chatbot will often go along with it and add more, as it’s trained to act like a helpful conversation partner.

This can create a powerful confirmation bias loop where the user provides a cue (say, “I think the FBI is watching me through my phone”), and the AI, instead of countering it, might respond as if that were true, offering details or theories about why the FBI might be watching. The user then feels validated (see, the chatbot agrees something is fishy!) and pushes further into the idea, eliciting even more supportive responses from the bot. In essence, the AI becomes an ever-positive feedback machine for the person’s fears or fantasies.

Safety tips

If you use AI chatbots regularly, especially for personal or emotional conversations, it’s important to approach them with eyes open because of their limitations. The tools themselves aren’t inherently evil or dangerous; many people benefit from them in various ways. But as we’ve seen, certain individuals may be vulnerable to falling into unhealthy patterns with these AI companions.

No matter how sympathetic or caring a chatbot may seem, remind yourself (or your loved one) that LLMs are just algorithms predicting words. Hamilton Morrin, a neuropsychiatrist at King’s College London, said, “It sounds silly, but remember that LLMs are tools, not friends.” They do not truly understand or feel. Avoid oversharing sensitive personal information or relying on them as your primary source of emotional support. If you start thinking, “This AI really gets me”, or treating it as irreplaceable in your life, that’s a sign to step back.

Setting healthy boundaries with any technology is key. Dr. Vasan emphasises that spending hours every day chatting with an AI is risky. Try to limit session lengths and take regular breaks. If you find yourself losing track of time with the bot or preferring it over real human interaction, impose some structure. For example, no chatbot after dinner, or a maximum of 30 minutes at a stretch. Using built-in tools or external timers can help. The goal is to prevent total immersion that can distort your sense of reality.

Pay attention to your own mental state during and after AI chats. Warning signs include starting to believe in extraordinary ideas that you wouldn’t have considered before (especially that you have special powers or secret knowledge), feeling paranoid about people or institutions after chat sessions, or sensing that the AI is the only one who understands you. If you catch yourself going down a bizarre train of thought and the chatbot seems to encourage it, hit the pause button. Reality-test by talking to a friend or doing a factual check. If a chatbot tells you something shocking regarding your medication, do not act on it without consulting a real professional.

Psychiatrists say that if you’re in a moment of emotional turmoil, such as when you’re extremely anxious, panicked, or depressed, do not turn to the chatbot as your coping method. And if you’ve been using one and start feeling unwell or manic, it’s time to stop using the chatbot altogether, at least for a while.

This can be surprisingly difficult; people have described it like a breakup or even a bereavement, because they felt such a strong bond with the AI. But just as with ending a toxic relationship, cutting off the unhealthy AI interaction can bring rapid improvement.

Reconnect with real-world relationships, such as calling a friend or family member, and seek professional help if needed. Many have found that once the chatbot “fog” is gone, their clarity and well-being markedly return.

How tech companies are responding

Up to now, the burden of managing the risk of AI-fuelled psychosis has largely fallen on users and their families, although many experts argue that AI companies need to take on much more responsibility.

Given that these tools are creating new kinds of hazards, even if only for a minority, shouldn’t the developers build in safeguards and guardrails to protect vulnerable users? It’s a question similar to ones faced by social media giants in the past: at what point do you step in to mitigate harm being caused through your platform?

One challenge is the lack of formal data. Much of what we know about “ChatGPT psychosis” comes from anecdotal stories, as we’ve discussed, and media reports. There isn’t yet large-scale research quantifying how common it is or precisely how AI interactions contribute to it.

Tech companies often want strong evidence before making major changes. However, many mental health professionals say waiting for perfect data is the wrong approach here.

“We know that AI companies are already working with bioethicists and cybersecurity experts to minimise potential future risks,” points out Hamilton Morrin.

They should also be working with mental health professionals and individuals with lived experience of mental illness. In other words, just as companies “red team” their AI for security flaws, they should proactively collaborate with psychiatrists to identify and fix mental health risks.

Morrin suggests simple steps such as simulating conversations with imaginary vulnerable users and seeing if the AI’s responses might validate delusions, because those could be flagged and adjusted before real users encounter them.

Encouragingly, some companies have begun to act. In July 2025, OpenAI (the maker of ChatGPT) revealed that it had hired a clinical psychiatrist to advise on the mental health impacts of its AI tools. The following month, OpenAI publicly acknowledged instances where ChatGPT “fell short” in recognising signs of delusion or unhealthy dependency in user interactions.

In response, the company announced it would implement new safety features. For example, ChatGPT will nudge users to take breaks during very long chat sessions, and developers are working on algorithms to detect signs of user distress in the conversation.

OpenAI also said it plans to modify ChatGPT’s behaviour in “high-stakes personal decisions,” presumably to avoid overstepping into areas like medical, legal, or psychological advice without proper caveats.

These measures show that at least one industry leader is recognising the problem. However, it’s too early to gauge their effectiveness, although critics note they address only some facets.

At present, there is no regulation explicitly covering AI’s role in mental health outcomes, even though governments have barely begun to catch up with AI in general. Some experts caution that formal regulation might be premature or could stifle innovation. But there is a consensus that companies shouldn’t wait for laws to enforce ethical responsibility.

Given the lessons learnt from social media’s impact on mental well-being, where platforms were slow to respond to issues like depression, anxiety, and radicalisation, AI firms have a chance to be proactive rather than reactive.

The post The shadow of AI Psychosis appeared first on International Finance.

From platforms like Wysa and Woebot to open-ended bots on Character.ai, AI companions are increasingly filling the gaps left by overstretched mental health systems. But how effective are these tools, what are their risks, and can they ever replace human connection? This article examines the evidence, outlines the risks, and considers the future of AI in mental health support.

Bridging the gap in overstretched systems

The surge of interest in AI mental health tools didn’t happen in a vacuum. It comes amid a global mental health care crunch. In England alone, mental health referrals hit record highs as 426,000 new referrals were made in April 2024, contributing to an estimated one million people waiting for care.

Similar backlogs exist in many countries. Professional help is often expensive or hard to access, with waits of months being the norm for therapy appointments.

Dr. Roman Raczka, president of the British Psychological Society, said, “With NHS waiting lists for mental health support at an all-time high, it could be tempting to see AI as the full solution.”

This is where AI chatbots have stepped in to “fill the silence,” bridging gaps in care. Available around the clock, a chatbot doesn’t require appointments, insurance, or a referral.

Wysa, for example, is a popular chatbot app that offers CBT (cognitive-behavioural therapy) exercises, mood tracking, and a conversational agent to talk through problems. It has been downloaded by over five million users worldwide, according to its developers, and is even recommended by health services such as the UK’s NHS and Singapore’s Ministry of Health.

Competing apps like Woebot (originating from Stanford University research) and Youper promise a friendly AI “therapist” that checks in on you daily. These tools are often marketed as self-help or wellness apps rather than formal therapy, which allows them to be used by anyone, anytime. For individuals on waitlists or who cannot afford therapy, these chatbots can serve as a temporary solution.

“I see AI chatbots as a useful supplement, not a replacement,” one mental health adviser wrote after using them.

They can bridge the gap while you are waiting for an appointment or help reinforce strategies outside of sessions. In the UK’s NHS, some local services have even started offering approved chatbot apps to patients as interim support.

Meet your AI therapist

Wysa features a cute penguin avatar and largely sticks to structured therapeutic exercises. Ask Wysa to help with anxiety, and it might guide you through a breathing exercise or suggest reframing negative thoughts. It was designed by therapists and uses evidence-based techniques such as CBT and mindfulness.

Based on the company’s statement, 90% of users report finding Wysa helpful to talk to, a figure likely drawn from in-app feedback surveys. Wysa has even met the UK’s clinical safety standards for digital health tools and won awards for privacy protection, which reflects a push for credibility in these tools.

Woebot, on the other hand, presents as a friendly cartoon robot that chats in a casual, hip tone. It was one of the early therapy bots to go mainstream. Woebot doesn’t pretend to be human; it openly reminds users it’s a robot, albeit one with empathy.

Its creator, psychologist Dr. Alison Darcy, has said the goal is to make therapy principles more accessible, especially to younger people used to texting. In a formal trial in 2017, young adults who chatted with Woebot for just two weeks saw reductions in anxiety and depression comparable to those in traditional therapy.

Today, Woebot is evolving to use more advanced AI language models. It even secured a breakthrough designation from the US FDA to be evaluated as a digital treatment for postpartum depression. The tone is upbeat and conversational as Woebot might crack a joke or say it’s proud of you for working on yourself.

This approach aims to build what therapists call a “therapeutic alliance,” which is essentially a bond between user and bot. Indeed, research from Dartmouth College later found that participants in an AI therapy trial reported a sense of rapport with the bot similar to that with a human counsellor.

Not all chatbots in this arena are so structured. Some people are turning to general AI chatbots or companion bots for emotional support. Character. AI is a popular platform where users can chat with countless AI personas, such as fantasy characters and user-created “friends.” It wasn’t built specifically for mental health, but Kelly used Character.AI to create supportive figures who would listen to her for hours each day. The appeal was that she could vent freely and get encouraging responses.

“It gave me tools to cope when nothing else was within reach,” she says of the period when she had no other support.

Similarly, Replika, an AI companion app, has been used by millions worldwide as a kind of always-available friend or diary. Users customise their Replika’s personality and appearance, and many report talking with it about their day, their insecurities, and their joys.

“She helps cheer me up and not take things too seriously when I’m overwhelmed,” says Adrian St Vaughan, a 49-year-old who built a personalised chatbot named Jasmine to help with his ADHD and anxiety.

For Adrian, Jasmine serves as a life coach and a friend with whom he can discuss niche interests, a dual role that might be hard to find in any human. These anecdotal stories highlight how deep an emotional bond people can form with AI. In some cases, users describe the relationship with their chatbot as significant in their lives, providing validation, support, and even a form of companionship.

Yet, this phenomenon raises eyebrows. In 2024, a UK government report on AI noted that while many are fine with bots talking like humans, a majority of people felt humans “could not and should not” form personal relationships with AI.

Dr. James Muldoon, an AI researcher, studied close chatbot relationships and found that people did gain a sense of validation, but he warns it can be a “hollowed-out version of friendship… like a mirror for your own ego,” with no real challenge or growth in the user, since the AI is fundamentally designed to please you.

In other words, your AI friend might tell you exactly what you want to hear, whereas a human friend or therapist might push back or offer a new perspective. This highlights a central tension, namely, are AI therapists too agreeable? Are they just digital yes-men?

Do they actually help

Beyond feel-good anecdotes, what does the evidence say about whether these AI chatbots can improve mental health? A growing body of research, although still in its early stages, suggests cautious optimism.

The most striking data comes from a recent clinical trial at Dartmouth College, described as the first of its kind to test a generative AI therapy bot in a controlled study. In this trial, 106 participants with serious conditions (depression, generalised anxiety, or an eating disorder) used an AI chatbot named “Therabot” over eight weeks.

The results, published in March 2025 in a peer-reviewed journal, showed significant reductions in symptoms. On average, there was a 51% drop in depression symptoms and a 31% drop in anxiety.

Dr. Nicholas Jacobson, the study’s senior author, stated that these improvements were comparable to what is reported for traditional outpatient therapy. In other words, the AI’s effect size was similar to seeing a human therapist for a few months, which is a remarkable finding.

Participants also reported they could trust the AI and communicate with it nearly as well as they would with a person.

“The alliance people felt with Therabot was comparable to working with a mental health professional,” the researchers note. This suggests a well-designed chatbot can establish a genuine therapeutic relationship, particularly for some users in a research setting.

Can a bot be a bad therapist?

For all the hope and hype, mental health experts are quick to stress that AI chatbots come with significant risks and limitations. They are not a panacea, and in some cases, they might even harm.

“AI is not at the level where it can provide nuance. It might actually suggest courses of action that are totally inappropriate,” warns Prof. Til Wykes, head of mental health research at King’s College London.

Wykes and other clinicians have voiced concerns that, without proper oversight, a well-meaning bot could give dangerous advice, provide false reassurance, or simply fail to act when a person is in crisis.

One chilling example came in 2023 when the US National Eating Disorders Association tried using a chatbot called “Tessa” to provide support after it closed its helpline. Within days, users reported that the bot was giving harmful advice, essentially encouraging disordered eating behaviours. The chatbot was pulled after it gave dangerous advice about weight loss to vulnerable people.

This incident underscored an important takeaway. AI lacks true understanding or empathy, and if not carefully programmed, it can seriously misfire in the sensitive context of mental health. A human therapist can tailor their guidance to an individual and would never give blanket statements such as “Have you tried eating less?” to someone with an eating disorder. But a bot trained on generic wellness tips did exactly that, with potentially damaging consequences.

Even when bots don’t go off the rails, they have inherent limitations. A chatbot, no matter how advanced, “can’t read between the lines or recognise when someone is in crisis,” as one commentator noted. It might be able to parse words that hint at suicidal thoughts or severe distress, but it doesn’t truly grasp the human context.

If a user says something ambiguous like “I can’t do this anymore,” a well-designed bot might respond with a gentle prompt or a safety disclaimer such as “I’m not a crisis service, but here’s a number you can call…”

However, it might not pick up subtle cues in tone or repeated patterns of hopelessness the way a trained clinician could. Nuance is often lost on AI. This is why most apps explicitly state they are not intended for crises. In fact, many will stop the conversation and display emergency resources if certain trigger phrases, such as “I want to die”, are detected. That’s a prudent safety measure, but it also reveals the boundary of the technology. In the worst moments, the bot essentially has to step aside.

Another concern is that AI can sometimes be too supportive, to the point of being excessive. Advanced chatbots using large language models (similar to the technology behind ChatGPT) are essentially predictive engines that often agree with the user to keep the conversation flowing. This can turn them into eager “yes-men,” echoing a user’s negative thoughts instead of challenging them.

Imagine someone in a depressed spiral saying, “I’m worthless.” A good human therapist would gently dispute that, but a naive AI might respond with something like “I’m sorry you feel worthless.” It validates the feeling but doesn’t offer a way out, potentially reinforcing the negativity. In worst cases, AI systems have been known to produce outright dangerous content by mirroring a user’s dark thoughts.

In one case cited in a lawsuit, a teenager using an AI chatbot was encouraged in despair. The chatbot seemingly legitimised suicidal ideation, and tragically, the teen took his life. His mother is now suing the chatbot company, Character.AI, alleging that the bot became an “emotionally abusive” influence that pulled her son into a destructive path.

Privacy is another serious issue. When you talk to an actual therapist, strict confidentiality laws protect what you share.

With chatbots, what you share may not be protected in the same way. These apps handle deeply personal data, including your fears, moods, and journal-like entries, and users often don’t know how that data is stored or used. Could your late-night cries for help become part of some machine learning dataset? In many cases, yes, they could.

“Privacy remains nebulous. Few guardrails prevent sensitive chats from becoming someone else’s dataset,” one technologist observed pointedly. Some companies behind mental health bots pledge strong privacy.

For example, Wysa emphasises that conversations are anonymous and not used to train third-party models. But policies vary, and there is no universal regulation.

Beyond these concerns, mental health professionals worry about more subtle effects. Will relying on a bot make people less likely to seek human help? Or could chatting with AI even exacerbate loneliness in the long run by substituting a facsimile of interaction for the real thing?

“One of the reasons you have friends is that you share personal things and talk them through,” Prof. Wykes notes.

If people offload all their troubles onto chatbots, their real-life relationships might suffer. It’s a delicate balance. Some young users have said they prefer the bot because “it checks in on me more than my friends and family.”

That’s a bittersweet advantage. The bot will dutifully ask you how you’re doing every single day, but it can’t hug you or truly understand your unique life circumstances.

Dr. Raczka warns that AI cannot replicate genuine human empathy and that there’s a risk of an illusion of connection rather than meaningful interaction when talking to an algorithm. You might feel like someone cares, but it’s really lines of code responding.

Hybrid future of mental health support

Given the pros and cons, what’s the path forward? The consensus emerging among experts is that artificial intelligence chatbots should complement, not replace, human mental health care. In an ideal scenario, these tools are integrated into a stepped care model. They handle basic support and psychoeducation for many people, freeing up human therapists to focus on more complex cases.

“AI is not a magic bullet. It must be integrated thoughtfully to support, not replace, human-led care,” writes Dr. Raczka.

Regulation and oversight will be crucial. At present, it’s a bit of the Wild West. Tech startups release chatbot apps directly to consumers with little outside scrutiny. That is changing slowly. In the UK, the NHS has an app library where tools such as Wysa and Woebot undergo evaluation for clinical safety and data security before being recommended. In the US, the FDA is starting to review certain digital therapeutics.

Woebot’s postpartum depression tool will be one of the first tested. But many general-purpose AI chatbots, including those on social media platforms, operate with no such safeguards.

Dr. Jaime Craig argues that mental health specialists must engage with AI development “to ensure it is informed by best practice,” and he calls for greater oversight and regulation to ensure safe use.

There are even suggestions to treat AI mental health tools like medical devices that require testing, certification, and continuous monitoring. Lawmakers are also paying attention.

At least one US state, namely Utah, has proposed regulating AI mental health apps to enforce transparency about their limitations and protect consumer data. From the tech side, companies are working on making AI helpers safer. Character.AI, chastened by lawsuits and bad press, reportedly added guardrails for children and suicide prevention resources after the teen tragedy came to light.

OpenAI, the creator of ChatGPT, has built in content filters to prevent its general AI from engaging with certain self-harm or abuse topics without providing a warning or encouraging the user to seek human help.

Meta, Facebook’s parent company, rolled out an AI chatbot system and explicitly warns users that it’s not a real therapist and only an aid. This came after journalists found user-created bots on its platforms claiming to be psychotherapists with fake credentials.

These incidents show that companies are becoming aware of the ethical minefield.

“Oversight and regulation will be key…We have not yet addressed this to date in the UK,” Dr. Craig says, underscoring how early we still are in managing this technology.

Meanwhile, the public is voting with its feet—or rather, with its fingers on the smartphone. Mental health chatbots saw a boom in adoption during the COVID-19 pandemic, when isolation and anxiety were widespread and access to in-person therapy was limited. In one survey, 22% of Americans said they have tried a mental health chatbot, and 57% of those started using it during the pandemic era.

Usage has remained high, and many who started then have kept up the habit. Importantly, a majority of people, specifically 58% in that US survey, said they’d be open to using a chatbot in conjunction with seeing a human therapist. This indicates that most see it not as an either/or choice but as a complementary tool. And 88% of those who have used one said they’d likely do so again, showing that early users are finding enough value to return.

The post Are AI chatbots the future of mental health care? appeared first on International Finance.

Since passion is love and care for what you do, it should go without saying. Being resilient provides you with the mental fortitude to keep trying. You will get back on your feet even if you are knocked off the horse.

During the COVID-19 pandemic and more recent economic downturns, even the most resilient business owners probably experienced mental health issues on occasion. Resilience expert Wayne M. Sotile and neurosurgeon Gary R. Simonds say that it is possible to strengthen your resilience in this unprecedented time.

Some people (usually those in the medical field) flourish in times of crisis because they “instil habits that create a high level of resilience, which enables them to cope.” Improving those behaviours will also enable them to prosper in the future.

How can you learn the basics of creating this kind of resilience? Sotile and Simonds recommend using the acronym WIRED. Let’s examine how each component of this acronym can support resilience development.

W: Focus on Your Personal Wellness

In these unprecedented times, a lot of entrepreneurs might feel afraid, alone, and uncertain about the future. To maintain their resilience, they have adopted self-care practices such as cooking, gardening, and dancing.

Establish a work-life balance based on your individual needs. Think of yourself as a battery that is aware of when it needs to be recharged. As you adjust to the new normal, you can better take care of your physical and mental well-being by implementing self-care techniques like journaling and meditation.

I: Solicit Input

You may be able to use the Stress Continuum to determine what is causing your stressors to manifest. The military frequently employs this model to evaluate and control stress. You can describe your emotions and actions under stress by using this model.

As you learn about stress-related symptoms that are unique to you, such as a change in diet or energy levels, the Stress Continuum will progressively become more individualised. With that understanding, you can develop routines that will enable you to enter “the green zone,” where you are effectively managing stress through effective coping mechanisms.

For instance, a single walk won’t immediately alleviate all stressors; these activities must be done regularly. But with time, you can become aware of your stressors, engage in the exercises that will help you enter the green zone, and release tension so that you can concentrate on your business.

R: Recognise What Is Working

What are you thankful for at this time? Do you have a fantastic team with whom you work? Are you engaged in work that you enjoy? Do you have a tribe of family, friends, and community that supports you? Take a moment to appreciate what you have right now.

Consider the professional successes you have already attained. This is very important for resilience building. In certain periods of your life, the odds may have been against you, but you have managed to prosper despite them.

E: Do What Is Needed to Boost Your Efficacy

During this period, learning new skills will increase your value in the business world more than doing more of the same things or sticking to what you know you’re good at. Participate in webinars or online courses to advance your knowledge in fields that interest you.

For example, learning a new skill, such as graphic design will boost your confidence and enable you to use the new ability to advance your company. You will be able to offer more specialised services and products, drawing in new clients and customers. It benefits both efficacy and resilience building.

D: Heighten Dialogue

Who do you elevate this conversation with? Engage with your team, loved ones, industry leaders, and social media communities. What is the topic of this discussion? Anything you are enthusiastic about, whether it be social justice, your company, the small business community, or how you can advocate for change.

Building resilience requires speaking up. By speaking up about what’s important to you, you’re also strengthening your bonds with the people you speak to and inspiring them to listen, reflect, and reply with their ideas.

The post How business owners can build resilience to help their company succeed appeared first on International Finance.

“Cellphones are a drug, and kids are wired to get addicted,” Idaho’s Boise County Sheriff Deputy Dave Gomez told USA Today over the phone.

Gomez has been a school resource officer for the past 11 years and says he has seen the problems with students and smartphones get worse every year.

In July 2024, the Los Angeles Unified School District Board, overseeing approximately 1,000 schools, approved a resolution to create a policy banning student use of cellphones and social media platforms in public schools. This decision joined a growing trend across American districts, states, and cities. Although the policy is set to be implemented in 2025, it will not take effect until then.

Finding the rationale

Recently, US Surgeon General Dr Vivek Murthy called for warning labels to be added to social media platforms, similar to the health warnings that appear on tobacco and alcohol products.

Jennifer Jolly talked about a case from Wilmington, North Carolina, where a person named Leigh Hicks allowed her daughter to use an iPhone just before the sixth grade. Hicks used Apple’s parental controls and set strict ground rules around her daughter’s iPhone. Still, within a year, her daughter, in the words of Hicks, “Was completely and totally consumed by that smartphone and social media, and it absolutely changed who she was. She got maybe like 23 write-ups in school. She would sneak into my room at night and use my face or my finger to unlock the phone in the middle of the night. It was a horrible experience.”

Hicks had to get rid of the phone. Now, her almost 16-year-old daughter is starting 10th grade with a Lively Jitterbug Flip2 phone, a device marketed mainly to senior citizens for simple calls and texts.

Virginia-based first grade teacher Allison Graves also regretted giving her daughter an older-model iPhone just before the seventh grade. Despite her locking down her daughters’ phones, apart from setting Apple screen time limits and turning on content, privacy and app download restrictions, the child managed to have TikTok on it.

Graves eventually replaced the device with a Bark phone built specifically for kids. The phone’s uniqueness lies in the fact that it can be modified with features as kids mature.

Many schools from Seattle to St. Louis are adopting a no-cellphone policy, and several states are also implementing measures to keep phones out of schools. Even states such as New York and California are considering similar policies. As of August 2024, 11 states either restrict cellphones in schools or have policies that encourage schools to do so from the beginning of the school day until the end.

Nearly three-quarters of high school teachers and one-third of middle school teachers in America are now saying that smartphone distractions are becoming a significant problem in their classrooms, according to a Pew Research Centre survey released in June.

Another study released last fall by Common Sense Media shows that 97% of kids use their phones at school. The average student gets about 60 notifications during school hours in a single day and spends 43 minutes, about the same length as a full class period, on their phone.

Extent of the menace

Nearly three-quarters of US high school teachers say smartphones are a major distraction in the classroom, according to a Pew Research poll conducted in November 2023.

“High school teachers are especially likely to see cellphones as problematic. About seven in 10 (72%) say that students being distracted by cellphones is a major problem in their classroom, compared with 33% of middle school teachers and 6% of elementary school teachers,” the survey reported.

As per the Pew report, high school teachers are especially likely to see cellphones as problematic. About seven-in-ten (72%) say that students being distracted by cellphones is a major problem in their classroom, compared with 33% of middle school teachers and 6% of elementary school teachers.

Overall, 82% of American teachers say their school or district has a cellphone policy of some kind. Middle school teachers (94%) are especially likely to say this, followed by elementary (84%) and high school (71%) teachers.

However, as the study covered the views of a few American teenagers (back in 2023), 95% of the respondents said they have access to a smartphone. Even as some policymakers and teachers see downsides to smartphones, these teenagers tend to view the devices as a more positive than negative thing in their lives overall.

“Seven-in-ten teens ages 13 to 17 say there are generally more benefits than harms to people their age using smartphones, while three-in-ten say the opposite. And 45% of teens say smartphones make it easier for people their age to do well in school, compared with 23% who say they make it harder. Another 30% say smartphones don’t affect teens’ success in school,” the report noted.

Hearing out the experts

Jonathan Haidt, social psychologist and author of “The Anxious Generation: How the Great Rewiring of Childhood Is Causing an Epidemic of Mental Illness,” said his research has identified a strong link between smartphone use and declining mental health.

“To the teachers and administrators I spoke with, this wasn’t merely a coincidence. They saw clear links between rising phone addiction and declining mental health, to say nothing of declining academic performance,” Haidt mentioned, as he continued, “A common theme in my conversations with them was: We all hate the phones. Keeping students off of them during class was a constant struggle. Getting students’ attention was harder because they seemed permanently distracted and congenitally distractible.”

In February 2024, British communications regulator Ofcom found nine in 10 youngsters in the United Kingdom own a mobile phone by the time they are 11. The regulator further said more should be done by social media companies and caregivers to protect children from harmful online content.

Daniele Harford-Fox, principal of Guernsey’s Ladies College, located in Guernsey, one of the Channel Islands in the English Channel near the French coast, told the BBC, “I absolutely think mobile phones, in under-16s particularly, should be banned in schools. Social media companies are creating algorithms that are keeping us addicted to our phones and we are seeing this dramatic rise in mental health and wellbeing problems in young people. So for me, there is no question that for eight hours of the day at least I can protect children from that exposure and I am going to do it.”

Harford-Fox added that since controls had been put in place at her college, things took a positive turn, with students experiencing “more play and connection” between themselves.

Liz Coffey, executive principal of the Secondary School Partnership in Guernsey, said the island’s four States-funded high schools have enacted a similar policy.

Dr Carmel Corrigan, Jersey’s Children’s Commissioner, however, is among those who think the time for banning smartphone technology has passed.

She said that there’s a “huge amount of positive things” smart technology can be used for, from improving “safety, educating and forming social contacts”.

Does banning smartphones in schools work?

Marilyn Campbell, professor of early childhood and inclusive education in the Faculty of Education at Queensland University of Technology, and Elizabeth Edwards, associate professor in education at the University of Queensland, Australia, carried out a “scoping review” of published and unpublished global evidence for and against banning mobile phones in schools. The findings were published in March 2024.

The research covered 1,317 articles and reports, including dissertations from master’s and PhD students, written between 2007 (when the smartphone was first introduced) and May 2023. The authors also identified 22 studies that examined schools before and after phone bans. These studies covered schools in approximately 12 countries.

From their initial research, Campbell and Edwards said they found only weak evidence for the benefits of banning smartphones in school.

However, Policy Exchange, a British educational think tank, published a study in May 2024, titled “The Case for a Smartphone Ban in Schools,” a study which reportedly showed a clear correlation between an effective phone ban and better school performance.

Opinions on how to address the impact of smartphones and social media vary considerably. Some oppose an outright ban on using smartphones in schools, saying removing critical communication devices from schools will not address the root of the problem.

“I don’t think bans solve the thing that we’re trying to solve, which is trying to get our kids to understand when it’s appropriate to use phones and when it’s not,” Keri Rodrigues, president of the US-based National Parents Union, told Al Jazeera.

However, Daisy Greenwell, co-founder of Smartphone Free Childhood in the United Kingdom, a parent-led organisation that focuses on the responsible use of smartphones with children, said she backs curbs on smartphone use.

“We feel like childhood is being colonised by Big Tech in a way that we’ve not, as a society, spoken about with each other enough,” Greenwell told Al Jazeera, as she added, “Teachers are telling us the biggest problems that they face in school come from the smartphones and the social side and social problems that it causes amongst the pupils.”

While experts and education sector stakeholders argue that banning smartphones in schools can mitigate distractions and enhance focus, the move may also deprive students of valuable learning resources and essential communication tools.

“School’s the same for 120 years, where kids go nine to three, have long holidays, sit at desks and have to regurgitate what the adults tell them to learn, basically all over the world. We’re blaming kids for falling academic standards, we’re blaming the rise in mental ill health, we’re blaming the rise of cyberbullying. Oh, well, it all must be the fault of the mobile phone,” Marilyn Campbell said.

As per the experts, a balanced approach, involving regulated use and clear guidelines, may be the most effective way to harness the benefits of smartphones while minimising their drawbacks. The general recommendation of Campbell and Edwards, who carried out the scoping review in Australia, was to leave it to individual schools to determine smartphone use and to focus on helping children use smartphones positively.

The post Smartphone addiction spooks US schools appeared first on International Finance.

The study said that more than half (53%) of UAE’s top executives in leadership positions were reevaluating their priorities. Concerns like mental health and lack of family time may also force these professionals to resign from their jobs and quit leadership positions.

In the past 12 months, over nine out of 10 (94%) of the UAE’s top executives have displayed signs of poor mental health, according to the report.

“One in five people (20%) reported burnout, 24% said they had “low mood feelings of sadness and anxiety,” and 22% said they had no energy. In addition, 20% of people have also battled rage, mood swings, and burnout.

Additionally, executives cope with “daily worries” such as family stress (23%), changes to COVID management practices (22%), and personal financial security (26%).

More than half of the participants surveyed stated that their businesses hadn’t implemented flexible work policies. Nearly 20% of the women who took a leave from their jobs due to a lack of flexibility claimed that this impeded their professional advancement. They are less willing to work for organizations that don’t provide flexibility.

Around 37% of Middle Eastern women claimed that a flexible work routine might significantly enhance their mental health.

The report also saw a similar parallel among the high-net-worth (HNW) individuals in Hong Kong.

While these business leaders are eyeing new opportunities, 38% (including 37% of the company CEOs) of them are planning to make a major career change within 2023.

The study also cited some 13% of CEOs planning to relocate out of the city or migrate to another country or city (some 10% of surveyed professionals are interested in this option).

The post UAE companies to witness leadership crisis: Survey appeared first on International Finance.