- The Prohuman
- Posts
- A new test for mental health chatbots
A new test for mental health chatbots
Plus: Washington may restrict open AI
Rimsha Bhardwaj, Hasan Toor & Ihtesham Haider
July 13, 2026
In partnership with
Hello, Prohuman
Today, we will talk about these stories:
Mental health AI still misses context
Open models face a policy deadline
Robinhood brings AI agents to crypto
You already have a take on which AI lab ships next.
Claude or Gemini? OpenAI or Anthropic? GPT-7 before year-end or not? If you read tech newsletters, you've already formed opinions on all of it.
Kalshi has real-money markets on which AI model leads benchmarks this week, which lab ships AGI first, when Anthropic releases Mythos, whether OpenAI raises ChatGPT pricing, and which company has the best coding model at year-end. These aren't abstract questions — they're live markets with real money on both sides, moving as labs ship, benchmarks drop, and announcements land.
The edge belongs to whoever actually follows this space. Not the casual observer — the person who reads model cards, tracks evals, and notices when a new release outperforms the field before the mainstream press catches up.
That person has a genuine edge. If that's you, Kalshi lets you act on it.
Fluent answers are not clinical judgment
Image Credits: MDPI
AI sounds capable here.
PsyEval tested 11 language models across medical knowledge, diagnosis, and emotional support using English and Chinese datasets.
GPT-4-turbo reached about 76% accuracy on English knowledge tasks, then dropped to roughly 73% on crisis-response questions.
The bigger weakness was conversation. Human counselors scored higher at asking follow-up questions that uncovered deeper concerns, while model performance changed sharply depending on the prompt format.
That matters because mental health support often happens late at night, with someone staring at a bright phone screen and deciding what to say next. A polished response can still miss risk, context, or the reason a person is asking.
The benchmark also exposed a difficult trade-off: cautious models sometimes refused to diagnose, while smaller models assigned labels more aggressively and risked overdiagnosis.
This makes current AI more useful as support infrastructure than as an independent clinical decision-maker. How should these systems respond when caution itself lowers their apparent performance?
Open AI models could face new restrictions
Image Credits: New York Times
Policy action could arrive quickly.
Lambert argues that the U.S. could restrict frontier open-weight models within six months, likely starting with Chinese systems and government use.
The immediate fight concerns distillation claims, especially Anthropic’s push for tighter controls after alleged API use by Chinese labs.
That case feels commercially convenient. APIs are still jailbroken, and banning downloadable weights would mostly protect incumbent labs while leaving determined overseas users with access.
The harder policy question begins when an open model reaches Claude Mythos-level capability and triggers a government review. At a desk late at night, a ban may look simple; in practice, it could strand U.S. startups that depend on cheaper models.
Meta, Microsoft, or Reflection could shift the discussion by releasing a strong American open model before agencies finalize restrictions.
Will policymakers regulate the capability itself, or follow the companies with the strongest lobbying operation?
Robinhood is testing hands-off crypto trading
The handoff is getting real.
Robinhood plans to let U.S. customers connect third-party AI agents that can analyze markets, build strategies, and place crypto trades automatically.
More than 70,000 accounts have already joined the beta across equities and options, with crypto access expected next.
That scale matters.
A user can set limits, close the app, and return later to a screen showing trades they never approved one by one. Robinhood says customers remain responsible when an agent misreads instructions, uses stale information, or loses the money assigned to it.
That risk is direct. This product could make automated trading feel routine, especially as OpenAI, Anthropic, and Grok agents plug into brokerage accounts.
The unresolved question is whether users will treat these systems like tools or trust them as decision-makers.
Prohuman team
Covers emerging technology, AI models, and the people building the next layer of the internet. |  Founder |
Writes about how new interfaces, reasoning models, and automation are reshaping human work. |  Founder |
Free Guides
Explore our free guides and products to get into AI and master it.
All of them are free to access and would stay free for you.
Feeling generous?
You know someone who loves breakthroughs as much as you do.
Share The Prohuman it’s how smart people stay one update ahead.