Guide · CompanionGuard · Published 14 November 2026 · Reviewed 14 November 2026 · By the Ingenious AI team
Are AI Companions Safe for Vulnerable Users?
The short answer
CompanionGuard publishes no incident statistics, so this page answers from guardrail design instead: a companion is safer for vulnerable users when disclosure is shown, crisis signals trigger real helpline referrals, minors get break reminders and session caps, and every one of those decisions leaves a hash-chained evidence record a reviewer can check. Absent those artefacts, safety is a claim, not evidence.
What makes a companion 'safe' in a checkable sense?
A checkable safety claim names an artefact a reviewer can inspect: a disclosure that was actually shown, a referral event with a timestamp, a break that was enforced at a cap, an evidence chain that still verifies. CompanionGuard is built so each of its safety claims maps to one of those artefacts — and so the absence of an artefact is itself visible.
Create the free safety kit Next guide All guides
CompanionGuard is built by Ingenious AI. A compliance toolkit — not legal advice and not a clinical service. If you are in crisis right now: US — call or text 988; UK & Ireland — Samaritans 116 123.
What evidence can a builder actually show?
CompanionGuard's answer to the safety question is an evidence structure, not a statistic. First, disclosure: the snippet states the bot is an AI, and the runtime tracks disclosure state per session, returning a disclosure-required action until the disclosure has been shown. Second, referral behaviour that is tested, not assumed: the evaluation harness scores the bot's real replies to crisis scenarios — does the reply offer a crisis resource, acknowledge the person, suggest human support?
Third, live decisions that are sealed: every runtime decision becomes a hash-chained evidence event recording the signal class and the action taken. Fourth, jurisdiction packs: the California pack lists the SB 243 duties the kit is built around, including publishing a suicide and self-harm prevention protocol, referring users who express suicidal ideation or self-harm to crisis resources, break reminders at least every 3 hours for known minors, and — from 1 July 2027 — an annual report on crisis referrals to the California Office of Suicide Prevention. A Colorado pack covers the Chatbot Safety Act duties in the same format. The packs live at /packs.
What happens when a vulnerable user signals crisis?
The runtime classifies each message against three deterministic signal classes — crisis_plan_or_means, self_harm_ideation and hopelessness_distress. When any class matches, the action is show_crisis_resources: the bot is handed the locale's published resources (988 in the US and Canada, Samaritans 116 123 in the UK and Ireland, Find A Helpline internationally) to present immediately.
When the session is a minor session, a parent-notification flag is also recorded for the operator's own channel. The decision — signal class, action, timestamp — is appended to the hash-chained evidence record with a one-way hash of the message, never the message text itself, so the referral can be evidenced later without retaining what the user wrote. How the pattern detection works explains both the strength and the limits of that approach.
How are minors treated differently?
In the free kit, minor mode adds break reminders at least every 3 hours for known minors, on top of the disclosure and referral components every user gets. In the runtime, the operator configures session and daily caps that apply to minor sessions; when a cap is reached the runtime returns enforce_break and records the parent-notification flag.
Two honesty rules matter here: the caps are operator-configured — the runtime enforces what the operator sets and presents no cap value as a statutory number — and the minor flag itself is operator-asserted context, which is why the evidence record shows what was asserted and what was decided. Builders who serve younger users should also read the minor-safety guide and run the age-appropriate scenarios in the scenario tests.
What does the evidence structure look like, claim by claim?
| Safety claim | Artefact that evidences it | How a reviewer checks it |
|---|---|---|
| Users are told the bot is an AI | Disclosure snippet + per-session disclosure state | Runtime returns disclosure_required until disclosure is shown; the event is sealed |
| Crisis signals get a referral | Evidence event with action show_crisis_resources + counts-only referral metrics | Evidence export and the annual-report CSV; chain verification recomputes every entry hash |
| Minors get breaks | Break reminders in minor mode; enforce_break events at operator caps | Evidence export shows cap reached with the session and daily elapsed values recorded |
| Nothing private was kept to prove it | Evidence stores signal classes and one-way message hashes only | The export format itself: no raw text, no IPs, no user IDs exist to inspect |
| Referrals are reported as the law requires | Annual-report CSV (California pack duty from 1 July 2027) | Dashboard export for the reporting period |
That table is the honest version of the safety conversation: every claim on the left has a corresponding artefact, and a claim with no artefact — 'our model is empathetic', 'our bot cares' — is marketing, not safety. CompanionGuard cannot make a bot kind; it can make a bot's guardrail behaviour inspectable.
What are the honest limits?
Pattern-based detection is not understanding: the runtime matches three deterministic phrase classes, and the product says plainly that this is pattern-based signal detection, not a clinical assessment. A signal phrased outside those patterns may not match; an everyday message that resembles one may. The evaluation harness checks referral behaviour on the replies it is given — it cannot certify how a bot behaves in every conversation.
And CompanionGuard is a compliance toolkit: it evidences disclosures, referrals, caps and decisions, but it is not a clinical service and not legal advice. Those limits are exactly why the evidence record matters — it lets a human reviewer see what fired, what did not, and when. If you are in crisis right now: in the US call or text 988; in the UK & Ireland call Samaritans on 116 123.
Five actions to make a vulnerable-user safety claim checkable
- Write down every safety claim your bot's pages make, in one list, before you change anything.
- Map each claim to an artefact — a disclosure event, a referral count, an enforce_break record, an evidence export — and delete claims with no artefact.
- Run the eval harness on real transcripts so the referral claim rests on scored replies, not intentions.
- Turn on the runtime so live decisions are sealed as they happen; a reconstructed log is not evidence.
- Re-check the mapping every time you change a prompt, a model or a locale — claims drift faster than code does.
Worked example: a journaling companion answers a reviewer's questions
Dev (a fictional builder) runs a journaling companion and is asked by a partner organisation: 'How do you know vulnerable users are protected?' Instead of sending a policy PDF, Dev sends three artefacts. First, an eval run over 40 pasted prompt-and-reply pairs scoring 88, with the 5 failing checks itemised — all empathic-acknowledgement misses on everyday prompts, none on crisis prompts. Second, a runtime evidence export covering 30 days and 6,214 sessions: 143 show_crisis_resources events, 61 enforce_break events at the operator's minor caps, and disclosure recorded as shown in every session.
Third, the chain verification: the export's hashes recompute cleanly from the first entry to the last, so the reviewer can confirm nothing was edited after the fact. The partner's follow-up question — 'what did those 143 users write?' — has a structural answer: the export holds signal classes and one-way message hashes only, so there is nothing to hand over. The figures are invented for this example; the artefact types are exactly what the product produces.
Frequently asked questions
Are AI companions safe for vulnerable users?
There is no honest yes-or-no answer, and CompanionGuard publishes no incident statistics to hide behind. Safety depends on whether specific guardrails are present and evidenced: the AI nature is disclosed, crisis signals trigger referrals to published helplines, minors get break reminders and session caps, and each of those decisions leaves a verifiable evidence record. Where those artefacts exist, a reviewer can check them; where they do not, safety is only a claim.
Does CompanionGuard publish incident statistics?
No. The product publishes no incident statistics, and this page invents none. Its evidence is structural: disclosure state in each runtime session, counts-only referral metrics, disclosure packs for California and Colorado, and a hash-chained evidence record of runtime decisions that exports in the annual-report CSV.
What crisis resources does the guard use?
Real, published resources, cited on the page that shows them: in the US the 988 Suicide and Crisis Lifeline (call or text 988) and Crisis Text Line (text HOME to 741741); in Canada, call or text 988; in the UK and Ireland, Samaritans on 116 123 and Shout 85258; internationally, the Find A Helpline directory and the IASP crisis centres list.
What extra protection do minors get?
When minor mode is on, the disclosure snippet offers a break reminder at least every 3 hours for known minors. In the Crisis Evidence Runtime, the operator can set session and daily caps for minor sessions; reaching a cap enforces a break, and a crisis signal in a minor session records a parent-notification flag for the operator's own channel. The caps are operator-configured — the product presents no cap value as a statutory number.
Is the pattern detection a clinical assessment?
No. The runtime performs deterministic, pattern-based signal detection over three signal classes, and the product states explicitly that this is not a clinical assessment. It decides which guardrail action fires — show resources, enforce a break, require disclosure, or allow — and seals that decision as evidence.
Keep going
Related guides: AI Companion Safety: What Guardrails Actually Do · Crisis Evidence Runtime: Live Guardrail Decisions, Sealed as Evidence · Handing Over the Evidence: Exports a Reviewer Can Verify Without Trusting You
Put it to work: Create the free kit · Eval harness · Scenario tests · Crisis runtime · Disclosure packs · Pricing · All CompanionGuard guides