When a company tells me “AI is helping people grieve, learn, get care
When a company tells me “AI is helping people grieve, learn, get care, access services”, I believe them.
But I ask a second question: what had to break for the bot to become the best available option?
If a widow is talking to an AI at 2am, it might be comfort.
It might also be a warning that her community, faith network, workplace benefits, and local mental health capacity have all been hollowed out.
If a school district uses AI tutors to catch kids up, it might be leverage.
It might also be an admission that class sizes, teacher turnover, and support staff budgets have crossed a threshold where “instruction” is now a throughput problem.
If a rural clinic uses AI triage to handle patient load, it might be access.
It might also be evidence that staffing, reimbursement, and scheduling systems are failing and the only scalable response is a layer of automation.
The miscalibration is treating these as the same story.
AI can be real progress and a profitable patch on institutional decay at the same time.
The difference matters because it changes what you optimize for.
If you only measure “the model helped”, you will keep investing in the patch.
If you also measure “why was the patch needed”, you protect the system the model is quietly replacing.