The hallucination issue started early and continued. In one of the more notable instances, the chatbot recommended contacting a network of law school alumni when a user inquired about where to find legal assistance in Alaska. Alaska does not have a law school. The system confidently, helpfully, and incorrectly pulled information from sources other than its carefully selected knowledge base of court probate documents, even though it was not supposed to. Tom Martin, a lawyer and law professor who created AVA through his business LawDroid, has put a lot of effort into limiting the chatbot’s access to pertinent court documents. However, limiting what a model will say is a truly challenging engineering problem, not a setting you can change, as anyone who has worked with large language models is aware.

The testing regimen provides a unique account of the project’s origins. In order to assess the accuracy and usefulness of AVA, the team first created a set of 91 questions that covered a variety of probate topics, such as which forms to use in particular circumstances. Given the stakes and the requirement for human review at every stage, the 91-question test proved to be too time-consuming to administer and assess. Eventually, it was narrowed down to 16 questions, some of which addressed common questions, some of which covered subjects on which AVA had previously provided incorrect answers, and some of which covered more complicated situations. Methodological details that have not been fully disclosed will determine whether that reduction is a retreat from rigor or a sensible prioritization.

The particular human texture that this project was attempting to serve is difficult to ignore. AVA’s initial iterations were said to be overly sympathetic, beginning with condolences that user testing showed were unwelcome. People didn’t want an AI chatbot to show sympathy when they were trying to process a bank account or transfer a car title after losing a loved one. They sought explanations. The difference between creating AI in theory and using it to help people in real-world challenging situations is captured by that little detail. The condolences were taken down. However, editing away the deeper issue—getting the answers correct—proved to be much more difficult.

Here, the larger context is important. According to a recent Deloitte report, less than 6% of local government practitioners currently view AI as a priority tool for service delivery. This can be explained by the experience of the Alaskan court system. It takes time, resources, and human oversight to even partially close the gap between using AI in a consumer product and using it in a public legal context, where the cost of an incorrect response is not a slightly irritated user but rather a harmed litigant who might miss a filing deadline or submit the wrong form. “It was just so very labor-intensive to do this,” Marz said to NBC News, “despite all the buzz about generative AI and everybody saying this is going to revolutionize self-help and democratize access to the courts.”

Although government technology projects typically have aspirational launch dates, AVA was supposed to launch in late January 2026. According to Marz, the team’s objectives had changed to something more constrained and verifiable rather than attempting to completely replicate what a human facilitator could accomplish. That recalibration isn’t exactly a failure. It is what responsible organizations do when the technology doesn’t perform as advertised. The goal was justified. The challenge was genuine.

The post Alaska Built an AI Chatbot to Help Grieving Families Navigate Probate. It Made Things Worse appeared first on Creative Learning Guild.