Adding a floating ChatGPT widget to your homepage is a major operational risk. Typically, these widgets are basic API wrappers fed a static FAQ document. While they seem capable of handling basic customer support, giving an unstructured conversational model direct contact with clients introduces significant liabilities.
The Unpredictability of Unstructured LLMs
Research from UC Berkeley on neural network reliability emphasizes that LLMs operate purely on word probability sequences; they have no concept of policy constraints, pricing structures, or contract law. When queried with leading questions, unstructured chatbots can easily fabricate discounts, misrepresent service terms, or confirm bookings that violate company policies.
The Air Canada Legal Precedent
In a landmark 2024 ruling, a passenger successfully sued Air Canada after their support chatbot hallucinated a bereavement discount policy that contradicted the airline's official guidelines. Air Canada argued that the chatbot was a separate legal entity and the passenger should have verified the terms on the main website. The court rejected this defense, holding the airline liable for all representations made by its automated systems. This established a clear precedent: businesses are legally bound by their AI's responses.
If your AI has the authority to converse with clients, it has the authority to legally bind your business to terms.
Designing Bounded JSON Operators
To eliminate this risk, AI must be removed from customer-facing interfaces and placed behind bounded schema filters. At CASRA, we structure AI as background Operators. Instead of chatting, the AI reads incoming emails, validates the data, and outputs structured JSON metadata. Below is a TypeScript interface defining a validation schema that restricts the AI to returning verified client actions, preventing free-text hallucinations:
interface QualifiedLeadIntake {
clientName: string;
clientEmail: string;
clientPhone: string;
requestedService: "maintenance" | "repair" | "installation";
requestedZipCode: string;
hasConfirmedDeposit: boolean;
rawRequestSummary: string; // validated and cleaned by database triggers
}By restricting AI to backend data extraction and routing, you get the speed of LLM automation without exposing your client relationships to hallucination risks.