An agency owner spends a few weekends building an AI prompt. It reads incoming emails, pulls out the policy number and the coverage requested, and drafts a certificate of insurance for his CSRs to send. He is proud of it, and he should be, because it works. Before rolling it out to the team he does the sensible thing and asks the assistant that helped him build it whether the setup is solid. Yes, it says. Enterprise grade. Bulletproof guardrails.
It would say that.
It has no way to know. It cannot see his mailboxes, his laptops, or his staff. And the guardrails it just graded are the ones it wrote for him twenty minutes earlier. Asking a model to audit its own instructions is asking a student to mark their own exam, with the added problem that the student is trained to be agreeable.
That moment is why we are writing this.
Before InsuraMate, our team worked inside a company certified to ISO 9001:2015 and ISO 27001. Quality management and information security were the daily job. You learn to ask the same dull questions about everything: who has access, where is the record, what happens when this fails, who patches it. We bring those questions to AI because most of the excitement around prompts skips them.
A prompt that works on your screen is not a system your agency can run on. The gap between the two is not intelligence. The model is clever enough. The gap is infrastructure, and infrastructure is the part nobody sees until it is missing.
One person, one screen, one run
A prompt runs once, for the person typing it, in the window where they typed it. Share it with five people and you have five copies, pasted into five accounts. Within a month, Maria has tightened the wording for her commercial clients, Tom has added a line because a carrier complained, and the new hire is running a version from a shared document nobody updated.
Ask which version is current and there is no answer. There are five, all called “the certificate prompt”, none of them the same, and no record of which one produced which document.
The day someone leaves
Your account manager hands in her notice on a Tuesday. She was the one who connected her mailbox to the prompt setup so it could read incoming requests. What does she still have on Thursday?
Whatever she had on Monday.
Connecting five mailboxes safely means OAuth, the mechanism where each person grants access through their own email provider’s login screen, so nobody hands over a password. What comes back is a token, a key to that one mailbox, stored encrypted on a server rather than in a text file or a browser extension. And on the morning someone resigns, you revoke that key with one action and it stops working everywhere.
A prompt has none of that. The access lives wherever she set it up, on her machine, under her password, and your offboarding checklist cannot reach it. If she pasted client emails into a personal account to run the prompt, those emails are still there.
Where do the logs go
A wrong certificate reaches a client. It names the wrong additional insured, or it shows limits from last year’s policy, and the client’s general contractor notices before you do. Now you need three facts, fast: what was sent, who sent it, and when. Then you need to prove it.
If the work happened in a chat window, you have a scrollback on one laptop, editable by anyone with the login, and not tied to the email that went out. That is not an audit trail. It is a memory. When your errors and omissions carrier asks how this happened, and they will, “we have a prompt” is not an answer they can file.
A system writes that audit trail itself: every input, every output, every check and its result, with a timestamp and a name, where nobody in the office can quietly tidy it up.
Responsibility does not move because software was involved. The agency is responsible either way. The only open question is whether you can show what happened.
Who patches the laptop
A prompt lives on a laptop. In a small agency that is the owner’s old laptop, a CSR’s home PC, and a producer’s tablet, each with its own idea of when to update. One of them will go six months without a security update, and that is the machine holding six months of client emails, policy numbers, and whatever got pasted into the chat window to make the prompt work.
Software your agency runs lives on a server that somebody is responsible for keeping current. When a security fix comes out, one person applies it, once, and everyone is covered by lunchtime. There is no version of that for a prompt. There is just hoping that five machines you do not control are all fine.
The rule the model can ignore
This is the one that confident answer got most wrong.
When you write “always verify the policy number before sending” into a prompt, you have made a request. You have not made a rule. The model reads it as text, alongside all the other text, and most of the time it will comply. Most of the time is not a standard an insurance agency can operate on. A model that can be instructed can be instructed otherwise, by a confusingly worded email, by a long conversation that pushed the instruction out of focus, by nothing in particular on a given Tuesday. There is no mechanism that makes it obey. There is only a tendency.
The guardrails have to live outside the model, in code that has no opinion.
Here is how we built it. The model is allowed to do exactly one thing: extract data from the source documents. Policy number, named insured, effective dates, limits, and nothing else. It does not decide whether the certificate is correct, and it is never asked whether its own extraction looks right. A certificate that needs a new holder added or special additional insured wording is never attempted at all. It goes straight to a person, because the system knows what it is not built to decide.
Then deterministic code takes over. Deterministic means the same input always gives the same output, with no judgement involved, like a spreadsheet formula. That code compares every extracted field against the policy record: policy number character for character, dates as dates, limits as numbers. If a single field does not match, the job stops and goes to a person. Not a warning, not a confidence score, a full stop and a human. The model never sees the result of that check and never gets a vote on it.
We have issued over three hundred certificates this way with zero errors. The model is good, but that is not the reason. The reason is that the model was never allowed to judge its own work. Something that cannot be persuaded did the judging.
What AI prompts are good for in an agency
We are not telling you to stop using these tools. For drafting a difficult email to a carrier, for thinking through how to explain a coverage gap to a client, a prompt in a chat window is genuinely useful for one person doing their own work and reading every output. We use them every day.
It is not a system for a team of five handling other people’s insurance data.
His prompt did work. Yours does too. It worked once, on your screen, while you were watching. That is not the same as working for five people, every day, when nobody is looking.
InsuraMate is that infrastructure, built for independent agencies.
InsuraMate AI