How an AI Website Chatbot Actually Works (and What to Check Before You Install One)

Almost every AI chatbot on the market is sold with the same four words: trained on your data. It is a comforting phrase and it is usually false. Nothing about your content is trained in the sense the word normally carries, and the difference is not pedantic — it decides how much of your content the bot can actually use, how fast an edit takes effect, and whether the thing can invent a refund policy you never wrote. This is what really happens between a visitor's question and the answer that appears on your site.

“Trained on your data” almost never means trained

Training a language model means changing its weights. It costs real money, takes hours at minimum, and no vendor is doing it per customer for the price of a website widget. What virtually every website chatbot does instead is put your content in front of the model at question time, inside the prompt. The model is unchanged; the context is what differs.

There are two common shapes of this. The first is retrieval: your content is chopped into chunks, converted to vectors, and the chunks closest to the visitor's question are fetched and pasted into the prompt. That is what people mean by RAG. The second is simpler — the whole knowledge base goes into the prompt on every request, with no retrieval step at all.

ChatKit is the second kind, and we say so on the product page in those words. Your entries are handed to the model with every single question, under instructions that say: answer only from this, and if the answer is not here, say the owner's fallback line. Nothing is embedded, indexed or vectorised anywhere in the codebase.

The ceiling is the number to ask for

Whichever shape a product uses, there is a limit on how much of your content reaches the model, and that limit is the single most useful number a vendor can give you. Ours is published on the product page and imported from the same module the chat route builds its prompt from, so it cannot drift: the model sees the first 40 entries in your sort order, up to 32,000 characters each and 60,000 characters across the whole base. Past that, entries are shortened or dropped, and the dashboard says which.

That is room for a full product catalogue and a policy set. It is not room for everything you have ever published, and a vendor who will not tell you where their ceiling sits has one anyway.

Categories and sort order are the real controls

Once you know your content is pasted into a prompt, two settings that look cosmetic turn out to be the ones that matter. Categories decide how the content is arranged before the model reads it; sort order decides who makes the cut when you are near the ceiling.

ChatKit groups entries by category before building the prompt, so everything you filed under policies arrives together, in a predictable place, on every question. Nine categories cover what a small business actually has to answer:

• general — hours, location, the basics
• about — who you are and what you do
• products — what you sell, specs, stock rules
• faq — the questions you answer daily
• policies — shipping, returns, refunds, cancellation
• tone — how the chatbot should sound
• pricing — rates, packages, what is included
• support — how to get help or book you
• other — anything that does not fit above

Sort order is the priority list. Put your highest-value answers first and keep each entry tight and single-topic: one entry per subject beats one long entry covering five, because a single sprawling entry burns the character budget four precise ones would have used better.

What good content looks like, by business type

The hardest part of running a website chatbot is not installing it. It is writing the thirty paragraphs it needs to be useful. Four shapes cover most businesses, described in the categories you would actually fill in.

An e-commerce store

Shipping, returns and refunds under policies. Sizing, materials and stock rules under products. Order tracking under faq. The three questions that fill your inbox then get answered at 11pm, in the buyer's language, on the product page where the doubt appeared rather than two emails later.

A service business — clinic, studio, trades, restaurant

Hours, location and parking under general. The service menu under pricing. How to book and the cancellation window under support. You get fewer “are you open on Sunday?” calls, and the fallback rate quietly tells you which detail you never got round to publishing.

A SaaS or app with docs

What the product does under products. Setup, integrations and limits under faq. Where to get help under support. Repetitive pre-sales questions get deflected without adding a help-desk seat, and the keyword list shows which part of the docs people cannot find.

An agency or freelancer running client sites

One chatbot per client site, each with its own knowledge base, colour and domain list — a support layer you can hand over at the end of a project. Check how the vendor bills this. Separate accounts per client gets expensive fast; a single account with per-chatbot slots and one shared balance does not.

Get it: The free /build-kb skill

The widget: a Shadow DOM is worth more than it sounds

A chat widget is a third-party script running inside your page, and two things can go wrong: it can break your site, or your site can break it. The fix in both directions is the Shadow DOM. The widget renders inside a boundary whose host rule resets all inherited styling, so your stylesheet cannot reach into the panel and the widget's styles cannot leak out onto your buttons or type scale.

:host { all: initial }

That one line is the whole isolation guarantee, and it is worth confirming a vendor has it. Without it a widget inherits whatever your theme does to buttons and inputs, and the results range from a squashed panel to an unreadable one.

The other widget details worth checking before you commit, because none of them are changeable afterwards: whether replies stream token by token or arrive as a block after a spinner; whether the thread survives a visitor clicking from your pricing page to your contact page (session storage is the usual mechanism); whether mobile gets a real layout or a shrunk desktop one; and whether the vendor's badge can be removed. Ours cannot — there is no white-label option in ChatKit at any price, which is a reason to stop reading now rather than a detail to discover later.

One more that surprises people: answer length is usually a fixed instruction, not a setting. ChatKit is told to answer in two or three sentences with no bullet lists and no headings, with a hard ceiling on output length behind that. On a support widget that is what visitors want. If you need a bot that emits long formatted documents, a chat bubble is the wrong shape of product.

Can somebody copy my snippet and spend my money?

This is the question that decides whether a self-service signup happens, and it deserves a named mechanism rather than a trust badge. Your embed snippet is visible in your page source to anyone who looks, and if billing is per message, a copied snippet is a copied invoice.

The control is a domain allow-list: you name the hostnames the chatbot may run on and anything else is refused. Two details separate a real one from a decorative one. First, enforcement has to be a server-side refusal on both the config load and the message send, not just CORS headers — CORS is a browser courtesy, and a script does not have to honour it. Second, find out what an empty list does. In ChatKit an empty list allows any origin, deliberately, so the widget works on a staging site before you have settled your domains — which means filling the list in is the last step before going live rather than one you can skip.

Rate limiting is the second half. Limits per visitor IP and per chatbot, per minute, evaluated before the chatbot record, your knowledge base or the model are touched, mean traffic that is going to be blocked never reaches anything that costs money. It is also worth asking what the limiter does when its own infrastructure fails. Ours fails open on purpose, because a database hiccup should not take a paying customer's chatbot offline, and every request that gets past it still has to clear the credit check behind it.

What happens when you run out

Every metered product has a zero, and the behaviour at zero is rarely on the pricing page. There are three plausible designs and only one of them is acceptable: the widget stops rendering and visitors see nothing; the widget renders and shows an error to your customers; or the account is suspended and something gets deleted.

The first is the right answer. On a ChatKit site the widget simply does not appear — no broken bubble, no error message aimed at your customers — and nothing is destroyed: knowledge base, styling, unlocks, conversations and transcripts stay exactly as they were, and adding credits brings the widget straight back. The warning comes earlier, in the dashboard, when five messages remain.

What gets stored, and who can read it

A chatbot that answers your customers is also a system of record for what your customers asked. Ours stores every visitor message and every reply as a full transcript, together with a randomly generated visitor id that is not tied to any identity, the IP address the conversation started from, and the URL of the page where the chat started. That retention is precisely what makes transcripts, exports, fallback rate and keyword analytics possible — you cannot have the analytics without the storage, and a vendor claiming both is worth a second look.

Source: Our privacy policy

Two things to establish before you install anything: whether you can export your own content and your conversations, and in what format. A knowledge base you cannot get back out is a knowledge base you rent. Ours exports as dated Markdown grouped by category, and that export is not behind a paywall.

It is also worth knowing who rates answers. In ChatKit thumbs-up and thumbs-down belong to the owner, in the dashboard — visitors never see a rating control. That matters for what the self-improvement features learn from: your judgement, not strangers'.

Self-improvement features, and the word “auto”

Most chatbot products now ship some form of learning loop. The question to ask is whether it writes. A feature that reads your badly rated replies, pairs each with the question that triggered it and drafts new knowledge-base entries is useful. A feature that then publishes those drafts to your live chatbot without asking is a liability with a friendly name.

Ours drafts up to five entries per run and every one is a pending suggestion: accept promotes it to a real entry, dismiss archives it, and where the correct answer is unknowable the draft writes a placeholder instead of inventing one. A suggestion that trips the injection checks needs a second, explicit confirmation before it can be accepted at all. Nothing reaches your visitors without a human clicking.

The seven questions worth asking any vendor

If you take nothing else from this, take the checklist. None of these are trick questions and all of them have a short factual answer, which is itself the test — a vendor who answers them with adjectives is telling you something.

• Is my content retrieved into the prompt, or pasted in whole? Neither is wrong; not knowing is.
• How much of my knowledge base actually reaches the model, in entries and characters — and what happens when I exceed it?
• What does the bot say when the answer is not in my content, and can I write that sentence myself?
• Is the widget style-isolated from my site, in both directions?
• Can someone copy my snippet onto another domain and spend my balance — and is the allow-list enforced on the server, not just in CORS?
• What happens at zero: does the widget disappear, show an error to my customers, or does something get deleted?
• What is stored, for how long, and can I export my content and my conversations?

What this is all for

A website chatbot is not a support team and it is not an AI strategy. It is a way of answering the same six questions at 11pm, in whatever language the visitor speaks, on the page where the doubt appeared — and of finding out, from the fallback rate, which seventh question you have never published an answer to. Judged against that, the mechanism questions above are not technical trivia. They are the difference between a widget that deflects work and one that quietly generates it.

If you would rather see the answers to all seven stated on a product page than extracted from a sales call, ours are on the ChatKit page, with the numbers imported from the same files the code runs on.

See: ChatKit — an AI chatbot for your website