How to add AI to an existing app or website
Most valuable AI work is not a new product. It is one well-chosen feature added to something people already use. Here is the process we follow.
1. Start with a job, not a model
"Add AI" is not a feature. Good AI features remove a specific, repeated task:
- Answering the same customer questions from your help docs
- Searching your content by meaning, not exact keywords
- Summarising long records, tickets or documents
- Pulling structured data out of emails, invoices or forms
- Drafting replies or descriptions a person then approves
Pick one, and write down how you will know it worked: tickets deflected, minutes saved, searches that now find a result.
2. Decide where the knowledge comes from
A general language model knows nothing about your business. To answer from your own content, use retrieval-augmented generation (RAG): your documents are split into chunks, indexed by meaning, and the most relevant chunks are given to the model with each question. The model answers from your content and can cite it.
RAG is usually better than fine-tuning for business content: it is cheaper, updates the moment your documents change, and lets you show sources.
3. Treat privacy as a design input
- Know exactly which data leaves your system and which provider receives it.
- Strip or mask personal data that the model does not need.
- Choose providers and regions that fit your obligations. In Europe, that means the GDPR, and increasingly the EU AI Act's transparency rules.
- Tell users when they are talking to an AI.
4. Pick the model last
Choose by the task. Large frontier models for reasoning and writing; small, fast models for classification and extraction at volume. Most products use more than one. Keep the provider behind your own interface so you can switch as prices and quality change.
5. Build guardrails in from the start
- Grounding: instruct the model to answer only from retrieved content, and to say "I don't know" otherwise.
- Human in the loop: anything that sends, pays or deletes should be confirmed by a person.
- Limits: rate limits and spending caps per user, so one script cannot run up a bill.
6. Test with real questions
Collect 50–100 real questions from your support inbox or search logs, with the right answers. Run every change against that set before it ships. This turns "it seems better" into a number, and catches regressions when you switch models.
7. Launch small, then widen
Release to a slice of users, watch the answers and costs, and fix the gaps your test set missed. Expect the first version to handle the common 70–80% of cases well; the long tail comes from iteration.
What it costs to run
Running cost scales with usage: how many requests, and how much text goes in and out of each. Retrieval keeps prompts short, caching repeated context cuts cost further, and routing simple requests to smaller models does the rest. Estimate from your real traffic before you commit.