Skip to content
Services

AI features that ship

AI in a product, not a demo. The hard part is never the first plausible output — it is knowing whether a change made things better, and keeping the feature usable when the model is slow, wrong or down.

When teams call us

If more than one of these is true, this is usually the right starting point.

Evaluation harness in weeks 1–2; the feature follows.

  • A prototype produces plausible answers nobody can verify
  • You cannot tell whether a prompt change helped or hurt
  • The feature works until the model has a bad day

What it covers

  • An evaluation harness built before the feature
  • Labelled golden sets and per-query scoring
  • Regression gates in CI, not in review
  • Latency budgets, fallbacks and graceful degradation
  • Citations and provenance where the answer matters

What you keep

  • The evaluation harness and its corpus
  • A retrieval or agent pipeline you can tune
  • Model-routing configuration you control
How it runs

How we approach it

The sequence this work follows, and why it is in that order.

  1. Harness before feature

    The scoring rig is built first, against a labelled set drawn from your real queries. Without it every prompt change is an opinion, and the team argues from screenshots.

  2. Establish the floor

    We measure the simplest thing that could work — often plain retrieval with no model in the loop. Plenty of features never need to beat it, and knowing that early saves the budget.

  3. Iterate against the score

    Retrieval, prompting and routing tuned as separate variables, each scored. Changes that do not move the number do not merge, however good the demo looked.

  4. Design for the bad day

    Latency budgets, fallbacks, and a defined behaviour when the model is slow, wrong or unavailable. The feature has to degrade into something useful rather than a spinner.

What we build with

The stack for this work

Chosen per engagement, not per fashion. If your team already runs something that works, we use that instead.

  • Models

    • Claude
    • GPT
    • Open-weight via vLLM
  • Retrieval

    • pgvector
    • Hybrid search
    • Rerankers
  • Evaluation

    • Golden sets
    • LLM-as-judge
    • Human calibration
  • Serving

    • Streaming
    • Fallback routing
    • Semantic caching
Before you ask

Questions we get about AI features that ship

Let’s talk

Need AI features that ship?

One week, and you keep the baseline and the plan either way.