Embedded Support Agent

Overview

This case study covers an AI support agent embedded inside a product. The details here are generalized and describe the design, not the product.

Customers reach the agent through a chat widget embedded in the product, or by email. The AI answers first. Human agents work the same conversations from a dedicated dashboard and can take over at any time.

Most support questions are about the customer's own account: why something is not running, what is paused, what a limit is set to, how to change a setting. A generic bot can only recite documentation, so this assistant is wired to the product itself. It reads the signed-in customer's real account data, diagnoses the problem, and makes the change if the customer confirms it.

Registered AI tools

40+

Pending action expiry

15 min

Channels: widget, email

2

Max tool steps per run

5

Architecture

The widget is a script bundle that runs inside the host product. It never talks to the product's own backend directly. It talks to the support API, and the support API calls the product on the customer's behalf.

flowchart TB
    subgraph host ["Host product (customer signed in)"]
        W["Support widget"]
    end

    subgraph api ["Support API"]
        P["AI pipeline"]
        DB[("PostgreSQL<br/>conversations, runs, action requests")]
        RT["Socket.IO<br/>delivery only"]
    end

    D["Support dashboard<br/>inbox, copilot, corrections"]
    LLM["LLM<br/>per-workspace model"]
    MEM["Cited knowledge memory"]
    PD["Product account data"]

    W -- "opaque session token" --> P
    P --> LLM
    P -- "memory search" --> MEM
    P -- "team-scoped read tools" --> PD
    P -- "pending, confirmed, executed" --> DB
    P -.-> RT
    RT -.-> W
    D <--> P

The widget renders the conversation, choice buttons and confirmation cards inside the product the customer already uses. The support API is the source of truth for conversations, identity, the AI pipeline and realtime events, and both clients are thin. The agent dashboard is the human side, with an inbox, assignment, notes, a teammate-facing copilot and review of AI answers. Postgres holds conversations, messages, citations, AI runs and action requests, and Redis backs presence and the Socket.IO adapter.

How a ticket works

1. The customer opens the widget inside the product

   They are already signed in, so there is no second login and no form asking who they are or which account they mean. The host product hands the widget its current session and active team.

2. The server verifies who they are

   The widget sends that session to the support API, which checks it with the host product and mints its own short-lived opaque token. The server trusts nothing the widget reports about itself, such as user, team or role. Every later request and socket connection uses the opaque token.

3. The AI looks things up

   For a how-to question it searches the knowledge memory and answers with citations. For an account question it calls read tools scoped to the verified team. Each run is limited to a handful of tool steps.

4. It asks when a request is ambiguous

   If the customer says "pause that one" and several items match, the assistant sends a choice with buttons instead of guessing. On email the same choice arrives as numbered text.

5. It proposes a change, the customer confirms

   For anything that modifies the account, the assistant creates a pending action. The widget shows a confirmation card with the exact before and after values.

6. The server re-checks, then executes

   The server never runs the payload the Confirm button carries. It claims the stored action in a single atomic update that succeeds only if the action belongs to this conversation and customer, is still pending and has not expired. A double click or a stale card finds nothing to claim. Only then does the server call the product and record the result.

7. Or it hands off

   Disputes, security problems, feature requests and frustrated customers go to a person. The handoff carries a reason, a topic and a note, and the assigned teammate is notified.

Memory and the customer's own data

The assistant works from two separate sources.

Knowledge memory answers how the product works. It is a hosted memory layer queried with hybrid search, which returns chunks and distilled memories along with their evidence. Each answer stores its citations against the message, so an admin can later see which memory produced it. Retrieval needed tuning on the real corpus: the relevance scores in that workspace sit far lower than a typical guide suggests, and the cutoff was re-fit until results came back.

Account data answers what is true for this customer right now, and the model never keeps it. Each question goes to a read tool that fetches fresh data for the verified team, parses the response against a schema and returns a typed result. If the team cannot be resolved, the tool says so instead of guessing.

Conversation history, internal notes and handoff context sit alongside these, so a teammate who takes over sees what the AI saw and did.

The tools

The pipeline registers more than forty tools in four groups, and the group decides how a tool may behave.

Group	What it covers	Behavior
Knowledge	Search the cited memory	Runs directly
Read and diagnose	The customer's resources and their settings, usage and account status	Runs directly, team-scoped, schema-parsed
Change	Pause or resume resources, edit their configuration and limits	Proposed as a pending action, executed only after confirmation
Conversation	Hand off to a human, close the conversation, leave an internal note	Handoff and close follow explicit rules; notes are internal only

A few rules sit on top of that:

• If the customer asks for something already in the requested state, such as a resource that is already paused, the assistant says so and skips the confirmation.
• A confirmation that is never clicked expires after 15 minutes, and an execution that stalls past two minutes is marked failed instead of hanging.
• Resuming a paused resource checks for active alerts first. If one is blocking, the assistant explains the blocker instead of attempting the resume.
• Settings changes show exact before and after values on the card, and those values are stored with the action for the audit trail.
• Tool failures return typed status objects, so a failing downstream call becomes a clear message instead of a crashed run.

Keeping the pipeline safe

• Each run is keyed to the customer message that triggered it, so a retry or a duplicate event cannot answer twice.
• Tool steps per run are capped, and a per-conversation rate limit catches reopen loops.
• When a teammate takes over, the in-flight AI run is cancelled and no AI message is written.
• If a run fails, the customer gets a plain message that a teammate will follow up instead of silence.
• Logs record action ids, targets, status, duration and error codes. Prompt bodies, message bodies, tokens and secrets are never logged.
• Every tool call is recorded against its run, and every action request records when it was confirmed, what ran and how it ended.

Closing the loop on answers

Admins can rate any AI or copilot answer as helpful or not. A negative rating opens a correction case that keeps a snapshot of the question, the answer and the memories it cited. An admin then writes a fix by hand or has one drafted, reviews it and approves it. Only an approved proposal changes the memory, and the apply step re-reads the current content first so it does not overwrite something that changed in the meantime. A wrong answer ends up as a reviewed correction to its source.

Outcome

The result is a working support platform where the assistant is part of the product. A customer can ask about their own account and get an answer based on their real data, ask for a change and approve it in one click, and reach a human with full context when the assistant should step back. The model cannot choose whose account it acts on. No change runs without a stored, confirmed request, and email cannot modify an account.

The tool layer, action lifecycle and pipeline each have automated tests. I am not publishing ticket-volume or resolution-rate figures here, because those figures are not mine to publish.