Skip to main content
The Agent component generally takes a prompt from a human or agent, and uses an LLM to generate a response. However, there are cases where you want to generate the reply from a human acting as an agent, such as for customer support. For full code, check out chat/human.ts

Saving a user message without generating a reply

You can save a message from a user without generating a reply by using the saveMessage function.

Saving a message from a human as an agent

Similarly, you can save a message from a human as an agent in the same way, using message for the role and content and agentName for attribution. To render a standalone operator reply separately from the previous agent turn, pass order: "next":
order: "next" allocates the next order atomically. Passing a number places the reply at that exact order. An unused numeric order starts at stepOrder: 0; an existing order appends at its next stepOrder.

Storing additional metadata about human agents

You can store additional metadata about human agents by using the saveMessage function, and adding the metadata field.

Deciding who responds next

You can choose whether the LLM or human responds next in a few ways:
  1. Explicitly store in the database whether the user or LLM is assigned to the thread.
  2. Using a call to a cheap and fast LLM to decide if the user question requires a human response.
  3. Using vector embeddings of the user question and message history to make the decision, based on a corpus of sample questions and what questions are better handled by humans.
  4. Have the LLM generate an object response that includes a field indicating whether the user question requires a human response.
  5. Providing a tool to the LLM to decide if the user question requires a human response. The human response is then the tool response message.

Human responses as tool calls

You can have the LLM generate a tool call to a human agent to provide context to answer the user question by providing a tool that doesn’t have a handler. Note: this generally happens when the LLM still intends to answer the question, but needs human intervention to do so, such as confirmation of a fact.