# Welcome to Wayfound Wayfound is the world's first AI Agent Supervisor platform. Our platform helps businesses supervise, evaluate, and optimize the performance of their AI Agents through a single-pane observability dashboard. Wayfound's AI Supervisor captures and analyzes every interaction of the agents you connect to the platform. The AI Supervisor assesses their performance and suggests improvements in near real-time. In addition to driving Agent behavior optimization, Wayfound helps you align your Agents with your company's values and brand messaging, ensuring they deliver consistent, business-relevant outcomes.
## The Wayfound platform Wayfound is organized into five tabs: [**Supervisor**](/supervisor/performance) provides a global view of your network of Agents, near real-time Agent performance reviews, access to guidelines that apply across all Agents, and an interface for facilitating meetings between agents [**Agents**](/agents/the-agents-page) allows you to connect and view agents [**Sessions**](/sessions/recordings) provides a record of Agent activity and suggestions for improving individual Agent performance [**Visitors**](/visitors) keeps track of users' interactions with Agents on the platform [**Settings**](/settings/organizations) allows you to manage users, organizations, actions, and integrations. The settings tab is only visible to admin users # Key Concepts Wayfound is your entry into the world of AI agent supervision. Here are the key concepts you need to get the most out of the platform. ### Agents Wayfound supports specialized agents powered by the latest large language models. Agents are designed to accomplish specific, clearly defined tasks. They can interact with users through text, images, and video formats. They can communicate with other agents for help with answering specific questions, and they can take action or access information in third-party systems. Agents are end-to-end, meaning that they can both interact with users and perform workflows. You can connect existing agents built on platforms like LangChain or CrewAI. ### Agent Networks The best agents are specialized to perform a specific task. Agent networks leverage multiple specialized agents that communicate and collaborate to handle complex tasks. Each agent focuses on its specific expertise while working within the broader network. See [Supervise Multi-Agent Systems](/applications/supervise-multi-agent-systems) ### AI Supervisor The AI Supervisor provides centralized oversight of your agent networks, offering agent performance analysis, agent relationship mapping, and network-wide behavior controls. Learn more in the Supervisor [Overview](/supervisor/overview) page. ### Performance The AI Supervisor continuously evaluates agent performance through user satisfaction metrics, knowledge gap analysis, guideline compliance monitoring, and tool utilization success rates. Learn more in the [Performance](/supervisor/performance) page. ### Guidelines Guidelines inform how the AI Manager evaluates agent performance. Wayfound offers two levels of guidelines: [Guidelines](/agents/guidelines) for individual agents and [Global Guidelines](/supervisor/global-guidelines) guidelines that apply across all agents in the organization. ### Sessions Sessions provide monitoring capabilities through [Recordings](/sessions/recordings) of agent-user interactions. ### Evaluation Rubric For each agent, the AI Supervisor maintains an **evaluation rubric** — a concise grading guide it distills from your guidelines and from the feedback you've given over time. The rubric captures how your organization interprets each guideline in practice, and every session is scored against it. This makes grading more consistent and increasingly tailored to your business. Learn more in [How the Supervisor Learns](/supervisor/how-the-supervisor-learns). ### Potential Issues **Potential issues** are recurring behaviors the AI Supervisor notices that aren't yet covered by any of your guidelines. You can promote a potential issue to a new guideline, confirm it as expected behavior, or dismiss it as a false positive. See [Potential Issues](/supervisor/potential-issues). ### Open Questions When the AI Supervisor is uncertain how to judge a behavior, it raises an **open question** for your team rather than guessing. Your answer teaches the Supervisor and refines how it evaluates future sessions. See [Open Questions](/supervisor/open-questions). # Getting Started ## **Planning and Building Your Agent** Before building your agent, carefully plan its purpose and requirements: 1. **Define your use case**: Focus on specific, single-task applications. Keep the scope narrow and let other agents or humans handle different tasks in a larger network of intelligence. 2. **Set clear objectives**: Identify the desired outcomes from agent interactions. 3. **Plan capabilities**: Determine the knowledge, behavior, and tools your agent needs to complete the desired tasks and how to equip the agent with them. 4. **Build your agent**: Use one of many available agent builders like LangChain to build your agent. Once your agent is built, you can connect it to Wayfound. ## Connecting your agents to Wayfound 1. **Add your agent to the platform**: Navigate to [The Agents Page](/agents/the-agents-page), click **+ Supervisor Agent**, and give it a name, role and goal. 1. The **Name** is how the supervisor is referenced throughout Wayfound 2. The **Role** describes the context in which the agent is running in your product such as how it interacts with users or describing where it sits within an agent workflow. The Wayfound Supervisor uses this context as part of the session analysis process. 3. The **Goal** describes the objective of the agent for each session run. This should be a verifiable goal. It is important that the goal has a clear success criteria that can be evaluated with the session data being provided to Wayfound. 2. **Create an API Key**. Users with admin status can create an API key on the [API](/implementation/api) page. 3. **Integrate your agent**: Add Wayfound to your external agent using the [Connecting Agents](/agents/connecting-agents) ## Agent Success Criteria 1. **Guidelines** describe the success criteria for how your agents accomplish their **Goal.** See the agent [Guidelines](/agents/guidelines) page for more information. 2. For Guidelines that apply to all agents see the [Global Guidelines](/supervisor/global-guidelines)page. 3. The Supervisor will use the Guideline definitions to roll up any violations of the guidelines across all sessions in it's nightly analysis. ## Supervisor Alignment 1. It is important to test your agent's Role, Goal, and Guideline definitions to ensure that the Wayfound Supervisor understands your guideline intent with your own session data. 2. We suggest that you use [Test Mode](/agents/test-mode) to upload a representative set of sessions that demonstrate both successful and unsuccessful session sessions. We suggest starting adding one guideline at a time while going through the define/test iteration process. 3. As part of the Test Mode analysis you will be setting your expected outcome of guidelines and be able to test multiple iteration runs against your test sessions to ensure that the agent's role, goal, and guidelines produce expected and reliable results. This is a sandboxed environment that will not impact your production environment. **Note:** This process is a fantastic way to start any new agent project. Even if you are early in the technical implementation of your agent, you can generate sample session data, build guidelines, and label each session/guidleine permutation with your expected output. You can think of this as "test driven development" for AI agent building where the business owner of the AI agent sets the success criteria that empowers both the business owner and engineering team with a common goal to work towards. ## Supervise your agent Once your agent is connected you can use the AI Supervisor to monitor and improve its performance on the [Performance](/supervisor/performance) page. 1. **Knowledge:** check knowledge gaps and fill them if necessary by adding missing information 2. **User Satisfaction:** Check user satisfaction scores and investigate any low ratings 3. **Issue management:** Examine potential issues raised by the AI Supervisor 4. **Follow-up analysis:** Delve deeper into agent performance by chatting with the AI Supervisor. You can start with questions recommended by Wayfound. ## **Using advanced features** Once your agents are running and managed, you can leverage Wayfound's full capabilities such as: 1. **Interaction-level analysis**: Use Recordings to review specific agent interactions. 2. **Engagement tracking**: Monitor Link clicks during agent interactions. 3. **Reports**: Set up agent reports to produce organizational insights. # Performance The Performance tab provides a comprehensive overview of your individual agents' performance, offering insights and areas for improvement. It is powered by Wayfound's AI Supervisor, which continually monitors your active agents. The Wayfound AI Supervisor is powered by state-of-the-art LLMs. This view is designed to help you quickly understand the strengths and weaknesses of your agents and identify directions for improvement.
The Performance tab provides insights for all agents in the organization with at least 5 user interactions. The performance of a given agent can be displayed by clicking or hidden by clicking next to each agent's name. ## **Assessment Outcomes** The AI Supervisor evaluates the performance of your agents by reading and rating its interactions with users. The overall results are displayed on the Performance tab to the right of each agent's name. Possible outcomes include: **Good to go:** The agent is meeting expectations in its interactions. However, the AI Supervisor can still raise potential issues and provide suggestions for improvement. **Needs review:** The agent's performance is satisfactory, but there are areas that require closer attention and potential improvement. The AI Supervisor will flag an agent as "Needs review" when any of the following are triggered: * User Rating 1-3 * Negative sentiment * Agent Goal was not successful * 1 or 2 or Knowledge Gaps * [Guidelines](/agents/guidelines) Violation where user specifies a "Needs review" priority **Needs attention:** the agent is facing significant challenges or issues that require immediate focus and resolution. The AI Supervisor will flag an agent as "Needs attention" when any of the following are triggered: * Action Failure * 3+ Knowledge Gaps * [Guidelines](/agents/guidelines) Violation where the user specifies a "Needs attention" priority More information can be accessed using the link to the [Suggestions](/sessions/suggestions) tab, which provides specific recommendations for improvement. ## **Assessment components** During its review of agent performance, the AI Supervisor identifies each agent's knowledge gaps, considers user satisfaction scores, and searches transcripts for specific issues. These components are all displayed in the tab. ### **User Satisfaction** Wayfound collects ratings given by users at the end of their interactions with agents. It summarizes them here with an overall average score and a distribution of scores. Click on the summary of user satisfaction scores to view a detailed breakdown of sessions by score. For each session, click to view the corresponding transcript.
### **Knowledge Gaps:** As part of its assessment, the AI Supervisor identifies the knowledge gaps that emerged in the agent's recent interactions. The Performance tab displays a graph of recordings that indicate knowledge gaps as a share of total recordings. Clicking the graph displays specific knowledge gaps on the right-hand side of the page:
Click each theme to expand with more details with summaries of relevant sessions. For each session, click to view the corresponding transcript. ### **Guideline Violations:** The AI Supervisor assesses the performance of each agent according to the custom [Guidelines](/agents/guidelines) you set for it. The performance tab displays the count and percentage of recordings where the agent was compliant vs in violation of the guidelines. Clicking on this chart provides more detail for each guideline, along with example sessions for each guideline violation. For each example session, click to view the corresponding transcript.
Users can provide feedback to improve the AI Supervisor's application of agent guidelines by opening a session containing a guideline violation. See [User Feedback](/user-feedback) for more information. The Wayfound Supervisor scores each session against your guidelines using an **evaluation rubric** it maintains for the agent — a grading guide that reflects how your organization has learned to apply each guideline. When you give feedback on a violation, the Supervisor folds it into that rubric so future scoring matches your intent. See [How the Supervisor Learns](/supervisor/how-the-supervisor-learns). ### **Action Failures:** The AI Supervisor monitors your agents' action calls and calculates their success rates. Click the overall summary of action failures to open a more detailed view of failure rates by action. Each action links to sessions where that action was called.
For each example session, click to view the corresponding transcript. Session transcripts display where an action was called. Click the action to show more detail about the request:
### **Potential issues:** As part of its review, the Wayfound Supervisor identifies **potential issues** — recurring behaviors or outcomes that stand out but aren't yet covered by any of your guidelines. Each potential issue includes a severity, the Supervisor's confidence, and references to the specific sessions that demonstrate it. Click a reference to open the session on the right, with its status, an explanation, per‑interaction suggestions, and the transcript. Potential issues are yours to triage. For each one you can: * **Promote to a guideline** — turn the pattern into a rule the Supervisor enforces going forward. * **Confirm as expected** — tell the Supervisor this behavior is intended; it folds the decision into how it evaluates the agent and stops flagging it. * **Dismiss** — mark it a false positive. The Supervisor won't re‑raise it unless the evidence grows substantially. Triaging potential issues teaches the Supervisor and sharpens future assessments. See Potential Issues for the full workflow. ### Follow-Up Analysis The performance tab displays a chat window where you can interact with the AI Supervisor agent for additional analysis. You can probe further on any of the insights it provides or ask questions about other aspects of the agent's performance. This allows the AI Supervisor to explain its assessment, enhancing your understanding and confidence in its analysis. Based on the feedback it provided, the AI Supervisor suggests custom follow-up questions. These suggestions are found below the key topics. In addition to the suggested questions, you can also prompt the AI Supervisor to: * Provide more details or evidence for a specific insight * Compare this agent's performance to similar agents in the organization * Suggest ways to leverage and expand on the highlighted strengths * Offer recommendations for resolving specific issues # Overview The Wayfound AI Supervisor provides visibility and control over entire network of agents. Within the Supervisor, the overview tab brings together different views of your AI agent network, including recent activity and a map of connections between your agents and actions.
### Recent Activity The overview tab provides quick statistics about your agents' usage over the past 30 days. This includes: **Agents**: the number of agents currently active in your organization. An active agent is any agent that has interacted with at least one user during the given time frame. A list of agents in your organization can be found further down the page or on [The Agents Page](/agents/the-agents-page) **Recordings:** the total number of recorded interactions between users and your agents in the given time frame. More data about individual recordings can be found in [Recordings](/sessions/recordings). **Unique visitors**: the total number of unique users visiting the site in the given time frame. **Meetings**: the total number of [Reports](/supervisor/reports) created between AI agents in your organization during the given time frame. ### Agent Network The overview tab also provides a visual representation of the AI agent networks in your organization. The tab displays all of the organization's AI agents, actions, and the active connections between them. The visual includes the following viewing features: * Scroll to zoom in and out * Click, hold, and drag the mouse to move * Drag and drop agents and tools to change their location in the map * Select an individual agent or action for more information (more information about actions requires admin level access to the organization) ### Agents Below the agent network view is a list of AI agents in your organization. You can sort the agents by any of the columns, and you can further customize the view using the **search** and **filter** buttons on the top-right corner of the table. Agents can also be viewed in [The Agents Page](/agents/the-agents-page). # Global Guidelines Global Guidelines define the oversight and organizational philosophy for all of your Agents. Global Guidelines are natural-language statements that apply to all of your Agents universally. The AI Supervisor uses global guidelines, in addition to its own judgment, to measure your agents' quality and alignment. These guidelines sit at the top of a two-level hierarchy. Below global guidelines are guidelines for individual agents. This hierarchical approach draws from cutting-edge research in the deployment and control of large language models (e.g. [Wallace et al 2024](https://arxiv.org/pdf/2404.13208)).
New global guidelines can be added by clicking **+ Add Guideline** below the currently active ones. Clicking the button reveals a menu displaying multiple kinds of guidelines to add: **Brand Voice and Tone:** establish the overall communication style for all agents across the organization. Use these guidelines to defines the language, formality level, and general demeanor that all agents share in common. AI interactions should embody. This ensures a consistent brand experience across all conversations with all agents in the organizations. **Company Values:** embed the organization's core principles and ethical standards into all AI interactions. It guides agents to make decisions and provide responses that align with the company's mission, vision, and values. **Restrictions/Limitations:** set universal boundaries for all agents in the network. It outlines topics, actions, or information that are off-limits for all AI interactions. This could include prohibitions on discussing confidential information, making commitments on behalf of the company, or engaging in any behavior that could be legally or ethically problematic. **Custom Command:** implement organization-wide protocols or behaviors that are unique to the company. This could include specific ways of handling certain situations or standard responses to common queries. Visit [Broken mention](broken://pages/5Uwq3VWAOzIH4QtYXDBk) for tips on writing good Global Guidelines. # Reports Reports let you turn the activity of your agents into insights you can act on. Instead of digging through individual sessions one agent at a time, you define a Report once - choosing which agents to draw from, what questions to answer, and which metrics to track - and Wayfound assembles the analysis for you cross every session those agents have handled. Clicking on the Reports tab opens an overview:
### Creating a Report Click **+ New Report** at the top of the page. Give the Report a name and description, then choose a starting template: * **Blank Report:** Start from scratch and build your own agenda from the ground up. * **Agent Improvement Report:** Comes pre-populated with an agenda focused on surfacing improvement opportunities (knowledge gaps, guideline violations, and suggested fixes) for the agents you selected. Select **Create Report** to continue to the configuration view.
### Configuring a Report The detail view is where you define exactly what the Report covers. You can update any of these settings and re-run the Report at any time. **Agents.** Choose which agents the Report draws from. The Report analyzes only the sessions belonging to the agents you select, so this defines the scope. **Special instructions.** Optional guidance that shapes the tone, focus, and format of the generated insights - for example, "Focus on enterprise customers" or "Keep each answer to three bullet points." **Timeframe.** Set the data window the Report looks at - last 7, 30, or 90 days, or all time. This timeframe applies to both the metrics it calculates and the sessions it nalyzes, so you can produce a weekly snapshot or a long-term trend from the same Report definition. **Agenda.** The heart of a Report. The agenda is a list of items the Report works through each time it runs. There are two kinds of agenda items: * **Narrative items.** Free-form questions or topics written in plain language (e.g. "What are the most common reasons contact support?"). Wayfound analyzes the relevant sessions and writes back a narrative answer, with links to the specific session it drew from so you can verify and dig deeper. * **Metric items.** Quantitative KPIs that are calculated deterministically from your session data (Covered in **Metrics** below) You can mix narrative and metric items freely in a single agenda.
### Metrics Metrics let a Report track quantative KPIs alongside its narrative insights. Unlike narrative items, metrics are computed deterministically from your session data pre-defined tags. **How metrics are defined.** Each metrics is built from an **expression** that references the **pre-defined tags** Wayfound applies to your agents' sessions. For example, if your agent tags sessions as **RESOLVED** and you want a resolution rate, your expression might be: **RESOLVED /** **TOTAL\_SESSIONS** **\* 100.** When the Report runs, Wayfound counts how many sessions in the timeframe carry each referenced tag, evaluates the expression, and stores the result. Each metric has a few settings: * **Name:** How the metric appears in the Report * **Expression:** The formula. Supports tag names, numbers, and the operators +, -, \*, /, and % with parenthesis. The reserved term **TOTAL\_SESSIONS** refers to the total number of sessions in scope. * **Format:** How the value is displayed (percentage, whole number, etc) * **Direction:** Whether higher or lower is better, so the Report can show whether a trend is improving or regressing.
### Scheduling and Email Delivery A Report can run automatically on a schedule so your team always has a current view without anyone clicking **Run.** * **Enable scheduling,** then choose a frequency - **daily** or **weekly.** For weekly schedules, pick the day of the week. Set the time of day and tiemzones the schedule should follow. * **Email delivery,** optionally have each scheduled run emailed to one or more recipients. Ad the email addresses you want to recieve the Report, and Wayfound sends the results as soon as the run completes. The detail view shows when the Report **last ran** and when it's **next scheduled** to run. ### Running a Report and Viewing Results To get a fresh read at any time, click **Run** on the Report's detail view. A Report runs against the latest sessions within its timeframe; **it may take a few minutes to complete.** Each run is saved to the Report's **History** so you can revisit it later. The results page presents: * **Metric titles** for each metric on the agenda, each with its current value and a sparkline. * **Narrative insights** for each narrative agenda item, written in plain language with links to the sessions they cite. * **Follow-up chat** on the right-hand side, where you can ask further questions about the results. The chat continues with the full context of the Report, so you can probe deeper.
# Report Sharing Wayfound enables users to share individual agents' Supervisor Reports with audiences outside the platform (with optional password). These reports provide comprehensive performance insights similar to those found in the Supervisor's [Performance](/supervisor/performance) tab.
Shared Supervisor Reports include the AI Supervisor's detailed analysis of agent performance, featuring: * User satisfaction summaries * Knowledge gap assessments * Action failure counts * Potential issues identified in the past 100 recordings * Key conversation topics from the past 100 recordings As in the [Performance](/supervisor/performance) tab, viewers can drill down for more detail by clicking on individual items. Viewers can also open illustrative chat recordings by clicking next to potential issues. ## How to Share a Report ### 1. Enable the feature Before users can share Supervisor Reports, an organization's admin user must enable the feature in the [Settings](/settings/organizations) page:
The Report Sharing feature is turned off by default. The feature requires administrator opt-in because generating reports creates pages with publicly accessible APIs (with optional password). ### 2. Generate links Once the feature is enabled, the platform displays share buttons next to each agent in the [Performance](/supervisor/performance) tab:
Clicking the button opens a pop-up for generating a share link:
Wayfound allows you to protect the manager report with an optional password. To create a password, enter it in the **Password** field and click **Set Password**:
When the password option is active, the shared report requests the password before it displays the manager insights:
Once a share link is generated, you can disable sharing at any time by revisiting the Share Report window. Here, you can also remove the report's optional password if you created one:
## Who can access shared reports? Without a password, Supervisor Reports can be accessed by anyone who obtains the active share link. We therefore advise caution when creating and sharing the links, as the reports contain agents' performance metrics, selected recordings, and API information. Setting a password allows users greater control over who can view the Supervisor Report. You can reset access to Supervisor Reports by disabling sharing and generating a new link; you can also reset the password while keeping the same link. # How the Supervisor Learns The AI Supervisor does more than grade each session in isolation. For every agent it maintains a living understanding — what the agent is for, where it struggles, and how your team wants its guidelines applied — and it uses that understanding to evaluate every new session. The result is an evaluation layer that gets more accurate and more tailored to your organization the longer it runs.
### Evidence memos Behind the scenes, the Supervisor keeps a small set of **evidence memos** for each agent — concise notes it updates as new sessions come in: * **The agent's purpose and scope** — what it's meant to do. * **What it does well** — strengths worth reinforcing. * **Recurring failure patterns** — issues that show up again and again. * **Knowledge gaps** — topics the agent repeatedly lacks information on. * **Evaluation principles** — the standards the agent should be judged by, including anything your team has clarified through feedback. Because these memos accumulate patterns across many sessions, the Supervisor recognizes recurring problems instead of rediscovering them one transcript at a time. ### The evaluation rubric From those memos and your published guidelines, the Supervisor distills a single **evaluation rubric** — a concise grading guide it applies to every future session. The rubric captures: * **Intent** — a short statement of what the agent is for. * **Guideline interpretation** — for each guideline, how your organization has learned to apply it in practice. Learned interpretation can only *narrow* a guideline (make it more precise), never broaden it beyond what you wrote. * **Broader patterns** — cross‑cutting signals to watch for, including any open potential issues. The rubric stays current automatically: it's re‑distilled whenever the Supervisor learns something new or when you change your guidelines. ### The feedback loop Your input is what makes the Supervisor smarter. Three things teach it: 1. **Teaching from a session** — correcting or reinforcing a verdict on a transcript. 2. **Answering open questions** — resolving cases the Supervisor flagged as ambiguous. 3. **Triaging potential issues** — promoting patterns into guidelines, confirming expected behavior, or dismissing false positives. Each of these updates the evidence memos, which re‑distill the rubric, which changes how the next session is scored. The Supervisor consolidates this learning continuously in the background, so improvements show up in subsequent assessments without any manual step. ### Auditability Everything the Supervisor uses to judge your agents is visible to organization admins in the **Memos** tab: the current evaluation rubric, the evidence memos behind it, and a version history that lets you trace any change back to the feedback and observations that caused it. \[screenshot: rubric version history / diff view] ### Availability Continuous learning is enabled for your organization and is available to organization admins. It's on by default for new organizations and is being rolled out across existing ones. If you're an admin and don't see the **Memos** and **Review** tabs yet, contact Wayfound to have it enabled. Until it's active, the AI Supervisor continues to provide session‑by‑session assessments as described in Performance. # Potential Issues ## Potential Issues **Potential issues** are recurring behaviors or outcomes the Wayfound Supervisor notices that aren't yet covered by any of your guidelines. They're how the Supervisor surfaces "you might want a rule about this" without waiting for you to anticipate every case.
Each potential issue includes: * **A description** of the pattern. * **Severity** and the Supervisor's **confidence**. * **Supporting evidence** — the specific sessions where the pattern appeared. Click through to read the transcripts. ### Reviewing a potential issue For each potential issue, you can: * **Promote to a guideline** — turn the pattern into a guideline the Supervisor enforces going forward. Wayfound pre‑fills a draft guideline from the issue so you can refine and publish it. * **Confirm as expected** — tell the Supervisor the behavior is intended. It records this as an evaluation principle for the agent and stops flagging it. * **Acknowledge** — mark that you're investigating so the Supervisor doesn't keep re‑raising it while you decide. * **Dismiss** — mark it a false positive. The Supervisor won't re‑raise it unless the supporting evidence grows substantially. Every action teaches the Supervisor. Promoting, confirming, and dismissing all feed back into how it evaluates the agent — see [How the Supervisor Learns](/supervisor/how-the-supervisor-learns). # Open Questions Sometimes the Wayfound Supervisor encounters a situation it genuinely can't judge on its own — feedback that conflicts with a guideline, or a new behavior that could be either good or bad. Instead of guessing, it raises an **open question** for your team.
Each open question includes the context that prompted it and, where relevant, a link to the session that raised it, so you can see exactly what the Supervisor is asking about. ### Answering For each open question you can: * **Answer** — provide your judgment in a sentence or two. The Supervisor folds your answer into the agent's evidence memos and evaluation rubric, and applies it to every similar session afterward. * **Dismiss** — if the question isn't worth resolving, dismiss it and the Supervisor moves on. Answering open questions is one of the highest‑leverage things you can do in Wayfound: a single answer can correct how thousands of future sessions are scored. See How the Supervisor Learns. # User Feedback ### Teach the Supervisor From a session transcript — and from the agent's Performance view — you can **Teach the Supervisor** to correct or reinforce its judgment: * **Correct a verdict** — e.g., "this wasn't actually a guideline violation," or "you missed one here." * **Reinforce a behavior** — tell the Supervisor a response was exactly right so it encourages more of it. * **Answer an open question** — resolve a case the Supervisor flagged as ambiguous. Every piece of feedback is folded into the agent's evidence memos and evaluation rubric, so the Supervisor applies your intent to *future* sessions automatically. Feedback compounds: the more you teach it, the more closely its grading matches how your team would judge the same interactions. See [How the Supervisor Learns](/supervisor/how-the-supervisor-learns). When evaluating agent performance, the AI Supervisor follows the [Guidelines](/agents/guidelines) set for each agent. The AI Supervisor may not always interpret the guidelines in ways that users anticipate when writing them. In cases such as this, Wayfound allows users to provide CLHF (continuous learning through human feedback) to the AI Supervisor to refine how it applies a guideline moving forward. Feedback can be found when opening a recording of a session where the AI Supervisor identified a guideline violation. Sessions can be accessed directly in the [Sessions](/sessions/recordings) page or through links in the [Performance](/supervisor/performance) page. Along with a transcript of the interaction, each sessions displays the AI Supervisor's review, including highlights of any potential guideline violations:
If the guideline violation does not align with your expectations, you can refine it by clicking the feedback button next to the guideline violation bullet point. This opens a User Feedback window:
The User Feedback window displays the relevant guideline, the message identified as in violation of the guideline, and the AI Supervisor's explanation of the violation. Below is a text box for providing user feedback. The AI Supervisor will consider all user feedback associated with each guideline when applying them in all future reviews. # Agent Supervision Best Practices Unlike conventional software which follows deterministic rules and procedures, AI agents use pattern recognition and reasoning to achieve their assigned goals. Whereas traditional software operates within strictly defined parameters and produces predictable but narrow results, AI agents use nondeterministic functions to operate in more complex and dynamic environments. As an AI agent interaction unfolds, an agent can continuously adapt based on user interactions and leverage natural language understanding to interpret requests in a way that mirrors human comprehension. While this nondeterministic nature of AI agents brings powerful flexibility, this approach to problem-solving introduces some unique management challenges. AI agents require continuous monitoring and improvement to ensure they remain aligned with your business goals and user expectations. ### Supervising Agents with Wayfound Wayfound's AI Supervisor platform addresses these challenges by providing comprehensive monitoring and evaluation tools specifically designed for AI agents. The AI Supervisor: * Analyzes agent-user interactions continuously and systematically * Evaluates performance against your custom guidelines * Identifies knowledge gaps and action failures * Flags potential issues and alerts you for review * Suggests specific improvements The AI Supervisor performs all of these tasks at scale, allowing you to deploy your AI agents across thousands of simultaneous interactions. However, it's important to note that the Wayfound AI Supervisor is itself an AI agent. This means that using it effectively benefits from following AI agent best practices. This includes iterating on your approach through continuous monitoring and feedback. ### Supervision is an ongoing process As with any AI agent, the key to success with Wayfound’s AI Supervisor is continuous iteration. To ensure that it captures all potential issues with your agents, the Supervisor is tuned to prioritize false positives over false negatives. The Wayfound platform allows you to refine how the Supervisor evaluates agents over time, focusing its attention on what matters most and reducing false positives. To achieve this, we recommend that users: **Review Alerts Regularly:** Check daily alerts for yellow flags (needs review) and address immediate red flags (needs attention) promptly. Use the Suggestions tab for specific recommendations. **Provide Feedback on Guidelines Application:** When the AI Supervisor flags a guideline violation that doesn't align with your expectations, use the User Feedback function directly from session transcripts. Clarify your intent to help the Supervisor better interpret guidelines in future assessments. Visit the [User Feedback](/user-feedback) documentation to learn more about providing feedback. **Refine Guidelines:** If you're receiving too many alerts, consider changing some red (needs attention) flags to yellow (needs review) if they're not critical. Provide more context in guidelines to help the Supervisor understand acceptable exceptions and adjust guideline priorities based on business impact. **Address Knowledge Gaps:** Prioritize knowledge gaps that appear frequently and update your agent's knowledge base in response to identified gaps. Review transcripts where knowledge gaps occur to understand the context. **Improve Tool Performance:** Monitor tool failure rates and patterns, optimize failing tools based on error messages and failed attempts, and consider alternative approaches for consistently problematic actions. **Answer Open Questions:** When the AI Supervisor raises an [open question](/supervisor/open-questions), answer it. These are the moments where a small amount of human judgment most improves future scoring — the Supervisor applies your answer to every similar session afterward. **Triage Potential Issues:** Regularly review [potential issues](/supervisor/potential-issues). Promote the ones you want enforced into guidelines, confirm the ones that are working as intended, and dismiss the noise. This is the fastest way to shape what the Supervisor watches for. **Teach from transcripts:** When you disagree with a verdict, don't just adjust guidelines — use [Teach the Supervisor](/user-feedback#teach-the-supervisor) on the session itself. It's the most direct way to correct interpretation, and it compounds over time. ### Measuring Progress We recommend that users track the following key indicators to measure improvement: **Alert Reduction:** Reduction in legitimate alerts over time. **Satisfaction Improvement:** Improved user satisfaction scores. **Knowledge Enhancement:** Decreased knowledge gaps. **Action Optimization:** Higher action success rates. Remember that supervision is an ongoing process. As your business evolves and user needs change, it is important to continue to refine your approach to agent supervision--just as you should refine the supervision and management of any part of the business. # Supervise Multi-Agent Systems For supervising multi-agent systems, Wayfound has the concept of **Applications.** One or more agents can be added to an application, and as sessions flow in to Wayfound, OpenTelemetry `trace` and `span` information is leveraged to create a holistic view of exactly what has occurred and when. Additionally, the Wayfound Supervisor can reason across the actions of every agent that participated in the application `trace`. To create an **Application**, go to the **Applications** section of Wayfound and click 'New Application'.
Provide a name and role and select one or more agents for the **Application**.
When viewing the list of Applications, simply click on the **Application** name to view `traces`.
The **Application** detail screen provides a quick overview of all the `traces` collected. For each `trace` you can see the number of sessions, the grades of each participating agent, the durations, and when each `trace` started and ended.
Clicking on a `trace` id will display the `trace` details drawer where you can view key analysis and the multi-agent timeline.
Click on an entry in the timeline to view the full agent analysis drawer.
**Contextualize Session Data** After an Application is created in Wayfound, the next step is to enhance your session data to include the `applicationId`, `trace_id`, and `span_id` . It is important to use the same `trace_id` across sessions where multiple agents are involved so that Wayfound can properly correlate the multi-agent session data. The required `span_id` and optional `parent_span_id` can be used to further group data within a trace. Here is an example of a Wayfound session with these fields included: ``` { "agentId": "22198fc5-1056-4a9d-b455-440db1249539", "applicationId": "7cf54242-0898-4ac1-a09c-c0e4be3f000a", "firstMessageAt": "2025-15-08T10:51:00.000Z", "lastMessageAt": "2025-15-08T10:51:00.000Z", "messages": [ { "timestamp": "2025-15-08T10:51:00.000Z", "event_type": "assistant_message", "attributes": { "content": "Hello, how can I help you today?" }, "context": { "trace_id": "123", "span_id": "abc" } } ] } ``` **Note:** The `context`, `trace_id`, and `span_id` are required fields when you add `applicationId` to a session. # The Agents Page Wayfound allows you to connect and manage a network of specialized agents. Each agent can perform a focused task, and agents can collaborate to accomplish more complex tasks. The Agents tab displays a list of all agents in your organization. It also allows you to connect and configure new ones.
### View and manage current agents Opening the agents tab displays a table of agents connected to your organization. Users can **sort** agents by name, platform, last updated date, last published date, and status. Users can also **search** for specific agents and **filter** by column using buttons in the table's upper-right corner. In the agents table, the **Actions** column contains a menu of actions that can be opened by clicking the ![](/files/Xgs7DYKyMON5A8Se1xQF)button. Within this menu, you can ![](/files/MDMbUINLmKGTjnhCXuhp)**view agent analytics,** and ![](/files/r5NUgS7yvN91TBNTRJL7)**activate** or ![](/files/dfePw7WixdHaghYT1iee)**deactivate agent**. ### Onboard an existing agent Wayfound lets you connect existing external agents to the platform. This includes those built on popular frameworks like LangChain. Adding an agent to Wayfound allows you to track, monitor, and manage it using the Wayfound platform. To connect an agent, you can use the [Wayfound Manager SDK](/agents/connecting-agents) to import recorded conversations from your external agent. To onboard an existing agent: 1. Click at the top-right corner of the Agents tab. 2. A new menu will appear. Name the agent and explain its role and goal. This information will inform the AI manager's performance evaluation of this agent. Click **Let's Go!**.
# Definition The definition page allows you to define your connected agent's key parameters. These parameters are used by the AI Supervisor to evaluate your agent's performance.
A connected agent's parameters include: **Agent Name:** The name displayed in [The Agents Page](/agents/the-agents-page) and the AI Supervisor's [Performance](/supervisor/performance) reports. **Role**: The agent's identity; it includes what the agent's job is and what its intended tasks include. During the AI Supervisor's assessment of the agent's performance, it evaluates how well the agent performs its role. **Goal:** The concrete outcomes intended for the agent's actions. The AI Supervisor will compare the outcomes of the agent's actual interactions with the goals listed here. **Sentiment Tuning:** Optionally tune sentiment analysis behavior by providing a custom description of how to reason about sentiment for your agent. You can also select if sentiment affects grade. When enabled negative sentiment will lower the agent's grade. Here, you can also customize the icon and color used to represent the agent on the Wayfound platform. # Test Mode Test Mode is a sandbox where you can test agent settings with the Wayfound Supervisor to ensure the settings meet your expectations. The Wayfound Supervisor takes into account your agent’s role, goal, and guidelines when analyzing sessions. Test Mode enables you to test your agent settings with the Wayfound Supervisor before publishing the settings to your production environment. ### **Setup Overview** #### 1. Select test sessions These are sessions that are the primary content of your test runs. Test sessions can be uploaded via the [Test Session API](https://wayfound-api.readme.io/reference/create-test-session) to facilitate testing before your agent is in production. You can also copy production sessions to become test sessions when you are managing an agent already released to production. #### **2. Set expected outcomes on test sessions** Specify for each guideline what your expected compliance status is for that guideline. It’s important to select sessions that have both compliant and non-compliant expected values to ensure the guideline language works for both cases. #### **3. Start test run** Select the sessions you want to test and specify the number of iterations to run each session. The more iterations you select the longer the analysis will take but you will also be able to test the stability of the guideline. Stability is important because the Wayfound Supervisor is conducting a probabilistic analysis of the guideline and you want to ensure that the analysis of the test sessions/guideline is converging to your expected outcome. ### **Select Test Sessions** When reviewing sessions the session drawer has a “Copy session for evaluation” button which will make a copy of this session and place it into the agent’s sandbox
The session is now listed as a “Test Session” in the Agent’s Test Mode pane after clicking on "Run Test from the Test Mode menu.
Clicking **View** to review the test sessions
### **Set Expected Outcomes** For each guideline and test session you can set the expected outcome by clicking on the ‘Test guideline’ button next to the guideline.
This will open a “Guideline Expectations” modal where you can set the expected outcome for each test session for the given guideline.
Repeat this process for each guideline and each test session. ### **Start Test Run** Select the sessions to test as well as the number of iterations to run. It is suggested to start with a lower number of iterations as you make guideline changes and gradually increase the iterations as the results are aligning with your expectations.
Click “Test Selected” and wait for processing to complete.
When processing is complete you can review the test results.
# Guidelines Guidelines define the objectives and goals of your agent. When the AI Supervisor evaluates your agent's [performance](/supervisor/performance) and sends alerts based on its performance, it takes these guidelines into consideration. By adding unique guidelines for each agent, you can customize how it is supervised according to its particular role.
Adding and updating guidelines To add more guidelines, click the **+ Add Guideline** button below the list of current guidelines. You may add the following kinds: **Prohibited Actions:** Specify actions the agent must never take, such as sharing sensitive information, making unauthorized promises, or accessing restricted systems. **Prohibited Words:** Define specific terms or phrases the agent should not use. This includes inappropriate vocabulary, competitor names, or terminology that could cause confusion. **Preferred Voice and Tone:** Establish the desired communication style through specific criteria like formality level, technical depth, and emotional resonance. **Other Evaluation criteria:** Create custom metrics to assess agent performance based on the agent's particular role and goal. You can update existing guidelines by clicking on their respective text field in the Guidelines page. ## Guideline Alerts Wayfound allows you to set the importance of each guideline, giving you control over how often the AI Supervisor alerts you about the issues it identifies. Guidelines have two possible alert levels: **Needs Review:** These are guidelines that, while important to follow, do not require urgent remediation if breached. The AI Supervisor will collect all potential "Needs Review" guidelines violations and alert uses about them once per day. **Needs Attention:** These are guidelines that may require an immediate response if breached. The AI Supervisor will alert users about "Needs Attention" guidelines as soon as it identifies a violation. ## Optimizing Your Guidelines Guidelines are most effective when they: **Focus on Specifics:** They focus on specific, measurable behaviors. **Differentiate Severity:** They clearly differentiate between critical (needs attention) and important (needs review) issues. **Provide Context:** They provide context about when exceptions are acceptable. **Align with Priorities:** They align with your business priorities and user needs. ## How the Supervisor interprets your guidelines You write guidelines in natural language, so real sessions inevitably raise edge cases. The Wayfound Supervisor learns your intended interpretation from the feedback you give and records it in the agent's **evaluation rubric** — a per‑guideline grading guide it applies to every future session. * When you correct a verdict (see [Teach the Supervisor](/user-feedback#teach-the-supervisor)), the Supervisor updates its interpretation of that guideline so it scores the same situation your way next time. * Learned interpretations can only **narrow** a guideline (clarify how it applies), never broaden it beyond what you wrote. * Only **published** guidelines are used to evaluate sessions. Draft edits don't affect scoring until you publish them. Learn more in [How the Supervisor Learns](/supervisor/how-the-supervisor-learns). # Tags ## How Tagging Works Tags are labels Wayfound applies to your agent's sessions to capture what each conversation was about. A session might be tagged `PRICING-QUESTION`, `BILLING-ISSUE`, or `FEATURE-REQUEST` — whatever categories matter to your business. Once your sessions are tagged, you can: * **Filter and search** your session list to find every conversation on a given topic. * **Build metrics** in Reports that count and trend tagged sessions over time (for example, the share of sessions that involved a pricing question).
## Pre-defined vs auto-generated There are two ways an agent can end up with tags: * **Pre-defined tags** — the tags *you* define on this page, each with a name and a description of when to apply it. When you've defined one or more pre-defined tags, the AI evaluates **only** those tags for new sessions, giving you a consistent, controlled vocabulary across every conversation. * **Auto-generated tags** — when an agent has **no** pre-defined tags, Wayfound instead generates a few descriptive tags per session on its own to summarize each conversation. These are useful out of the box but vary from session to session. You can also combine the two — see **Including auto-generated tags** below. ## Defining a pre-defined tag On the **Tags** page, the **Pre-Defined Tags** section is where you build your tag set. Click **Add Pre-Defined Tag** to create one, then fill in: * **Tag Name** — a short, uppercase label (e.g., `PRICING-QUESTION`). Names are automatically converted to uppercase and spaces become underscores. Use only letters, numbers, underscores, and hyphens; each name must be unique on the agent, and shorter names read best in the session list. * **Description (when to apply this tag)** — a natural-language description telling the AI exactly when this tag should be applied. This is the most important field: the clearer and more specific it is, the more accurately the tag will be applied. Aim for roughly 10–500 characters. *Example:* "Apply this tag when the user asks about pricing, costs, payment options, or billing information." * **Active** — a toggle that controls whether the tag is currently evaluated. Turn it off to pause a tag without deleting it; inactive tags are skipped when new sessions are tagged. Use the trash icon on a tag card to remove a tag entirely. ## Including auto-generated tags Once you've added at least one pre-defined tag, an **Also include auto-generated tags** toggle appears at the top of the section. When enabled, Wayfound applies your matching pre-defined tags **and** adds a small number of AI-generated tags to capture anything your pre-defined set didn't cover. Leave it off to keep sessions strictly to your defined vocabulary; turn it on when you want broader coverage and are comfortable with some variation. ## Publishing your tags Tags are part of your agent's configuration, so they follow the same draft-and-publish flow as the rest of Agent Settings. Edits you make here are saved to your **draft**; fields that differ from the published version are highlighted so you can see what's changed. **Publish the agent** to put your tags into effect for live sessions — only published tags are used to evaluate real traffic, while draft tags apply to test runs. > Tags are applied to sessions **going forward**, as new sessions are processed. Changing your tags doesn't re-tag sessions that were already analyzed. ## Using your tags Once sessions are being tagged, your tags show up in two places: * **Session list** — tags appear as badges on each session, and the **Tags** filter lets you include sessions *with* selected tags or exclude sessions that carry them, so you can zero in on a topic quickly. * **Reports & Metrics** — metric expressions in a Report reference your tag names to count and trend tagged sessions. For example, a metric expression like `PRICING-QUESTION / TOTAL_SESSIONS * 100` reports the percentage of sessions that involved a pricing question. See the **Reports** documentation for how to build metrics from your tags. # Connecting Agents The Wayfound Manager SDK provides a simple interface for connecting agents to Wayfound. It allows developers to seamlessly integrate Wayfound's AI agent supervision capabilities with their third-party agents using Python, JavaScript or simple REST API endpoints directly. The **Connection** page provides instructions and information for connecting your third-party agent to Wayfound. It includes the agent's unique ID, which is required for use of the Wayfound SDK.
### Getting started {% hint style="info" %} First, you should have at least one third-party agent connected to the Wayfound platform. If you have not yet connected an agent, visit [The Agents Page](/agents/the-agents-page) and connect an agent. {% endhint %} #### Connecting Agents with Python You can install the Wayfound SDK using pip: ```python pip install wayfound ``` To use the Wayfound Manager SDK, you'll need a Wayfound API key and an agent ID. The Agent ID is available on the agent's Connection page. API keys can be obtained by users with admin permissions from the [API page](/implementation/api). The SDK provides methods to record new messages, update existing recordings, and add details like user ratings and handoffs. For example: ```python from wayfound import Session wayfound_api_key = "" wayfound_agent_id = "" wayfound_session = Session(wayfound_api_key=wayfound_api_key, agent_id=wayfound_agent_id) messages = [] messages.append({ "timestamp": "2025-05-07T10:00:00Z", "event_type": "assistant_message", "attributes": { "content": "Hello, how can I help you today?", } }) messages.append({ "timestamp": "2025-05-07T10:00:04Z", "event_type": "user_message", "attributes": { "content": "What's the current status of Project Alpha?" } }) result = wayfound_session.create(messages=messages) ``` #### Connecting Agents with JavaScript: You can install the Wayfound SDK using npm: ``` npm install wayfound ``` To use the Wayfound Manager SDK, you'll need a Wayfound API key and an agent ID. The Agent ID is available on the agent's Connection page. API keys can be obtained by users with admin permissions from the [API page](/implementation/api). The SDK provides methods to record new messages, update existing recordings, and add details like user ratings and handoffs. For example: ```javascript import { Session } from "wayfound"; const wayfoundApiKey = ""; const agentId = ""; const session = new Session({ wayfoundApiKey, agentId }); const messages = [ { timestamp: "2025-05-07T10:00:00Z", event_type: "assistant_message", attributes: { content: "Hello, how can I help you today?", }, }, { timestamp: "2025-05-07T10:00:10Z", event_type: "user_message", attributes: { content: "What's the current status of Project Alpha?", }, }, ]; const response = await session.create({ messages }); ``` ### Resources * PyPI Project: * npm Package: * Python Source Code: * Javascript Source Code: # Connecting to Agentforce The Salesforce Integration allows you to connect your Salesforce Agentforce agents with Wayfound, enabling comprehensive performance monitoring and management. This integration synchronizes your Salesforce agents with Wayfound's powerful analytics and evaluation tools, providing valuable insights to improve agent effectiveness. ### Before You Connect **Salesforce Requirements** - What must be enabled in Salesforce before starting: * Agentforce enabled with at least one active agent * Data Cloud provisioned * Agent Analytics enabled in Data Cloud **Who Should Authorize the Connection?** * A System Administrator role with have all of the required permissions * For non-admin roles the following permissions are needed: * API Enabled * Data Cloud User permission set * Read access to BotDefinition * Approve Uninstalled Connected Apps or Use Any API Client You need to be an active Wayfound user and admin as well as a Salesforce user with required permissions to connect Salesforce via OAuth. ### Connecting to Salesforce To connect Salesforce to Wayfound, visit the **Agentforce** page in Wayfound's Settings tab:
In the Agentforce page, click . This will take you through Salesforce OAuth:
Once you have logged into Salesforce and authorized Wayfound, you will be taken back to Wayfound's Agentforce page:
### Salesforce Configuration To connect Agentforce agents to Wayfound, agents must be enabled within Salesforce, in addition to agent analytics for Data Cloud. When Wayfound detects that each of these conditions are met, it will display a blue checkmark next to them. #### Enable Agentforce If Agentforce is not turned on in Salesforce, visit the Agentforce Agents setup page by clicking . This will take you to the following page:
Here, take the following actions: 1. Turn on Agentforce 2. Either create a **New Agent** or enable the **Agentforce (Default) Agent** After taking these actions, refresh Wayfound's Agentforce settings page you will see a blue checkmark next to **Enable Agents**. #### Enable Data Cloud If Data Cloud is not enabled, click to take you to visit the Einstein Feedback and Monitoring Setup page in Salesforce:
{% hint style="info" %} **NOTE: Access to this page requires that you have already enabled Data Cloud** {% endhint %} Here, turn on **Agent Analytics**. It may take some time for Salesforce to update this setting. Return to Wayfound's Agentforce page to confirm that Agent Analytics have been activated. If so, Wayfound will display a blue check mark next to **Enable Data Cloud**. ### What Wayfound Accesses When you authorize the connection, Wayfound requests three OAuth scopes from Salesforce: | Scope | What It Does | | --------------- | ---------------------------------------------------------------------------- | | `api` | Reads your list of Agentforce agents | | `cdp_api` | Reads agent conversation transcripts from Data Cloud | | `refresh_token` | Keeps the connection active so Wayfound can sync sessions in the background. | All access is **read-only**. Wayfound never writes data back to Salesforce. ### Syncing Agents with Wayfound Once Salesforce is correctly set up in Wayfound, you can **activate** your agents on the platform. To do so, activate **Supervise in Wayfound** for each agent you would like to sync with Wayfound:
Once an Agentforce agent is activated, it appear on [The Agents Page](/agents/the-agents-page) as any other agent. Now, you can add a role, goal, and guidelines to your agent to give the Wayfound AI Supervisor the context it needs for analyzing its performance:
### Session Recordings with Wayfound Every hour, Wayfound will sync with your Salesforce instance to pull your active agents' latest sessions. {% hint style="warning" %} Once the agent is activated it can take up to an hour for the first conversations to appear. In addition, conversations will only be pulled into Wayfound once they are in Data Cloud. This means it can take some time for conversations to appear. In the worst case, this may take several hours. {% endhint %} You can view your Agentforce Agents' session recordings in [Recordings](/sessions/recordings). ### Stop Syncing Agents If you would like to disconnect an Agentforce agent from Wayfound, deactivate **Supervise in Wayfound** for the agent in Wayfound's Agentforce page. Disconnected agents can always be reconnected on this page. When reconnected, guidelines and previous sessions will be restored. You can always turn it back on again afterwards and you will not lose any guidelines or previous sessions synced to Wayfound. {% hint style="info" %} When an agent is reconnected, Wayfound will not download all sessions that have taken place during the period between the agent's deactivation and reactivation. Instead, Wayfound will only download agent sessions from the last six hours. {% endhint %} ### Disabling Salesforce To disconnect all Agentforce agents, click in Wayfound's Agentforce page. This will deactivate all agents and Wayfound will no longer be able to sync any agents unless Salesforce is reconnected.
### Troubleshooting | Problem | Likely Cause | | ---------------------------------------- | --------------------------------------------------------------------------- | | OAuth authorization fails immediately | User doesn't have API Enabled or lacks permission to approve connected apps | | Connection succeeds but no agents appear | Agentforce isn't enabled or no agents are active | | Agents appear but no conversations sync | Data Cloud or Agent Analytics isn't enabled | | "Access Denied" or scope errors | User is missing the Data Cloud User permission set | # Connecting to Intercom #### Connect your Intercom workspace In Wayfound, go to Settings > Intercom and click Connect to Intercom. You'll be redirected to Intercom to authorize access, then returned to Wayfound automatically. #### Activate Fin Integration Once connected, you'll see your Fin AI Agent listed. Toggle the Supervised switch to start monitoring. Wayfound will begin syncing your Fin conversations automatically every 15 minutes.
# Connecting Tool Calls After [Connecting Agents](/agents/connecting-agents) to Wayfound, you can also connect tool call information. This allows Wayfound to track tool call success rates in [Performance](/supervisor/performance) reports. You can do so by adding a "tool\_call" event to "messages": ```json { "timestamp": "2025-05-07T10:00:03Z", "event_type": "tool_call", "label": "Tool Call: FlightService.SearchFlights", "description": "Searching for flights from SFO to JFK", "attributes": { "success": true, "tool_name": "FlightService", "latency_ms": 120, "tool_input": { "to": "SFO", "from": "JFK", "dates": [ "2025-06-10", "2025-06-15" ] }, "tool_output": [ { "price": 320, "flight_id": "FS123" }, { "price": 290, "flight_id": "FS456" } ] } } ``` This will add the following information to the agent's Wayfound session transcript:
Clicking the arrow will reveal the details of the tool call event:
Note that tool calls should be logged regardless of success or failure for accurate performance tracking. # Connecting User Feedback After [Connecting Agents](/agents/connecting-agents) to Wayfound, you can also track user feedback ratings. This allows Wayfound to generate satisfaction metrics in [Performance](/supervisor/performance) reports. You can do so by adding a "user\_feedback" event to "messages". Two types of user feedback are supported: "stars" and "thumbs\_up\_down": ```json { "timestamp": "2025-05-07T10:05:05Z", "event_type": "user_feedback", "label": "User Feedback (Thumbs)", "description": "User gave a thumbs-up on risk report", "attributes": { "feedback_type": "thumbs_up_down", "rating": "thumbs_up", "comment": "This was great!" } }, { "timestamp": "2025-05-07T10:05:10Z", "event_type": "user_feedback", "label": "User Feedback (Stars)", "description": "User rated overall experience", "attributes": { "feedback_type": "stars", "rating": 5 } } ``` Examples of user feedback events are shown here as they appear in a session transcript:
Note that multiple feedback entries can be included for the same conversation, and feedback can be submitted at any point during or after the interaction. # Connecting other Events In addition to [Connecting Tool Calls](/agents/connecting-tool-calls) and [Connecting User Feedback](/agents/connecting-user-feedback) to your agents' session recordings in Wayfound, you can also connect other kinds of events. These can be any kind of data that is relevant to the agents' interactions. The AI Supervisor can reason about this additional data in the session when analyzing your agents' activity. ### Event Types There are 14 supported event types (including the stop-gap "custom\_event"): * `assistant_message` * `user_message` * `system_message` * `developer_message` * `structured_message` * `reasoning_step` * `tool_call` * `agent_call` * `user_feedback` * `button_click` * `link_click` * `agent_handoff` * `human_review` * `custom_event` ### Example events with descriptions Below is an example session transcript that uses all the event types along with brief explanations of why they are being used: #### 1. `user_message` Records the user’s request for Project Alpha status. ```json { "timestamp": "2025-05-07T10:00:00Z", "event_type": "user_message", "label": "User Message", "description": "User asks for project Alpha status", "attributes": { "content": "What’s the current status of Project Alpha?" } } ``` *** #### 2. `reasoning_step` Logs the internal thought about which service to call first. ```json { "timestamp": "2025-05-07T10:00:01Z", "event_type": "reasoning_step", "label": "Reasoning Step", "description": "Determine which service to call first", "attributes": { "thought": "Fetch high-level project data, then drill into trends." } } ``` *** #### 3. `developer_message` Shows how you parsed the user’s intent. ```json { "timestamp": "2025-05-07T10:00:02Z", "event_type": "developer_message", "label": "Dev Log", "description": "Parsed user intent", "attributes": { "content": "Intent identified: get_project_status, projectId=Alpha" } } ``` *** #### 4. `tool_call` (PMService.getProjectData) Fetches the high-level summary for Project Alpha. ```json { "timestamp": "2025-05-07T10:00:03Z", "event_type": "tool_call", "label": "Tool Call: PMService.getProjectData", "description": "Fetched summary for Project Alpha", "attributes": { "tool_name": "PMService", "tool_input": { "projectId": "Alpha" }, "tool_output": { "status": "On Track", "completion": 80 }, "success": true, "latency_ms": 95 } } ``` *** #### 5. `assistant_message` Delivers the summary back to the user. ```json { "timestamp": "2025-05-07T10:00:04Z", "event_type": "assistant_message", "label": "Assistant Summary", "description": "Reported high-level status", "attributes": { "content": "Project Alpha is On Track with 80% completion to date.", "model_name": "gpt-4", "tokens_total": 18, "latency_ms": 60 } } ``` *** #### 6. `structured_message` Offers the user structured next-step options. ```json { "timestamp": "2025-05-07T10:00:05Z", "event_type": "structured_message", "label": "Options Presented", "description": "Offered next steps", "attributes": { "options": [ { "title": "Show trend chart", "value": "trend" }, { "title": "Get risk assessment", "value": "risk" } ] } } ``` *** #### 7. `button_click` Records the user choosing “trend.” ```json { "timestamp": "2025-05-07T10:00:06Z", "event_type": "button_click", "label": "Button Click", "description": "User chose to view trend", "attributes": { "button_id": "trend" } } ``` *** #### 8. `tool_call` (AnalyticsService.getCompletionTrend) Fetches week-by-week completion data. ```json { "timestamp": "2025-05-07T10:00:07Z", "event_type": "tool_call", "label": "Tool Call: AnalyticsService.getCompletionTrend", "description": "Fetched completion trend data", "attributes": { "tool_name": "AnalyticsService", "tool_input": { "projectId": "Alpha" }, "tool_output": [ { "week": 1, "completion": 50 }, { "week": 2, "completion": 60 }, { "week": 3, "completion": 75 }, { "week": 4, "completion": 80 } ], "success": true, "latency_ms": 110 } } ``` *** #### 9. `reasoning_step` Decides whether to invoke the risk-evaluation agent. ```json { "timestamp": "2025-05-07T10:00:08Z", "event_type": "reasoning_step", "label": "Reasoning Step", "description": "Decide whether to call risk agent", "attributes": { "thought": "Trend looks healthy—next get risk score." } } ``` *** #### 10. `agent_call` (RiskAgent) Requests a detailed risk assessment from your RiskAgent. ```json { "timestamp": "2025-05-07T10:00:09Z", "event_type": "agent_call", "label": "Agent Call: RiskAgent", "description": "Requested project risk evaluation", "attributes": { "external_id": "agent-42", "input": { "projectId": "Alpha" } } } ``` *** #### 11. `assistant_message` Presents the risk score to the user. ```json { "timestamp": "2025-05-07T10:00:10Z", "event_type": "assistant_message", "label": "Assistant Risk Report", "description": "Delivered risk assessment", "attributes": { "content": "Risk score for Project Alpha is 2.1/5 (Low), with schedule variance at 5%.", "model_name": "gpt-4", "tokens_total": 22, "latency_ms": 70 } } ``` *** #### 12. `link_click` Logs the user clicking through to the web dashboard. ```json { "timestamp": "2025-05-07T10:00:11Z", "event_type": "link_click", "label": "Link Click", "description": "User clicked through to dashboard", "attributes": { "url": "https://dashboard.example.com/projects/Alpha" } } ``` *** #### 13. `system_message` Notes a background system action (scheduling a report). ```json { "timestamp": "2025-05-07T10:00:12Z", "event_type": "system_message", "label": "System Message", "description": "Scheduled weekly status email", "attributes": { "content": "Weekly report for Project Alpha will be sent every Monday at 09:00." } } ``` *** #### 14. `custom_event` Tracks a domain-specific audit entry. ```json { "timestamp": "2025-05-07T10:00:13Z", "event_type": "custom_event", "label": "Custom Event", "description": "Audit log entry", "attributes": { "event_name": "audit_log", "details": { "action": "view_status", "user": "user-007" } } } ``` *** #### 15. `agent_handoff` Marks the transfer to a human agent. ```json { "timestamp": "2025-05-07T10:00:14Z", "event_type": "agent_handoff", "label": "Agent Handoff", "description": "Transferred to human for budget questions", "attributes": { "reason": "budget_inquiry" } } ``` *** #### 16. `human_review` Captures a supervisor’s follow-up on that handoff. ```json { "timestamp": "2025-05-07T10:05:00Z", "event_type": "human_review", "label": "Human Review", "description": "Supervisor added budget commentary", "attributes": { "approver_id": "supervisor-99", "status": "note_added" } } ``` *** #### 17. `user_feedback` (Thumbs) Logs a thumbs-up/down on the risk report. ```json { "timestamp": "2025-05-07T10:05:05Z", "event_type": "user_feedback", "label": "User Feedback (Thumbs)", "description": "User gave a thumbs-up on risk report", "attributes": { "feedback_type": "thumbs_up_down", "rating": "thumbs_up", "related_span_id": "span-10" } } ``` *** #### 18. `user_feedback` (Stars) Captures an overall star rating for the session. ```json { "timestamp": "2025-05-07T10:05:10Z", "event_type": "user_feedback", "label": "User Feedback (Stars)", "description": "User rated overall experience", "attributes": { "feedback_type": "stars", "rating": 5 } } ``` *** You can copy and adapt each block to instrument your own agent’s session logging for Wayfound—just plug in your timestamps, labels, descriptions, and attribute data as you go. # OpenTelemetry Event Data Wayfound supports sending optional [OpenTelemetry](https://opentelemetry.io/) fields along with each event in a session. Here is an example that is using the supported fields (context, resource, and instrumentation) along with sub-fields. Note that `resource` and `instrumentation` are optional fields: ```json { "timestamp": "2025-05-08T17:00:00.000Z", "context": { "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736", "span_id": "00f067aa0ba902b7", "parent_span_id": "3d7a2b8f9e1c4f2a" }, "resource": { "service.name": "chat-interface-agent", "service.version": "3.2.1", "deployment.environment": "production", "hostname": "chat-frontend-02.example.com" }, "instrumentation": { "name": "wayfound-otel-sdk", "version": "2.0.0" }, "event_type": "user_message", "label": "User Message", "description": "User asks for account balance", "attributes": { "content": "What’s my current checking account balance?", "language": "en-US" } } ``` This data will be returned whenever a session is retrieved through the Wayfound API. Additionally, Wayfound will leverage this data when used with multi-agent Wayfound Applications. **Important**: the OpenTelemetry data is not included when Wayfound analyzes the session. # Recordings The Recordings tab provides a detailed view of individual interactions between users and your AI agents.
### View Options You can customize the table of chat recordings using the drop-down menus at the top-right corner of the window: **Agent Selection**: Select which agent to analyze. You can choose from any agent in your current organization. **Time Range**: Choose the time period for the data you want included in the assessment, from the past 24 hours to all time. You can further customize the view using the **search** and **filter** buttons on the top-right of the table ### Session data The table of sessions displays columns of different data pertaining to each interaction: **Recording:** Click **View** to open a transcript of the full conversation on the right-hand side of the window. This window also displays the session analysis and the Wayfound AI Supervisor's explanation and suggestions for improvement **Tags:** Keywords categorizing the content of each conversation **Labels:** Labels applied to session recordings **Grade:** The AI supervisor evaluates the quality of each recording. It assigns one of the following grades:
GradeExplanation
Hot to go!The agent is meeting expectations in its interactions. However, the AI Supervisor can still raise potential issues and provide suggestions for improvement.
Needs reviewThe agent's performance is satisfactory, but there are areas that require closer attention and potential improvement.
Needs attentionThe agent is facing significant challenges or issues that require immediate focus and resolution.
**Messages:** Number of exchanges recorded between the user and agent **Events:** Flags when the agent invoked an action in the recording **Visitor:** Displays the user's email, if connected, or the user's unique ID **Duration:** Length of the conversation **Last Updated:** Date and time the session was last updated ### Delete a recording Users with admin status can delete recordings on Wayfound. To delete a recording, open it by clicking "View" in the Recordings page. Then, click the trash can next to the recording timestamp. Type "DELETE" in the pop-up window to confirm and delete the recording. Note that deleted recordings cannot be recovered. # Suggestions The suggestions tab enables you to continually improve the performance of your AI agents over time. As you build and deploy AI agents, their interactions with users and workflows will scale beyond a level that humans can monitor on their own. To solve this challenge, Wayfound provides an automatic assessment of your AI agents using its AI Supervisor Agent, which continually monitors and evaluates your agents' interactions. It provides you with insights about your agents, serving as your interface with AI agents at scale.
### Suggestions components **Suggested behaviors** are recommendations to improve the quality of an agent's interactions. These recommendations are written as prompts using principles from[Broken mention](broken://pages/SnCYl8B4ygcDHjhSoaj2#how-to-create-effective-behaviors). They can be leveraged as directives in the connected agent. **Suggested knowledge** identifies gaps in knowledge that, when filled, can enhance the agent's performance. We recommend that you consider each gap and the possible sources of information that can be used to fill them. **Evidence‑backed and traceable.** Suggestions are drawn from the understanding the Wayfound Supervisor builds across many sessions, not a single transcript. Each suggestion links back to the sessions and observations that motivated it, so you can see *why* it's recommending a change before you act on it. # Accounts The accounts page displays information about all of the user accounts that have interacted with your organization's agents. Use this page to learn how users are interacting with your various agents.
You can sort the table by any of its columns and further customize the view using the **search** and **filter** buttons on the top-right of the table. Clicking on an account ID opens a table displaying information about each of the respective account's sessions. # Visitors The visitors page displays information about all users who have interacted with your organization's agents. Use this page to learn how users are interacting with your various agents.
You can sort the table by any of its columns and further customize the view using the **search** and **filter** buttons on the top-right of the table. Clicking on a visitor ID opens a table displaying information about each of the respective visitor's sessions. # Agent Skill Wayfound publishes an open-source skill that teaches AI coding agents how to integrate Wayfound into your application. When a developer asks their coding agent to "add Wayfound supervision" or "send session transcripts to Wayfound," the agent automatically knows how to wire up the SDK and API. #### What is an Agent Skill? An agent skill is a portable set of instructions that gives AI coding agents new capabilities. Skills follow the [https://agentskills.io](https://agentskills.io/) open standard and are supported by leading AI development tools including Claude Code, Codex, Cursor, Gemini CLI, and many others. The Wayfound skill teaches your coding agent: \- How to install and configure the Wayfound Python and JavaScript SDKs \- How to format session messages using all 14 supported event types \- How to send completed or streaming session transcripts to Wayfound via SDK or REST API \- How to set up visitor tracking, account tracking, and session metadata ### Install #### Claude Code This is available as a Claude Code plugin marketplace. To install: 1\. Add the marketplace: `/plugin marketplace add Wayfound-AI/wayfound-agent-skills` 2\. Open `/plugin`, go to the Marketplaces tab, select wayfound-skills, browse plugins, and install wayfound #### Other Agents Any agent that supports the [https://agentskills.io](https://agentskills.io/) format can use this skill. Check your agent's documentation for how it discovers and loads skills. #### Source The skill is open source and available on GitHub: # Overview ## Wayfound MCP: The New Standard for AI Agent Integration Wayfound’s Model Context Protocol (MCP) server is your gateway to seamless AI agent management and integration. As the new standard rapidly adopted across enterprises, MCP simplifies your AI strategy, enabling effortless deployment, real-time performance monitoring, and consistent compliance. \ **Why MCP?** Model Context Protocol is transforming how businesses integrate and manage AI agents. Wayfound’s MCP server provides a secure, centralized, and streamlined way to embed AI directly into your operational workflows, significantly reducing the complexity and friction traditionally associated with AI deployments. \ **How Wayfound’s MCP Server Works** Our MCP implementation offers specialized tools designed to empower both technical and non-technical users: #### `evaluate_session` * Automatically validate AI-generated outputs before finalizing critical actions. * Ensure adherence to business guidelines on tone, formatting, privacy, and more. *Example*: Validate AI generated customer email responses for compliance before sending, preventing guideline violations and enhancing customer satisfaction. #### `get_supervisor_analysis_for_agent` * Gain real-time insights into AI agent performance directly through popular MCP clients like Claude Desktop, Cursor, and Slack. * Effortlessly track guideline adherence, performance issues, and knowledge gaps. *Example*: Quickly identify the number and types of compliance issues within your Customer Support agents over the past week. #### `get_session_analysis` * Get the full session analysis and transcript (if allowed) for a specific session ID. * This includes all messages, metadata, ratings, knowledge gaps, guideline violations, sentiment, and other session data. #### `get_improvement_suggestions_for_agent` * Provide developers actionable recommendations based on real-time analytics embedded directly into IDEs like Claude Code, Cursor, Windsurf, or Zed. * Continuously improve AI agent performance through practical insights. *Example*: Automatically suggest improvements to address common performance issues based on ongoing session analysis. #### `list_agents` * Retrieve a complete list of all Agents being supervised by Wayfound for your organization. *Example*: Get a quick status update on your organization’s agents. #### `get_agent_details` * Retrieve the details of a specific Agent in your Wayfound organization. * Includes the agent’s role, goal, guidelines, and sentiment tuning. *Example*: Understand how each agent is configured and what their purpose in your organization is. #### `get_session_tags_for_agent` * Retrieve the tags and labels for an agent. *Example*: Get a list of tags or labels that can be used for questions related to tag or label filtering. #### `list_sessions` * Retrieve sessions with provided include or exclude filtering for tags or labels. *Example*: Get a list of sessions for a given tag/label query. ### Business Benefits of Wayfound MCP * Rapid Deployment: Dramatically reduce the time required to integrate and operationalize AI agents. * Enhanced Productivity: Automate routine compliance and management tasks, allowing your team to focus on strategic priorities. * Proactive Compliance: Consistently maintain high standards and quickly address guideline deviations. ### Who Should Use Wayfound MCP? * Business Leaders: Gain clarity and understanding over AI agent operations without technical complexity. * Developers: Receive precise, real-time improvement recommendations directly within your workflow. Take direct action using your MCP enabled coding agent like Claude Code or Cursor. * Compliance Teams: Ensure continuous adherence to guidelines and regulations effortlessly. # Using ### MCP Server Address The Wayfound MCP server can be accessed at: ``` https://app.wayfound.ai/mcp ``` ### Authentication You can authenticate into Wayfound using either OAuth or an API key. Both methods can be done on the Wayfound Settings/Connections screen You can create an MCP API key in the Wayfound Settings/API screen
#### OAuth An organization admin can enable OAuth for their organization by toggling the "Wayfound MCP Server OAuth" switch in the blue banner. #### API Key The MCP API key must be passed as an Authorization bearer token. Be sure to use Key Type: MCP
### Cursor Example Add the following to your mcp.json file. Choose the example that matches your authentication method: #### API Key ``` { "mcpServers": { "wayfound": { "command": "npx", "args": [ "mcp-remote", "https://app.wayfound.ai/mcp", "--header", "Authorization:${AUTH_HEADER}" ], "env": { "AUTH_HEADER": "Bearer " } } } } ``` #### OAuth ``` { "mcpServers": { "Wayfound": { "url": "https://app.wayfound.ai/mcp", "transport": "http" } } } ``` ### Claude Code Example Run the following command line to add Wayfound MCP server to Claude Code: #### API Key ``` claude mcp add --transport http wayfound https://app.wayfound.ai/mcp --header "Authorization: Bearer " ``` #### OAuth ``` claude mcp add --transport http wayfound https://app.wayfound.ai/mcp ``` # Organizations ### Navigating Organizations (All Users) An organization is an instance of the platform. Organizations can host multiple users, and every user in an organization can view and manage all of its agents. All users in an organization can also access the organization's currently active tools and integrations. Customers may want to create multiple organizations on the platform in order to split users and/or agents. Customers can create multiple organizations with an Enterprise plan. To do so, contact Wayfound. An individual user can also join multiple organizations on the Wayfound platform. If you belong to multiple organizations, you can switch between them without signing out. To do so, hover the mouse over your email at the bottom left corner of the screen. A menu of organizations will appear. Select the organization you wish to use. You can always check which organization you're in by mousing over your name and viewing the menu. ### Organizations settings (Admin only) Users with admin permissions can view their organization details in the settings menu. In the organizations tab, you can update the company name and view your Customer Support ID, Wayfound Plan, and the amount of Credits used. # User Management Those with admin level permissions can access the Users page in the settings menu. This page displays a table of all users and their statuses. The page also allows you to manage users and their roles in your organization.
### Roles There are currently two roles on the platform: **Users** can access the Supervisor, Agents, Sessions, and Visitors pages. They can create and manage agents and use integrations and actions. **Admins** have all abilities available to users. They can also access their organization's settings, where they can manage users, add or remove actions and integrations, and view API information. ### Adding Users You can invite new users by clicking at the top-right corner of the page. Enter the new user's email address in the pop-up window to send them an invitation to join the platform. Check the **Admin** box to give them the admin role. When left unchecked, the user will join the organization as a "user." ### Changing a user's status To change a user's status on the platform, an admin can click the edit user button on the right-hand side of the table. This opens the Edit User menu:
Here, you can check or un-check Admin to switch the user's status. You can also toggle whether the user receives real-time and daily agent alerts for agents. # Okta Integration # Inline Supervision ## Inline Supervision Wayfound normally analyzes sessions asynchronously: your agent posts a completed session, the API returns immediately, and the Supervisor's analysis shows up in the dashboard a short time later. That's the right default for observability, but sometimes you want the analysis *before* your agent takes its next step. **Inline supervision** turns Wayfound into an active checkpoint in your agent's control flow. By setting `async: false` when creating a session, the API blocks until the Supervisor has finished analyzing it and returns the full analysis with grade, guideline results, sentiment, and more in the response. Your code can then branch on those results: hand work to the next agent, retry a bad response, escalate to a human, or proceed. ### How it works Create a session the same way you always do — `POST https://app.wayfound.ai/api/v2/sessions` with your agent ID and messages — but add `"async": false` to the request body: ```json { "agentId": "your-agent-uuid", "async": false, "messages": [ { "timestamp": "2026-07-23T10:00:00Z", "event_type": "user_message", "attributes": { "content": "Can you cancel my subscription and refund this month?" } }, { "timestamp": "2026-07-23T10:00:04Z", "event_type": "assistant_message", "attributes": { "content": "I've submitted your cancellation. A full refund for this month will be processed within 3–5 business days." } } ] } ``` Instead of returning right away with just a session ID, the request blocks until analysis completes and returns the analyzed session. The fields your inline logic will care about most: ```json { "id": "session-uuid", "agentId": "agent-uuid", "grade": { "grade": "B", "explanation": "The agent resolved the request but promised a refund timeline it cannot guarantee." }, "compliance": [ { "guideline": "Never commit to a specific refund timeline; refunds are processed by the billing team.", "guidelineType": "content", "guidelineSource": "agent", "guidelinePriority": "red", "messageId": "assistant_1", "result": { "compliant": false, "reason": "The assistant promised a refund within 3–5 business days, which commits to a specific timeline." } }, { "guideline": "Always confirm the user's intent before making account changes.", "guidelinePriority": "yellow", "result": { "compliant": true, "reason": "The assistant acted on an explicit, unambiguous request." } } ], "sentiment": { "value": "neutral", "reason": "..." }, "knowledgeGaps": [], "tags": ["cancellation", "refund"] } ``` #### Reading guideline results Every guideline configured for the agent (including global guidelines) appears in the `compliance` array. Each entry's `result.compliant` is one of three values: | Value | Meaning | | ------- | ----------------------------------------------------- | | `true` | The session complied with the guideline | | `false` | The session violated the guideline | | `null` | Not enough conversation to determine (not applicable) | For gating decisions, match entries by their `guideline` text of the guideline as you authored it in Wayfound and treat `null` however your use case demands (usually as a pass, since the guideline didn't come into play). > **Tip:** In most inline-supervision setups you gate on one or two specific **red-priority** guidelines and let everything else flow to the dashboard asynchronously. The fewer guidelines you gate on, the simpler and more predictable your control flow. ### Pattern 1: Orchestrator gate In a multi-agent system, an orchestrator assembles context and dispatches work to specialized workers. Inline supervision lets the orchestrator verify its own output. For example: the plan, the assembled context, the task framing against your guidelines *before* a worker acts on it. ``` User request │ ▼ Orchestrator assembles context / plan │ ▼ POST /api/v2/sessions (async: false) ──► Wayfound analyzes │ ├─ gated guideline compliant ──► dispatch to worker agent └─ gated guideline violated ──► revise plan / escalate / halt ``` ```python import requests WAYFOUND_URL = "https://app.wayfound.ai/api/v2/sessions" HEADERS = { "Authorization": f"Bearer {WAYFOUND_API_KEY}", "Content-Type": "application/json", } GATED_GUIDELINE = "Worker tasks must never include raw customer payment details." def check_before_dispatch(orchestrator_messages): response = requests.post( WAYFOUND_URL, headers=HEADERS, json={ "agentId": ORCHESTRATOR_AGENT_ID, "async": False, "messages": orchestrator_messages, }, timeout=90, ) if response.status_code == 202: # Analysis didn't finish within the synchronous window — # poll GET /api/v2/sessions/{id}, or apply your fallback policy. return handle_pending(response.json()["id"]) response.raise_for_status() session = response.json() gate = next( (g for g in session["compliance"] if g["guideline"] == GATED_GUIDELINE), None, ) # compliant is True, False, or None (guideline not applicable) if gate and gate["result"]["compliant"] is False: return { "proceed": False, "reason": gate["result"]["reason"], "sessionId": session["id"], } return {"proceed": True, "sessionId": session["id"]} result = check_before_dispatch(messages) if result["proceed"]: dispatch_to_worker(task) else: revise_and_retry(task, feedback=result["reason"]) ``` Because every check is a real Wayfound session, you also get a complete audit trail: every dispatch decision, including the ones that were blocked is recorded, graded, and visible in the dashboard. ### Pattern 2: Grade-and-retry before the user sees a response For user-facing agents, inline supervision can act as a quality gate between your LLM and the user: generate a candidate response, have Wayfound grade it, and only show it to the user if the guideline you care about passed. If it failed, feed the violation reason back into the prompt and regenerate. ``` User message ──► LLM generates candidate response │ ▼ POST /api/v2/sessions (async: false) │ ┌───────────┴────────────┐ ▼ ▼ guideline passed guideline failed │ │ ▼ ▼ send to user regenerate with the violation reason as feedback (max N tries) ``` ```typescript const GATED_GUIDELINE = 'Responses must never provide specific legal or tax advice.'; const MAX_ATTEMPTS = 3; async function respondWithSupervision(userMessage: string): Promise { let feedback: string | null = null; for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) { const candidate = await generateResponse(userMessage, feedback); const res = await fetch('https://app.wayfound.ai/api/v2/sessions', { method: 'POST', headers: { Authorization: `Bearer ${process.env.WAYFOUND_API_KEY}`, 'Content-Type': 'application/json', }, body: JSON.stringify({ agentId: AGENT_ID, async: false, metadata: { attempt }, messages: [ { timestamp: new Date().toISOString(), event_type: 'user_message', attributes: { content: userMessage }, }, { timestamp: new Date().toISOString(), event_type: 'assistant_message', attributes: { content: candidate }, }, ], }), }); if (res.status === 202) { // Analysis still running past the synchronous window. // Fallback policy: hold the response, or release it and // review asynchronously — your call per use case. return applyPendingPolicy(candidate, (await res.json()).id); } const session = await res.json(); const gate = session.compliance?.find((g: any) => g.guideline === GATED_GUIDELINE); // Passed (or guideline wasn't applicable) — release the response. if (!gate || gate.result.compliant !== false) { return candidate; } // Failed — feed the Supervisor's reason back into the next attempt. feedback = gate.result.reason; } return fallbackResponse(); // e.g. safe canned reply or human handoff } ``` Each attempt is recorded as its own session, so you can see in the dashboard exactly how often the gate fires and what the rejected candidates looked like. Passing an attempt counter in `metadata` makes retries easy to correlate. ### Handling latency and the pending case Synchronous analysis takes as long as the Supervisor needs to evaluate the transcript — typically several seconds, longer for long sessions. Two things to build for: * **Set a generous client timeout.** The request blocks while analysis runs; a 90-second client timeout is a reasonable starting point. * **Handle `202 Accepted`.** If analysis doesn't finish within the synchronous window, the API returns `202` with `{ "id": "...", "status": "processing" }` instead of holding the connection open. Poll `GET /api/v2/sessions/{id}` until `lastProcessedAt` is set, or fall back to your default behavior. Decide your fallback policy up front. This is an important design choice in an inline-supervision architecture: * **Fail closed** (block until you have a verdict) for high-stakes gates: compliance-sensitive content, irreversible actions, financial commitments. * **Fail open** (proceed, review asynchronously) for latency-sensitive user experiences where the gate is a quality improvement rather than a hard requirement. ### Best practices * **Gate on few, specific guidelines.** Inline logic should hinge on one or two red-priority guidelines with crisp, testable language. Broad or subjective guidelines belong in your asynchronous review flow, not in a gate. * **Match on exact guideline text.** The `guideline` field contains the guideline exactly as authored. If you edit the guideline in Wayfound, update the string in your code — or centralize it in configuration. * **Treat `null` deliberately.** `compliant: null` means the guideline didn't apply to this session. For most gates that's a pass; for gates like "the response must always include a disclaimer," you may want to treat it as a failure. * **Use `async: false` only where the verdict changes behavior.** Everything else should stay asynchronous — you keep full observability either way, without adding latency. * **Test your gates in Test Mode.** Use [Test Mode](https://docs.wayfound.ai/agents/test-mode) with expected outcomes to verify your gated guidelines fire when they should before wiring them into production control flow. For the complete request and response schema, see the [Create Session API reference](https://wayfound-api.readme.io/reference/create-completed-session). # API Wayfound offers APIs to connect your agents to the platform. You can generate an API key on the **Connections tab** in the **Settings** page. Access to this page requires admin permissions on the Wayfound platform. [Learn more about implementing Wayfound's APIs here](https://wayfound-api.readme.io/reference/get-agents). For more information about connecting agents to Wayfound in order to leverage the API see here: {% content-ref url="/pages/LMfFTqvWL1AxGaLXj0Qt" %} [Connecting Agents](/agents/connecting-agents) {% endcontent-ref %} ## Creating and Configuring Supervisor Agents Examples for creating supervisor agents and setting their **role**, **goal**, and **guidelines** through the Wayfound public **v2** API. ### Authentication All requests use a Bearer token in the `Authorization` header. The token is a Wayfound **API key** (a UUID), created under **Settings → Connections**. ``` Authorization: Bearer 550e8400-e29b-41d4-a716-446655440000 Content-Type: application/json ``` Notes: * The key must be UUID-format or you get `401 Unauthorized: Invalid API key`. * **MCP-type keys are rejected.** Use a standard API key. * The key scopes every request to its organization automatically. ### Fields | API field | Type | Required | Notes | | ------------ | --------- | -------- | ------------------------------------------------------------ | | `name` | string | yes | Display name of the agent. | | `role` | string | no | Who the agent is / its persona. Stored as the description. | | `goal` | string | no | What the agent is trying to accomplish. | | `guidelines` | object\[] | no | Rules the supervisor evaluates sessions against (see below). | ### Guideline Object | Field | Type | Required | Notes | | ---------- | ------ | -------- | ---------------------------------------------------- | | `type` | string | yes | One of the guideline types below. | | `content` | string | yes | The rule text. Cannot be empty / whitespace-only. | | `priority` | string | yes | `"medium"` or `"high"`. | | `context` | string | no | Optional extra context for the rule. Defaults to "". | #### Valid Types | `type` | Meaning | | ------------------ | -------------------- | | `prohibitedAction` | Prohibited actions | | `prohibitedWords` | Prohibited words | | `preferredVoice` | Preferred voice/tone | | `formatting` | Formatting rules | | `security` | Security rules | | `bestPractices` | Best practices | | `aiDisclosure` | AI disclosure | | `otherEvaluation` | Other evaluation | **Valid `priority` values:** `"medium"` or `"high"`. ### 1. Create a supervisor agent (role + goal + guidelines) `POST /api/v2/agents` ```bash curl -X POST https://app.wayfound.ai/api/v2/agents \ -H "Authorization: Bearer $WAYFOUND_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "Banking Assistant", "role": "Front-line virtual assistant for Atlas Bank retail customers.", "goal": "Answer everyday banking questions quickly and safely while verifying identity before sharing any account-specific details.", "guidelines": [ { "type": "security", "content": "Verify the customer'\''s identity (name plus one security factor) before sharing any account-specific details such as balance, transactions, or card numbers.", "priority": "high" }, { "type": "prohibitedAction", "content": "Never read back a full card number, full account number, or SSN. Only the last 4 digits.", "priority": "high" }, { "type": "aiDisclosure", "content": "Always identify yourself as an AI assistant. Never imply or claim to be a human employee.", "priority": "high" }, { "type": "bestPractices", "content": "For a disputed transaction, hand off to the Disputes & Transactions specialist.", "priority": "medium" } ] }' ``` **Response — `200 OK`** ```json { "id": "8c0e89d0-2c7a-4c3e-9c6b-9f8a5d1e7c2a", "createdAt": "2026-06-23T14:32:15.123Z" } ``` `id` is the agent's UUID — use it for all subsequent calls. #### Minimal create (name only) `name` is the only required field. Role, goal, and guidelines can be added later with a PUT. ```bash curl -X POST https://app.wayfound.ai/api/v2/agents \ -H "Authorization: Bearer $WAYFOUND_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "Disputes & Transactions" }' ``` *** ### 2. Update role, goal, or guidelines `PUT /api/v2/agents/{agentId}` Send only the fields you want to change. Each PUT republishes the agent. **Update the goal only:** ```bash curl -X PUT https://app.wayfound.ai/api/v2/agents/8c0e89d0-2c7a-4c3e-9c6b-9f8a5d1e7c2a \ -H "Authorization: Bearer $WAYFOUND_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "goal": "Resolve account questions and money-movement requests within one session, escalating anything that fails identity verification." }' ``` **Replace the full guideline set:** > `guidelines` is replaced wholesale, not merged. Always send the complete list you want the agent to have. ```bash curl -X PUT https://app.wayfound.ai/api/v2/agents/8c0e89d0-2c7a-4c3e-9c6b-9f8a5d1e7c2a \ -H "Authorization: Bearer $WAYFOUND_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "role": "Atlas Bank specialist for disputes and money movement.", "guidelines": [ { "type": "security", "content": "Re-verify identity before opening a dispute or moving money.", "priority": "high" }, { "type": "bestPractices", "content": "For a disputed transaction, open a dispute case and explain the timeline to the customer.", "priority": "medium" } ] }' ``` **Response — `200 OK`** ```json { "id": "8c0e89d0-2c7a-4c3e-9c6b-9f8a5d1e7c2a", "createdAt": "2026-06-23T14:32:15.123Z", "updatedAt": "2026-06-23T15:04:51.880Z" } ``` > **Archiving is exclusive.** `{ "archived": true }` (or `false`) must be sent on its own — combining it with any other field returns `400`. ### 3. Read agents back **List all agents:** ```bash curl https://app.wayfound.ai/api/v2/agents \ -H "Authorization: Bearer $WAYFOUND_API_KEY" ``` **Get one agent (add `?detail=full` for sections/directives):** ```bash curl https://app.wayfound.ai/api/v2/agents/8c0e89d0-2c7a-4c3e-9c6b-9f8a5d1e7c2a \ -H "Authorization: Bearer $WAYFOUND_API_KEY" ``` **Response (shape):** ```json { "id": "8c0e89d0-2c7a-4c3e-9c6b-9f8a5d1e7c2a", "createdAt": "2026-06-23T14:32:15.123Z", "updatedAt": "2026-06-23T15:04:51.880Z", "name": "Banking Assistant", "role": "Front-line virtual assistant for Atlas Bank retail customers.", "goal": "Answer everyday banking questions quickly and safely...", "guidelines": [ { "uuid": "a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d", "type": "security", "priority": "high", "content": "Verify the customer's identity..." } ] } ``` Note that on read, `priority` comes back as `"medium"`/`"high"`. ### Validation & error reference | Status | Body | Cause | | ------ | --------------------------------------------------------------------------------- | --------------------------------------------- | | `400` | `{ "error": "Missing name field" }` | `name` omitted on create. | | `400` | `{ "error": "Invalid guideline type: . Valid types are: ..." }` | `type` not in the allowed list. | | `400` | `{ "error": "Invalid priority: . Valid priorities are: medium, high" }` | `priority` not `medium`/`high`. | | `400` | `{ "error": "Guideline content cannot be empty for