User Feedback

When evaluating agent performance, the AI Supervisor follows the Guidelines set for each agent. The AI Supervisor may not always interpret the guidelines in ways that users anticipate when writing them. In cases such as this, Wayfound allows users to provide CLHF (continuous learning through human feedback) to the AI Supervisor to refine how it applies a guideline moving forward.

Feedback can be found when opening a recording of a session where the AI Supervisor identified a guideline violation. Sessions can be accessed directly in the Sessions page or through links in the Performance page. Along with a transcript of the interaction, each sessions displays the AI Supervisor's review, including highlights of any potential guideline violations:

If the guideline violation does not align with your expectations, you can refine it by clicking the feedback button next to the guideline violation bullet point. This opens a User Feedback window:

The User Feedback window displays the relevant guideline, the message identified as in violation of the guideline, and the AI Supervisor's explanation of the violation. Below is a text box for providing user feedback. The AI Supervisor will consider all user feedback associated with each guideline when applying them in all future reviews.