Perplexity Research has unveiled a novel approach to improving AI agent performance by training its computer agent on real-world user mistakes. The technique, detailed in a recent post-training study, involves leveraging actual user sessions—including those that result in failures—to refine the model's decision-making capabilities.
Rejection Sampling and Self-Distillation
The method combines rejection sampling fine-tuning with hint-guided self-distillation. Rejection sampling allows the system to identify and learn from incorrect tool calls, while self-distillation helps the model internalize the correct behavior through a process of knowledge transfer from a larger, more capable teacher model to a smaller, more efficient student model. This hybrid approach ensures that the AI agent not only avoids repeating past errors but also improves its understanding of when and how to correctly execute tasks.
Performance Gains in Live Testing
In a live A/B test, the technique demonstrated measurable improvements. Tool-call failures dropped from 2.24% to 1.77%, representing a statistically significant 21.2% reduction. This success underscores the value of incorporating real user feedback into the training loop, particularly in dynamic environments where agents must adapt to unexpected or ambiguous inputs.
Implications for AI Development
This work marks a shift toward more robust and adaptive AI systems, especially in applications involving complex tool interactions. By learning from its own missteps, Perplexity's agent becomes more reliable and efficient, setting a new standard for how AI models can be fine-tuned in production environments. The approach could be especially impactful for AI assistants and agents that rely on tool usage to perform tasks, such as code generation, data analysis, or customer service automation.