AWS
1 listing
Skills
Write evaluations for AI agents with Strands Evals
This skill from the Strands Agents project helps a coding agent write evaluations with the strands-agents-evals Python package. It covers setting up cases and experiments, choosing evaluators such as output quality, trajectory, helpfulness, faithfulness, tool selection or goal success, scoring from traces in CloudWatch, OpenSearch, OpenInference or LangChain, simulating users and tools, and finding the root cause of failures. You need the strands-agents-evals package installed. You get working evaluation code and test cases for your agent.