We open-sourced the evaluation and tracing framework behind AgentX.
It gives you:
- Tracing of agent runs with OpenTelemetry-compatible spans that understand reasoning steps, tool calls, and retrieval, not just HTTP requests.
- Offline evals against golden datasets and online evals on production traffic.
- LLM-as-judge scoring with grounding, factuality, and hallucination metrics, plus human feedback loops to tune the judges themselves.
- Model portability analysis, so you can see how an agent behaves when you swap the underlying model.
The code is on GitHub at AgentX-ai/AgentX-Trace-Eval. Stars, issues, and pull requests are all welcome.