Skip to main content
Judge the path and the report, half each. Based on the trajectory preset. A research agent: it searches, reads and follows leads in whatever order the evidence suggests, then writes a report. No single step is right or wrong on its own, so the path is judged as a whole, as the trajectory preset does: one judge call per trace and subject sees the subject’s step at each point beside the baseline’s, and asks whether the subject could have stood in for the baseline; guidance tells it how much a different path matters here. The report is what the user reads, so a criteria group grades the final reply on its own terms, and replies: mean brings that into the trace’s score: the mean of the verdict and the report’s score. Compare the coding agent, which combines a verdict with rules that must hold at every reply instead (all-pass). An example to learn from, not a starting point: start a config from ambral tasksets scoring preset, and take from this one the choices that fit your agent.