Import Trajectories from Braintrust
If your agent already sends traces to Braintrust, Zep can import those traces as Trajectories. You do not have to add capture code to the agent first. Imported Trajectories go through the same summary, compilation, admission, and retrieval steps as captured Trajectories. An import does not create or admit a Skill directly.
You can do these steps in the Dashboard or with the SDK. In the Dashboard, add the Braintrust credential in the project settings under Trace Connections. Then open the Agent and add an import.
How an import works
- A trace connection stores one Braintrust credential for the project. Zep verifies the credential before it stores it, and Zep never returns the credential.
- A Trajectory import selects Braintrust traces for one Agent. A one-time import reads a list of trace IDs or the traces that match a filter. A scheduled import reads new traces that match a filter at a fixed interval.
- A mapping tells Zep how to set the task family, the objective, and the outcome of each Trajectory from the trace data.
- Each run of the import creates or updates one Trajectory for each trace.
Connect Braintrust
Use a Braintrust service token that can read only the projects that you import. A Braintrust API key also works.
The project list is paged. The SDK reads the next pages when you iterate. If you change the credential of a connection later, the new credential must belong to the same Braintrust organization. Zep returns HTTP 409 Conflict for a credential from a different organization.
Map traces to Trajectories
The mapping object sets the Trajectory fields from each trace:
In these examples, agent is an Agent that you created, as in the quickstart.
Use a Braintrust score that your evaluation already computes. A score rule sets external verification, so a successful run can increase the trust of a Skill. A Trajectory with outcome unknown gives a summary, but it gives weak evidence for admission. For more about outcomes and verification, see Capture Trajectories.
To test a mapping before you save an import, read one trace with the mapping. The preview field of the response shows the task family, objective, outcome, and verification that the mapping gives. If the run will skip the trace, preview shows the skip reason.
To find trace IDs, list the traces of a project with trace_connection.trace.list. The list accepts a filter on start time, tags, metadata, scores, and errors.
Import traces one time
A one-time import starts a run immediately. Use it to learn from a set of traces that you selected. Send up to 1,000 trace IDs, or send a filter instead of trace_ids. A filter must contain started_after, started_before, or both.
Import new traces on a schedule
A scheduled import keeps learning from production traffic without a change to the agent. An import with a schedule is a scheduled import. The filter must contain a time bound, as in a one-time import. Each run reads the traces that match the filter and that changed after the previous run. Zep waits until a trace stops changing before it imports the trace.
The interval is 1, 4, 12, or 24 hours. start_from is now or an RFC 3339 time. To also import earlier traces, set start_from to a time in the past. To start a run before the next interval, call agent.trajectory_import.run.create.
Check the results
Each run reports counts of the traces that it imported, updated, skipped, or could not import, and a count for each skip reason. Use agent.trajectory_import.run.list and agent.trajectory_import.run.get to read the runs. Use agent.trajectory_import.run.item.list to read the result and the reason for each trace. A skip is a decision of a mapping rule, for example a trace with no task family. A skip is not an error.
To list the Trajectories of one import, call agent.trajectory.list with import_uuid. Then follow the learning state of the Agent, as you do for captured Trajectories.
Review imported evidence
Imported Trajectories follow the approval settings of the Agent. To require human review for each candidate that uses evidence from one import, set require_review=True on the import. This rule applies also when the Agent approval is auto. For more about approval, see Configure Learning and Admission.
Pause or delete an import
When you delete an import, Zep keeps its Trajectories by default. When you set trajectories="delete", Zep deletes them asynchronously. When you delete a trace connection, Zep keeps all of the imported Trajectories.