Import Trajectories from Braintrust
If your agent already sends traces to Braintrust, Zep can import those traces as Trajectories. You do not have to add capture code to the agent first. Imported Trajectories go through the same summary, compilation, admission, and retrieval steps as captured Trajectories. An import does not create or admit a Skill directly.
You can do these steps in the Dashboard or with the SDK. In the Dashboard, add the Braintrust credential in the project settings under Trace Connections. Then open the Agent and add an import.
How an import works
- A trace connection stores one Braintrust credential for the project. Zep verifies the credential before it stores it, and Zep never returns the credential.
- A Trajectory import selects Braintrust traces for one Agent. A one-time import reads a list of trace IDs or the traces that match a filter. A scheduled import reads new traces that match a filter at a fixed interval.
- A mapping tells Zep how to set the task family, the objective, and the outcome of each Trajectory from the trace data.
- Each run of the import creates or updates one Trajectory for each trace.
Connect Braintrust
Use a Braintrust service token that can read only the projects that you import. A Braintrust API key also works.
The project list is paged. The SDK reads the next pages when you iterate. If you change the credential of a connection later, the new credential must belong to the same Braintrust organization. Zep returns HTTP 409 Conflict for a credential from a different organization.
Map traces to Trajectories
The mapping object sets the Trajectory fields from each trace:
In these examples, agent is an Agent that you created, as in the quickstart.
Use a Braintrust score that your evaluation already computes. Set score_verifier to true to make the score an external verification. Then a successful run can support a Skill and increase its trust. Without score_verifier, a score rule sets agent verification, and Zep does not compile Skills from those Trajectories. A Trajectory with outcome unknown gives a summary, but it gives weak evidence for admission. For more about outcomes and verification, see Capture Trajectories.
To test a mapping before you save an import, read one trace with the mapping. The preview field of the response shows the task family, objective, outcome, and verification that the mapping gives. If the run will skip the trace, preview shows the skip reason.
To find trace IDs, list the traces of a project with trace_connection.trace.list. The list accepts a filter on start time, tags, metadata, scores, and errors.
Import traces one time
A one-time import starts a run immediately. Use it to learn from a set of traces that you selected. Send up to 1,000 trace IDs, or send a filter instead of trace_ids. A filter must contain started_after, started_before, or both.
Import new traces on a schedule
A scheduled import keeps learning from production traffic without a change to the agent. An import with a schedule is a scheduled import. The filter must contain a time bound, as in a one-time import, but the runs do not use that bound. start_from sets the earliest start time of the traces that the runs read. Each run reads the traces that match the other filter fields and that changed after the previous run. After a trace closes, Zep waits for settle_minutes with no new activity before it imports the trace.
An online scorer can add a score after a trace closes. Set settle_minutes to a time that is longer than the scorer delay. If Zep imports a trace before its score arrives, the Trajectory has no score verification, and Zep does not compile Skills from it.
The schedule has these fields:
interval_hours: 1, 4, 12, or 24.start_from:nowor an RFC 3339 time.nowis the time when you create the import. To also import earlier traces, setstart_fromto a time in the past.settle_minutes: the time, from 0 to 1,440 minutes, that a closed trace must have no new activity before Zep imports it. If you do not set this field, the value is 0.max_open_hours: the maximum time, from 1 to 168 hours, that Zep waits for an open trace. This field is required.
To start a run before the next interval, call agent.trajectory_import.run.create.
Check the results
Each run reports counts of the traces that it imported, updated, skipped, or could not import, and a count for each skip reason. Use agent.trajectory_import.run.list and agent.trajectory_import.run.get to read the runs. Use agent.trajectory_import.run.item.list to read the result and the reason for each trace. A skip is a decision of a mapping rule, for example a trace with no task family. A skip is not an error.
To list the Trajectories of one import, call agent.trajectory.list with import_uuid. Then follow the learning state of the Agent, as you do for captured Trajectories.
Review imported evidence
Imported Trajectories follow the approval settings of the Agent. To require human review for each candidate that uses evidence from one import, set require_review=True on the import. This rule applies also when the Agent approval is auto. For more about approval, see Configure Learning and Admission.
Pause or delete an import
- To stop the scheduled runs, call
agent.trajectory_import.pause. - To start the scheduled runs again, call
agent.trajectory_import.resume. - To delete the import and keep its Trajectories, call
agent.trajectory_import.delete. - To delete the import and its Trajectories, call
agent.trajectory_import.deletewithtrajectories="delete".
When you delete an import, Zep keeps its Trajectories by default. When you set trajectories="delete", Zep deletes them asynchronously. When you delete a trace connection, Zep keeps all of the imported Trajectories.