Run Async Vs Sync Tasks and Evals
Experiments can be run as either Synchronous or Asynchronous.
We recommend:
Synchronous: Slower but easier to debug. When you are building your tests these are inherently easier to debug. Start with synchronous and then make them asynchronous.
Asynchronous: Faster. When timing and speed of the tests matter. Make the tasks and/or Evals asynchronous and you can 10x the speed of your runs.
Synchronous
Code errors for synchronous tasks break at the line of the error in the code. They are easier to debug and we recommend using these to develop your tasks and Evals.
The synchronous running of an experiment runs one after another.
The above shows a synchronous run of a task and Eval with a linear run of task and Eval.
The task and eval functions defined above are synchronous because they do not have 'async'.
Async
The asynchronous running of an experiment runs paralell.
The above shows a asynchronous run of a task and Eval with a parallel run of task and Eval.
Last updated