Optimize to speed up inference evaluation tests

[betogaona7]

ATM, the evaluation is being done sequentially, we load the model, process the 1000 prompts from the db, and then unload the model to release the memory, so we can load the next one.

Instead, of processing prompt after prompt, we could explore adding support to process batches aiming an speed up.

[betogaona7]

Batching didn't work, will continue finalizing the threaded_evaluation demo

[betogaona7]

Time processing 10 prompts:
sequential: 324s
batched (size=2): 316s
threads (concurrent calls): 355s