Remove DP in inference #26
Description
Some parts of the codebase use DP to distribute inference workload over multiple devices. This is inefficient as DP re-distributes the model to all devices on every batch.
We should check where it is uses and change to manually placing models on devices and manually sending and syncing batches.