When predicting, we are just loading all the predictions into memory. This will run OOM fast on smaller machines.