PERFORMANCE: Make AIP runnable through Dask or another platform to parallize the parsing

[lfdversluis]

Each sub-set of data and each data source can be processed in parallel. Dask can be used to parallelize this.

[lfdversluis]

https://joblib.readthedocs.io/en/latest/ Seems promising.

[lfdversluis]

Perhaps investigating if the XML file and the JSON files of Semantic Scholar / AMiner can be processed at an item-level parallelization might me interesting. With joblib linked above, file-level parallelization becomes possible, yet the JSON files are structured in such a way that each line in the file is one (standalone) JSON object. Perhaps parsing these in parallel is even faster.

[lfdversluis]

Setting up some benchmarks + regression tests might be a nice idea as well.