Inference against a custom dataset

[rstarmer]

If possible extend PyKoi (or a similar toolset) to inference against any/all of the following datasets:

• ArXiv
• pubmed
• wikipedia

[sdake]

@rstarmer do you permit changes in the input format? We generally have far more elaborate user stories. I guess we didn't do that here. Currently, for storage of retrieval information, jsonl is an easy path forward. Let me see if I can enhance this user story a bit.

Please also note, that processing realnews took quite a bit of time. I have found the wikipedia jsonl files, and it will take about (roughly guessing) the same compute....