Usage without HuggingFace?

[kiseloMlyako]

Would it be possible to use an API for the model calls and tokenizations instead of HF models?

[bencw99]

Is there any model in particular you were hoping to use? As long as the model provides the ability to run inference and obtain log-probabilities, it's possible to run ContextCite. However, many API-only models (e.g., OpenAI) don't provide the ability to run inference over an arbitrary tokens (as we need to do for the ablations performed by ContextCite), so it's not possible to run ContextCite with these.

[kiseloMlyako]

Oh I see, my hopes were for the usage of the OpenAI API. Read through the blog posts, they are really well explained, and saw that even when using OpenAI in the examples it was for the initial generation only, but is then substituted for the following generations. Thank you very much, amazing project!