v0.9.6 - fix Llama 3 tokenizer

Breaking changes

None

New features

None

Bug fixes

• cappr.huggingface is compatible with Llama 3/3.1's tokenizer. It works around this issue using code from this PR (with small modifications). See the updated list of supported architectures here.

v0.9.5 - address deprecation of HF KV tuple

Breaking changes

None

New features

• cappr.huggingface.classify internally passes in a DynamicCache object if possible. This change gets rid of a warning you might see when running previous versions of CAPPr:

We detected that you are passing `past_key_values` as a tuple and this is deprecated and will be removed in v4.43. Please use an appropriate `Cache` class (https://huggingface.co/docs/transformers/v4.41.3/en/internal/generation_utils#transformers.Cache)

Bug fixes

None

v0.9.4 - don't repeat KV if possible

Breaking changes

None

New features

• cappr.huggingface.classify doesn't copy the prompt's KVs when broadcasting the prompt to completions if batch_size=1 or if you pass in a single prompt. Instead, it repeats a view of it. This change saves memory for tasks where there are many completions. For example, in the Banking 77 demo, peak reserved CUDA memory goes from 13.8 GB to 8.3 GB (~40% decrease), and peak allocated CUDA memory goes from 9.3 GB to 7.7 GB (~17% decrease).

Bug fixes

None

v0.9.3 - log-probs is an array when possible

Breaking changes

None

New features

• The agg_log_probs function returns a numpy array instead of a list of numpy arrays if there are a constant number of completions.

Bug fixes

None

v0.9.2 - arbitrary token log-prob aggregation

Breaking changes

None

New features

• The agg_log_probs function applies func instead of np.exp ∘ func. So if you want average token log-probabilities, set func=np.mean

Bug fixes

None

v0.9.1 - no setup.py

Breaking changes

• There's no setup.py file, in case you were relying on that.

New features

None

Bug fixes

None

v0.9.0 - don't require openai, tiktoken

Breaking changes

• pip install cappr will no longer install openai, tiktoken. Install them yourself, or install them using pip install "cappr[openai]". For previous versions of cappr, if you needed to install cappr without these dependencies, you had to run:

python -m pip install \
    "numpy>=1.21.0" \
    "tqdm>=4.27.0" && \
python -m pip install --no-deps cappr

• cappr.openai.api.Model no longer includes the deprecated text-* models

New features

None

Bug fixes

None

v0.8.8 - default axis for posterior prob

Breaking changes

None

New features

• The axis of posterior_prob defaults to the last one

Bug fixes

None

v0.8.7 - Llama CPP no need for logits_all=True

Breaking changes

None

New features

• You no longer need to instantiate your Llama CPP model with logits_all=True

Bug fixes

None

v0.8.6 - support LongLLaMA

Breaking changes

• Setting the internal past attribute of the cache to None now will cause an error to be raised if you try to use it again. Please use the original model instead

New features

• Support LongLLaMA
• repr for cached model
• Don't check logits from Llama CPP

Bug fixes

None