how to fit other llm

[zhanpengjie]

great work，I wonder how to train with other llm such as qwen2.5 3B？

[Ingvarstep]

@zhanpengjie , thanks. To use other llms such as qwen, you need to install llm2vec package. Ideally, you need to train it using masked next token prediction. After that, you can specify a model as a backbone for gliclass. We also open-sourced our pre-training dataset, here you can find it: https://huggingface.co/datasets/knowledgator/gliclass-v1.0.