HuggingFace Models in Javascript: Using different tools, it is possible to use LLM models in Javascript projects, all locally
- Hugginface
- Nestjs
- MongoDB
- Python
- ONNX
- Transformerjs
$ yarn installcopy .env.example and rename to .env
MONGODB_ATLAS_URI='uri'cd src/
python -m scripts.convert --quantize --model_id Xenova/all-MiniLM-L6-v2Use Conda environment
conda create --name vectors python=3.10.12
conda activate vectors
pip install transformers==4.33.2
pip install onnxruntime==1.15.0
pip install optimum==1.13.2
pip install tqdm
pip install onnx==1.13.1
# development
$ yarn run start
# watch mode
$ yarn run start:dev
# production mode
$ yarn run start:prodPOST http://localhost:3000/vectors
GET http://localhost:3000/vectors/search?word=red- Twitter/X - @GeraDeluxer
Nest is MIT licensed.