Add Local Model Support

[RedLordezh7Venom]

Local Model support is necessary for privacy, there are the following ways

• Ollama, llama.cpp : bundled with the installation, but hard to setup, with dependencies
• Transformers : could lead to dependency issues

best :

• Llamafile : Use llamaindex with llamafile for easy setup and run LLM, no other deps , just one file runs the model,
• it is based on llama.cpp, ggml , compose gcc , that bundles all of them into one exe file for linux, windows, android
• just install model and run, easily integratable into the setup

[RedLordezh7Venom]

Fix : use in langchain, with gemma
1st. groq, then llamafile

[RedLordezh7Venom]

Edit

llamafile is not helping at all, the concept of llamafile is good but doesn't work as expected, need interface for it, ollama is still better

[RedLordezh7Venom]

suggested llamafile : qwen2.5 coder 1.5b instruct, or ollama gemma1b