create dev environment:
- conda create poke-llm
- conda activate poke-llm
- pip install -r requirements.txt
- ollama list
- ollama create model-name -f Modelfile
- use finetune_model.py to perform LoRA finetune with model on .JSON dataset. Script will output a lora file
- Merge the finetuned lora file with the base model to create the finetuned model. This will output a merged folder
- Navigate to llama.cpp and use the script 'convert_hf_to_gguf_update.py' with command - python3 convert_hf_to_gguf_update.py --outtype f16 --outfile ./your-model.gguf ./merged-model
- python pokemon-chatbot.py in /scripts