Immediate focus points

[Kishlay-notabot]

1. Ease of installation (terminal usage).
2. GUI development (contributors are welcome and much needed!)
3. more important features listed in src/technical.md
4. Cross platform support.

[Kishlay-notabot]

Finetuning! Model kinda sucks at some cases now that I dont use langextract.

[Kishlay-notabot]

The base model performance has dropped significantly ever since I have stopped using LangExtract. Currently trying out two implementations, the base one being on the main branch and the other on feature/chunking.

[Kishlay-notabot]

I wrote eval scripts for 2 of my implementation branches. Both have the same input prompts but different response structures.

feature/chunking branch implementation:

Model: Qwen3-0.6B-Q8_0.gguf
Total Tests: 45
Success (Perfect): 2 (4.4%)
Failed (Critical): 31 (68.9%)
Warning (Minor Issues): 12 (26.7%)

main branch implementation:

Model: Qwen3-0.6B-Q8_0.gguf
Total Tests: 45
Success (Perfect): 6 (13.3%)
Failed (Critical): 31 (68.9%)
Warning (Minor Issues): 8 (17.8%)

1. The chunking ones have lesser success rates because the output json is complex, relatively.
2. The number for failed ones magically doesn't change. (something to do with temp? but the prompt and structure is different though)
3. I'll run both the tests again just to make sure I didn't mix the numbers up because same failure count is weird. I have detailed test results json files, I'll push them later on with clean code.

Personally if I had to choose one out of the two, I'd still use the chunking one because it is more robust and easier to have a grasp on, in terms of control in keeping track of. randomness of the output etc.

[Kishlay-notabot]

Upon testing with the legacy non chunking implementation (llama.cpp json schema), doubling the context length and providing many examples to the system prompt gives amazing results. The score jumped from 13/30 to 24/30, but the tokens/second slashed from 500t/s to 250t/s (local inference on Mac M4 Air)