Feature request: Support for structured outputs (or at least passing additional arguments)

[dhicks]

As of December, Ollama has support for structured outputs. ollamar supports this via the output and format arguments: https://hauselin.github.io/ollama-r/articles/ollamar.html#structured-outputs. It doesn't seem like llm_custom() has a way to access this feature currently.

[dhicks]

Spent a few minutes digging around the code and thinking about how to approach this. Adding ellipses and a llm_args argument to a tall stack of function calls seemed too fragile. I realized that use_llm() could accept arbitrary arguments, so something like this seemed promising:

format <- list(
  type = "object",
  properties = list(
    name = list(type = "string"),
    capital = list(type = "string"),
    languages = list(type = "array",
                     items = list(type = "string")
    )
  ),
  required = list("name", "capital", "languages")
)

llm_use('ollama', 'llama3.2', 
              output = 'structured', 
              format = format)

llm_vec_custom('', 'tell me about Canada')

However, m_backend_submit.mall_ollama() assumes we're working with text/character responses at least twice: once on line 31 as output = "text" and on 24 as map_here <- map_chr. The first case results in an argument collision before calling the LLM, and the second in a type error when parsing the response.

So one approach might be to (a) set output = "text" when calling llm_use() unless the user explicitly specifies otherwise; (b) remove the line where output = "text" is set in m_backend_submit(), and (c) add some logic to check m_defaults_args(backend)$output in appropriate places.

Possibly llm_use() just needs to add a few lines above line 91 (start of the m_defaults_set() call) to take care of (a). But I don't have any more time to poke at this today. Hopefully these notes are useful!