MLX Tool Call Failures due to Jinja Convert Failure

[james-333i]

When trying to use tools with MLX models (such as the exact weather example in the README), this runs through the function "applyChatTemplate" in swift-transformer Tokenizer. When it hits this block:

        if let tools {
            context["tools"] = try .array(tools.map { try Value(any: $0) })
        }

It throws:

runtime("Cannot convert value of type Optional to Jinja Value")

NOTE: The only change I made to the example was updating the description to:
Get the weather forecast for any location worldwide (city, state, or address). ALWAYS use this tool when a user asks about current weather or weather forecasts.

So that Qwen would recognize it MUST run this tool.

Here is the tools object:

Printing description of tools:
▿ Optional<Array<Dictionary<String, Any>>>
  ▿ some : 1 element
    ▿ 0 : 2 elements
      ▿ 0 : 2 elements
        - key : "function"
        ▿ value : 3 elements
          ▿ 0 : 2 elements
            - key : "name"
            - value : "get_weather"
          ▿ 1 : 2 elements
            - key : "description"
            - value : "Get the weather forecast for any location worldwide (city, state, or address). ALWAYS use this tool when a user asks about current weather or weather forecasts."
          ▿ 2 : 2 elements
            - key : "parameters"
            ▿ value : JSONValue
              ▿ object : 6 elements
                ▿ 0 : 2 elements
                  - key : "properties"
                  ▿ value : JSONValue
                    ▿ object : 1 element
                      ▿ 0 : 2 elements
                        - key : "city"
                        ▿ value : JSONValue
                          ▿ object : 2 elements
                            ▿ 0 : 2 elements
                              - key : "type"
                              ▿ value : JSONValue
                                - string : "string"
                            ▿ 1 : 2 elements
                              - key : "description"
                              ▿ value : JSONValue
                                - string : "The city to fetch the weather for"
                ▿ 1 : 2 elements
                  - key : "required"
                  ▿ value : JSONValue
                    ▿ array : 1 element
                      ▿ 0 : JSONValue
                        - string : "city"
                ▿ 2 : 2 elements
                  - key : "description"
                  ▿ value : JSONValue
                    - string : "Generated Arguments"
                ▿ 3 : 2 elements
                  - key : "type"
                  ▿ value : JSONValue
                    - string : "object"
                ▿ 4 : 2 elements
                  - key : "additionalProperties"
                  ▿ value : JSONValue
                    - bool : false
                ▿ 5 : 2 elements
                  - key : "$defs"
                  ▿ value : JSONValue
                    ▿ object : 1 element
                      ▿ 0 : 2 elements
                        - key : "Infinity_Vaultz.WeatherTool.Arguments"
                        ▿ value : JSONValue
                          ▿ object : 5 elements
                            ▿ 0 : 2 elements
                              - key : "type"
                              ▿ value : JSONValue
                                - string : "object"
                            ▿ 1 : 2 elements
                              - key : "properties"
                              ▿ value : JSONValue
                                ▿ object : 1 element
                                  ▿ 0 : 2 elements
                                    - key : "city"
                                    ▿ value : JSONValue
                                      ▿ object : 2 elements
                                        ▿ 0 : 2 elements
                                          - key : "description"
                                          ▿ value : JSONValue
                                            - string : "The city to fetch the weather for"
                                        ▿ 1 : 2 elements
                                          - key : "type"
                                          ▿ value : JSONValue
                                            - string : "string"
                            ▿ 2 : 2 elements
                              - key : "required"
                              ▿ value : JSONValue
                                ▿ array : 1 element
                                  ▿ 0 : JSONValue
                                    - string : "city"
                            ▿ 3 : 2 elements
                              - key : "additionalProperties"
                              ▿ value : JSONValue
                                - bool : false
                            ▿ 4 : 2 elements
                              - key : "description"
                              ▿ value : JSONValue
                                - string : "Generated Arguments"
      ▿ 1 : 2 elements
        - key : "type"
        - value : "function"

This issue is caused by:

    private func convertToolToMLXSpec(_ tool: any Tool) -> ToolSpec {
        // Convert AnyLanguageModel's GenerationSchema to Sendable dictionary
        // using MLXLMCommon.JSONValue which is already Sendable
        let parametersValue: JSONValue
        do {
            let resolvedSchema = tool.parameters.withResolvedRoot() ?? tool.parameters
            let data = try JSONEncoder().encode(resolvedSchema)
            parametersValue = try JSONDecoder().decode(JSONValue.self, from: data)
        } catch {
            parametersValue = .object(["type": .string("object"), "properties": .object([:]), "required": .array([])])
        }

        return [
            "type": "function",
            "function": [
                "name": tool.name,
                "description": tool.description,
                "parameters": parametersValue,
            ] as [String: any Sendable],
        ]
    }

Suggested change is:

1. Add a helper to convert JSONValue → Foundation types:

extension JSONValue {
    var foundationObject: Any {
        switch self {
        case .string(let s): return s
        case .number(let n): return n
        case .bool(let b): return b
        case .array(let a): return a.map { $0.foundationObject }
        case .object(let o): return o.mapValues { $0.foundationObject }
        case .null: return NSNull()
        }
    }
}

2. Use foundationObject instead:

•       "parameters": parametersValue,
  

•       "parameters": parametersValue.foundationObject(),
  

The crash occurs because parameters is a JSONValue enum, which swift-transformers Value(any:) cannot convert. Converting JSONValue recursively to Foundation types before passing to Jinja fixes the crash, and allows nested schemas to work correctly.