tool names vs. implementations

[ultronozm]

This question might be premature, but I was wondering if you have thought about decoupling tool names (as presented to LLMs) from their implementations. One benefit would be to allow multiple implementations of common tools, such as "edit_buffer".

A related question is whether you want PR's to adhere to the format of examples currently in the repo, where they are implemented inline in the tool definition. Does this work well with debugging? I haven't tried yet. The alternative would be to put the main definitions in some separate elisp function, like at https://github.com/ultronozm/codel.el. For that approach, I'm curious if you have any thoughts on namespacing (llm-tool-collection-whatever, I guess?).

[skissue]

This question might be premature, but I was wondering if you have thought about decoupling tool names (as presented to LLMs) from their implementations. One benefit would be to allow multiple implementations of common tools, such as "edit_buffer".

Not premature at all! That is a shortcoming I realized a while after writing the implementation in the repository right now, and something I plan to address.

A related question is whether you want PR's to adhere to the format of examples currently in the repo, where they are implemented inline in the tool definition. Does this work well with debugging? I haven't tried yet. The alternative would be to put the main definitions in some separate elisp function, like at https://github.com/ultronozm/codel.el. For that approach, I'm curious if you have any thoughts on namespacing (llm-tool-collection-whatever, I guess?).

I'm planning to make some structural changes to how the tools are organized; until then, I'll likely hold off on accepting PRs. I hadn't considered debugging being an issue with inline definitions, but that is a good point! It's something I'll keep in mind.

I've been experimenting with some reorganization schemes locally; I was planning to get something "final" into the upstream repository today, but unfortunately life caught up to me—it'll have to wait for tomorrow!

Let me know if you'd like to provide some feedback on the schemes I'm considering. I'll likely expand the macro to define the function separately under a namespace, and perhaps put the tool properties in a similarly- or identical-named symbol in variable space. Still thinking, but I'll try to get something live soon!

[ultronozm]

Sure, I'd be happy to offer whatever feedback I can

[skissue]

I went ahead and pushed some of my preliminary work to the restructuring branch if you'd like to take a look. I've only converted the first tool for testing purposes; I'll convert the rest to the new macro format when I have time. I'm also planning to add support for "tags" in addition to categories, which would be used solely for referencing many tools at once (I think categories are a bit rigid, since tools can only be in one category).

I'm not too sure about using the llm-tc/ namespace; I know it's against convention, but I feel that llm-tool-collection is a bit long.

Curious if you have any feedback or insight! I'm not sure if this structure is the best, so I'm open to suggestions.

[ultronozm]

Thanks for sharing, looks pretty good. I took a look and made a few adjustments to the DSL (see https://github.com/ultronozm/llm-tool-collection/tree/restructuring):

• added edebug instrumentation
• tweaked the way args are specified, e.g., allowing:

(llm-tool-collection-deftool view-buffer
  (:description "View contents of BUFFER-NAME with optional OFFSET and LIMIT.
OFFSET specifies the starting line (0-based).
LIMIT specifies the maximum number of lines to return."
                :category "buffers")
  ((buffer-name :type string :description "Name of the buffer to view.")
   &optional
   (offset :type integer :description "Line number to start reading from (0-based).")
   (limit :type integer :description "Maximum number of lines to return."))
  (with-current-buffer buffer-name
    (let* ((lines (split-string (buffer-string) "\n"))
           (total-lines (length lines))
           (start (min (or offset 0) total-lines))
           (end (min (+ start (or limit total-lines)) total-lines))
           (selected-lines (seq-subseq lines start end)))
      (string-join selected-lines "\n"))))

I had the thought that it might be useful to let users specify in their configs what they want to name the tools from the packages, but this might be overcomplicating, and could be addressed later. I'll let you know if I play around further

[ultronozm]

I pushed a few more small enhancements to the same branch, adding imenu support and an llm-tool-collection-after-tool-define-hook. I've been experimenting with the tools at codel.el, and so far the collection infrastructure is working smoothly. Debugging is straightforward. The hook makes it easy to have tools update in the frontend immediately after evaluating an updated deftool form, which is quite convenient.

Overall, the new structure is shaping up nicely!

[skissue]

Awesome! These changes are great; your Lisp skills certainly exceed mine :). Would you be willing to submit a PR to my branch? There are some small nitpicks I'd like to discuss before pulling your changes. After that, I'm excited to merge this into the main branch so we can start collecting tools 🎉.