Skip to content

index.md

Latest commit

 

History

History
72 lines (49 loc) · 3.09 KB

index.md

File metadata and controls

72 lines (49 loc) · 3.09 KB

Functions

Functions (tools) are the main building blocks of NeMo Agent toolkit and define the logic of your workflow.

In NeMo Agent toolkit, functions are a core abstraction that offer type-safe, asynchronous operations with support for both single and streaming outputs. They wrap callable objects (like Python functions or coroutines) and enhance them with:

  • Type validation and conversion
  • Schema-based input/output validation via Pydantic models
  • Unified interfaces to improve composability
  • Support for both streaming and non-streaming (single) outputs

Key Concepts

Type Safety

Functions use Python's type annotation system to:

  • Validate inputs and outputs
  • Convert between different types using converters
  • Generate input and output schemas that provide runtime information about the function's input and output types

Dual Output Modes

Functions support two output modes:

  • Single Output - For operations that produce a single result
  • Streaming Output - For operations that produce multiple results

A function can support either or both modes.

Input and Output Schemas

Every function has schemas to define the input and output types. Every function has:

  • An input schema
  • A streaming output schema (optional)
  • A single output schema (optional)

These schemas are Pydantic BaseModel classes that provide runtime validation and documentation. Pydantic models are used because they provide a way to validate and coerce values at runtime while also providing a way to document the schema properties of the input and output values.

Asynchronous Operation

All function operations are asynchronous. To invoke a function, use one of the following methods:

  • {py:meth}~nat.builder.function.Function.ainvoke - For single output operations
  • {py:meth}~nat.builder.function.Function.astream - For streaming output operations

Using asynchronous operations allows for better performance and scalability when processing a large number of functions in parallel. In most cases, applications that integrate LLMs are IO bound and can benefit from cooperative multitasking. Asynchronous operations also provide a natural mechanism (using ContextVars) for maintaining application state between multiple function invocations simultaneously.

Writing Functions

For information about writing functions, refer to the Writing Custom Functions document.

Using Functions

./code-execution.md
./text-to-sql.md