Tool Calling :: Spring AI Reference (original) (raw)

Spring AI provides built-in support for specifying tools (i.e. ToolCallback(s)) from methods in two ways:

declaratively, using the @Tool annotation
programmatically, using the low-level MethodToolCallback implementation.

Declarative Specification: `@Tool`

You can turn a method into a tool by annotating it with @Tool.

class DateTimeTools {

    @Tool(description = "Get the current date and time in the user's timezone")
    String getCurrentDateTime() {
        return LocalDateTime.now().atZone(LocaleContextHolder.getTimeZone().toZoneId()).toString();
    }

}

The @Tool annotation allows you to provide key information about the tool:

name: The name of the tool. If not provided, the method name will be used. AI models use this name to identify the tool when calling it. Therefore, it’s not allowed to have two tools with the same name in the same class. The name must be unique across all the tools available to the model for a specific chat request.
description: The description for the tool, which can be used by the model to understand when and how to call the tool. If not provided, the method name will be used as the tool description. However, it’s strongly recommended to provide a detailed description because that’s paramount for the model to understand the tool’s purpose and how to use it. Failing in providing a good description can lead to the model not using the tool when it should or using it incorrectly.
returnDirect: Whether the tool result should be returned directly to the client or passed back to the model. See Return Direct for more details.
resultConverter: The ToolCallResultConverter implementation to use for converting the result of a tool call to a String object to send back to the AI model. See Result Conversion for more details.

The method can be either static or instance, and it can have any visibility (public, protected, package-private, or private). The class that contains the method can be either a top-level class or a nested class, and it can also have any visibility (as long as it’s accessible where you’re planning to instantiate it).

| | Spring AI provides built-in support for AOT compilation of the @Tool-annotated methods as long as the class containing the methods is a Spring bean (e.g. @Component). Otherwise, you’ll need to provide the necessary configuration to the GraalVM compiler. For example, by annotating the class with @RegisterReflection(memberCategories = MemberCategory.INVOKE_DECLARED_METHODS). | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

You can define any number of arguments for the method (including no argument) with most types (primitives, POJOs, enums, lists, arrays, maps, and so on). Similarly, the method can return most types, including void. If the method returns a value, the return type must be a serializable type, as the result will be serialized and sent back to the model.

| | Some types are not supported. See Method Tool Limitations for more details. | | ----------------------------------------------------------------------------------------------------------------- |

Spring AI will generate the JSON schema for the input parameters of the @Tool-annotated method automatically. The schema is used by the model to understand how to call the tool and prepare the tool request. The @ToolParam annotation can be used to provide additional information about the input parameters, such as a description or whether the parameter is required or optional. By default, all input parameters are considered required.

import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;

class DateTimeTools {

    @Tool(description = "Set a user alarm for the given time")
    void setAlarm(@ToolParam(description = "Time in ISO-8601 format") String time) {
        LocalDateTime alarmTime = LocalDateTime.parse(time, DateTimeFormatter.ISO_DATE_TIME);
        System.out.println("Alarm set for " + alarmTime);
    }

}

The @ToolParam annotation allows you to provide key information about a tool parameter:

description: The description for the parameter, which can be used by the model to understand better how to use it. For example, what format the parameter should be in, what values are allowed, and so on.
required: Whether the parameter is required or optional. By default, all parameters are considered required.

If a parameter is annotated as @Nullable, it will be considered optional unless explicitly marked as required using the @ToolParam annotation.

Besides the @ToolParam annotation, you can also use the @Schema annotation from Swagger or @JsonProperty from Jackson. See JSON Schema for more details.

Adding Tools to `ChatClient`

When using the declarative specification approach, you can pass the tool class instance to the tools() method when invoking a ChatClient. Such tools will only be available for the specific chat request they are added to.

ChatClient.create(chatModel)
    .prompt("What day is tomorrow?")
    .tools(new DateTimeTools())
    .call()
    .content();

Under the hood, the ChatClient will generate a ToolCallback from each @Tool-annotated method in the tool class instance and pass them to the model. In case you prefer to generate the ToolCallback(s) yourself, you can use the ToolCallbacks utility class.

ToolCallback[] dateTimeTools = ToolCallbacks.from(new DateTimeTools());

Adding Default Tools to `ChatClient`

When using the declarative specification approach, you can add default tools to a ChatClient.Builder by passing the tool class instance to the defaultTools() method. If both default and runtime tools are provided, the runtime tools will completely override the default tools.

| | Default tools are shared across all the chat requests performed by all the ChatClient instances built from the same ChatClient.Builder. They are useful for tools that are commonly used across different chat requests, but they can also be dangerous if not used carefully, risking to make them available when they shouldn’t. | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

ChatModel chatModel = ...
ChatClient chatClient = ChatClient.builder(chatModel)
    .defaultTools(new DateTimeTools())
    .build();

Adding Tools to `ChatModel`

When using the declarative specification approach, you can pass the tool class instance to the toolCallbacks() method of the ToolCallingChatOptions you use to call a ChatModel. Such tools will only be available for the specific chat request they are added to.

ChatModel chatModel = ...
ToolCallback[] dateTimeTools = ToolCallbacks.from(new DateTimeTools());
ChatOptions chatOptions = ToolCallingChatOptions.builder()
    .toolCallbacks(dateTimeTools)
    .build();
Prompt prompt = new Prompt("What day is tomorrow?", chatOptions);
chatModel.call(prompt);

Adding Default Tools to `ChatModel`

When using the declarative specification approach, you can add default tools to ChatModel at construction time by passing the tool class instance to the toolCallbacks() method of the ToolCallingChatOptions instance used to create the ChatModel. If both default and runtime tools are provided, the runtime tools will completely override the default tools.

| | Default tools are shared across all the chat requests performed by that ChatModel instance. They are useful for tools that are commonly used across different chat requests, but they can also be dangerous if not used carefully, risking to make them available when they shouldn’t. | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

ToolCallback[] dateTimeTools = ToolCallbacks.from(new DateTimeTools());
ChatModel chatModel = OllamaChatModel.builder()
    .ollamaApi(OllamaApi.builder().build())
    .defaultOptions(ToolCallingChatOptions.builder()
            .toolCallbacks(dateTimeTools)
            .build())
    .build();

Programmatic Specification: `MethodToolCallback`

You can turn a method into a tool by building a MethodToolCallback programmatically.

class DateTimeTools {

    String getCurrentDateTime() {
        return LocalDateTime.now().atZone(LocaleContextHolder.getTimeZone().toZoneId()).toString();
    }

}

The MethodToolCallback.Builder allows you to build a MethodToolCallback instance and provide key information about the tool:

toolDefinition: The ToolDefinition instance that defines the tool name, description, and input schema. You can build it using the ToolDefinition.Builder class. Required.
toolMetadata: The ToolMetadata instance that defines additional settings such as whether the result should be returned directly to the client, and the result converter to use. You can build it using the ToolMetadata.Builder class.
toolMethod: The Method instance that represents the tool method. Required.
toolObject: The object instance that contains the tool method. If the method is static, you can omit this parameter.
toolCallResultConverter: The ToolCallResultConverter instance to use for converting the result of a tool call to a String object to send back to the AI model. If not provided, the default converter will be used (DefaultToolCallResultConverter).

The ToolDefinition.Builder allows you to build a ToolDefinition instance and define the tool name, description, and input schema:

name: The name of the tool. If not provided, the method name will be used. AI models use this name to identify the tool when calling it. Therefore, it’s not allowed to have two tools with the same name in the same class. The name must be unique across all the tools available to the model for a specific chat request.
description: The description for the tool, which can be used by the model to understand when and how to call the tool. If not provided, the method name will be used as the tool description. However, it’s strongly recommended to provide a detailed description because that’s paramount for the model to understand the tool’s purpose and how to use it. Failing in providing a good description can lead to the model not using the tool when it should or using it incorrectly.
inputSchema: The JSON schema for the input parameters of the tool. If not provided, the schema will be generated automatically based on the method parameters. You can use the @ToolParam annotation to provide additional information about the input parameters, such as a description or whether the parameter is required or optional. By default, all input parameters are considered required. See JSON Schema for more details.

The ToolMetadata.Builder allows you to build a ToolMetadata instance and define additional settings for the tool:

returnDirect: Whether the tool result should be returned directly to the client or passed back to the model. See Return Direct for more details.

Method method = ReflectionUtils.findMethod(DateTimeTools.class, "getCurrentDateTime");
ToolCallback toolCallback = MethodToolCallback.builder()
    .toolDefinition(ToolDefinition.builder(method)
            .description("Get the current date and time in the user's timezone")
            .build())
    .toolMethod(method)
    .toolObject(new DateTimeTools())
    .build();

| | Spring AI provides built-in support for AOT compilation of the tool methods as long as the class containing the methods is a Spring bean (e.g. @Component). Otherwise, you’ll need to provide the necessary configuration to the GraalVM compiler. For example, by annotating the class with @RegisterReflection(memberCategories = MemberCategory.INVOKE_DECLARED_METHODS). | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

| | Some types are not supported. See Method Tool Limitations for more details. | | ----------------------------------------------------------------------------------------------------------------- |

If the method is static, you can omit the toolObject() method, as it’s not needed.

class DateTimeTools {

    static String getCurrentDateTime() {
        return LocalDateTime.now().atZone(LocaleContextHolder.getTimeZone().toZoneId()).toString();
    }

}

Method method = ReflectionUtils.findMethod(DateTimeTools.class, "getCurrentDateTime");
ToolCallback toolCallback = MethodToolCallback.builder()
    .toolDefinition(ToolDefinition.builder(method)
            .description("Get the current date and time in the user's timezone")
            .build())
    .toolMethod(method)
    .build();

Spring AI will generate the JSON schema for the input parameters of the method automatically. The schema is used by the model to understand how to call the tool and prepare the tool request. The @ToolParam annotation can be used to provide additional information about the input parameters, such as a description or whether the parameter is required or optional. By default, all input parameters are considered required.

import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import org.springframework.ai.tool.annotation.ToolParam;

class DateTimeTools {

    void setAlarm(@ToolParam(description = "Time in ISO-8601 format") String time) {
        LocalDateTime alarmTime = LocalDateTime.parse(time, DateTimeFormatter.ISO_DATE_TIME);
        System.out.println("Alarm set for " + alarmTime);
    }

}

The @ToolParam annotation allows you to provide key information about a tool parameter:

description: The description for the parameter, which can be used by the model to understand better how to use it. For example, what format the parameter should be in, what values are allowed, and so on.
required: Whether the parameter is required or optional. By default, all parameters are considered required.

If a parameter is annotated as @Nullable, it will be considered optional unless explicitly marked as required using the @ToolParam annotation.

Besides the @ToolParam annotation, you can also use the @Schema annotation from Swagger or @JsonProperty from Jackson. See JSON Schema for more details.

Adding Tools to `ChatClient` and `ChatModel`

When using the programmatic specification approach, you can pass the MethodToolCallback instance to the tools() method of ChatClient. The tool will only be available for the specific chat request it’s added to.

ToolCallback toolCallback = ...
ChatClient.create(chatModel)
    .prompt("What day is tomorrow?")
    .tools(toolCallback)
    .call()
    .content();

Adding Default Tools to `ChatClient`

When using the programmatic specification approach, you can add default tools to a ChatClient.Builder by passing the MethodToolCallback instance to the defaultTools() method. If both default and runtime tools are provided, the runtime tools will completely override the default tools.

ChatModel chatModel = ...
ToolCallback toolCallback = ...
ChatClient chatClient = ChatClient.builder(chatModel)
    .defaultTools(toolCallback)
    .build();

Adding Tools to `ChatModel`

When using the programmatic specification approach, you can pass the MethodToolCallback instance to the toolCallbacks() method of the ToolCallingChatOptions you use to call a ChatModel. The tool will only be available for the specific chat request it’s added to.

ChatModel chatModel = ...
ToolCallback toolCallback = ...
ChatOptions chatOptions = ToolCallingChatOptions.builder()
    .toolCallbacks(toolCallback)
    .build():
Prompt prompt = new Prompt("What day is tomorrow?", chatOptions);
chatModel.call(prompt);

Adding Default Tools to `ChatModel`

When using the programmatic specification approach, you can add default tools to a ChatModel at construction time by passing the MethodToolCallback instance to the toolCallbacks() method of the ToolCallingChatOptions instance used to create the ChatModel. If both default and runtime tools are provided, the runtime tools will completely override the default tools.

ToolCallback toolCallback = ...
ChatModel chatModel = OllamaChatModel.builder()
    .ollamaApi(OllamaApi.builder().build())
    .defaultOptions(ToolCallingChatOptions.builder()
            .toolCallbacks(toolCallback)
            .build())
    .build();

Method Tool Limitations

The following types are not currently supported as parameters or return types for methods used as tools:

Optional
Asynchronous types (e.g. CompletableFuture, Future)
Reactive types (e.g. Flow, Mono, Flux)
Functional types (e.g. Function, Supplier, Consumer).

Functional types are supported using the function-based tool specification approach. See Functions as Tools for more details.

Tool Calling :: Spring AI Reference (original) (raw)

Declarative Specification: @Tool

Adding Tools to ChatClient

Adding Default Tools to ChatClient

Adding Tools to ChatModel

Adding Default Tools to ChatModel

Programmatic Specification: MethodToolCallback

Adding Tools to ChatClient and ChatModel

Adding Default Tools to ChatClient

Adding Tools to ChatModel

Adding Default Tools to ChatModel

Method Tool Limitations

Declarative Specification: `@Tool`

Adding Tools to `ChatClient`

Adding Default Tools to `ChatClient`

Adding Tools to `ChatModel`

Adding Default Tools to `ChatModel`

Programmatic Specification: `MethodToolCallback`

Adding Tools to `ChatClient` and `ChatModel`

Adding Default Tools to `ChatClient`

Adding Tools to `ChatModel`

Adding Default Tools to `ChatModel`