LangChain4j Chat (original) (raw)

Since Camel 4.5

Both producer and consumer are supported

The LangChain4j Chat Component allows you to integrate with any Large Language Model (LLM) supported by LangChain4j.

Maven users will need to add the following dependency to their pom.xml for this component:

<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-langchain4j-chat</artifactId>
    <version>x.x.x</version>
    <!-- use the same version as your Camel core version -->
</dependency>

URI format

langchain4j-chat:chatId[?options]

Where chatId can be any string to uniquely identify the endpoint

Configuring Options

Camel components are configured on two separate levels:

• component level
• endpoint level

Configuring Component Options

At the component level, you set general and shared configurations that are, then, inherited by the endpoints. It is the highest configuration level.

For example, a component may have security settings, credentials for authentication, urls for network connection and so forth.

Some components only have a few options, and others may have many. Because components typically have pre-configured defaults that are commonly used, then you may often only need to configure a few options on a component; or none at all.

You can configure components using:

• the Component DSL.
• in a configuration file (application.properties, *.yaml files, etc).
• directly in the Java code.

Configuring Endpoint Options

You usually spend more time setting up endpoints because they have many options. These options help you customize what you want the endpoint to do. The options are also categorized into whether the endpoint is used as a consumer (from), as a producer (to), or both.

Configuring endpoints is most often done directly in the endpoint URI as path and query parameters. You can also use the Endpoint DSL and DataFormat DSL as a type safe way of configuring endpoints and data formats in Java.

Property placeholders provide a few benefits:

• They help prevent using hardcoded urls, port numbers, sensitive information, and other settings.
• They allow externalizing the configuration from the code.
• They help the code to become more flexible and reusable.

The following two sections list all the options, firstly for the component followed by the endpoint.

Component Options

The LangChain4j Chat component supports 6 options, which are listed below.

Endpoint Options

The LangChain4j Chat endpoint is configured using URI syntax:

With the following path and query parameters:

Path Parameters (1 parameters)

Query Parameters (6 parameters)

The LangChain4j Chat component supports 2 message header(s), which is/are listed below:

Spring Boot Auto-Configuration

When using langchain4j-chat with Spring Boot make sure to use the following Maven dependency to have support for auto configuration:

<dependency>
  <groupId>org.apache.camel.springboot</groupId>
  <artifactId>camel-langchain4j-chat-starter</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel core version -->
</dependency>

The component supports 7 options, which are listed below.

Usage

Using a specific Chat Model

The Camel LangChain4j chat component provides an abstraction for interacting with various types of Large Language Models (LLMs) supported by LangChain4j.

Integrating with specific LLM

To integrate with a specific LLM, users should follow the steps described below, which explain how to integrate with OpenAI.

Using LangChain4j Spring Boot Starters (Recommended for Spring Boot)

When using Camel with Spring Boot, you can leverage LangChain4j’s Spring Boot starters for automatic configuration.

Add the dependency for LangChain4j OpenAI Spring Boot starter:

pom.xml

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-open-ai-spring-boot-starter</artifactId>
    <version>1.10.0</version>
    <!-- use the same version as your LangChain4j version -->
</dependency>

Configure the OpenAI Chat Model in application.properties or application.yml:

application.properties

langchain4j.open-ai.chat-model.api-key=${OPENAI_API_KEY}
langchain4j.open-ai.chat-model.model-name=gpt-3.5-turbo
langchain4j.open-ai.chat-model.temperature=0.3
langchain4j.open-ai.chat-model.timeout=3000s

application.yml

langchain4j:
  open-ai:
    chat-model:
      api-key: ${OPENAI_API_KEY}
      model-name: gpt-3.5-turbo
      temperature: 0.3
      timeout: 3000s

The ChatLanguageModel bean will be automatically configured and available in the Spring context. Use it in your Camel routes:

• Java
• YAML
• XML

from("direct:chat")
    .to("langchain4j-chat:test?chatModel=#chatLanguageModel");

- route:
    from:
      uri: "direct:chat"
    steps:
      - to:
          uri: "langchain4j-chat:test"
          parameters:
            chatModel: "#chatLanguageModel"

<route>
  <from uri="direct:chat"/>
  <to uri="langchain4j-chat:test?chatModel=#chatLanguageModel"/>
</route>

| | LangChain4j Spring Boot starters provide auto-configuration for various LLM providers including: langchain4j-open-ai-spring-boot-starter - OpenAI langchain4j-azure-open-ai-spring-boot-starter - Azure OpenAI langchain4j-google-ai-gemini-spring-boot-starter - Google Gemini langchain4j-ollama-spring-boot-starter - Ollama langchain4j-anthropic-spring-boot-starter - Anthropic Claude langchain4j-mistral-ai-spring-boot-starter - Mistral AI langchain4j-hugging-face-spring-boot-starter - Hugging Face | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

Manual Configuration (Alternative)

Alternatively, you can manually initialize the Chat Model and add it to the Camel Registry:

Add the dependency for LangChain4j OpenAI support:

pom.xml

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-open-ai</artifactId>
    <version>1.10.0</version>
    <!-- use the same version as your LangChain4j version -->
</dependency>

Initialize the OpenAI Chat Model, and add it to the Camel Registry:

ChatLanguageModel model = OpenAiChatModel.builder()
    .apiKey(openApiKey)
    .modelName(GPT_3_5_TURBO)
    .temperature(0.3)
    .timeout(ofSeconds(3000))
    .build();
context.getRegistry().bind("myChatModel", model);

Use the model in the Camel LangChain4j Chat Producer:

• Java
• YAML

from("direct:chat")
    .to("langchain4j-chat:test?chatModel=#myChatModel");

- route:
    from:
      uri: "direct:chat"
    steps:
      - to:
          uri: "langchain4j-chat:test"
          parameters:
            chatModel: "#myChatModel"

| | To switch to another Large Language Model and its corresponding dependency, replace the langchain4j-open-ai dependency with the appropriate dependency for the desired model. Update the initialization parameters accordingly in the code snippet provided above. | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

Send a prompt with variables

To send a prompt with variables, use the Operation type LangChain4jChatOperations.CHAT_SINGLE_MESSAGE_WITH_PROMPT. This operation allows you to send a single prompt message with dynamic variables, which will be replaced with values provided in the request.

Route example:

 from("direct:chat")
      .to("langchain4j-chat:test?chatModel=#chatModel&chatOperation=CHAT_SINGLE_MESSAGE_WITH_PROMPT")

Usage example:

var promptTemplate = "Create a recipe for a {{dishType}} with the following ingredients: {{ingredients}}";

Map<String, Object> variables = new HashMap<>();
variables.put("dishType", "oven dish");
variables.put("ingredients", "potato, tomato, feta, olive oil");

String response = template.requestBodyAndHeader("direct:chat", variables,
                LangChain4jChat.Headers.PROMPT_TEMPLATE, promptTemplate, String.class);

Chat with history

You can send a new prompt along with the chat message history by passing all messages in a list of type dev.langchain4j.data.message.ChatMessage. Use the Operation type LangChain4jChatOperations.CHAT_MULTIPLE_MESSAGES. This operation allows you to continue the conversation with the context of previous messages.

Route example:

 from("direct:chat")
      .to("langchain4j-chat:test?chatModel=#chatModel&chatOperation=CHAT_MULTIPLE_MESSAGES")

Usage example:

List<ChatMessage> messages = new ArrayList<>();
messages.add(new SystemMessage("You are asked to provide recommendations for a restaurant based on user reviews."));
// Add more chat messages as needed

String response = template.requestBody("direct:send-multiple", messages, String.class);

Retrieval Augmented Generation (RAG)

Use the RAG feature to enrich exchanges with data retrieved from any type of Camel endpoint. The feature is compatible with all LangChain4 Chat operations and is ideal for orchestrating the RAG workflow, utilizing the extensive library of components and Enterprise Integration Patterns (EIPs) available in Apache Camel.

There are two ways for utilizing the RAG feature:

Using RAG with Content Enricher and LangChain4jRagAggregatorStrategy

Enrich the exchange by retrieving a list of strings using any Camel producer. The LangChain4jRagAggregatorStrategy is specifically designed to augment data within LangChain4j chat producers.

Usage example:

// Create an instance of the RAG aggregator strategy
LangChain4jRagAggregatorStrategy aggregatorStrategy = new LangChain4jRagAggregatorStrategy();

from("direct:test")
     .enrich("direct:rag", aggregatorStrategy)
     .to("langchain4j-chat:test1?chatOperation=CHAT_SIMPLE_MESSAGE");

  from("direct:rag")
          .process(exchange -> {
                List<String> augmentedData = List.of("data 1", "data 2" );
                exchange.getIn().setBody(augmentedData);
              });

| | This method leverages a separate Camel route to fetch and process the augmented data. | | ---------------------------------------------------------------------------------------- |

It is possible to enrich the message from multiple sources within the same exchange.

Usage example:

// Create an instance of the RAG aggregator strategy
LangChain4jRagAggregatorStrategy aggregatorStrategy = new LangChain4jRagAggregatorStrategy();

from("direct:test")
     .enrich("direct:rag-from-source-1", aggregatorStrategy)
     .enrich("direct:rag-from-source-2", aggregatorStrategy)
     .to("langchain4j-chat:test1?chatOperation=CHAT_SIMPLE_MESSAGE");

Directly add augmented data into the header. This method is particularly efficient for straightforward use cases where the augmented data is predefined or static. You must add augmented data as a List of dev.langchain4j.rag.content.Content directly inside the header CamelLangChain4jChatAugmentedData.

Usage example:

import dev.langchain4j.rag.content.Content;

...

Content augmentedContent = new Content("data test");
List<Content> contents = List.of(augmentedContent);

String response = template.requestBodyAndHeader("direct:send-multiple", messages, LangChain4jChat.Headers.AUGMENTED_DATA , contents, String.class);