Simplify LLM Integration with LangChain4j AI Services

Until now in this tutorial series, we've focused on low-level components such as ChatModel, ChatMessage, and ChatMemory. While this gives you fine-grained control and maximum flexibility, it also leads to writing a lot of boilerplate code. Building LLM-powered applications typically requires coordinating several components like prompt templates, memory management, LLMs, output parsers, and RAG components (embedding models and stores) and often involves multiple steps or interactions. Managing all this complexity at a low level can quickly become tedious.

 

To help you concentrate on what truly matters, your business logic, not the low-level mechanics AI Services are here to help.

 

1. What Are AI Services?

AI Services abstract away the complexities of working with LLMs and related components by offering a simplified, declarative API.

 

Think of them as similar in concept to Spring Data JPA or Retrofit, you define an interface for your desired functionality, and LangChain4j provides a proxy implementation behind the scenes. In essence, AI Services are part of your application's service layer they deliver AI capabilities with minimal setup.

 

Following are the Key Capabilities of AI Services:

·      Input formatting for LLMs

·      Output parsing from LLMs

·      Support for advanced features like:

o   Chat memory

o   Tool integration

o   Retrieval-Augmented Generation (RAG)

 

AI Services are well-suited for building both stateful chatbots with multi-turn interactions and stateless automation tasks where each LLM call is independent.

 

2. Your First and Simplest AI Service

Let’s walk through creating a basic AI Service using LangChain4j. This is the most minimal setup to get started, perfect for quick experimentation or learning.

 

2.1. Define the AI Service Interface

Start by declaring a plain Java interface that describes the interaction you want with your AI. In this example, the interface has a single method chat, which accepts a String as input and returns a String as the response. 

public interface MyAssistant {
    String chat(String message);
}

LangChain4j will generate the implementation of this interface for you behind the scenes.

 

2.2. Configure the Chat Model

Next, configure a low-level component, specifically, a ChatModel. This model will power your AI Service and handle communication with the underlying LLM (in this case, a model served via Ollama).

 

ChatModel model = OllamaChatModel.builder()
    .baseUrl("http://localhost:11434")   // The local endpoint where Ollama is running
    .modelName("llama3.2")               // The model name you want to use
    .build();

 

This sets up everything the service needs to communicate with your LLM.

 

2.3 Create the AI Service Instance

Now, with your interface and model ready, you can create the AI service using AiServices.create(...). This dynamically generates a proxy implementation of your interface.

 

MyAssistant assistant = AiServices.create(MyAssistant.class, model);

2.4 Use the Assistant

Once the assistant is created, you can use it just like any other Java object. Here's an example of sending a message and printing the AI’s response.

 

String response = assistant.chat("Hi, tell me about LLMs in 30 words maximum");
System.out.println(response);

This simple setup lets you start building AI-powered features without worrying about the low-level plumbing. You get a clean, declarative interface and LangChain4j handles the rest.

 

Find the below working application.

 

MyAssistant.java

 

package com.sample.app.aiservices;

public interface MyAssistant {
    String chat(String message);
}

 

AiServiceHelloWorld.java

package com.sample.app.aiservices;

import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.ollama.OllamaChatModel;
import dev.langchain4j.service.AiServices;

public class AiServiceHelloWorld {
    public static void main(String[] args) {
        ChatModel model = OllamaChatModel.builder().baseUrl("http://localhost:11434").modelName("llama3.2").build();

        MyAssistant assistant = AiServices.create(MyAssistant.class, model);
        String response = assistant.chat("Hi, tell me about LLMs in 30 words maximum");
        System.out.println(response);

    }
}

 

Output

Large Language Models (LLMs) are AI algorithms that process and generate human-like language based on vast amounts of training data. They're trained to predict the next word or character.

How Does It Work?

In the previous example, we passed the interface class (MyAssistant.class) along with a configured low-level component (ChatModel) to the AiServices.create(...) method. Behind the scenes, LangChain4j uses this information to generate a proxy object that implements the provided interface. Currently, this proxy is created using Java reflection.

 

This proxy handles all necessary input and output conversions automatically.

 

·      Your interface method (chat(String message)) accepts a simple String.

·      However, under the hood, the ChatModel expects input in the form of a ChatMessage.

·      The proxy takes care of this mismatch by wrapping the input string into a UserMessage and then passing it to the model.

·      The model processes the request and returns an AiMessage.

·      Since the return type of the method is declared as String, the proxy converts the AiMessage content back into a String before returning it to the caller.

 

This seamless conversion makes it possible to interact with complex LLM models using simple Java method calls abstracting away the complexity and making development faster and cleaner.

 

Previous                                                    Next                                                    Home