Getting Started with LangChain4j and Ollama: A Step-by-Step Guide

LangChain4j is the Java-centric counterpart of LangChain (Python) for building LLM-powered applications. It abstracts LLMs, vector stores, tools, memory, and agents for seamless integration into Java projects.

 

Ollama is a local LLM runtime that lets you run models like LLaMA2, Mistral, or Code LLaMA on your machine with a simple HTTP interface.

 

In this guide, you'll learn how to:

 

·      Set up a Java project using LangChain4j

·      Integrate it with a locally running Ollama model

·      Send a prompt and receive a response from the LLM

 

Prerequisites

a. Setup Ollama and pull llama3.2 mode. Refer this link to get more details.

 

b. After installing Ollama in your computer pull llama3.2 model

ollama run llama3.2

 

c. Java21

 

Follow below step-by-step procedure to build the complete working application.

 

Step 1: Create new maven project langchain4j-learning.

 

Step 2: Update pom.xml with maven dependencies.

 

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.sample.app</groupId>
    <artifactId>langchain4j-learning</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>21</maven.compiler.source>
        <maven.compiler.target>21</maven.compiler.target>
        <java.version>21</java.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>dev.langchain4j</groupId>
                <artifactId>langchain4j-bom</artifactId>
                <version>1.0.1</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>

        <dependency>
            <groupId>dev.langchain4j</groupId>
            <artifactId>langchain4j</artifactId>
        </dependency>

        <!--
        https://mvnrepository.com/artifact/dev.langchain4j/langchain4j-ollama -->
        <dependency>
            <groupId>dev.langchain4j</groupId>
            <artifactId>langchain4j-ollama</artifactId>
        </dependency>

    </dependencies>
</project>

Step 3: Create a package com.sample.app.ollama and define HelloWorld class.

 

HelloWorld.java

 

package com.sample.app.ollama;

import dev.langchain4j.model.ollama.OllamaLanguageModel;
import dev.langchain4j.model.output.Response;

public class HelloWorld {
    public static void main(String[] args) {
        OllamaLanguageModel model = OllamaLanguageModel.builder().baseUrl("http://localhost:11434") 
                .modelName("llama3.2")
                .build();

        String prompt = "Tell me some Interesting Fact About LLMs in maximum 30 words";
        Response<String> response = model.generate(prompt);

        System.out.println("Response: " + response.content());
    }
}

 

Run HelloWorld application, you will see following messages in the console.

Response: LLMs (Large Language Models) can process and generate vast amounts of text, with some models holding over 1 trillion parameters, making them one of the most complex AI systems.

OllamaLanguageModel model = OllamaLanguageModel.builder()

        .baseUrl("http://localhost:11434")

        .modelName("llama3.2")

        .build();

 

Above statement build an instance of the Ollama model integration from LangChain4j.

 

Here:

·      baseUrl("http://localhost:11434"): Specifies the local endpoint where Ollama is running. This is the default Ollama server address.

·      modelName("llama3.2"): Specifies the name of the model to use. You should make sure llama3.2 is actually availabel in your system

ollama pull llama3.2

 

String prompt = "Tell me some Interesting Fact About LLMs in maximum 30 words";

This defines the input prompt to send to the LLM (LLaMA3.2). It instructs the model to reply with a fact about LLMs, using no more than 30 words.

 

Response<String> response = model.generate(prompt);

This sends the prompt to the model and gets a Response<String> object back. generate(prompt) handles the HTTP request to the Ollama server and returns the output as a response.

 

Response<T> is a wrapper object that encapsulates not only the generated content but also various useful details. For example, it includes:

 

·      Content: The actual result returned by the model

·      Token usage: Counts for inputTokenCount, outputTokenCount, and totalTokenCount

·      Finish reason: Indicates why the generation stopped, such as STOP, LENGTH, TOOL_EXECUTION, CONTENT_FILTER, or OTHER

·      Metadata: Additional information related to the response

 

  

Previous                                                    Next                                                    Home