Split Large Documents for LLMs: Understanding DocumentSplitter in LangChain4j

With the rise of Large Language Models (LLMs), integrating them into Java applications has become increasingly popular. However, working with large documents comes with a challenge due to the limited context window of LLMs. LangChain4j offers a clean solution through its DocumentSplitter interface, which allows developers to split large documents into smaller, manageable text segments before feeding them to an LLM. This blog post explains the importance of document splitting and walks through the LangChain4j DocumentSplitter interface.

 

Why Document Splitting?

Imagine you are trying to feed a 50 page document into an LLM in one go. You’ll likely hit the model’s token limit. Not only this is inefficient, but it also leads to errors or loss of crucial information.

 

Instead, by splitting the document into smaller parts (text segments), we can

·      Stay within the token limit

·      Select only the most relevant segments for the task

·      Improve LLM response quality and performance

 

DocumentSplitter Interface 

LangChain4j provides the DocumentSplitter interface to define how documents should be split.

public interface DocumentSplitter {

    /**
     * Splits a single Document into a list of TextSegment objects.
     */
    List<TextSegment> split(Document document);

    /**
     * Splits a list of Documents into segments.
     * A convenient default method.
     */
    default List<TextSegment> splitAll(List<Document> documents) {
        return documents.stream()
                .flatMap(document -> split(document).stream())
                .collect(toList());
    }
}

What is a TextSegment?

A TextSegment is a smaller, manageable portion of the original document. These segments are enriched with metadata and used in retrieval-augmented generation (RAG) pipelines, making them easier to send to an LLM.

 

Different Document Splitters supported by Langchain4j

LangChain4j provides a versatile DocumentSplitter interface along with several built-in implementations for different splitting strategies. Let’s explore each of them:

 

·      DocumentByParagraphSplitter: Splits the document based on paragraph boundaries.

·      DocumentByLineSplitter: Splits the content line by line.

·      DocumentBySentenceSplitter: Breaks the document into individual sentences.

·      DocumentByWordSplitter: Divides the text into separate words.

·      DocumentByCharacterSplitter: Splits the text at the character level.

·      DocumentByRegexSplitter:Uses regular expressions to define custom splitting patterns.

·      Recursive Splitter created using DocumentSplitters.recursive(...),

 

Following snippet split the document by paragraph.

 

DocumentByParagraphSplitterDemo.java

 

package com.sample.app.document.splitters;

import java.util.List;

import dev.langchain4j.data.document.Document;
import dev.langchain4j.data.document.splitter.DocumentByParagraphSplitter;
import dev.langchain4j.data.segment.TextSegment;

public class DocumentByParagraphSplitterDemo {
  private static final String CONTENT = """
      LangChain4j is a powerful Java framework that helps developers integrate Large Language Models (LLMs) into their applications. It offers components like document loaders, text segmenters, retrievers, and chat interfaces, making it easy to build LLM-powered apps in Java.
      Text segmentation is a crucial step in building scalable and efficient LLM solutions. Rather than sending an entire document to the LLM, which might exceed token limits or introduce irrelevant noise, it's more practical to split the content into meaningful segments.
      For instance, if you’re building a document search engine, you want the LLM to search through relevant pieces instead of parsing a 100-page document at once. Segments can be paragraphs, sentences, or even semantically meaningful sections, depending on your use case.
      Let’s say you are analyzing a user manual. The introduction, setup instructions, troubleshooting steps, and safety guidelines can be treated as separate segments. This makes it easier to retrieve and serve only the part of the document that answers the user's query.
      LangChain4j supports different splitting strategies such as fixed-size chunking, recursive splitting, or custom logic based on punctuation or semantics. Choosing the right strategy impacts the quality of responses generated by the LLM.
      In summary, proper segmentation reduces cost, enhances response relevance, and ensures faster processing times. It also improves traceability, making it easier to understand why the LLM gave a certain response.
            """;

  public static void main(String[] args) {
    Document document = Document.from(CONTENT);
    DocumentByParagraphSplitter splitter = new DocumentByParagraphSplitter(200, 30);

    List<TextSegment> textSegments = splitter.split(document);
    int seqNo = 1;
    for (TextSegment textSegment : textSegments) {
      System.out.println(seqNo + "." + textSegment.text());
      seqNo++;
    }

  }

}

Output

1.LangChain4j is a powerful Java framework that helps developers integrate Large Language Models (LLMs) into their applications.
2.It offers components like document loaders, text segmenters, retrievers, and chat interfaces, making it easy to build LLM-powered apps in Java.
3.Text segmentation is a crucial step in building scalable and efficient LLM solutions.
4.Rather than sending an entire document to the LLM, which might exceed token limits or introduce irrelevant noise, it's more practical to split the content into meaningful segments.
5.For instance, if you’re building a document search engine, you want the LLM to search through relevant pieces instead of parsing a 100-page document at once.
6.Segments can be paragraphs, sentences, or even semantically meaningful sections, depending on your use case. Let’s say you are analyzing a user manual.
7.The introduction, setup instructions, troubleshooting steps, and safety guidelines can be treated as separate segments.
8.This makes it easier to retrieve and serve only the part of the document that answers the user's query.
9.LangChain4j supports different splitting strategies such as fixed-size chunking, recursive splitting, or custom logic based on punctuation or semantics.
10.Choosing the right strategy impacts the quality of responses generated by the LLM. In summary, proper segmentation reduces cost, enhances response relevance, and ensures faster processing times.
11.It also improves traceability, making it easier to understand why the LLM gave a certain response.

How DocumentSplitter Works in LangChain4j?

To begin, we need to create a DocumentSplitter by specifying the desired size of each TextSegment, along with an optional overlap in characters or tokens to help preserve context between segments.

 

Next, we should invoke the split(Document) or splitAll(List<Document>) methods to split the content. The splitter processes the document(s) into smaller units based on the splitting strategy used.

 

For example,

 

·      DocumentByParagraphSplitter divides the content into paragraphs using two or more consecutive newline characters as boundaries.

·      DocumentBySentenceSplitter detects sentence boundaries using the OpenNLP library.

·      Other strategies split by lines, words, characters, or custom patterns.

 

After splitting into these smaller units, the splitter groups them into TextSegments, each containing as many units as possible without exceeding the specified size limit.

 

If any individual unit is still too large to fit within a segment, the splitter delegates the task to a sub-splitter, which applies a more granular splitting method. Each resulting TextSegment inherits all metadata from the original Document, and an additional metadata field called "index" is added to indicate its position in the sequence starting from index = 0 for the first segment, index = 1 for the second, and so on.

 

Previous                                                    Next                                                    Home