Best practices for segmenting overlapping streaming data for vector indexing

A static PDF report has titles, tables, and section breaks you can parse ahead of time — that's the structure-aware approach covered in lesson 1 of this course. A live chat transcript, an application log file, or a continuously-edited wiki page doesn't have that kind of fixed structure to detect; new text just keeps getting appended. For that kind of data, the standard technique is a sliding window: split on a target character count, but let consecutive chunks overlap by a small amount so no sentence gets orphaned exactly at a cut point.

How overlap actually works

LangChain's RecursiveCharacterTextSplitter is the standard open-source implementation of this pattern. It takes a chunk_size (the target character count per chunk) and a chunk_overlap (how many characters of the previous chunk get repeated at the start of the next one). Per LangChain's own documentation, overlapping chunks "helps to mitigate loss of information when context is divided between chunks" — the repeated slice means a sentence that falls right on a chunk boundary still appears whole in at least one of the two adjacent chunks.

The splitter is called "recursive" because it doesn't cut blindly at the character count. It tries a list of separators in order — by default paragraph breaks (\n\n), then line breaks (\n), then spaces, and only individual characters as a last resort — stopping as soon as the resulting piece is small enough. That means it prefers to break between paragraphs or sentences and only falls back to a mid-word cut when the text has no natural break within the target size.

A common starting point, per LangChain's own examples, is a chunk_overlap around 10-20% of chunk_size (e.g. 200 characters of overlap on a 1,000-character chunk). Overlap that's too small brings back the original problem — boundary content still gets lost; overlap that's too large means you're storing and embedding the same text repeatedly, which adds index size and retrieval noise without adding new information.

# Sliding-window chunking for a continuously-appended log or transcript
from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,      # target characters per chunk
    chunk_overlap=200,    # ~20% overlap between adjacent chunks
    separators=["\n\n", "\n", " ", ""],
)

# New log lines or transcript text keep appending to this string
running_transcript = open("support_chat_log.txt").read()

chunks = splitter.split_text(running_transcript)
print(f"Split into {len(chunks)} overlapping chunks")

Overlap is a mitigation, not a substitute for reviewing your chunk size against your actual content. If a single idea in your data routinely runs longer than your chunk_size, no amount of overlap fully fixes that — the real fix is raising the chunk size or switching to a structure-aware approach like lesson 1's, if the data actually has structure to detect.