Pipeline

The runtime pipeline is built around lazy loading, content-aware chunking, embedding-based relevance and strict token-budget selection.

Pipeline#

Collect eager inputs and lazy files#

Strings and byte inputs are stored immediately, while file paths stay lazy until the optimizer decides to read them.

Preview files for query filtering#

If a query exists and there are more than five lazy files, Forgetless reads fast previews and embeds them to keep only the most relevant paths.

Read only the selected files#

PDFs, images, text and code are loaded in parallel. PDFs use pdftotext when available, and images can optionally invoke the vision model.

Chunk by content type#

The chunker adapts to text, code, markdown and structured data, then labels every chunk with priority and source metadata.

Score and rank chunks#

Embeddings, priority boosts, position heuristics, and conversation-style recency combine into the final ordering.

Select within budget and assemble output#

Top chunks are kept until the token limit is met, then the final context is grouped by source and optionally polished with the local text model.

Scoring signals#

Signal	Source	Notes
Priority	user input	Critical content outranks everything else and survives budget pressure first.
Algorithmic relevance	chunk scoring	The exported `algorithmic` breakdown field currently tracks the position-based heuristic.
Semantic similarity	embeddings	Preview text and chunk embeddings are compared to the query when one is present.
Recency factor	scoring breakdown	Conversation-like chunks receive a recency-biased factor during ranking.
Optional LLM polish	context_llm	A local SmolLM2 pass can reorganize selected chunks after budget selection.

Models used locally#

Purpose	Model	When it loads
Embeddings	all-MiniLM-L6-v2	Used for semantic similarity and preview filtering.
Text polish	HuggingFaceTB/SmolLM2-135M-Instruct	Only when `context_llm(true)` is enabled.
Vision	HuggingFaceTB/SmolVLM-256M-Instruct	Only when `vision_llm(true)` is enabled.

Output shape#

text

## system.md
You are preparing a release summary.

---

## design-review.pdf
Important design decisions and open questions...

---

## src/lib.rs
Public exports and library entry points...

Inputs and Priorities

Control what the LLM context optimizer keeps using WithPriority, FileWithPriority, byte payloads, and lazy file inputs to rank and preserve critical content.

Chunking

Configure content-aware chunking for LLMs: pick presets by content type and tune deduplication, overlap, and target token size for accurate ranking.

Introduction

Learn how Forgetless, a Rust LLM context optimizer, ranks and compresses oversized prompts and files into budget-aware context for production workflows.