Pipeline
The runtime pipeline is built around lazy loading, content-aware chunking, embedding-based relevance and strict token-budget selection.
Pipeline
Collect eager inputs and lazy files
Strings and byte inputs are stored immediately, while file paths stay lazy until the optimizer decides to read them.
Preview files for query filtering
If a query exists and there are more than five lazy files, Forgetless reads fast previews and embeds them to keep only the most relevant paths.
Read only the selected files
PDFs, images, text and code are loaded in parallel. PDFs use `pdftotext` when available, and images can optionally invoke the vision model.
Chunk by content type
The chunker adapts to text, code, markdown and structured data, then labels every chunk with priority and source metadata.
Score and rank chunks
Embeddings, priority boosts, position heuristics, and conversation-style recency combine into the final ordering.
Select within budget and assemble output
Top chunks are kept until the token limit is met, then the final context is grouped by source and optionally polished with the local text model.
Scoring signals
| Signal | Source | Notes |
|---|---|---|
| Priority | user input | Critical content outranks everything else and survives budget pressure first. |
| Algorithmic relevance | chunk scoring | The exported `algorithmic` breakdown field currently tracks the position-based heuristic. |
| Semantic similarity | embeddings | Preview text and chunk embeddings are compared to the query when one is present. |
| Recency factor | scoring breakdown | Conversation-like chunks receive a recency-biased factor during ranking. |
| Optional LLM polish | context_llm | A local SmolLM2 pass can reorganize selected chunks after budget selection. |
Models used locally
| Purpose | Model | When it loads |
|---|---|---|
| Embeddings | all-MiniLM-L6-v2 | Used for semantic similarity and preview filtering. |
| Text polish | HuggingFaceTB/SmolLM2-135M-Instruct | Only when `context_llm(true)` is enabled. |
| Vision | HuggingFaceTB/SmolVLM-256M-Instruct | Only when `vision_llm(true)` is enabled. |
Output shape
1## system.md2You are preparing a release summary.34---56## design-review.pdf7Important design decisions and open questions...89---1011## src/lib.rs12Public exports and library entry points...