Configuration

Config covers the runtime switches most projects need. Lower-level tuning is available through ForgetlessConfig, ChunkConfig, ScoringConfig, and TokenizerModel.

Runtime options#

rust
use forgetless::{Config, Forgetless};

let result = Forgetless::new()
    .config(
        Config::default()
            .context_limit(96_000)
            .chunk_size(256)
            .vision_llm(true)
            .context_llm(true)
            .parallel(true)
            .cache(true),
    )
    .add_file("diagram.png")
    .run()
    .await?;
FieldTypeDefaultEffect
context_limitusize128,000Maximum output token budget.
vision_llmboolfalseInitializes SmolVLM-256M for image description.
context_llmboolfalseInitializes SmolLM2-135M for post-selection polishing.
chunk_sizeusize512Syncs the chunk target token size.
parallelbooltrueStored on config; the current builder path still uses parallel file reads regardless.
cachebooltrueStored on config; the current global embedding cache is not toggled by this flag yet.

parallel and cache are part of the public config surface, but in the current source they are stored rather than used to branch runtime behavior.

Local model behavior#

rust
use forgetless::{Config, Forgetless};

// Fast path: embeddings + heuristics only
let fast = Forgetless::new()
    .config(Config::default().context_limit(64_000))
    .add_file("paper.pdf");

// Add local image descriptions
let with_vision = Forgetless::new()
    .config(Config::default().vision_llm(true))
    .add_file("diagram.png");

// Add local text polishing
let with_context_llm = Forgetless::new()
    .config(Config::default().context_llm(true))
    .add(long_context);

Vision mode loads HuggingFaceTB/SmolVLM-256M-Instruct. Context polish mode loads HuggingFaceTB/SmolLM2-135M-Instruct and only runs after chunk selection.

Chunk tuning#

rust
use forgetless::{ChunkConfig, Config, ForgetlessConfig, ScoringConfig};

let advanced = ForgetlessConfig::new(
    Config::default().context_limit(64_000),
)
.with_chunk(
    ChunkConfig::for_quality()
        .with_target_tokens(256)
        .with_max_tokens(512)
        .with_min_tokens(10),
)
.with_scoring(ScoringConfig {
    semantic_weight: 0.6,
    keyword_weight: 0.25,
    priority_weight: 0.15,
});
PresetTargetWhen to use it
ChunkConfig::default()512General-purpose prompt compression.
ChunkConfig::for_code()256More granular code selection.
ChunkConfig::for_conversation()200Message-oriented chat history.
ChunkConfig::for_speed()1000Faster processing with larger chunks.
ChunkConfig::for_quality()256Smaller chunks for more precise selection.

Tokenizer options#

rust
use forgetless::{Config, ForgetlessConfig, TokenizerModel};

let config = ForgetlessConfig::new(Config::default())
    .with_tokenizer(TokenizerModel::Custom {
        chars_per_token_x100: 400,
    });

The default tokenizer uses cl100k_base. A custom characters-per-token ratio is available when you want a lightweight estimate instead.

Tokenizer

Count LLM tokens in Rust with cl100k_base or a custom chars-per-token mode, plus truncation helpers and image token estimation to plan context budgets.

Results

Read the Forgetless result types: OptimizedContext, selected chunks, score breakdowns, token statistics, and the compression ratio returned after each run.

HTTP Server

Run the optional Forgetless HTTP server built on Axum: a multipart API for language-agnostic context optimization with a health endpoint and file priorities.