Getting Started

Install the crate, choose a token budget, attach content and files, then call run() to produce an optimized context block.

Install the crate#

bash
cargo add forgetless

The current crate line in the repository is 0.1.1.

toml
[dependencies]
forgetless = "0.1.1"

Optional features#

bash
# HTTP server
cargo add forgetless --features server

# Apple Silicon GPU inference
cargo add forgetless --features metal

# NVIDIA GPU inference
cargo add forgetless --features cuda

# CPU acceleration
cargo add forgetless --features accelerate
cargo add forgetless --features mkl

Build the first pipeline#

rust
use forgetless::{Config, Forgetless, WithPriority};

let result = Forgetless::new()
    .config(Config::default().context_limit(48_000))
    .add(WithPriority::critical(system_prompt))
    .add(conversation_history)
    .add_file("spec.pdf")
    .add_file("screenshot.png")
    .query("What changed in the API contract?")
    .run()
    .await?;

println!("{}", result.content);
println!("compression: {:.1}x", result.compression_ratio());

Attach content and files with priorities#

rust
use forgetless::{FileWithPriority, Forgetless, WithPriority};

let result = Forgetless::new()
    .add_pinned("You are preparing a release summary.")
    .add(WithPriority::high(recent_messages))
    .add(WithPriority::low(archived_notes))
    .add_file(FileWithPriority::critical("system.md"))
    .add_file(FileWithPriority::high("design-review.pdf"))
    .add_files(["src/lib.rs", "src/builder.rs"])
    .run()
    .await?;

Add in-memory bytes#

rust
use forgetless::{Forgetless, Priority};

let png_bytes = std::fs::read("diagram.png")?;

let result = Forgetless::new()
    .add_bytes(b"temporary notes", "text/plain")
    .add_bytes_p(&png_bytes, "image/png", Priority::High)
    .run()
    .await?;

In the current builder implementation, non-text byte payloads are turned into placeholder content records rather than being parsed as rich binary inputs.

Runtime configuration#

OptionDefaultWhat it changes
context_limit128,000Maximum output token budget.
vision_llmfalseLoads the local SmolVLM vision model for image descriptions.
context_llmfalseLoads the local SmolLM2-135M model for post-selection polishing.
chunk_size512Target chunk size passed into the chunker.
paralleltrueStored on config; current file reads still use parallel processing.
cachetrueStored on config; the current global embedding cache is not toggled by this flag.
rust
use forgetless::{Config, Forgetless};

let result = Forgetless::new()
    .config(
        Config::default()
            .context_limit(96_000)
            .chunk_size(256)
            .vision_llm(true)
            .context_llm(true)
            .parallel(true)
            .cache(true),
    )
    .add_file("diagram.png")
    .run()
    .await?;

Read the result#

rust
result.content
result.total_tokens
result.chunks

result.stats.input_tokens
result.stats.output_tokens
result.stats.chunks_processed
result.stats.chunks_selected
result.stats.processing_time_ms

result.compression_ratio()

Introduction

Learn how Forgetless, a Rust LLM context optimizer, ranks and compresses oversized prompts and files into budget-aware context for production workflows.

Builder

Reference for the Forgetless builder API in Rust: chain config, add, add_file, and query, then call run to execute context optimization.

Inputs and Priorities

Control what the LLM context optimizer keeps using WithPriority, FileWithPriority, byte payloads, and lazy file inputs to rank and preserve critical content.