Exports

src/lib.rs re-exports the main builder, config types, chunking helpers, tokenizer, result structs, and optional AI utilities from one crate root.

Crate root#

rust
pub use core::config::{Config, ForgetlessConfig, ScoringConfig};
pub use core::error::{Error, Result};
pub use core::types::{OptimizationStats, OptimizedContext, PolishedContext, ScoredChunk, ScoreBreakdown};

pub use builder::Forgetless;

pub use input::content::{ContentInput, FileWithPriority, IntoContent, IntoFileContent, WithPriority};
pub use input::file::read_file_content;

pub use processing::chunking::{Chunk, ChunkConfig, Chunker, ContentType};
pub use processing::scoring::Priority;
pub use processing::token::{TokenCounter, TokenizerModel};

pub use ai::embeddings::{cosine_similarity, embed_batch, embed_text, EmbeddingCache};
pub use ai::llm::{LLMConfig, Quantization, LLM};
pub use ai::vision::{describe_image, describe_image_with_prompt, init_vision, is_vision_ready};

Export groups#

GroupMain items
BuilderForgetless
ConfigConfig, ForgetlessConfig, ScoringConfig
InputsWithPriority, FileWithPriority, ContentInput
ProcessingChunk, ChunkConfig, Chunker, ContentType, Priority
TokenizationTokenCounter, TokenizerModel
ResultsOptimizedContext, PolishedContext, OptimizationStats, ScoredChunk, ScoreBreakdown
ErrorsError, Result
AIEmbeddingCache, embed_text, embed_batch, LLM, LLMConfig, Quantization, vision helpers

Quick imports#

rust
use forgetless::{
    Config,
    Error,
    FileWithPriority,
    Forgetless,
    Priority,
    TokenCounter,
    TokenizerModel,
    WithPriority,
};

Configuration

Configure Forgetless token budgets, scoring weights, chunk behavior, local model options, and feature toggles to tune LLM context for your workload.

Tokenizer

Count LLM tokens in Rust with cl100k_base or a custom chars-per-token mode, plus truncation helpers and image token estimation to plan context budgets.

Results

Read the Forgetless result types: OptimizedContext, selected chunks, score breakdowns, token statistics, and the compression ratio returned after each run.