Async Context Compression¶

Filter v1.2.2

Reduces token consumption in long conversations through intelligent summarization while maintaining conversational coherence.

Overview¶

The Async Context Compression filter helps manage token usage in long conversations by:

• Intelligently summarizing older messages
• Preserving important context
• Reducing API costs
• Maintaining conversation coherence

This is especially useful for:

• Long-running conversations
• Complex multi-turn discussions
• Cost optimization
• Token limit management

Features¶

• Smart Compression: AI-powered context summarization
• Async Processing: Non-blocking background compression
• Context Preservation: Keeps important information
• Cost Reduction: Minimize token usage
• Frontend Debugging: Debug logs in browser console
• Enhanced Error Reporting: Clear error status notifications
• Open WebUI v0.7.x Compatibility: Dynamic DB session handling
• Improved Compatibility: Summary role changed to assistant
• Enhanced Stability: Resolved race conditions in state management
• Preflight Context Check: Validates context fit before sending
• Structure-Aware Trimming: Preserves document structure
• Native Tool Output Trimming: Trims verbose tool outputs (Note: Non-native tool outputs are not fully injected into context)
• Detailed Token Logging: Granular token breakdown
• Smart Model Matching: Inherit config from base models
• Multimodal Support: Images are preserved but tokens are NOT calculated

Installation¶

1. Download the plugin file: async_context_compression.py
2. Upload to OpenWebUI: Admin Panel → Settings → Functions
3. Configure compression settings
4. Enable the filter

How It Works¶

graph TD
    A[Incoming Messages] --> B{Token Count > Threshold?}
    B -->|No| C[Pass Through]
    B -->|Yes| D[Summarize Older Messages]
    D --> E[Preserve Recent Messages]
    E --> F[Combine Summary + Recent]
    F --> G[Send to LLM]

Configuration¶

Example¶

Before Compression¶

[Message 1] User: Tell me about Python...
[Message 2] AI: Python is a programming language...
[Message 3] User: What about its history?
[Message 4] AI: Python was created by Guido...
[Message 5] User: And its features?
[Message 6] AI: Python has many features...
... (many more messages)
[Message 20] User: Current question

After Compression¶

[Summary] Previous conversation covered Python basics,
history, features, and common use cases...

[Message 18] User: Recent question about decorators
[Message 19] AI: Decorators in Python are...
[Message 20] User: Current question

Requirements¶

Troubleshooting¶

Source Code¶

View on GitHub