mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-09-29 04:12:38 +00:00
Enhanced ChatGPT Clone: Features Agents, MCP, DeepSeek, Anthropic, AWS, OpenAI, Responses API, Azure, Groq, o1, GPT-5, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, Code Interpreter, langchain, DALL-E-3, OpenAPI Actions, Functions, Secure Multi-User Auth, Presets, open-source for self-hosting. Active.
https://librechat.ai/
aianthropicartifactsawsazurechatgptchatgpt-cloneclaudeclonedeepseekgeminigooglegpt-5librechatmcpo1openairesponses-apivisionwebui
58cdd9cd8f
* ⚡ feat: Coalesce Redis Streaming Delta Publications into Windowed Batches Every streamed delta currently costs two Redis EVALs (durable append + sequence-allocating publish), and the publish round trip is awaited inside the provider-stream consumption loop. Behind STREAM_DELTA_COALESCE_MS (default off), message/reasoning/run-step deltas now buffer for a small window and flush as one CHUNK_BATCH frame: a single INCRBY reserves consecutive per-event sequences and one EVAL publishes the batch, while a matching batched XADD keeps the durable chunk log on the same cadence so the resume frontier's log-vs-counter timing assumptions hold. Subscribers unpack batch frames at ingress into individually sequenced chunks, so the reorder buffer, duplicate drop, and force-flush behavior are unchanged. Durable, steer-receipt, created, and terminal emissions stay on the awaited per-event path and act as ordering barriers that flush any pending window first; terminal claims flush both sides before the status CAS so a warm tail cannot fence against its own completion. Benchmarked on local Redis (per-scenario RESETSTAT, INFO cpu/commandstats): at 100-200 ev/s a 25ms window cuts EVAL calls 67-82% and Redis engine CPU 52-70%; at the incident's 40 ev/s it halves EVALs while a 20ms window batches nothing (avg 1.0/frame). Producer await stall drops from ~0.9ms/delta to ~0.05ms/delta, matching the previously measured 16-18% USE_REDIS_STREAMS wall-time overhead. Delivery p95 stays under one window (27-28ms at 25ms). * 📝 docs: Document STREAM_DELTA_COALESCE_MS in .env.example * 🚧 fix: Drain Coalesced Windows Before Abort and Shutdown Terminal CAS abortJob and the graceful-shutdown finalizer claim terminal state through their own CAS calls rather than claimTerminalJob, so the pre-CAS coalescer flush did not cover them: a window tail buffered at abort time flushed against the already-aborted status, fenced (-1), and the false receipts retired the healthy runtime and error-closed subscribers before the abort FINAL frame. Extract the flush into flushCoalescedStreamBuffers and call it from all three terminal paths that can interrupt a live emitter (claim, abort, shutdown); the abort call sits ahead of the content snapshot so a chunk-log reconstruction also observes the flushed tail. Regression test aborts mid-window and asserts the tail is delivered with no subscriber error (fails without the fix). Paused-state terminals (approval expiry, pause-persistence timeout) need no flush: the pause's durable barrier already drained the window and nothing streams while paused. * 🛡️ fix: Keep Fence Retire a Lost-Signal Backstop on Aborted Runtimes A cross-replica abort claims its terminal CAS on the aborting replica, so the owner cannot drain its coalesced window pre-CAS; the window flush then fences against the aborted status. When the flush timer lands in the CAS-to-FINAL gap, the false receipts retired the owner runtime and detached its SSE handlers, so the abort FINAL published moments later was dropped and attached clients hung until client-side reconnect. The stop signal reaching the owner (~1ms pub/sub) is proof the abort/replacement flow owns terminal delivery and cleanup, so retireRuntimeAfterDurableFence now returns early for runtimes whose abort signal already landed. The forced teardown remains exactly for its original purpose: a fence observed by a NOT-yet-aborted owner, which is the lost-signal case. Regression test pins the race deterministically via the abort beforePublish hook (which runs between the CAS and the FINAL), forcing the owner flush there: without the guard the FINAL is dropped and the subscriber never completes; with it the FINAL delivers cleanly. * 🧰 fix: Gate, Isolate, and Bound the Coalesced Delta Path Three hardening fixes for the coalescing prototype. The manager now enables the fire-and-forget delta path only when the configured services actually batch — presence of flushPendingChunks/flushPendingAppends is the advertisement — so a custom transport that only implements emitChunk keeps the awaited per-event ordering contract even with STREAM_DELTA_COALESCE_MS set, and a batching transport is never paired with a per-event store (which would let the durable log trail the sequence counter by a full window). Batch unpack isolates each event: a throwing subscriber callback now degrades exactly like a lost individual frame (that sequence stalls until the reorder force-flush) instead of discarding the batch tail whose sequences were already reserved. And the emitter tracks outstanding coalesced receipts per stream, awaiting one once 256 accumulate: healthy settlement is a window plus a round trip so the count sits in single digits and the await never runs, while a stalled Redis now paces the producer exactly like the flag-off awaited path instead of accumulating batches, resolver closures, and queued commands without bound. Unit tests cover the capability gate (hint shape and await behavior for capable, incapable, and window-off configurations) and the backpressure threshold; an integration test pins the unpack isolation (fails without it: the batch tail vanishes instead of recovering via force-flush). Benchmark re-run confirms the counter and gate cost nothing measurable: identical EVAL counts and the serial drain still enqueue-bound. * 🎛️ fix: Make STREAM_DELTA_COALESCE_MS the Single Coalescing Switch The per-instance coalesceWindowMs constructor overrides could disagree with the environment the manager reads: overrides without the env silently did nothing, and an enabled env with an override of 0 selected the un-awaited manager path while both services published and appended per-event. Nothing in the repo passed these options, so remove them — the transport, the job store, and the manager now read STREAM_DELTA_COALESCE_MS through one resolver, making a half-enabled process unrepresentable rather than documented against. The capability-presence gate remains for services that do not implement batching at all. * 🧪 fix: Observe Abort Tail Delivery Before Terminal Teardown in Test The same-replica abort test waited for the coalesced tail only after abortJob returned, but abortJob's finally-block cleanup tears down local subscription state and publish receipts acknowledge Redis execution, not subscriber delivery. Single-node pub/sub delivers sub-millisecond so the frames always won locally; under the CI Redis Cluster they cross the cluster bus and lost the race, timing out the assertion. Await delivery concurrently with the abort instead — the pre-CAS flush publishes the tail several round trips before the teardown, so observing during the call is deterministic in both topologies. Test-only change. |
||
|---|---|---|
| .devcontainer | ||
| .do/gitnexus | ||
| .github | ||
| .husky | ||
| .vscode | ||
| api | ||
| client | ||
| config | ||
| e2e | ||
| helm | ||
| otel/langfuse-fanout | ||
| packages | ||
| redis-config | ||
| scripts | ||
| skill | ||
| src/tests | ||
| utils | ||
| .dockerignore | ||
| .env.example | ||
| .gitattributes | ||
| .gitignore | ||
| .nvmrc | ||
| .prettierrc | ||
| AGENTS.md | ||
| bun.lock | ||
| CLAUDE.md | ||
| CONTEXT.md | ||
| deploy-compose.langfuse-fanout.yml | ||
| deploy-compose.yml | ||
| docker-compose.langfuse-fanout.yml | ||
| docker-compose.override.yml.example | ||
| docker-compose.yml | ||
| Dockerfile | ||
| Dockerfile.multi | ||
| eslint.config.mjs | ||
| librechat.example.yaml | ||
| LICENSE | ||
| package-lock.json | ||
| package.json | ||
| rag.yml | ||
| README.md | ||
| README.zh.md | ||
| tool-intent-spec.md | ||
| turbo.json | ||
LibreChat
English · 中文
✨ Features
-
🖥️ UI & Experience inspired by ChatGPT with enhanced design and features
-
🤖 AI Model Selection:
- Anthropic (Claude), AWS Bedrock, OpenAI, Azure OpenAI, Google, Vertex AI, OpenAI Responses API (incl. Azure)
- Custom Endpoints: Use any OpenAI-compatible API with LibreChat, no proxy required
- Compatible with Local & Remote AI Providers:
- Ollama, groq, Cohere, Mistral AI, Apple MLX, koboldcpp, together.ai,
- OpenRouter, Helicone, Perplexity, ShuttleAI, Deepseek, Qwen, and more
-
- Secure, Sandboxed Execution in Python, Node.js (JS/TS), Go, C/C++, Java, PHP, Rust, and Fortran
- Seamless File Handling: Upload, process, and download files directly
- No Privacy Concerns: Fully isolated and secure execution
- Open-Source & Self-Hostable: powered by ClickHouse/code-interpreter
-
🔦 Agents & Tools Integration:
- LibreChat Agents:
- No-Code Custom Assistants: Build specialized, AI-driven helpers
- Agent Marketplace: Discover and deploy community-built agents
- Collaborative Sharing: Share agents with specific users and groups
- Flexible & Extensible: Use MCP Servers, tools, file search, code execution, and more
- Skills: Create reusable
SKILL.mdinstruction bundles for manual, automatic, or always-on agent workflows - Subagents: Delegate focused work to isolated child agent runs with their own context windows
- Compatible with Custom Endpoints, OpenAI, Azure, Anthropic, AWS Bedrock, Google, Vertex AI, Responses API, and more
- Model Context Protocol (MCP) Support for Tools
- LibreChat Agents:
-
🔍 Web Search:
- Search the internet and retrieve relevant information to enhance your AI context
- Combines search providers, content scrapers, and result rerankers for optimal results
- Customizable Jina Reranking: Configure custom Jina API URLs for reranking services
- Learn More →
-
🪄 Generative UI with Code Artifacts:
- Code Artifacts allow creation of React, HTML, and Mermaid diagrams directly in chat
-
🎨 Image Generation & Editing
- Text-to-image and image-to-image with GPT-Image-1
- Text-to-image with DALL-E (3/2), Stable Diffusion, Flux, or any MCP server
- Produce stunning visuals from prompts or refine existing images with a single instruction
-
💾 Presets & Context Management:
- Create, Save, & Share Custom Presets
- Switch between AI Endpoints and Presets mid-chat
- Edit, Resubmit, and Continue Messages with Conversation branching
- Create and share prompts with specific users and groups
- Fork Messages & Conversations for Advanced Context control
-
💬 Multimodal & File Interactions:
- Upload and analyze images with Claude 3, GPT-4.5, GPT-4o, o1, Llama-Vision, and Gemini 📸
- Chat with Files using Custom Endpoints, OpenAI, Azure, Anthropic, AWS Bedrock, & Google 🗃️
-
🌎 Multilingual UI:
- English, 中文 (简体), 中文 (繁體), العربية, Deutsch, Español, Français, Italiano
- Polski, Português (PT), Português (BR), Русский, 日本語, Svenska, 한국어, Tiếng Việt
- Türkçe, Nederlands, עברית, Català, Čeština, Dansk, Eesti, فارسی
- Suomi, Magyar, Հայերեն, Bahasa Indonesia, ქართული, Latviešu, ไทย, ئۇيغۇرچە
-
🧠 Reasoning UI:
- Dynamic Reasoning UI for Chain-of-Thought/Reasoning AI models like DeepSeek-R1
-
🎨 Customizable Interface:
- Customizable Dropdown & Interface that adapts to both power users and newcomers
-
- Never lose a response: AI responses automatically reconnect and resume if your connection drops
- Multi-Tab & Multi-Device Sync: Open the same chat in multiple tabs or pick up on another device
- Production-Ready: Works from single-server setups to horizontally scaled deployments with Redis
-
🗣️ Speech & Audio:
- Chat hands-free with Speech-to-Text and Text-to-Speech
- Automatically send and play Audio
- Supports OpenAI, Azure OpenAI, and Elevenlabs
-
📥 Import & Export Conversations:
- Import Conversations from LibreChat, ChatGPT, Chatbot UI
- Export conversations as screenshots, markdown, text, json
-
🔍 Search & Discovery:
- Search all messages/conversations
-
👥 Multi-User & Secure Access:
- Multi-User, Secure Authentication with OAuth2, LDAP, & Email Login Support
- Built-in Moderation, and Token spend tools
-
🎛️ Admin Panel:
- Browser-based UI to manage users, groups, roles, and configuration overrides
- Edit settings and per-role/group permissions live, without redeploying
- Bundled with the Docker Compose stacks for one-command setup
-
⚙️ Configuration & Deployment:
- Configure Proxy, Reverse Proxy, Docker, & many Deployment options
- Use S3 with CloudFront for stable media links, edge delivery, signed cookies, and secured downloads
- Use completely local or deploy on the cloud
-
📖 Open-Source & Community:
- Completely Open-Source & Built in Public
- Community-driven development, support, and feedback
For a thorough review of our features, see our docs here 📚
🪶 All-In-One AI Conversations with LibreChat
LibreChat is a self-hosted AI chat platform that unifies all major AI providers in a single, privacy-focused interface.
Beyond chat, LibreChat provides AI Agents, Model Context Protocol (MCP) support, Artifacts, Code Interpreter, custom actions, conversation search, and enterprise-ready multi-user authentication.
Open source, actively developed, and built for anyone who values control over their AI infrastructure.
🌐 Resources
GitHub Repo:
- RAG API: github.com/danny-avila/rag_api
- Website: github.com/LibreChat-AI/librechat.ai
Other:
- Website: librechat.ai
- Documentation: librechat.ai/docs
- Blog: librechat.ai/blog
📝 Changelog
Keep up with the latest updates by visiting the releases page and notes:
⚠️ Please consult the changelog for breaking changes before updating.
⭐ Star History
✨ Contributions
Contributions, suggestions, bug reports and fixes are welcome!
For new features, components, or extensions, please open an issue and discuss before sending a PR.
If you'd like to help translate LibreChat into your language, we'd love your contribution! Improving our translations not only makes LibreChat more accessible to users around the world but also enhances the overall user experience. Please check out our Translation Guide.
💖 This project exists in its current state thanks to all the people who contribute
🎉 Special Thanks
We thank Locize for their translation management tools that support multiple languages in LibreChat.