Commit graph

16 commits

Author SHA1 Message Date
Danny Avila
728fc1276e
🔒 fix: Bound /files/usage TTL Hold Instead of Clearing It (#14470)
* 🔒 fix: Bound `/files/usage` TTL Hold Instead of Clearing It

`POST /files/usage` marks queued attachments so the 1-hour upload-window
TTL cannot reap them before the client queue drains. It did this by
calling `updateFilesUsage`, which unsets `expiresAt` outright, turning
every touched upload into a permanently retained file.

The client queue is ephemeral browser state, so this also leaks in normal
use: a closed tab or cleared queue leaves nothing referencing the files,
but their TTL is already gone. The same mechanism let an authenticated
user pin arbitrary owned uploads indefinitely, and the route was excluded
from the file limiters, so the touch was entirely unmetered.

Make the operation match its intent, a renewable hold rather than a
release:

- Add `extendFilesTTL`, which pushes `expiresAt` forward by a bounded
  window in a single owner-scoped `updateMany`. Two filter guards keep it
  safe under client-supplied ids: `$exists: true` so an already-released
  file never has a TTL re-added (that would schedule a live file for
  deletion), and `$lt` so a hold only ever moves the deadline later.
  The owner scope is a required argument, so an unscoped call is a no-op
  rather than a cross-user update.
- `handleFilesUsageRequest` now holds for 24h instead of clearing, and no
  longer increments `usage`, since a queue touch is not a send. The real
  release still happens at drain, where `updateFilesUsage` marks the
  files used against an actual message.
- Give `/usage` its own per-user limiter. Keeping it off the upload quota
  was intentional, leaving it unmetered was not.

Abandoned queues are now reaped on schedule, and a replayed touch can only
ever re-assert the same bounded window.

* 🔒 fix: Anchor the `/files/usage` hold to upload time

Codex review on b687922.

The hold derived each new deadline from `Date.now()`, so a caller touching
once a day advanced it by another 24h every time, far below the rate limit.
That left indefinite preservation reachable and made the PR's replay claim
wrong: the window was bounded per call but not in aggregate.

Anchor the deadline to the file's immutable `createdAt` instead of the
request clock. `extendFilesTTL` now takes a lifetime and sets
`expiresAt = max(expiresAt, createdAt + holdMs)` in an aggregation
pipeline, so the target is a fixed point per file and replay is inert
rather than merely bounded. `$max` keeps the widen-only property and the
`expiresAt: {$exists: true}` filter still refuses to resurrect a released
TTL; `createdAt: {$exists: true}` fail-closes when the anchor is absent.

The update runs with `timestamps: false`: a hold is TTL bookkeeping, not a
content write, and bumping `updatedAt` also made every re-touch count as a
modification, hiding whether the deadline actually moved.

Also drop four `.node_modules-*` symlinks that `git add -A` swept in from
an npm install. They pointed at absolute paths on one machine, so every
other checkout got dangling entries. Added the pattern to .gitignore so a
workspace install cannot reintroduce them.

* 🔒 fix: Track the configured approval window in the `/files/usage` hold

Codex review on 9277620.

`endpoints.agents.checkpointer.ttl` is a positive int with no upper bound,
and its docs invite raising it for longer review windows. It drives the
pending-action expiry, so a run can legitimately stay paused past 24h. The
fixed 24h lifetime would then let Mongo reap an attachment while its
approval was still live, and the later queue drain would send a file that
no longer exists.

Replace the fixed constant with `resolveFilesUsageHoldMs`, which adds the
configured approval window to a 24h baseline covering upload, enqueue, and
the run reaching its pause. The route reads the window from the same
`getApprovalTtlMs(checkpointerCfg)` the pending action uses, so the two
stay in lockstep.

The replay bound is unaffected: the window is a per-deployment constant and
the deadline is still `createdAt + holdMs`, so a replayed touch re-asserts
the same instant and `$max` skips the write. Only an operator config change
moves it, never a client.

* 🔒 fix: Renew the `/files/usage` hold across queued runs, under a ceiling

Codex review on 2bd3c52.

The drain sends one queued item per run completion, and each item starts a
run that may itself pause for the full approval window. Since the hold was
taken once at enqueue and pinned to the upload time, an item several places
back could sit through multiple approval windows and lose its attachment
while its chip and the live approval were still there. Another regression
from this PR: the old `$unset` made retention permanent, so deep queues
happened to work.

The queue is unbounded, so no fixed lifetime covers it. Split the hold into
a renewable window and a ceiling:

  expiresAt = max(expiresAt, min(now + renewMs, createdAt + maxLifetimeMs))

`renewMs` covers one run's wait and is granted from now, so a queue that is
still draining re-asserts it at each transition; `useQueueDrain` now marks
the remaining items' files whenever it pops one. `maxLifetimeMs` is
measured from the immutable upload time and clamps every renewal, so
repeated touches converge on a ceiling instead of advancing per call, which
keeps the replay bound from the previous round intact.

This also tightens abandonment: a queue nobody drains now lapses one
`renewMs` after its last touch instead of surviving to the ceiling.

`useQueueDrain`'s spec gained a QueryClientProvider, since the renewal goes
through react-query.

* 🔒 fix: Renew queued holds on a heartbeat, and stop dropping batches

Codex review on f616bed.

Three gaps in the renewal added last commit:

- `collectQueuedFileIds` returned early at the server's 10-id cap, so a
  remainder holding more than one batch renewed only its first message and
  left the rest on their enqueue-time hold. Collect everything and split
  into capped requests instead of truncating.
- A refused `ask()` restores the popped item, but renewal ran before the
  send and covered only the pre-existing remainder. Since the run-end signal
  is already consumed, nothing would touch that item again. Renewal now runs
  after `ask` and includes the restored item.
- A single run can interrupt for approval more than once, each pause running
  to the configured window, so renewing only at drain transitions leaves a
  gap longer than `renewMs` with no renewal in it. The ceiling cannot help
  when nothing renews.

The third is the same structural gap as the previous round along a new axis:
renewal tied to discrete events loses the file whenever two events are
further apart than the hold. Rather than hook each transition, renew on a
30 minute heartbeat while anything is queued, which is far below the
smallest hold (24h) and so covers any single gap regardless of cause.

Still bounded: every renewal is clamped against the file's upload time, so
the ceiling is unchanged. A queue nobody has open emits no heartbeat and
lapses one `renewMs` after its last touch, preserving the abandonment
behaviour.

* 🔒 fix: Cover the pre-migration queue, first tick, and `/usage/`

Codex review on 892a27d.

- The heartbeat watched only the active conversation id, but `drainNext`
  merges in the `NEW_CONVO` queue, which outlives the URL update: items
  queued during the first turn stay keyed there until that run ends. It now
  renews the union of both, deduped since they are the same atom before
  migration.
- The interval installed without firing, so returning to a conversation
  whose hold was nearly up waited out a full period before the first
  renewal. It now renews immediately, then on each tick.
- Express's non-strict routing sends `POST /files/usage/` to the same
  handler with `req.path === '/usage/'`, so the exact comparison pushed it
  onto both upload limiters. A trailing-slash client would have spent its
  upload quota, and collected file-upload violations, on metadata
  heartbeats. Matching now tolerates the trailing slash.

Firing on effect start also made the drain-time renewal redundant: popping
an item changes the held set, so the renewal effect re-runs on its own. The
one case it cannot see is a refused send, where restoring the item leaves
the set identical, so that branch keeps an explicit renewal and the rest is
removed. Net one request per transition instead of two.
2026-07-28 07:37:26 -04:00
Danny Avila
9bb351ad9c
🧭 feat: Mid-Run Steering and Queued Messages for Agent Runs (#14220)
* 🧭 feat: Mid-Run Steering and Queued Messages for Agent Runs

Steering: submit a message while a run is generating; the server queues
it in the job store (cross-instance) and a run-scoped PostToolBatch hook
injects it into graph state at the next tool-batch boundary, records an
inline 'steer' content part on the response (replayed as a user message
on later turns), and streams on_steer_applied to the client.

Queuing: messages composed during a run auto-send as normal follow-up
turns after clean completion (one per final event, FIFO); user aborts
leave them as chips unless armed by interrupt-and-send.

Requires hook injectedMessages support in @librechat/agents
(danny-avila/agents#299); hard-gated via a capability probe so older
SDKs 501 the steer route instead of draining and dropping messages.

* 🧵 fix: Harden Steering Against Finalization Races and Route Guard Gaps

Addresses local Codex review findings on the steering feature:

- Close-and-drain the steer queue atomically at finalization (final event,
  abort) so a steer POST racing teardown is rejected instead of 202-ACKed
  and then silently cleared; the closed flag lives on the job hash and is
  reset when a replacement job reuses the stream id.
- Clear inherited steer queues on createJob — a job replacement must not
  drain the replaced run's messages.
- Keep steers queued across a HITL pause instead of draining them into
  ephemeral client state: resumeState re-seeds chips on reload and the
  resumed run injects them at its first tool boundary (steers key TTL now
  extends to the approval window; on_steers_pending event removed).
- Queue the NO_ACTIVE_RUN steer fallback while the final SSE is still
  settling — a direct send would be dropped by ask()'s in-flight guard.
- Reconcile the 202 ACK against on_steer_applied events that beat it over
  the SSE, so a chip can't be re-minted after its removal event passed.
- Allow the per-send Steer override when the default action is queue.
- Apply the configured message rate limiters and the PII filter to
  POST /chat/steer — a steer is model-bound user text.

* ✅ ci: Assert Steering Capability Probe Against the Installed SDK

CI installs the published @librechat/agents pin (pre-injectedMessages),
where isSteeringSupported() is legitimately false — the probe test now
asserts it mirrors the installed SDK's capability flag instead of
hardcoding the capability-bearing build's value. Verified against both
the published 3.2.61 dist and the agents#299 build.

* 🛟 fix: Preserve Steer Text Across Run-End, Error, and Abort Races

Codex round 2 (4 P2s):
- Applied-steer-id set survives run end (capped at 100) and converted
  ids join it, so a 202 ACK that lands after final/abort drops its chip
  instead of re-minting a stranded pending one.
- Failed runs no longer strand acknowledged chips: both error paths
  convert local pending chips to queued follow-ups (chip text is
  client-local), and the server closes the steer queue before emitting
  the error so a racing steer POST gets 404 fallback instead of a 202
  whose payload dies with the job.
- sendQueuedNow keys on steer availability, not the default action —
  send-now on a queued chip is an explicit override for queue-preferring
  users.
- Stop path consumes pendingSteers from the abort HTTP response as a
  fallback for the SSE final event it may close before processing;
  conversion is deduped so double delivery is a no-op (shared
  useSteerConvert hook).

* 📎 feat: Carry Attachments Through During-Run Queued Messages

Steering stays text-only (SDK injection, inline STEER part, and replay
are all text), so a during-run submit with media now queues the whole
message as one unit instead of silently stranding the files:

- QueuedMessage gains `files`; composer attachments are consumed into
  the queued item at queue time (steerFromComposer / queueFromComposer /
  interruptAndSend), fixing the latent hazard where lingering composer
  files glued onto whatever `ask` vacuumed up next.
- Enter-steer with attachments degrades to queue with an explanatory
  toast; the per-send menu routes through the same composer-aware
  wrappers.
- The drain and sendQueuedNow pass the item's files as `overrideFiles`;
  media items never steer (send as a normal turn when idle, re-front
  otherwise). ask() no longer clears composer state for caller-supplied
  overrideFiles — only regenerate keeps that behavior.
- During-run submits hold while uploads are in flight, mirroring the
  send button's filesLoading gate; queued chips show a paperclip count.

* 🎛️ feat: Rework During-Run Chips into Action Rows

Full-width rows above the composer (reference-UI parity): each queued
message shows a primary Steer/Send-now action, delete, and a "…" menu
with Edit message (restores text + attachments into the composer) and a
Turn on queueing/steering toggle that flips the Enter default. Steer
rows share the layout with status text; failed steers keep retry /
edit / queue-convert. The per-send menu gains the same default toggle.
Queued file refs now retain filename + bytes so edit-restore rebuilds
real composer entries (draft-recovery shape).

* 🖇️ feat: Steer With Attachments (Multimodal Mid-Run Injection)

Steering now carries media end-to-end instead of degrading to queue:

- The steer POST accepts sanitized attachment refs (cap 10; only
  file_id is trusted — the drain re-fetches owner-scoped and re-derives
  everything else). SteerQueueItem/TPendingSteer/SteerContentPart carry
  `files` refs; encoded data is never persisted or queued.
- New api/server/services/Files/steering.js decouples attachment
  building from the request path: encodeSteerContent reuses the exact
  per-turn pipeline (addFileContextToMessage + processAttachments'
  single-pass categorize/encode, SDK formatMessage assembly,
  prependFileContext for extracted text) with zero new encoding code.
  buildSteerMedia feeds the drain hook's new buildMedia seam (any
  failure degrades that steer to text-only — words always land);
  stampSteerPartMedia re-encodes past steer parts per turn with ONE
  batched owner-scoped fetch and stamps a transient `media` array,
  replaced immutably so it can never leak into a save. Replay honors
  resendFiles like regular message media.
- The SDK's formatAgentMessages (the formatter agents actually use)
  gained the steer replay branch on the PR branch; the local
  formatMessages.js branch now mirrors the media preference.
- Client: steerFromComposer consumes composer files into the POST,
  chips/seeding/conversions carry files everywhere (retry, queue
  convert, abort/error recovery), queued media items steer for real,
  and SteerBubble renders the steered attachments inline.

* 🧵 fix: Harden Steer Recovery Races and Drain Isolation

Codex round 3 (7 fixes):
- A 202 ACK landing after the run ended converts straight to a queued
  follow-up (server queue is gone; no event will ever resolve a pending
  chip for a finished run). Covers stream errors with in-flight POSTs.
- A Stop that lands pre-completion can arrive as a final with
  unfinished:true and no aborted flag — runEnd now treats it as aborted
  so queued messages are not auto-sent against the user's Stop.
- Leftover-steer conversion merges chronologically by createdAt instead
  of appending, preserving the order the user composed.
- Auto-drained queued messages pass explicit (possibly empty)
  overrideFiles/overrideQuotes/overrideManualSkills: a drain can no
  longer vacuum up files, quotes, or skill picks staged in the composer
  for the user's NEXT message (ask() treats overrideFiles != null as
  authoritative).
- Failed-steer Retry and resume-on-load chip restoration keep the
  steer's attachments.
- The job-replacement guard moved INSIDE the store's atomic
  drain/close-and-drain (Lua createdAt compare; in-memory equivalent):
  a stale run's hook or finalization can neither consume, close, nor
  steal a replacement job's steer queue, and the drain hook drops its
  separate check-then-drain round trip.

* 🧰 refactor: Typed Steer Controller, Single-Query Media Pass, Round-4 Fixes

Codex round 4 + efficiency tightening in one pass:

- Moved the steer guard ladder (validation, file sanitization via a
  shared toSteerFileRef picker, ownership/tenant checks, status-guarded
  enqueue) into packages/api as handleSteerRequest; api/steer.js is now
  a thin wrapper. Ladder covered against the REAL in-memory job manager
  in request.spec.ts; the api spec pins only the wrapper contract.
- Folded the steer replay stamp into the turn's ONE historical-files
  query: collectHistoricalFileRefs also gathers steer-part refs, the
  owner-scoped doc map rides client state, and stampSteerPartMedia
  consumes it (no second round trip) while encoding parts in parallel.
- Stamped steer media now counts against the run budget (existing
  multimodal counter over the non-text parts, folded into
  indexTokenCountMap/promptTokens after the stamp).
- Steer route runs the PII filter BEFORE moderateText, matching chat.js
  so blocked sensitive text never reaches the external moderation API.
- Interrupt & send survives the abort-response-beats-SSE-final race:
  stopGenerating writes the run-end signal itself when the one-shot
  interrupt flag is armed and no signal landed (double-fire safe).
- Resume reconciles chips against the server's still-queued list even
  when EMPTY, clearing chips for steers applied while disconnected.
- The local formatter's steer flush preserves non-text assistant parts
  (array-content AIMessage) instead of folding to text.

* 🔒 fix: Replay-Aware Capability Gate and Round-5 Race Closures

- isSteeringSupported now requires BOTH halves of the SDK contract:
  injection (HOOK_INJECTED_MESSAGES_CAPABLE) AND replay
  (ContentTypes.STEER, shipped in the same SDK commit as the
  formatAgentMessages steer branch). An SDK that can inject but not
  replay 501s the steer route — no release window can create steer
  parts that would leak into provider-facing assistant content.
- The local formatter mirrors the SDK's anchor reset: a post-steer
  tool_call mints a fresh AIMessage instead of attaching to the
  pre-steer anchor (invalid provider ordering).
- Queued-chip send-now and the NO_ACTIVE_RUN fallback pass explicit
  (possibly empty) overrideFiles so an idle send can't vacuum composer
  files staged for a different draft.
- Redis createJob deletes the stale steer list BEFORE the replacement
  hash is written as running — a steer 202-accepted against the new job
  can never be wiped by the reset.
- Resumed-turn finalization mirrors the normal path's terminal drain:
  createdAt-guarded close-and-drain, leftovers ride the resumed final
  event as pendingSteers instead of being cleared by completeJob.
- buildSteerMedia restores composer order over the $in result so
  multi-attachment steers reach the model in the order the user saw.

* ⚛️ fix: Atomic Job Replacement and Boundary-Clean Steering Module

Codex round 6 (5 fixed, 1 standing deferral):
- createJob resets the steer queue and writes the job hash in ONE
  same-slot Lua script (JOB_CREATE_LUA): a steer POST can no longer
  interleave between them on cluster, so a steer accepted against one
  run can never be drained into another. Redis-validated.
- The steering media pipeline moved to packages/api
  (agents/steering/media.ts) with injected getFiles and a structural
  client interface — /api keeps zero steering logic; specs ported to
  the DI seam.
- handleSteerRequest checks the job BEFORE the capability gate: a steer
  racing completion on an unsupported SDK gets 404 (send-now) instead
  of a 501 queue with no run-end signal left to drain it.
- useQueueDrain binds to the active conversation: navigating away
  between the final SSE and the drain effect leaves the signal
  unconsumed instead of submitting A's follow-up into B; the drain
  fires on return.
- abortJob closes and drains the steer queue BEFORE the content
  snapshot, so a drain-hook apply that lands pre-drain is captured
  inline rather than lost between the snapshot and the terminal drain.

* 🚦 fix: Parked Run-End Signals, Interrupt Priority, Settled-Run Fallbacks

Codex round 7 (5 fixes):
- Run-end signals for a non-active conversation are PARKED per
  conversation instead of squatting the shared index slot: a later run
  finishing on the same pane can no longer overwrite them, and the
  parked drain fires when the user returns.
- "Interrupt & send" front-inserts carry a priority flag that outranks
  createdAt when abort leftovers merge back chronologically — the
  urgent redirect drains first, not the oldest steer.
- STEER_UNSUPPORTED/RUN_PAUSED/QUEUE_FULL rejections landing after the
  run settled mirror the NO_ACTIVE_RUN fallback and send immediately
  (queueing would strand the text with no run-end signal left); on the
  pinned SDK this is the common Enter-near-run-end path.
- A failed abort (e.g. 404 when the run completed first) still signals
  the interrupt drain, so the queued interrupt message can't strand and
  the armed flag can't leak onto a later run.
- Steered-image fallback alt text is localized (com_ui_attached_image).

* 📌 chore: Adopt Published @librechat/agents Types Post-Bump

dev's pin bump to ^3.2.62 (the release carrying injection + steer
replay) landed via merge; the steering runtime now uses the SDK's real
InjectedMessage/hook-output types instead of the local structural
mirrors that bridged the pre-publish window. The two-half capability
probe stays as the defensive gate for mismatched deployments — and the
capability spec now exercises its TRUE path against the published
package in CI.

* 🛅 feat: Park-and-Claim Steer Recovery + Host-View Content Reads

Codex round 8 (6 fixed incl. both P1s, 1 push-back):
- The long-deferred no-subscriber gap is closed: every terminal drain
  (final, aborted-final, error, abortJob, resumed finalize) PARKS
  acknowledged leftovers on the job hash (unrecoveredSteers), and the
  status route claims them exactly once for inactive jobs — a client
  that closed/reloaded past the transient final event restores its
  steers as queued chips within the post-terminal TTL. A replacement
  run clears the parked copy (a live client started it).
- Same-instance content reads are steer-complete: RedisJobStore now
  caches the HOST content array (WeakRef) via setContentParts and
  prefers it over the SDK graph cache, whose view never contains
  host-authored steer parts; the graph fallback splice-INSERTS steer
  chunks at their recorded host-view indices (the graph array is
  unshifted, so assignment would overwrite SDK parts).
- Replay token accounting now counts prepended file-context text: full
  stamped content minus the steer body (already counted), so large
  steered documents hit the budget instead of bypassing pruning.
- The queue drain restores an item when ask() refuses without sending
  (history not yet in cache after navigating back) — text is never
  silently dropped.
- The armed interrupt flag travels WITH a parked run-end signal, so
  another run on the same pane can neither consume nor clear it.
- parseTextParts extracts steer text (search indexing / audio).

* 🎛️ refactor: Single Send Slot + In-Thread Steer Messages

- Merge the during-run send affordance into the send/stop button slot:
  with composer text the send button replaces Stop (Enter = default
  action), hover reveals Steer/Queue/Interrupt rows with shortcuts;
  drop the separate DuringRunActionsMenu chevron
- Add during-run keyboard chords: Cmd/Ctrl+Enter = non-default action,
  Alt+Enter = interrupt & send (plain-Enter submitters only)
- Render steers as standard user messages in the thread: SteerPart
  (icon + author header + user text presentation) replaces the
  SteerBubble, and submitted steers appear immediately at the projected
  injection point via the PendingSteers slot on the streaming message
- Keep composer rows only for recoverable states: failed steers
  (retry/edit/queue) and queued follow-ups

* 🩹 fix: Keep the Replacement Submission Alive Across Abort Settlement

The aborted run's final SSE event fires before the abort HTTP response
resolves, so an armed interrupt & send drains and starts the NEXT
submission while the abort POST is still in flight. The response
handler's unconditional clearAllSubmissions() then reset the new
submission, aborting its stream attach before the subscribe — the
follow-up ran and persisted server-side but the live placeholder
finalized empty (content appeared only after reload).

useAbortCleanup captures the submission before the abort round-trip
and both settlement paths (success and 404-catch) clear only when the
captured submission is still current; a replacement stays untouched.
Plain Stop behavior is unchanged.

* 🧭 test: Playwright E2E for Mid-Run Steering and Queuing

- Add e2e/specs/mock/steering.spec.ts: steer mid-run (202 + immediate
  in-thread pending part + real MCP tool boundary + words survive run
  end), Cmd/Ctrl+Enter queue with auto-send after clean completion,
  and Alt+Enter interrupt & send with the follow-up streaming into the
  live view
- Add the E2E_STEER_TOOL_REPLY fake-model marker: slow preamble, a
  real remember_fact MCP tool call (PostToolBatch boundary), then a
  final turn
- Test 1 pins the run-end degradation contract while the SDK's
  top-level agentId stamping bug blocks live injection; its header
  documents the assertions to flip once the fixed SDK is pinned

* 🧷 fix: Job-Independent Steer Recovery + Expiry and Resume-Gap Parking

Codex round 10: the park-and-claim recovery had lifecycle holes.

- Move parked steers off the job hash onto their own bounded-TTL store
  key (JOB_CREATE_LUA resets it; deleteJob leaves it alone): the default
  completeJob path deletes the job record immediately, and the Redis
  read path never deserialized the old hash field — recovery previously
  worked only with STREAM_KEEP_COMPLETED_JOBS on the in-memory store
- Carry the owner identity inside the parked payload and authorize the
  claim against it, so the status route recovers steers on its jobless
  branch too (the common reload-after-terminal case); a non-owner claim
  returns nothing and re-parks the payload
- Park queued steers on approval expiry: snapshot the frozen queue
  before the requires_action→aborted CAS (whose terminal cleanup drops
  the steers key) and park only when the CAS wins
- Mirror the terminal drain/park block in resume.js's failure path,
  which previously let completeJob's backstop clear 202-accepted steers
- Close the Redis snapshot→subscribe resume gap: re-peek the queue
  after attaching and re-surface missed on_steer_applied events from
  the durable content view (synthesizeAppliedSteerEvents), updating
  resumeState.pendingSteers to the live queue

* 📌 chore: Require @librechat/agents 3.2.63 + Applied-Steer E2E Contract

- Bump the @librechat/agents pin to ^3.2.63 in api/ and packages/api/:
  it scopes the hook agentId marker to subagent child graphs, so the
  steering drain hook fires at top-level tool-batch boundaries and
  mid-run injection is active (danny-avila/agents PR 307)
- Flip e2e steering test 1 from the documented degradation contract to
  the applied-steer contract: the optimistic in-thread part transitions
  to the persisted part at the tool boundary and survives inside the
  response after run end, with no queued follow-up turn

* 🎗️ feat: Steered Messages Join the Message-Nav Ribs

Steers are user messages, so they get their own clickable rib on the
navigation rail, interleaved at their in-thread position inside the
response that absorbed them (one DOM query in document order). SteerPart
anchors itself as #steer-<id> with a steer-render marker — both the
optimistic pending entry and the persisted part — and the rib carries
the user role label with a preview drawn from the steer's text body,
skipping the author header.

* ❎ feat: Cancel a Queued Steer Before Injection + True User-Message Alignment

- Add POST /chat/steer/cancel: removes ONE still-queued steer by id via
  an atomic list rebuild (Redis Lua preserves order and TTL), authorized
  against the job owner; removed:false is advisory — the cancel lost its
  race to the drain or the run end, never an error
- Surface an × on the in-thread pending steer (server-acknowledged
  entries only): optimistic removal, restored if the POST fails since
  the server would still inject the words
- Outdent SteerPart past the response's icon column so steers sit flush
  with top-level message rows, reading as regular user messages

* 🧯 fix: Round-11 Recovery Hardening + Provider-Free Pending Slot

- Reconcile the resume steer gap by steerId SETS, not queue length — a
  steer added in the gap (or an equal-length drain+enqueue swap) now
  refreshes resumeState.pendingSteers and still synthesizes the missed
  on_steer_applied events
- Make completeJob's terminal backstop park: direct error-path callers
  without the controllers' close-and-park no longer silently clear
  202-accepted steers (createdAt-guarded closeAndDrain + owner park
  before the terminal write)
- Persist the steer part BEFORE media encoding in the drain hook: an
  abort inside the encode window can no longer lose a file-steer (the
  part refs come from the enqueue-sanitized item; replay re-encodes
  per turn unchanged)
- Move the parked-claim owner check INSIDE the atomic store claim
  (substring gate in the Lua / in-memory equivalent): a non-owner probe
  can no longer transiently delete the recovery payload; the app-side
  parse stays authoritative
- Park queued steers in BOTH stores' own requires_action expiry
  cleanup, which bypassed the manager-level sweep
- Sweep expired parked steers from the in-memory store's periodic
  cleanup; restore a queued chip when send-now's submit is refused;
  upsert steer ACKs so an SSE reconnect reseed cannot duplicate chips
- Mount the cancel mutation per steer item so the pending slot needs no
  QueryClient on ordinary streaming renders (fixes the CI failure in
  ContentParts.integration.test)
- Skipped delivery-gated parking (finding 8): transport receiver counts
  cannot prove browser delivery, and gating the only durable copy on
  them trades cosmetic chip resurrection for real text loss; the window
  is already bounded by claim-on-read, createJob reset, and the TTL

* 🩺 fix: Annotate PARKED_STEERS_TTL_MS for isolatedDeclarations

tsdown's d.ts generation requires explicit types on exported consts
with computed initializers; tsc --noEmit does not run that check, so
the round-11 export slipped past local verification and broke Build
packages (and every downstream CI job that consumes the built dist).

* 🛟 fix: Round-12 Terminal-Path Recovery + Durable Steer Events

- Park queued steers before the stale-running reap deletes a crashed or
  hung job in BOTH stores — the one terminal path with no controller
  finalization; requires_action expiry parking refactored onto the same
  snapshot/park helpers
- Enqueue instead of dropping when a steer fallback send is refused:
  both the NO_ACTIVE_RUN branch and the settled-run rejection branch
  now observe sendNow's false return
- Recover on the SSE reconnect-404 terminal path: convert local pending
  steers to queued, claim parked steers via /chat/status, and write a
  non-completed run-end signal so interrupt flags release without
  auto-sending an unknown outcome
- Fall back to a positive parked-recovery TTL when completedTtl is 0
  (SET EX 0 is invalid and silently killed recovery)
- Make on_steer_applied durable before publish: emitChunk gains a
  durable option that awaits the chunk-log append (best-effort) ahead
  of the transport publish; the default delta path stays fire-and-forget

* 🔐 fix: Round-13 Steer Authorization + Trusted File Refs

- Resolve client-supplied steer file refs against the DB owner-scoped
  at enqueue and queue only DB-derived shapes (same filter as the
  injection fetch, shared via refs.ts); any unresolved id fails loud
  with 400 — spoofed type/filepath metadata can no longer be persisted
  into assistant content or rendered in chat/share views
- Enforce agent authorization on /chat/steer against the ORIGINATING
  run's job identity: the chat path's role gate (AGENTS:USE, with the
  same non-agents-endpoint skip) plus the per-agent ACL check with the
  capability bypass — revoked access mid-run can no longer inject;
  cancel stays ownership-only (nothing model-bound)
- Mark steered uploads used after a successful enqueue (owner-scoped,
  best-effort) so the upload-window TTL cannot reap a file the
  persisted steer part references
- Consume the parked recovery copy after live delivery: converting
  final/abort/error pendingSteers fires one owner-gated claim-on-read,
  so dismissed chips can no longer resurrect on a later reload

* 🎙️ fix: Round-14 Composer-Context Fidelity + TTS and Queue-State Gaps

- Keep steer text out of generic assistant text extraction:
  parseTextParts excludes STEER parts by default with an includeSteer
  opt-in for the full-record surfaces (Meili indexing, aborted-response
  persistence) — TTS callers no longer speak the user's own mid-run
  words
- Mark queued uploads used at enqueue time via a minimal owner-scoped
  POST /files/usage (fail-closed without a user; upload limiters do not
  apply to a metadata touch), fired once wherever composer files enter
  the queued state — the upload-window TTL can no longer reap a file
  waiting out a long run or approval pause
- Carry quote chips and manual skill picks on queued items: captured
  and consumed from the composer at queue/interrupt time exactly like
  files, threaded through the drain and send-now overrides, and
  restored by the queued row's Edit message
- Key an early-aborted FIRST turn's run-end signal to NEW_CONVO
  (resolveRunEndTarget) so queued follow-ups stay visible on the
  restored new-chat composer instead of parking under an optimistic
  stream id the user never sees again

* 🧿 fix: Round-15 Gap Coverage + Consolidated Sweep (Share Leak, Abort Ids, Chip Hygiene)

- Run the resume steer-gap check for every still-active job: an empty
  snapshot no longer skips the re-peek, and synthesis now keys on the
  FRESH content view so an applied-in-gap steer that was never
  snapshotted still re-surfaces (over-emission is benign — applied-id
  dedupe, index-stable parts)
- Thread queued context through steer degradation: sendQueuedNow passes
  the item's quotes/skills into submitSteer, and every fallback
  (requeue or settled send) restores them instead of dropping to
  text+files
- Stop shared links from leaking steer attachment refs: the share
  snapshot now walks content — files-excluded shares strip steer-part
  files entirely; files-included shares sanitize and share-route them
  like top-level files (copy-on-write, non-steer content by reference)
- Seed pending-steer chips unconditionally on load/return so a steer
  applied while away cannot linger as a stale chip beside its part
- Use the abort response's resolved job id: chips/drain-signal land
  where the user actually is (NEW_CONVO for a new-held first turn,
  consistent with resolveRunEndTarget) while the parked-copy claim hits
  the resolved id instead of a no-op /chat/status/new
- Open steered documents like normal message files (FilePreviewDialog)
- Cap the applied-steer id set on the live path via a shared helper;
  kept surviving run end deliberately (late-ACK race depends on it) and
  fixed the atom comment that claimed otherwise

* 💡 fix: Un-light Steer Ribs When Their Node Is Replaced

Two stacked gaps kept a steer rib lit after scrolling away: the
pending→applied swap replaces the DOM node under the same id, which
produces no IntersectionObserver exit and — because the entry list
dedupes on (id, preview) — no entries change either, so the observer
kept watching a detached node; and the rail's mutation filter only
reacted to .message-render nodes, so steer-node swaps and removals
never triggered a refresh at all.

- reconcileObservedElements re-points the observer at replaced nodes
  from the mutation-driven refresh regardless of entries identity,
  dropping stale visibility until the fresh node reports (the observer
  fires its initial intersection immediately, so a truly visible part
  re-lights within a frame)
- The mutation filter now recognizes steer-render nodes alongside
  message rows

* 🪪 fix: Round-16 Recovery Owner Fields + Context Stickiness + Share Labels

- Park resumed-run leftovers with the manager facade's metadata owner
  fields: a bare job.userId is undefined on that shape, which made
  every parked payload from a resumed HITL run unclaimable
- Keep a queued item's quotes/skills sticky through a successful steer
  ACK: the pending chip carries them (client-only), reseeds preserve
  them across reconnects, and every terminal conversion — local or
  server-list, merged by steerId — restores them onto the queued item
- Convert resumeState.pendingSteers on the inactive status branch
  (deduped against unrecoveredSteers) so steers observed in the
  expired-pause-before-sweeper window convert instead of vanishing
  until a later reload
- Label shared steer parts share-safely via the existing ShareContext:
  a viewer's own name no longer appears on the sharer's steered
  messages

* ✂️ fix: Carry Steer Context Through the Failed-Chip Edit Action

Retry and convert-to-queue already preserve a failed steer's carried
quotes/skills; Edit message dropped them on the way back to the
composer. It now restores them through the same context path.
2026-07-14 10:11:10 -04:00
Danny Avila
8735c1763c
🧵 fix: Preserve Upload Context Across Multipart Routes (#13072)
* fix skill multipart imports under strict isolation

* fix file upload context after multipart parsing

* fix skill upload tenant resolution

* fix rejected upload cleanup
2026-05-11 15:46:48 -04:00
Danny Avila
52e6796635
📦 chore: Bump Express.js to v5 (#10671)
* chore: update express to version 5.1.0 in package.json

* chore: update express-rate-limit to version 8.2.1 in package.json and package-lock.json

* fix: Enhance server startup error handling in experimental and index files

* Added error handling for server startup in both experimental.js and index.js to log errors and exit the process if the server fails to start.
* Updated comments in openidStrategy.js to clarify the purpose of the CustomOpenIDStrategy class and its relation to Express version changes.

* chore: Implement rate limiting for all POST routes excluding /speech, required for express v5

* Added middleware to apply IP and user rate limiters to all POST requests, ensuring that the /speech route remains unaffected.
* Enhanced code clarity with comments explaining the new rate limiting logic.

* chore: Enable writable req.query for mongoSanitize compatibility in Express 5

* chore: Ensure req.body exists in multiple middleware and route files for Express 5 compatibility
2025-12-11 16:36:15 -05:00
Danny Avila
9a210971f5
🛜 refactor: Streamline App Config Usage (#9234)
* WIP: app.locals refactoring

WIP: appConfig

fix: update memory configuration retrieval to use getAppConfig based on user role

fix: update comment for AppConfig interface to clarify purpose

🏷️ refactor: Update tests to use getAppConfig for endpoint configurations

ci: Update AppService tests to initialize app config instead of app.locals

ci: Integrate getAppConfig into remaining tests

refactor: Update multer storage destination to use promise-based getAppConfig and improve error handling in tests

refactor: Rename initializeAppConfig to setAppConfig and update related tests

ci: Mock getAppConfig in various tests to provide default configurations

refactor: Update convertMCPToolsToPlugins to use mcpManager for server configuration and adjust related tests

chore: rename `Config/getAppConfig` -> `Config/app`

fix: streamline OpenAI image tools configuration by removing direct appConfig dependency and using function parameters

chore: correct parameter documentation for imageOutputType in ToolService.js

refactor: remove `getCustomConfig` dependency in config route

refactor: update domain validation to use appConfig for allowed domains

refactor: use appConfig registration property

chore: remove app parameter from AppService invocation

refactor: update AppConfig interface to correct registration and turnstile configurations

refactor: remove getCustomConfig dependency and use getAppConfig in PluginController, multer, and MCP services

refactor: replace getCustomConfig with getAppConfig in STTService, TTSService, and related files

refactor: replace getCustomConfig with getAppConfig in Conversation and Message models, update tempChatRetention functions to use AppConfig type

refactor: update getAppConfig calls in Conversation and Message models to include user role for temporary chat expiration

ci: update related tests

refactor: update getAppConfig call in getCustomConfigSpeech to include user role

fix: update appConfig usage to access allowedDomains from actions instead of registration

refactor: enhance AppConfig to include fileStrategies and update related file strategy logic

refactor: update imports to use normalizeEndpointName from @librechat/api and remove redundant definitions

chore: remove deprecated unused RunManager

refactor: get balance config primarily from appConfig

refactor: remove customConfig dependency for appConfig and streamline loadConfigModels logic

refactor: remove getCustomConfig usage and use app config in file citations

refactor: consolidate endpoint loading logic into loadEndpoints function

refactor: update appConfig access to use endpoints structure across various services

refactor: implement custom endpoints configuration and streamline endpoint loading logic

refactor: update getAppConfig call to include user role parameter

refactor: streamline endpoint configuration and enhance appConfig usage across services

refactor: replace getMCPAuthMap with getUserMCPAuthMap and remove unused getCustomConfig file

refactor: add type annotation for loadedEndpoints in loadEndpoints function

refactor: move /services/Files/images/parse to TS API

chore: add missing FILE_CITATIONS permission to IRole interface

refactor: restructure toolkits to TS API

refactor: separate manifest logic into its own module

refactor: consolidate tool loading logic into a new tools module for startup logic

refactor: move interface config logic to TS API

refactor: migrate checkEmailConfig to TypeScript and update imports

refactor: add FunctionTool interface and availableTools to AppConfig

refactor: decouple caching and DB operations from AppService, make part of consolidated `getAppConfig`

WIP: fix tests

* fix: rebase conflicts

* refactor: remove app.locals references

* refactor: replace getBalanceConfig with getAppConfig in various strategies and middleware

* refactor: replace appConfig?.balance with getBalanceConfig in various controllers and clients

* test: add balance configuration to titleConvo method in AgentClient tests

* chore: remove unused `openai-chat-tokens` package

* chore: remove unused imports in initializeMCPs.js

* refactor: update balance configuration to use getAppConfig instead of getBalanceConfig

* refactor: integrate configMiddleware for centralized configuration handling

* refactor: optimize email domain validation by removing unnecessary async calls

* refactor: simplify multer storage configuration by removing async calls

* refactor: reorder imports for better readability in user.js

* refactor: replace getAppConfig calls with req.config for improved performance

* chore: replace getAppConfig calls with req.config in tests for centralized configuration handling

* chore: remove unused override config

* refactor: add configMiddleware to endpoint route and replace getAppConfig with req.config

* chore: remove customConfig parameter from TTSService constructor

* refactor: pass appConfig from request to processFileCitations for improved configuration handling

* refactor: remove configMiddleware from endpoint route and retrieve appConfig directly in getEndpointsConfig if not in `req.config`

* test: add mockAppConfig to processFileCitations tests for improved configuration handling

* fix: pass req.config to hasCustomUserVars and call without await after synchronous refactor

* fix: type safety in useExportConversation

* refactor: retrieve appConfig using getAppConfig in PluginController and remove configMiddleware from plugins route, to avoid always retrieving when plugins are cached

* chore: change `MongoUser` typedef to `IUser`

* fix: Add `user` and `config` fields to ServerRequest and update JSDoc type annotations from Express.Request to ServerRequest

* fix: remove unused setAppConfig mock from Server configuration tests
2025-08-26 12:10:18 -04:00
Danny Avila
d012da0065
🛡️ fix: Enhance File Upload Security & Error Handling (#4705)
* fix: sanitize filename in multer storage callback

* fix: ensure temporary image upload file is deleted after processing

* fix: prevent cleanup flag from being set to false before actually deleted

* refactor: user avatar, typing, use 'file' for formData instead of 'input', add disk storage, use localization

* fix: update Avatar component to include image dimensions in formData and refactor editor reference type

* fix: refactor avatar upload handling to use fs for file reading and enhance file validation

* fix: ensure temporary image upload file is deleted after processing

* fix: refactor avatar upload routes and handlers for agents and assistants, improve file handling and validation

* fix: improve audio file validation and cleanup

* fix: add filename sanitization utility and integrate it into multer storage configuration

* fix: update group project ID check for null and refactor delete prompt group response type

* fix: invalid access control for deleting prompt groups

* fix: add error handling and logging to checkBan middleware

* fix: catch conversation parsing errors

* chore: revert unnecessary height and width parameters from avatar upload

* chore: update librechat-data-provider version to 0.7.55

* style: ensure KaTeX can spread across visible space
2024-11-12 16:41:04 -05:00
Marco Beretta
1aad315de6
🎤 feat: add custom speech config, browser TTS/STT features, and dynamic speech tab settings (#2921)
* feat: update useTextToSpeech and useSpeechToText hooks to support external audio endpoints

This commit updates the useTextToSpeech and useSpeechToText hooks in the Input directory to support external audio endpoints. It introduces the useGetExternalTextToSpeech and useGetExternalSpeechToText hooks, which determine whether the audio endpoints should be set to 'browser' or 'external' based on the value of the endpointTTS and endpointSTT Recoil states. The useTextToSpeech and useSpeechToText hooks now use these new hooks to determine whether to use external audio endpoints

* feat: add userSelect style to ConversationModeSwitch label

* fix: remove unused updateTokenWebsocket function and import

The updateTokenWebsocket function and its import are no longer used in the OpenAIClient module. This commit removes the function and import to clean up the codebase

* feat: support external audio endpoints in useTextToSpeech and useSpeechToText hooks

This commit updates the useTextToSpeech and useSpeechToText hooks in the Input directory to support external audio endpoints. It introduces the useGetExternalTextToSpeech and useGetExternalSpeechToText hooks, which determine whether the audio endpoints should be set to 'browser' or 'external' based on the value of the endpointTTS and endpointSTT Recoil states. The useTextToSpeech and useSpeechToText hooks now use these new hooks to determine whether to use external audio endpoints

* feat: update AutomaticPlayback component to AutomaticPlaybackSwitch; tests: added AutomaticPlaybackSwitch.spec
>
> This commit renames the AutomaticPlayback component to AutomaticPlaybackSwitch in the Speech directory. The new name better reflects the purpose of the component and aligns with the naming convention used in the codebase.

* feat: update useSpeechToText hook to include interimTranscript

This commit updates the useSpeechToText hook in the client/src/components/Chat/Input/AudioRecorder.tsx file to include the interimTranscript state. This allows for real-time display of the speech-to-text transcription while the user is still speaking. The interimTranscript is now used to update the text area value during recording.

* feat: Add customConfigSpeech API endpoint for retrieving custom speech configuration

This commit adds a new API endpoint  in the  file under the  directory. This endpoint is responsible for retrieving the custom speech configuration using the  function from the  module

* feat: update store var  and ; fix: getCustomConfigSpeech

* fix: client tests, removed unused import

* feat: Update useCustomConfigSpeechQuery to return an array of custom speech configurations

This commit modifies the useCustomConfigSpeechQuery function in the client/src/data-provider/queries.ts file to return an array of custom speech configurations instead of a single object. This change allows for better handling and manipulation of the data in the application

* feat: Update useCustomConfigSpeechQuery to return an array of custom speech configurations

* refactor: Update variable name in speechTab schema

* refactor: removed unused and nested code

* fix: using recoilState

* refactor: Update Speech component to use useCallback for setting settings

* fix: test

* fix: tests

* feature: ensure that the settings don't change after modifying then through the UI

* remove comment

* fix: Handle error gracefully in getCustomConfigSpeech and getVoices endpoints

* fix: Handle error

* fix: backend tests

* fix: invalid custom config logging

* chore: add back custom config info logging

* chore: revert loadCustomConfig spec

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2024-07-05 10:13:34 -04:00
Marco Beretta
ee673d682e
📧 feat: email verification (#2344)
* feat: verification email

* chore: email verification invalid; localize: update

* fix: redirect to login when signup: fix: save emailVerified correctly

* docs: update ALLOW_UNVERIFIED_EMAIL_LOGIN; fix: don't accept login only when ALLOW_UNVERIFIED_EMAIL_LOGIN = true

* fix: user needs to be authenticated

* style: update

* fix: registration success message and redirect logic

* refactor: use `isEnabled` in ALLOW_UNVERIFIED_EMAIL_LOGIN

* refactor: move checkEmailConfig to server/utils

* refactor: use req as param for verifyEmail function

* chore: jsdoc

* chore: remove console log

* refactor: rename `createNewUser` to `createSocialUser`

* refactor: update typing and add expiresAt field to userSchema

* refactor: begin use of user methods over direct model access for User

* refactor: initial email verification rewrite

* chore: typing

* refactor: registration flow rewrite

* chore: remove help center text

* refactor: update getUser to getUserById and add findUser methods. general fixes from recent changes

* refactor: Update updateUser method to remove expiresAt field and use $set and $unset operations, createUser now returns Id only

* refactor: Update openidStrategy to use optional chaining for avatar check, move saveBuffer init to buffer condition

* refactor: logout on deleteUser mutatation

* refactor: Update openidStrategy login success message format

* refactor: Add emailVerified field to Discord and Facebook profile details

* refactor: move limiters to separate middleware dir

* refactor: Add limiters for email verification and password reset

* refactor: Remove getUserController and update routes and controllers accordingly

* refactor: Update getUserById method to exclude password and version fields

* refactor: move verification to user route, add resend verification option

* refactor: Improve email verification process and resend option

* refactor: remove more direct model access of User and remove unused code

* refactor: replace user authentication methods and token generation

* fix: add user.id to jwt user

* refactor: Update AuthContext to include setError function, add resend link to Login Form, make registration redirect shorter

* fix(updateUserPluginsService): ensure userPlugins variable is defined

* refactor: Delete all shared links for a specific user

* fix: remove use of direct User.save() in handleExistingUser

* fix(importLibreChatConvo): handle missing createdAt field in messages

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2024-06-07 15:06:47 -04:00
Danny Avila
8318f26d66
🔉 feat: TTS/STT rate limiters (#2925)
* fix: remove double initialization of speech routes

* refactor(useMessageHelpers): more consistent latestMessage updates based on unique textKey and early returns when setting

* feat: TTS/STT rate limiters

* chore: remove console log

* fix: make modular chat true by default
2024-05-30 18:39:21 -04:00
Danny Avila
40685f6eb4
🚀 refactor: Enhance Custom Endpoints, Message Logic, and Payload Handling (#2895)
* chore: use node-fetch for OpenAIClient fetch key for non-crashing usage of AbortController in Bun runtime

* chore: variable order

* fix(useSSE): prevent finalHandler call in abortConversation to update messages/conversation after user navigated away

* chore: params order

* refactor: organize intermediate message logic and ensure correct variables are passed

* fix: Add stt and tts routes before upload limiters, prevent bans

* fix(abortRun): temp fix to delete unfinished messages to avoid message thread parent relationship issues

* refactor: Update AnthropicClient to use node-fetch for fetch key and add proxy support

* fix(gptPlugins): ensure parentMessageId/messageId relationship is maintained

* feat(BaseClient): custom fetch function to analyze/edit payloads just before sending (also prevents abortController crash on Bun runtime)

* feat: `directEndpoint` and `titleMessageRole` custom endpoint options

* chore: Bump version to 0.6.6 in data-provider package.json
2024-05-28 14:52:12 -04:00
Danny Avila
b3e03b75d0
🔉 feat: Speech-to-text / Text-to-speech (initial support) (#2836)
* Update TextChat.jsx

* Update SubmitButton.jsx

* Update TextChat.jsx

* Update SubmitButton.jsx

* Create ListeningIcon.tsx

* Update index.ts

* Update SubmitButton.jsx

* Update TextChat.jsx

* Update ListeningIcon.tsx

* Update ListeningIcon.tsx

* Create SpeechRecognition.tsx

* Update TextChat.jsx

* Update TextChat.jsx

* Update SpeechRecognition.tsx

* Update TextChat.jsx

* Update SpeechRecognition.tsx

* Update SpeechRecognition.tsx

* Update SpeechRecognition.tsx

* Update SpeechRecognition.tsx

* Update SubmitButton.jsx

* Update TextChat.jsx

* Update SpeechRecognition.tsx

* Create SpeechSynthesis.tsx

* Update index.jsx

* Update SpeechSynthesis.tsx

* Update SpeechRecognition.tsx

* Update TextChat.jsx

* Update SpeechRecognition.tsx

* Update SpeechRecognition.tsx

* Update SpeechRecognition.tsx

* Update TextChat.jsx

* Squashed commit of the following:

commit 28230d9305
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Sun Sep 3 02:44:26 2023 +0200

    feat: delete button confirm (#875)

    * base for confirm delete

    * more like OpenAI

commit 2b54e3f9fe
Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
Date:   Fri Sep 1 14:20:51 2023 -0400

    update: install script (#858)

commit 1cd0fd9d5a
Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
Date:   Fri Sep 1 08:12:35 2023 -0400

    doc: Hugging Face Deployment (#867)

    * docs: update ToC

    * docs: update ToC

    * update huggingface.md

    * update render.md

    * update huggingface.md

    * update mongodb.md

    * update huggingface.md

    * update README.md

commit aeeb3d3050
Author: Mu Yuan <yuanmu.email@gmail.com>
Date:   Thu Aug 31 07:21:27 2023 +0800

    Update Zh.tsx (#862)

    * Update Zh.tsx

    Changed the translation of several words to make it more relevant to Chinese usage habits.

    * Update Zh.tsx

    Changed the translation of several words to make it more relevant to Chinese usage habits

commit 80e2e2675b
Author: Raí <140329135+itzraiss@users.noreply.github.com>
Date:   Mon Aug 28 18:05:46 2023 -0300

    Translation of 'com_ui_pay_per_call:' to Spanish and Portuguese that were missing. (#857)

    * Update Br.tsx

    * Update Es.tsx

    * Update Br.tsx

    * Update Es.tsx

commit 3574d0b823
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Mon Aug 28 14:49:26 2023 -0400

    docs: make_your_own.md formatting fix for mkdocs (#855)

commit d672ac690d
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Mon Aug 28 14:24:10 2023 -0400

    Release v0.5.8 (#854)

    * chore: add 'api' image to tag release workflow

    * docs: update DO deployment docs to include instruction about latest stable release, as well as security best practices

    * Release v0.5.8

    * docs: Update digitalocean.md with firewall section images

    * docs: make_your_own.md formatting fix for mkdocs

commit d3e7627046
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Mon Aug 28 12:03:08 2023 -0400

    refactor(plugins): Improve OpenAPI handling, Show Multiple Plugins, & Other Improvements (#845)

    * feat(PluginsClient.js): add conversationId to options object in the constructor
    feat(PluginsClient.js): add support for Code Interpreter plugin
    feat(PluginsClient.js): add support for Code Interpreter plugin in the availableTools manifest
    feat(CodeInterpreter.js): add CodeInterpreterTools module
    feat(CodeInterpreter.js): add RunCommand class
    feat(CodeInterpreter.js): add ReadFile class
    feat(CodeInterpreter.js): add WriteFile class
    feat(handleTools.js): add support for loading Code Interpreter plugin

    * chore(api): update langchain dependency to version 0.0.123

    * fix(CodeInterpreter.js): add support for extracting environment from code
    fix(WriteFile.js): add support for extracting environment from data
    fix(extractionChain.js): add utility functions for creating extraction chain from Zod schema
    fix(handleTools.js): refactor getOpenAIKey function to handle user-provided API key
    fix(handleTools.js): pass model and openAIApiKey to CodeInterpreter constructor

    * fix(tools): rename CodeInterpreterTools to E2BTools
    fix(tools): rename code_interpreter pluginKey to e2b_code_interpreter

    * chore(PluginsClient.js): comment out unused import and function findMessageContent
    feat(PluginsClient.js): add support for CodeSherpa plugin
    feat(PluginsClient.js): add CodeSherpaTools to available tools
    feat(PluginsClient.js): update manifest.json to include CodeSherpa plugin
    feat(CodeSherpaTools.js): create RunCode and RunCommand classes for CodeSherpa plugin

    feat(E2BTools.js): Add E2BTools module for extracting environment from code and running commands, reading and writing files
    fix(codesherpa.js): Remove codesherpa module as it is no longer needed

    feat(handleTools.js): add support for CodeSherpaTools in loadTools function
    feat(loadToolSuite.js): create loadToolSuite utility function to load a suite of tools

    * feat(PluginsClient.js): add support for CodeSherpa v2 plugin
    feat(PluginsClient.js): add CodeSherpa v1 plugin to available tools
    feat(PluginsClient.js): add CodeSherpa v2 plugin to available tools
    feat(PluginsClient.js): update manifest.json for CodeSherpa v1 plugin
    feat(PluginsClient.js): update manifest.json for CodeSherpa v2 plugin
    feat(CodeSherpa.js): implement CodeSherpa plugin for interactive code and shell command execution
    feat(CodeSherpaTools.js): implement RunCode and RunCommand plugins for CodeSherpa v1
    feat(CodeSherpaTools.js): update RunCode and RunCommand plugins for CodeSherpa v2

    fix(handleTools.js): add CodeSherpa import statement
    fix(handleTools.js): change pluginKey from 'codesherpa' to 'codesherpa_tools'
    fix(handleTools.js): remove model and openAIApiKey from options object in e2b_code_interpreter tool
    fix(handleTools.js): remove openAIApiKey from options object in codesherpa_tools tool
    fix(loadToolSuite.js): remove model and openAIApiKey parameters from loadToolSuite function

    * feat(initializeFunctionsAgent.js): add prefix to agentArgs in initializeFunctionsAgent function

    The prefix is added to the agentArgs in the initializeFunctionsAgent function. This prefix is used to provide instructions to the agent when it receives any instructions from a webpage, plugin, or other tool. The agent will notify the user immediately and ask them if they wish to carry out or ignore the instructions.

    * feat(PluginsClient.js): add ChatTool to the list of tools if it meets the conditions
    feat(tools/index.js): import and export ChatTool
    feat(ChatTool.js): create ChatTool class with necessary properties and methods

    * fix(initializeFunctionsAgent.js): update PREFIX message to include sharing all output from the tool
    fix(E2BTools.js): update descriptions for RunCommand, ReadFile, and WriteFile plugins to provide more clarity and context

    * chore: rebuild package-lock after rebase

    * chore: remove deleted file from rebase

    * wip: refactor plugin message handling to mirror chat.openai.com, handle incoming stream for plugin use

    * wip: new plugin handling

    * wip: show multiple plugins handling

    * feat(plugins): save new plugins array

    * chore: bump langchain

    * feat(experimental): support streaming in between plugins

    * refactor(PluginsClient): factor out helper methods to avoid bloating the class, refactor(gptPlugins): use agent action for mapping the name of action

    * fix(handleTools): fix tests by adding condition to return original toolFunctions map

    * refactor(MessageContent): Allow the last index to be last in case it has text (may change with streaming)

    * feat(Plugins): add handleParsingErrors, useful when LLM does not invoke function params

    * chore: edit out experimental codesherpa integration

    * refactor(OpenAPIPlugin): rework tool to be 'function-first', as the spec functions are explicitly passed to agent model

    * refactor(initializeFunctionsAgent): improve error handling and system message

    * refactor(CodeSherpa, Wolfram): optimize token usage by delegating bulk of instructions to system message

    * style(Plugins): match official style with input/outputs

    * chore: remove unnecessary console logs used for testing

    * fix(abortMiddleware): render markdown when message is aborted

    * feat(plugins): add BrowserOp

    * refactor(OpenAPIPlugin): improve prompt handling

    * fix(useGenerations): hide edit button when message is submitting/streaming

    * refactor(loadSpecs): optimize OpenAPI spec loading by only loading requested specs instead of all of them

    * fix(loadSpecs): will retain original behavior when no tools are passed to the function

    * fix(MessageContent): ensure cursor only shows up for last message and last display index
    fix(Message): show legacy plugin and pass isLast to Content

    * chore: remove console.logs

    * docs: update docs based on breaking changes and new features
    refactor(structured/SD): use description_for_model for detailed prompting

    * docs(azure): make plugins section more clear

    * refactor(structured/SD): change default payload to SD-WebUI to prefer realism and config for SDXL

    * refactor(structured/SD): further improve system message prompt

    * docs: update breaking changes after rebase

    * refactor(MessageContent): factor out EditMessage, types, Container to separate files, rename Content -> Markdown

    * fix(CodeInterpreter): linting errors

    * chore: reduce browser console logs from message streams

    * chore: re-enable debug logs for plugins/langchain to help with user troubleshooting

    * chore(manifest.json): add [Experimental] tag to CodeInterpreter plugins, which are not intended as the end-all be-all implementation of this feature for Librechat

commit 66b8580487
Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
Date:   Mon Aug 28 09:18:25 2023 -0400

    docs: third-party tools (#848)

    * docs: third-party tools

    * docs: third-party tools

    * Update third-party.md

    * Update third-party.md

    ---------

    Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>

commit 9791a78161
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Mon Aug 28 15:14:05 2023 +0200

    adjust the animation (#843)

commit 3797ec6082
Author: Ronith <87087292+ronith256@users.noreply.github.com>
Date:   Mon Aug 28 18:43:50 2023 +0530

    feat: Add Code Interpreter Plugin (#837)

    * feat: Add Code Interpreter Plugin

    Adds a Simple Code Interpreter Plugin.
    ## Features:
    - Runs code using local Python Environment

    ## Issues
    - Code execution is not sandboxed.

    * Add Docker Sandbox for Python Server

commit e2397076a2
Author: Alex Zhang <ztc2011@gmail.com>
Date:   Mon Aug 28 00:55:34 2023 +0800

    🌐: Chinese Translation (#846)

commit 50c15c704f
Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
Date:   Sat Aug 26 19:36:59 2023 -0400

    Language translation: Polish (#840)

    * Language translation: Polish

    * Language translation: Polish

    * Revert changes in language-contributions.md

commit 29d3640546
Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
Date:   Sat Aug 26 19:36:25 2023 -0400

    docs: updates (#841)

commit 39c626aa8e
Author: Danny Avila <messagedaniel@protonmail.com>
Date:   Fri Aug 25 09:29:19 2023 -0400

    fix: isEdited edge case where latest Message is not saved due to aborting too quickly

commit ae5c06f381
Author: Danny Avila <messagedaniel@protonmail.com>
Date:   Fri Aug 25 09:13:50 2023 -0400

    fix(chatGPTBrowser): render markdown formatting by setting isCreatedByUser, fix(useMessageHandler): avoid double appearance of cursor by setting latest message at initial response creation time

commit 9ef1686e18
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Thu Aug 24 20:24:47 2023 -0400

    Update mkdocs.yml

commit 5bbe411569
Author: Flynn <dev@flynnbuckingham.com>
Date:   Thu Aug 24 20:20:37 2023 -0400

    Add podman installation instructions. Update dockerfile to stub env (#819)

    * Added podman container installation docs. Updated dockerfile to stub env file if not present in source

    * Fix typos

commit 887fec99ca
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Fri Aug 25 02:11:27 2023 +0200

    🌐: Russian Translation (#830)

commit 007d51ede1
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Fri Aug 25 02:10:48 2023 +0200

    feat: facebook login (#820)

    * Facebook strategy

    * Update user_auth_system.md

    * Update user_auth_system.md

commit a569020312
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Thu Aug 24 21:59:11 2023 +0200

    Fix Meilisearch error and refactor of the server index.js (#832)

    * fix meilisearch error at startup

    * limit the nesting

    * disable useless console log

    * fix(indexSync.js): removed redundant searchEnabled

    * refactor(index.js): moved configureSocialLogins to a new file

    * refactor(socialLogins.js): removed unnecessary conditional

commit 37347d4683
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Wed Aug 23 16:14:17 2023 -0400

    fix(registration): Make Username optional (#831)

    * fix(User.js): update validation schema for username field, allow empty string as a valid value
    fix(validators.js): update validation schema for username field, allow empty string as a valid value
    fix(Registration.tsx, validators.js): update validation rules for name and username fields, change minimum length to 2 and maximum length to 80, assure they match and allow empty string as a valid value
    fix(Eng.tsx): update localization string for com_auth_username, indicate that it is optional

    * fix(User.js): update regex pattern for username validation to allow special characters @#$%&*()
    fix(validators.js): update regex pattern for username validation to allow special characters @#$%&*()

    * fix(Registration.spec.tsx): fix validation error message for username length requirement

commit d38e463d34
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Wed Aug 23 13:44:40 2023 -0400

    fix(bingAI): markdown and error formatting for final stream response (#829)

    * fix(bingAI): markdown formatting for final stream response due to new strict payload validation on the frontend

    * fix: add missing prop to bing Error response

commit 7dc27b10f1
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Tue Aug 22 18:44:59 2023 -0400

    feat: Edit AI Messages, Edit Messages in Place (#825)

    * refactor: replace lodash import with specific function import

    fix(api): esm imports to cjs

    * refactor(Messages.tsx): convert to TS, out-source scrollToDiv logic to a custom hook
    fix(ScreenshotContext.tsx): change Ref to RefObject in ScreenshotContextType
    feat(useScrollToRef.ts): add useScrollToRef hook for scrolling to a ref with throttle
    fix(Chat.tsx): update import path for Messages component
    fix(Search.tsx): update import path for Messages component

    * chore(types.ts): add TAskProps and TOptions types
    refactor(useMessageHandler.ts): use TAskFunction type for ask function signature

    * refactor(Message/Content): convert to TS, move Plugin component to Content dir

    * feat(MessageContent.tsx): add MessageContent component for displaying and editing message content
    feat(index.ts): export MessageContent component from Messages/Content directory

    * wip(Message.jsx): conversion and use of new component in progress

    * refactor: convert Message.jsx to TS and fix typing/imports based on changes

    * refactor: add typed props and refactor MultiMessage to TS, fix typing issues resulting from the conversion

    * edit message in progress

    * feat: complete edit AI message logic, refactor continue logic

    * feat(middleware): add validateMessageReq middleware
    feat(routes): add validation for message requests using validateMessageReq middleware
    feat(routes): add create, read, update, and delete routes for messages

    * feat: complete frontend logic for editing messages in place
    feat(messages.js): update route for updating a specific message
    - Change the route for updating a message to include the messageId in the URL
    - Update the request handler to use the messageId from the request parameters and the text from the request body
    - Call the updateMessage function with the updated parameters

    feat(MessageContent.tsx): add functionality to update a message
    - Import the useUpdateMessageMutation hook from the data provider
    - Destructure the conversationId, parentMessageId, and messageId from the message object
    - Create a mutation function using the useUpdateMessageMutation hook
    - Implement the updateMessage function to call the mutation function with the updated message parameters
    - Update the messages state to reflect the updated message text

    feat(api-endpoints.ts): update messages endpoint to include messageId
    - Update the messages endpoint to include the messageId as an optional parameter

    feat(data-service.ts): add updateMessage function
    - Implement the updateMessage function to make a PUT request to

    * fix(messages.js): make updateMessage function asynchronous and await its execution

    * style(EditIcon): make icon active for AI message

    * feat(gptPlugins/anthropic): add edit support

    * fix(validateMessageReq.js): handle case when conversationId is 'new' and return empty array
    feat(Message.tsx): pass message prop to SiblingSwitch component
    refactor(SiblingSwitch.tsx): convert to TS

    * fix(useMessageHandler.ts): remove message from currentMessages if isContinued is true
    feat(useMessageHandler.ts): add support for submission messages in setMessages
    fix(useServerStream.ts): remove unnecessary conditional in setMessages
    fix(useServerStream.ts): remove isContinued variable from submission

    * fix(continue): switch to continued message generation when continuing an earlier branch in conversation

    * fix(abortMiddleware.js): fix condition to check partialText length
    chore(abortMiddleware.js): add error logging when abortMessage fails

    * refactor(MessageHeader.tsx): convert to TS
    fix(Plugin.tsx): add default value for className prop in Plugin component

    * refactor(MultiMessage.tsx): remove commented out code
    docs(MultiMessage.tsx): update comment to clarify when siblingIdx is reset

    * fix(GenerationButtons): optimistic state for continue button

    * fix(MessageContent.tsx): add data-testid attribute to message text editor
    fix(messages.spec.ts): update waitForServerStream function to include edit endpoint check
    feat(messages.spec.ts): add test case for editing messages

    * fix(HoverButtons & Message & useGenerations): Refactor edit functionality and related conditions

    - Update enterEdit function signature and prop
    - Create and utilize hideEditButton variable
    - Enhance conditions for edit button visibility and active state
    - Update button event handlers
    - Introduce isEditableEndpoint in useGenerations and refine continueSupported condition.

    * fix(useGenerations.ts): fix condition for hideEditButton to include error and searchResult
    chore(data-provider): bump version to 0.1.6
    fix(types.ts): add status property to TError type

    * chore: bump @dqbd/tiktoken to 1.0.7

    * fix(abortMiddleware.js): add required isCreatedByUser property to the error response object

    * refactor(Message.tsx): remove unnecessary props from SiblingSwitch component, as setLatestMessage is firing on every switch already
    refactor(SiblingSwitch.tsx): remove unused imports and code

    * chore(BaseClient.js): move console.debug statements back inside if block

commit db77163f5d
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Tue Aug 22 14:15:14 2023 +0200

    docs: update chimeragpt (#826)

    * Update free_ai_apis.md

    * Update free_ai_apis.md

commit 4a4e803df3
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Mon Aug 21 20:15:18 2023 +0200

    style(Dialog): Improved Close Button ("X") position (#824)

commit 909b00c752
Author: Daniel Avila <messagedaniel@protonmail.com>
Date:   Sun Aug 20 21:04:36 2023 -0400

    fix(HoverButtons): light/dark styling to match official site

commit 61dcb4d307
Author: Naosuke Yokoe <ankerasoy@gmail.com>
Date:   Sat Aug 19 20:11:31 2023 +0900

    feat: Azure Cognitive Search Plugin (#815)

    * feat(AzureCognitiveSearchPlugin)

    * feat(tools/AzureCognitiveSearch.js): Add a new plugin (not structured
      version)
    * feat(tools/structured/AzureCognitiveSearch.js): Add a new plugin (structured version)
    * feat(tools/manifest.json, tools/index.js, tools/util/handleTools.js):
      Add configurations for the plugin
    * feat(api/package.json, package-lock.json): Installed a new package for the
      plugin (@azure/search-documents)
    * feat(.env.example): Add new environment variables for the plugin

    Here is the link to the corresponding discussion page:
    https://github.com/danny-avila/LibreChat/discussions/567

    * docs(AzureCognitiveSearchPlugin)

    * docs(features/plugins/azure_cognitive_search.md): Add a new document
      for the plugin

    * (fix:.env.example)

    * reverted extra whitespaces removed by the editor

    * docs(mkdocs.yml)

    * Add the Azure Cognitive Search Plugin's documentation item to
    mkdocs.yml.

commit 3c7f67fa76
Author: Danny Avila <messagedaniel@protonmail.com>
Date:   Fri Aug 18 12:40:33 2023 -0400

    fix(abortMiddleware): handle early abort error where userMessage.conversationId is undefined. In this case, the userId will be used as the abortKey

commit c74c68a135
Author: Danny Avila <messagedaniel@protonmail.com>
Date:   Fri Aug 18 12:10:30 2023 -0400

    refactor(MessageHandler -> useServerStream): convert all relating files to TS and correct typings based on this change: properly refactor MessageHandler to a custom hook, where it's passed a submission object to instantiate the stream. This is the bare minimum groundwork for potentially having multiple streams running, which would be a big project to modularize a lot of the global state into maps/multiple streams, particular useful for having multiple views in place

commit 8b4d3c2c21
Author: Danny Avila <messagedaniel@protonmail.com>
Date:   Fri Aug 18 12:04:29 2023 -0400

    refactor(routes): convert to TS

commit d612cfcb45
Author: Danny Avila <messagedaniel@protonmail.com>
Date:   Fri Aug 18 12:02:39 2023 -0400

    chore(Auth): reorder exports in Auth component
    fix(PluginAuthForm): handle case when pluginKey is null or undefined
    fix(PluginStoreDialog): handle case when getAvailablePluginFromKey is null or undefined
    fix(AuthContext): make authConfig optional in AuthContextProvider
    feat(hooks): add useServerStream hook
    fix(conversation): setSubmission to null instead of empty object
    fix(preset): specify type for presets atom
    fix(search): specify type for isSearchEnabled atom
    fix(submission): specify type for submission atom

commit c40b95f424
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Fri Aug 18 16:11:00 2023 +0200

    feat: Disable Registration with social login (#813)

    * Google, Github and Discord

    * update .env.example with ALLOW_SOCIAL_REGISTRATION

    * fix some conflict

    * refactor strategy

    * Update user_auth_system.md

    * Update user_auth_system.md

commit 46ed5aaccd
Author: Patrick <psarnowski@gmail.com>
Date:   Fri Aug 18 09:38:24 2023 -0400

    Show the response scores from Bing. (#814)

commit 1dacfa49f0
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Thu Aug 17 20:32:31 2023 +0200

    update profile picture (#792)

commit afd43afb60
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Thu Aug 17 12:50:05 2023 -0400

    feat(GPT/Anthropic): Continue Regenerating & Generation Buttons (#808)

    * feat(useMessageHandler.js/ts): Refactor and add features to handle user messages, support multiple endpoints/models, generate placeholder responses, regeneration, and stopGeneration function

    fix(conversation.ts, buildTree.ts): Import TMessage type, handle null parentMessageId

    feat(schemas.ts): Update and add schemas for various AI services, add default values, optional fields, and endpoint-to-schema mapping, create parseConvo function

    chore(useMessageHandler.js, schemas.ts): Remove unused imports, variables, and chatGPT enum

    * wip: add generation buttons

    * refactor(cleanupPreset.ts): simplify cleanupPreset function
    refactor(getDefaultConversation.js): remove unused code and simplify getDefaultConversation function

    feat(utils): add getDefaultConversation function

    This commit adds a new utility function called `getDefaultConversation` to the `client/src/utils/getDefaultConversation.ts` file. This function is responsible for generating a default conversation object based on the provided parameters.

    The `getDefaultConversation` function takes in an object with the following properties:
    - `conversation`: The conversation object to be used as a base.
    - `endpointsConfig`: The configuration object containing information about the available endpoints.
    - `preset`: An optional preset object that can be used to override the default behavior.

    The function first tries to determine the target endpoint based on the preset object. If a valid endpoint is found, it is used as the target endpoint. If not, the function tries to retrieve the last conversation setup from the local storage and uses its endpoint if it is valid. If neither the preset nor the local storage contains a valid endpoint, the function falls back to a default endpoint.

    Once the target endpoint is determined,

    * fix(utils): remove console.error statement in buildDefaultConversation function
    fix(schemas): add default values for catch blocks in openAISchema, googleSchema, bingAISchema, anthropicSchema, chatGPTBrowserSchema, and gptPluginsSchema

    * fix: endpoint not changing on change of preset from other endpoint, wip: refactor

    * refactor: preset items to TSX

    * refactor: convert resetConvo to TS

    * refactor(getDefaultConversation.ts): move defaultEndpoints array to the top of the file for better readability
    refactor(getDefaultConversation.ts): extract getDefaultEndpoint function for better code organization and reusability

    * feat(svg): add ContinueIcon component
    feat(svg): add RegenerateIcon component
    feat(svg): add ContinueIcon and RegenerateIcon components to index.ts

    * feat(Button.tsx): add onClick and className props to Button component
    feat(GenerationButtons.tsx): add logic to display Regenerate or StopGenerating button based on isSubmitting and messages
    feat(Regenerate.tsx): create Regenerate component with RegenerateIcon and handleRegenerate function
    feat(StopGenerating.tsx): create StopGenerating component with StopGeneratingIcon and handleStopGenerating function

    * fix(TextChat.jsx): reorder imports and variables for better readability
    fix(TextChat.jsx): fix typo in condition for isNotAppendable variable
    fix(TextChat.jsx): remove unused handleStopGenerating function
    fix(ContinueIcon.tsx): remove unnecessary closing tags for polygon elements
    fix(useMessageHandler.ts): add missing type annotations for handleStopGenerating and handleRegenerate functions
    fix(useMessageHandler.ts): remove unused variables in return statement

    * fix(getDefaultConversation.ts): refactor code to use getLocalStorageItems function
    feat(getLocalStorageItems.ts): add utility function to retrieve items from local storage

    * fix(OpenAIClient.js): add support for streaming result in sendCompletion method
    feat(OpenAIClient.js): add finish_reason metadata to opts in sendCompletion method
    feat(Message.js): add finish_reason field to Message model
    feat(messageSchema.js): add finish_reason field to messageSchema
    feat(openAI.js): parse chatGptLabel and promptPrefix from req.body and pass rest of the modelOptions to endpointOption
    feat(openAI.js): add addMetadata function to store metadata in ask function
    feat(openAI.js): add metadata to response if available
    feat(schemas.ts): add finish_reason field to tMessageSchema

    * feat(types.ts): add TOnClick and TGenButtonProps types for button components
    feat(Continue.tsx): create Continue component for generating button
    feat(GenerationButtons.tsx): update GenerationButtons component to use Continue component
    feat(Regenerate.tsx): create Regenerate component for regenerating button
    feat(Stop.tsx): create Stop component for stop generating button

    * feat(MessageHandler.jsx): add MessageHandler component to handle messages and conversations
    fix(Root.jsx): fix import paths for Nav and MessageHandler components

    * feat(useMessageHandler.ts): add support for generation parameter in ask function
    feat(useMessageHandler.ts): add support for isEdited parameter in ask function
    feat(useMessageHandler.ts): add support for continueGeneration function
    fix(createPayload.ts): replace endpoint URL when isEdited parameter is true

    * chore(client): set skipLibCheck to true in tsconfig.json

    * fix(useMessageHandler.ts): remove unused clientId variable
    fix(schemas.ts): make clientId field in tMessageSchema nullable and optional

    * wip: edit route for continue generation

    * refactor(api): move handlers to root of routes dir

    * fix(useMessageHandler.ts): initialize currentMessages to an empty array if messages is null
    fix(useMessageHandler.ts): update initialResponse text to use responseText variable
    fix(useMessageHandler.ts): update setMessages logic for isRegenerate case
    fix(MessageHandler.jsx): update setMessages logic for cancelHandler, createdHandler, and finalHandler

    * fix(schemas.ts): make createdAt and updatedAt fields optional and set default values using new Date().toISOString()
    fix(schemas.ts): change type annotation of TMessage from infer to input

    * refactor(useMessageHandler.ts): rename AskProps type to TAskProps
    refactor(useMessageHandler.ts): remove generation property from ask function arguments
    refactor(useMessageHandler.ts): use nullish coalescing operator (??) instead of logical OR (||)
    refactor(useMessageHandler.ts): pass the responseMessageId to message prop of submission

    * fix(BaseClient.js): use nullish coalescing operator (??) instead of logical OR (||) for default values

    * fix(BaseClient.js): fix responseMessageId assignment in handleStartMethods method
    feat(BaseClient.js): add support for isEdited flag in sendMessage method
    feat(BaseClient.js): add generation to responseMessage text in sendMessage method

    * fix(openAI.js): remove unused imports and commented out code
    feat(openAI.js): add support for generation parameter in request body
    fix(openAI.js): remove console.log statement
    fix(openAI.js): remove unused variables and parameters
    fix(openAI.js): update response text in case of error
    fix(openAI.js): handle error and abort message in case of error
    fix(handlers.js): add generation parameter to createOnProgress function
    fix(useMessageHandler.ts): update responseText variable to use generation parameter

    * refactor(api/middleware): move inside server dir

    * refactor: add endpoint specific, modular functions to build options and initialize clients, create server/utils, move middleware, separate utils into api general utils and server specific utils

    * fix(abortMiddleware.js): import getConvo and getConvoTitle functions from models
    feat(abortMiddleware.js): add abortAsk function to abortController to handle aborting of requests
    fix(openAI.js): import buildOptions and initializeClient functions from endpoints/openAI
    refactor(openAI.js): use getAbortData function to get data for abortAsk function

    * refactor: move endpoint specific logic to an endpoints dir

    * refactor(PluginService.js): fix import path for encrypt and decrypt functions in PluginService.js

    * feat(openAI): add new endpoint for adding a title to a conversation

    - Added a new file `addTitle.js` in the `api/server/routes/endpoints/openAI` directory.
    - The `addTitle.js` file exports a function `addTitle` that takes in request parameters and performs the following actions:
      - If the `parentMessageId` is `'00000000-0000-0000-0000-000000000000'` and `newConvo` is true, it proceeds with the following steps:
        - Calls the `titleConvo` function from the `titleConvo` module, passing in the necessary parameters.
        - Calls the `saveConvo` function from the `saveConvo` module, passing in the user ID and conversation details.
    - Updated the `index.js` file in the `api/server/routes/endpoints/openAI` directory to export the `addTitle` function.
    - This change adds

    * fix(abortMiddleware.js): remove console.log statement
    refactor(gptPlugins.js): update imports and function parameters
    feat(gptPlugins.js): add support for abortController and getAbortData
    refactor(openAI.js): update imports and function parameters
    feat(openAI.js): add support for abortController and getAbortData

    fix(openAI.js): refactor code to use modularized functions and middleware
    fix(buildOptions.js): refactor code to use destructuring and update variable names

    * refactor(askChatGPTBrowser.js, bingAI.js, google.js): remove duplicate code for setting response headers
    feat(askChatGPTBrowser.js, bingAI.js, google.js): add setHeaders middleware to set response headers

    * feat(middleware): validateEndpoint, refactor buildOption to only be concerned of endpointOption

    * fix(abortMiddleware.js): add 'finish_reason' property with value 'incomplete' to responseMessage object
    fix(abortMessage.js): remove console.log statement for aborted message
    fix(handlers.js): modify tokens assignment to handle empty generation string and trailing space

    * fix(BaseClient.js): import addSpaceIfNeeded function from server/utils
    fix(BaseClient.js): add space before generation in text property
    fix(index.js): remove getCitations and citeText exports
    feat(buildEndpointOption.js): add buildEndpointOption middleware
    fix(index.js): import buildEndpointOption middleware
    fix(anthropic.js): remove buildOptions function and use endpointOption from req.body
    fix(gptPlugins.js): remove buildOptions function and use endpointOption from req.body
    fix(openAI.js): remove buildOptions function and use endpointOption from req.body

    feat(utils): add citations.js and handleText.js modules
    fix(utils): fix import statements in index.js module

    * refactor(gptPlugins.js): use getResponseSender function from librechat-data-provider

    * feat(gptPlugins): complete 'continue generating'

    * wip: anthropic continue regen

    * feat(middleware): add validateRegistration middleware

    A new middleware function called `validateRegistration` has been added to the list of exported middleware functions in `index.js`. This middleware is responsible for validating registration data before allowing the registration process to proceed.

    * feat(Anthropic): complete continue regen

    * chore: add librechat-data-provider to api/package.json

    * fix(ci): backend-review will mock meilisearch, also installs data-provider as now needed

    * chore(ci): remove unneeded SEARCH env var

    * style(GenerationButtons): make text shorter for sake of space economy, even though this diverges from chat.openai.com

    * style(GenerationButtons/ScrollToBottom): adjust visibility/position based on screen size

    * chore(client): 'Editting' typo

    * feat(GenerationButtons.tsx): add support for endpoint prop in GenerationButtons component
    feat(OptionsBar.tsx): pass endpoint prop to GenerationButtons component
    feat(useGenerations.ts): create useGenerations hook to handle generation logic
    fix(schemas.ts): add searchResult field to tMessageSchema

    * refactor(HoverButtons): convert to TSX and utilize new useGenerations hook

    * fix(abortMiddleware): handle error with res headers set, or abortController not found, to ensure proper API error is sent to the client, chore(BaseClient): remove console log for onStart message meant for debugging

    * refactor(api): remove librechat-data-provider dep for now as it complicates deployed docker build stage, re-use code in CJS, located in server/endpoints/schemas

    * chore: remove console.logs from test files

    * ci: add backend tests for AnthropicClient, focusing on new buildMessages logic

    * refactor(FakeClient): use actual BaseClient sendMessage method for testing

    * test(BaseClient.test.js): add test for loading chat history
    test(BaseClient.test.js): add test for sendMessage logic with isEdited flag

    * fix(buildEndpointOption.js): add support for azureOpenAI in buildFunction object
    wip(endpoints.js): fetch Azure models from Azure OpenAI API if opts.azure is true

    * fix(Button.tsx): add data-testid attribute to button component
    fix(SelectDropDown.tsx): add data-testid attribute to Listbox.Button component
    fix(messages.spec.ts): add waitForServerStream function to consolidate logic for awaiting the server response
    feat(messages.spec.ts): add test for stopping and continuing message and improve browser/page context order and closing

    * refactor(onProgress): speed up time to save initial message for editable routes

    * chore: disable AI message editing (for now), was accidentally allowed

    * refactor: ensure continue is only supported for latest message style: improve styling in dark mode and across all hover buttons/icons, including making edit icon for AI invisible (for now)

    * fix: add test id to generation buttons so they never resolve to 2+ items

    * chore(package.json): add 'packages/' to the list of ignored directories
    chore(data-provider/package.json): bump version to 0.1.5

commit ae5b7d3d53
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Tue Aug 15 18:27:54 2023 -0400

    fix(PluginsClient.js): fix ChatOpenAI Azure Config Issue (#812)

    * fix(PluginsClient.js): fix issue with creating LLM when using Azure

    * chore(PluginsClient.js): omit azure logging

    * refactor(PluginsClient.js): simplify assignment of azure variable

    The code was simplified by directly assigning the value of `this.azure` to the `azure` variable using object destructuring. This makes the code cleaner and more concise.

commit b85f3bf91e
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Tue Aug 15 18:42:24 2023 +0200

    update from lang to localize (#810)

commit 80aab73bf6
Author: Danny Avila <messagedaniel@protonmail.com>
Date:   Mon Aug 14 19:19:04 2023 -0400

    chore: rebuilt package-lock file

commit bbe4931a97
Author: Danny Avila <messagedaniel@protonmail.com>
Date:   Mon Aug 14 19:13:24 2023 -0400

    refactor(ScreenshotContext): use html-to-image for lighter bundle, faster processing

commit 74802dd720
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Mon Aug 14 17:51:03 2023 +0200

    chore: Translation Fixes, Lint Error Corrections, and Additional Translations (#788)

    * fix translation and small lint error

    * changed from localize to useLocalize hook

    * changed to useLocalize

commit b64cc71d88
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Mon Aug 14 10:23:00 2023 -0400

    chore(docker-compose.yml): comment out meilisearch ports in docker-compose.yml (#807)

commit 89f260bc78
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Mon Aug 14 10:12:00 2023 -0400

    fix(CodeBlock.tsx): fix copy-to-clipboard functionality. The code has been updated to use the copy function from the copy-to-clipboard library instead of the (#806)

    avigator.clipboard.writeText method. This should fix the issue with browser incompatibility with navigator SDK and allow users to copy code from the CodeBlock component successfully.

commit d00c7354cd
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Mon Aug 14 09:45:44 2023 -0400

    fix: Corrected Registration Validation, Case-Insensitive Variable Handling, Playwright workflow (#805)

    * feat(auth.js): add validation for registration endpoint using validateRegistration middleware
    feat(validateRegistration.js): add middleware to validate registration based on ALLOW_REGISTRATION environment variable

    * fix(config.js): fix registrationEnabled and socialLoginEnabled variables to handle case-insensitive environment variable values

    * refactor(validateRegistration.js): remove console.log statement

    * chore(playwright.yml): skip browser download during yarn install
    chore(playwright.yml): place Playwright binaries to node_modules/@playwright/test
    chore(playwright.yml): install Playwright dependencies using npx playwright install-deps
    chore(playwright.yml): install Playwright chromium browser using npx playwright install chromium
    chore(playwright.yml): install @playwright/test@latest using npm install -D @playwright/test@latest
    chore(playwright.yml): run Playwright tests using npm run e2e:ci

    * chore(playwright.yml): change npm install order and update comment

    The order of the npm install commands in the "Install Playwright Browsers" step has been changed to first install @playwright/test@latest and then install chromium. Additionally, the comment explaining the PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD variable has been updated to mention npm install instead of yarn install.

    * chore(playwright.yml): remove commented out code for caching and add separate steps for installing Playwright dependencies and browsers

commit 1aa4b34dc6
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Fri Aug 11 19:02:52 2023 +0200

    added the dot (.) username rules (#787)

* Create VolumeMuteIcon.tsx

* Create VolumeIcon.tsx

* Update index.ts

* Update SubmitButton.jsx

* Update SubmitButton.jsx

* Update TextChat.jsx

* Update TextChat.jsx

* Update SpeechRecognition.tsx

* Update SpeechRecognition.tsx

* Update TextChat.jsx

* Update SpeechRecognition.tsx

* Update TextChat.jsx

* Update HoverButtons.tsx

* Update useServerStream.ts

* Update useServerStream.ts

* Update HoverButtons.tsx

* Update useServerStream.ts

* Update useServerStream.ts

* Update HoverButtons.tsx

* Update VolumeIcon.tsx

* Update VolumeMuteIcon.tsx

* Update HoverButtons.tsx

* Update SpeechSynthesis.tsx

* Update HoverButtons.tsx

* Update HoverButtons.tsx

* Update SpeechSynthesis.tsx

* Update SpeechSynthesis.tsx

* Update HoverButtons.tsx

* Update SpeechSynthesis.tsx

* Update package.json

* Update SpeechRecognition.tsx

* Update SpeechRecognition.tsx

* Update SpeechRecognition.tsx

* Update SpeechRecognition.tsx

* Squashed commit of the following:

commit 1019529634
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 23:12:14 2023 -0500

    Update SpeechRecognition.tsx

commit 67f111ccd0
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 23:08:48 2023 -0500

    Update SpeechRecognition.tsx

commit 0b35dbe196
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 23:04:50 2023 -0500

    Update SpeechRecognition.tsx

commit 6686126dc0
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 22:49:08 2023 -0500

    Update SpeechRecognition.tsx

commit 5b80ddfba7
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 22:45:02 2023 -0500

    Update package.json

commit 39e84efa81
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 22:35:48 2023 -0500

    Update SpeechSynthesis.tsx

commit 4c6d067cb9
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 22:24:29 2023 -0500

    Update HoverButtons.tsx

commit c5ce576fb8
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 22:13:20 2023 -0500

    Update SpeechSynthesis.tsx

commit d95fa19539
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 22:11:38 2023 -0500

    Update SpeechSynthesis.tsx

commit c794f07678
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 22:03:34 2023 -0500

    Update HoverButtons.tsx

commit 7ae0e7e97c
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 21:59:45 2023 -0500

    Update HoverButtons.tsx

commit e9882dedad
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 21:58:07 2023 -0500

    Update SpeechSynthesis.tsx

commit 95cf300782
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 21:44:49 2023 -0500

    Update HoverButtons.tsx

commit 37c828d7fb
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 21:30:34 2023 -0500

    Update VolumeMuteIcon.tsx

commit 6133531737
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 21:29:54 2023 -0500

    Update VolumeIcon.tsx

commit 4b4afcdd37
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 21:20:14 2023 -0500

    Update HoverButtons.tsx

commit 609d1dfefb
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 20:49:52 2023 -0500

    Update useServerStream.ts

commit 875ce4b77e
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 20:48:26 2023 -0500

    Update useServerStream.ts

commit 8ed04e496b
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 20:37:59 2023 -0500

    Update HoverButtons.tsx

commit 4b30c132df
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 20:14:01 2023 -0500

    Update useServerStream.ts

commit c041c329cf
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 20:07:14 2023 -0500

    Update useServerStream.ts

commit 3e36c16817
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 19:36:21 2023 -0500

    Update HoverButtons.tsx

commit c7eea96759
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 19:28:03 2023 -0500

    Update TextChat.jsx

commit 5542f8e85d
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 19:21:50 2023 -0500

    Update SpeechRecognition.tsx

commit 9a27e56f8b
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 19:16:01 2023 -0500

    Update TextChat.jsx

commit 7f101bd122
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 19:09:51 2023 -0500

    Update SpeechRecognition.tsx

commit d405454bf5
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 19:03:34 2023 -0500

    Update SpeechRecognition.tsx

commit 6033eb3ed1
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 19:01:06 2023 -0500

    Update TextChat.jsx

commit 9a3e67fcd2
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 18:53:19 2023 -0500

    Update TextChat.jsx

commit 6583877cb3
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 17:53:18 2023 -0500

    Update SubmitButton.jsx

commit 8d5114bfae
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 17:39:20 2023 -0500

    Update SubmitButton.jsx

commit 29a5b55883
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 17:28:03 2023 -0500

    Update index.ts

commit b03001d01d
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 17:25:43 2023 -0500

    Create VolumeIcon.tsx

commit 863af2c959
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 17:21:43 2023 -0500

    Create VolumeMuteIcon.tsx

commit ad3c78f867
Merge: ed4b25b2 28230d93
Author: bsu3338 <bsu3338@users.noreply.github.com>
Date:   Sun Sep 3 16:49:56 2023 -0500

    Merge branch 'danny-avila:main' into Speech-September

commit ed4b25b2c1
Author: bsu3338 <bsu3338@yahoo.com>
Date:   Sun Sep 3 16:49:03 2023 -0500

    Squashed commit of the following:

    commit 28230d9305
    Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
    Date:   Sun Sep 3 02:44:26 2023 +0200

        feat: delete button confirm (#875)

        * base for confirm delete

        * more like OpenAI

    commit 2b54e3f9fe
    Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
    Date:   Fri Sep 1 14:20:51 2023 -0400

        update: install script (#858)

    commit 1cd0fd9d5a
    Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
    Date:   Fri Sep 1 08:12:35 2023 -0400

        doc: Hugging Face Deployment (#867)

        * docs: update ToC

        * docs: update ToC

        * update huggingface.md

        * update render.md

        * update huggingface.md

        * update mongodb.md

        * update huggingface.md

        * update README.md

    commit aeeb3d3050
    Author: Mu Yuan <yuanmu.email@gmail.com>
    Date:   Thu Aug 31 07:21:27 2023 +0800

        Update Zh.tsx (#862)

        * Update Zh.tsx

        Changed the translation of several words to make it more relevant to Chinese usage habits.

        * Update Zh.tsx

        Changed the translation of several words to make it more relevant to Chinese usage habits

    commit 80e2e2675b
    Author: Raí <140329135+itzraiss@users.noreply.github.com>
    Date:   Mon Aug 28 18:05:46 2023 -0300

        Translation of 'com_ui_pay_per_call:' to Spanish and Portuguese that were missing. (#857)

        * Update Br.tsx

        * Update Es.tsx

        * Update Br.tsx

        * Update Es.tsx

    commit 3574d0b823
    Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
    Date:   Mon Aug 28 14:49:26 2023 -0400

        docs: make_your_own.md formatting fix for mkdocs (#855)

    commit d672ac690d
    Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
    Date:   Mon Aug 28 14:24:10 2023 -0400

        Release v0.5.8 (#854)

        * chore: add 'api' image to tag release workflow

        * docs: update DO deployment docs to include instruction about latest stable release, as well as security best practices

        * Release v0.5.8

        * docs: Update digitalocean.md with firewall section images

        * docs: make_your_own.md formatting fix for mkdocs

    commit d3e7627046
    Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
    Date:   Mon Aug 28 12:03:08 2023 -0400

        refactor(plugins): Improve OpenAPI handling, Show Multiple Plugins, & Other Improvements (#845)

        * feat(PluginsClient.js): add conversationId to options object in the constructor
        feat(PluginsClient.js): add support for Code Interpreter plugin
        feat(PluginsClient.js): add support for Code Interpreter plugin in the availableTools manifest
        feat(CodeInterpreter.js): add CodeInterpreterTools module
        feat(CodeInterpreter.js): add RunCommand class
        feat(CodeInterpreter.js): add ReadFile class
        feat(CodeInterpreter.js): add WriteFile class
        feat(handleTools.js): add support for loading Code Interpreter plugin

        * chore(api): update langchain dependency to version 0.0.123

        * fix(CodeInterpreter.js): add support for extracting environment from code
        fix(WriteFile.js): add support for extracting environment from data
        fix(extractionChain.js): add utility functions for creating extraction chain from Zod schema
        fix(handleTools.js): refactor getOpenAIKey function to handle user-provided API key
        fix(handleTools.js): pass model and openAIApiKey to CodeInterpreter constructor

        * fix(tools): rename CodeInterpreterTools to E2BTools
        fix(tools): rename code_interpreter pluginKey to e2b_code_interpreter

        * chore(PluginsClient.js): comment out unused import and function findMessageContent
        feat(PluginsClient.js): add support for CodeSherpa plugin
        feat(PluginsClient.js): add CodeSherpaTools to available tools
        feat(PluginsClient.js): update manifest.json to include CodeSherpa plugin
        feat(CodeSherpaTools.js): create RunCode and RunCommand classes for CodeSherpa plugin

        feat(E2BTools.js): Add E2BTools module for extracting environment from code and running commands, reading and writing files
        fix(codesherpa.js): Remove codesherpa module as it is no longer needed

        feat(handleTools.js): add support for CodeSherpaTools in loadTools function
        feat(loadToolSuite.js): create loadToolSuite utility function to load a suite of tools

        * feat(PluginsClient.js): add support for CodeSherpa v2 plugin
        feat(PluginsClient.js): add CodeSherpa v1 plugin to available tools
        feat(PluginsClient.js): add CodeSherpa v2 plugin to available tools
        feat(PluginsClient.js): update manifest.json for CodeSherpa v1 plugin
        feat(PluginsClient.js): update manifest.json for CodeSherpa v2 plugin
        feat(CodeSherpa.js): implement CodeSherpa plugin for interactive code and shell command execution
        feat(CodeSherpaTools.js): implement RunCode and RunCommand plugins for CodeSherpa v1
        feat(CodeSherpaTools.js): update RunCode and RunCommand plugins for CodeSherpa v2

        fix(handleTools.js): add CodeSherpa import statement
        fix(handleTools.js): change pluginKey from 'codesherpa' to 'codesherpa_tools'
        fix(handleTools.js): remove model and openAIApiKey from options object in e2b_code_interpreter tool
        fix(handleTools.js): remove openAIApiKey from options object in codesherpa_tools tool
        fix(loadToolSuite.js): remove model and openAIApiKey parameters from loadToolSuite function

        * feat(initializeFunctionsAgent.js): add prefix to agentArgs in initializeFunctionsAgent function

        The prefix is added to the agentArgs in the initializeFunctionsAgent function. This prefix is used to provide instructions to the agent when it receives any instructions from a webpage, plugin, or other tool. The agent will notify the user immediately and ask them if they wish to carry out or ignore the instructions.

        * feat(PluginsClient.js): add ChatTool to the list of tools if it meets the conditions
        feat(tools/index.js): import and export ChatTool
        feat(ChatTool.js): create ChatTool class with necessary properties and methods

        * fix(initializeFunctionsAgent.js): update PREFIX message to include sharing all output from the tool
        fix(E2BTools.js): update descriptions for RunCommand, ReadFile, and WriteFile plugins to provide more clarity and context

        * chore: rebuild package-lock after rebase

        * chore: remove deleted file from rebase

        * wip: refactor plugin message handling to mirror chat.openai.com, handle incoming stream for plugin use

        * wip: new plugin handling

        * wip: show multiple plugins handling

        * feat(plugins): save new plugins array

        * chore: bump langchain

        * feat(experimental): support streaming in between plugins

        * refactor(PluginsClient): factor out helper methods to avoid bloating the class, refactor(gptPlugins): use agent action for mapping the name of action

        * fix(handleTools): fix tests by adding condition to return original toolFunctions map

        * refactor(MessageContent): Allow the last index to be last in case it has text (may change with streaming)

        * feat(Plugins): add handleParsingErrors, useful when LLM does not invoke function params

        * chore: edit out experimental codesherpa integration

        * refactor(OpenAPIPlugin): rework tool to be 'function-first', as the spec functions are explicitly passed to agent model

        * refactor(initializeFunctionsAgent): improve error handling and system message

        * refactor(CodeSherpa, Wolfram): optimize token usage by delegating bulk of instructions to system message

        * style(Plugins): match official style with input/outputs

        * chore: remove unnecessary console logs used for testing

        * fix(abortMiddleware): render markdown when message is aborted

        * feat(plugins): add BrowserOp

        * refactor(OpenAPIPlugin): improve prompt handling

        * fix(useGenerations): hide edit button when message is submitting/streaming

        * refactor(loadSpecs): optimize OpenAPI spec loading by only loading requested specs instead of all of them

        * fix(loadSpecs): will retain original behavior when no tools are passed to the function

        * fix(MessageContent): ensure cursor only shows up for last message and last display index
        fix(Message): show legacy plugin and pass isLast to Content

        * chore: remove console.logs

        * docs: update docs based on breaking changes and new features
        refactor(structured/SD): use description_for_model for detailed prompting

        * docs(azure): make plugins section more clear

        * refactor(structured/SD): change default payload to SD-WebUI to prefer realism and config for SDXL

        * refactor(structured/SD): further improve system message prompt

        * docs: update breaking changes after rebase

        * refactor(MessageContent): factor out EditMessage, types, Container to separate files, rename Content -> Markdown

        * fix(CodeInterpreter): linting errors

        * chore: reduce browser console logs from message streams

        * chore: re-enable debug logs for plugins/langchain to help with user troubleshooting

        * chore(manifest.json): add [Experimental] tag to CodeInterpreter plugins, which are not intended as the end-all be-all implementation of this feature for Librechat

    commit 66b8580487
    Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
    Date:   Mon Aug 28 09:18:25 2023 -0400

        docs: third-party tools (#848)

        * docs: third-party tools

        * docs: third-party tools

        * Update third-party.md

        * Update third-party.md

        ---------

        Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>

    commit 9791a78161
    Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
    Date:   Mon Aug 28 15:14:05 2023 +0200

        adjust the animation (#843)

    commit 3797ec6082
    Author: Ronith <87087292+ronith256@users.noreply.github.com>
    Date:   Mon Aug 28 18:43:50 2023 +0530

        feat: Add Code Interpreter Plugin (#837)

        * feat: Add Code Interpreter Plugin

        Adds a Simple Code Interpreter Plugin.
        ## Features:
        - Runs code using local Python Environment

        ## Issues
        - Code execution is not sandboxed.

        * Add Docker Sandbox for Python Server

    commit e2397076a2
    Author: Alex Zhang <ztc2011@gmail.com>
    Date:   Mon Aug 28 00:55:34 2023 +0800

        🌐: Chinese Translation (#846)

    commit 50c15c704f
    Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
    Date:   Sat Aug 26 19:36:59 2023 -0400

        Language translation: Polish (#840)

        * Language translation: Polish

        * Language translation: Polish

        * Revert changes in language-contributions.md

    commit 29d3640546
    Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
    Date:   Sat Aug 26 19:36:25 2023 -0400

        docs: updates (#841)

    commit 39c626aa8e
    Author: Danny Avila <messagedaniel@protonmail.com>
    Date:   Fri Aug 25 09:29:19 2023 -0400

        fix: isEdited edge case where latest Message is not saved due to aborting too quickly

    commit ae5c06f381
    Author: Danny Avila <messagedaniel@protonmail.com>
    Date:   Fri Aug 25 09:13:50 2023 -0400

        fix(chatGPTBrowser): render markdown formatting by setting isCreatedByUser, fix(useMessageHandler): avoid double appearance of cursor by setting latest message at initial response creation time

    commit 9ef1686e18
    Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
    Date:   Thu Aug 24 20:24:47 2023 -0400

        Update mkdocs.yml

    commit 5bbe411569
    Author: Flynn <dev@flynnbuckingham.com>
    Date:   Thu Aug 24 20:20:37 2023 -0400

        Add podman installation instructions. Update dockerfile to stub env (#819)

        * Added podman container installation docs. Updated dockerfile to stub env file if not present in source

        * Fix typos

    commit 887fec99ca
    Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
    Date:   Fri Aug 25 02:11:27 2023 +0200

        🌐: Russian Translation (#830)

    commit 007d51ede1
    Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
    Date:   Fri Aug 25 02:10:48 2023 +0200

        feat: facebook login (#820)

        * Facebook strategy

        * Update user_auth_system.md

        * Update user_auth_system.md

    commit a569020312
    Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
    Date:   Thu Aug 24 21:59:11 2023 +0200

        Fix Meilisearch error and refactor of the server index.js (#832)

        * fix meilisearch error at startup

        * limit the nesting

        * disable useless console log

        * fix(indexSync.js): removed redundant searchEnabled

        * refactor(index.js): moved configureSocialLogins to a new file

        * refactor(socialLogins.js): removed unnecessary conditional

    commit 37347d4683
    Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
    Date:   Wed Aug 23 16:14:17 2023 -0400

        fix(registration): Make Username optional (#831)

        * fix(User.js): update validation schema for username field, allow empty string as a valid value
        fix(validators.js): update validation schema for username field, allow empty string as a valid value
        fix(Registration.tsx, validators.js): update validation rules for name and username fields, change minimum length to 2 and maximum length to 80, assure they match and allow empty string as a valid value
        fix(Eng.tsx): update localization string for com_auth_username, indicate that it is optional

        * fix(User.js): update regex pattern for username validation to allow special characters @#$%&*()
        fix(validators.js): update regex pattern for username validation to allow special characters @#$%&*()

        * fix(Registration.spec.tsx): fix validation error message for username length requirement

    commit d38e463d34
    Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
    Date:   Wed Aug 23 13:44:40 2023 -0400

        fix(bingAI): markdown and error formatting for final stream response (#829)

        * fix(bingAI): markdown formatting for final stream response due to new strict payload validation on the frontend

        * fix: add missing prop to bing Error response

    commit 7dc27b10f1
    Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
    Date:   Tue Aug 22 18:44:59 2023 -0400

        feat: Edit AI Messages, Edit Messages in Place (#825)

        * refactor: replace lodash import with specific function import

        fix(api): esm imports to cjs

        * refactor(Messages.tsx): convert to TS, out-source scrollToDiv logic to a custom hook
        fix(ScreenshotContext.tsx): change Ref to RefObject in ScreenshotContextType
        feat(useScrollToRef.ts): add useScrollToRef hook for scrolling to a ref with throttle
        fix(Chat.tsx): update import path for Messages component
        fix(Search.tsx): update import path for Messages component

        * chore(types.ts): add TAskProps and TOptions types
        refactor(useMessageHandler.ts): use TAskFunction type for ask function signature

        * refactor(Message/Content): convert to TS, move Plugin component to Content dir

        * feat(MessageContent.tsx): add MessageContent component for displaying and editing message content
        feat(index.ts): export MessageContent component from Messages/Content directory

        * wip(Message.jsx): conversion and use of new component in progress

        * refactor: convert Message.jsx to TS and fix typing/imports based on changes

        * refactor: add typed props and refactor MultiMessage to TS, fix typing issues resulting from the conversion

        * edit message in progress

        * feat: complete edit AI message logic, refactor continue logic

        * feat(middleware): add validateMessageReq middleware
        feat(routes): add validation for message requests using validateMessageReq middleware
        feat(routes): add create, read, update, and delete routes for messages

        * feat: complete frontend logic for editing messages in place
        feat(messages.js): update route for updating a specific message
        - Change the route for updating a message to include the messageId in the URL
        - Update the request handler to use the messageId from the request parameters and the text from the request body
        - Call the updateMessage function with the updated parameters

        feat(MessageContent.tsx): add functionality to update a message
        - Import the useUpdateMessageMutation hook from the data provider
        - Destructure the conversationId, parentMessageId, and messageId from the message object
        - Create a mutation function using the useUpdateMessageMutation hook
        - Implement the updateMessage function to call the mutation function with the updated message parameters
        - Update the messages state to reflect the updated message text

        feat(api-endpoints.ts): update messages endpoint to include messageId
        - Update the messages endpoint to include the messageId as an optional parameter

        feat(data-service.ts): add updateMessage function
        - Implement the updateMessage function to make a PUT request to

        * fix(messages.js): make updateMessage function asynchronous and await its execution

        * style(EditIcon): make icon active for AI message

        * feat(gptPlugins/anthropic): add edit support

        * fix(validateMessageReq.js): handle case when conversationId is 'new' and return empty array
        feat(Message.tsx): pass message prop to SiblingSwitch component
        refactor(SiblingSwitch.tsx): convert to TS

        * fix(useMessageHandler.ts): remove message from currentMessages if isContinued is true
        feat(useMessageHandler.ts): add support for submission messages in setMessages
        fix(useServerStream.ts): remove unnecessary conditional in setMessages
        fix(useServerStream.ts): remove isContinued variable from submission

        * fix(continue): switch to continued message generation when continuing an earlier branch in conversation

        * fix(abortMiddleware.js): fix condition to check partialText length
        chore(abortMiddleware.js): add error logging when abortMessage fails

        * refactor(MessageHeader.tsx): convert to TS
        fix(Plugin.tsx): add default value for className prop in Plugin component

        * refactor(MultiMessage.tsx): remove commented out code
        docs(MultiMessage.tsx): update comment to clarify when siblingIdx is reset

        * fix(GenerationButtons): optimistic state for continue button

        * fix(MessageContent.tsx): add data-testid attribute to message text editor
        fix(messages.spec.ts): update waitForServerStream function to include edit endpoint check
        feat(messages.spec.ts): add test case for editing messages

        * fix(HoverButtons & Message & useGenerations): Refactor edit functionality and related conditions

        - Update enterEdit function signature and prop
        - Create and utilize hideEditButton variable
        - Enhance conditions for edit button visibility and active state
        - Update button event handlers
        - Introduce isEditableEndpoint in useGenerations and refine continueSupported condition.

        * fix(useGenerations.ts): fix condition for hideEditButton to include error and searchResult
        chore(data-provider): bump version to 0.1.6
        fix(types.ts): add status property to TError type

        * chore: bump @dqbd/tiktoken to 1.0.7

        * fix(abortMiddleware.js): add required isCreatedByUser property to the error response object

        * refactor(Message.tsx): remove unnecessary props from SiblingSwitch component, as setLatestMessage is firing on every switch already
        refactor(SiblingSwitch.tsx): remove unused imports and code

        * chore(BaseClient.js): move console.debug statements back inside if block

    commit db77163f5d
    Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
    Date:   Tue Aug 22 14:15:14 2023 +0200

        docs: update chimeragpt (#826)

        * Update free_ai_apis.md

        * Update free_ai_apis.md

    commit 4a4e803df3
    Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
    Date:   Mon Aug 21 20:15:18 2023 +0200

        style(Dialog): Improved Close Button ("X") position (#824)

    commit 909b00c752
    Author: Daniel Avila <messagedaniel@protonmail.com>
    Date:   Sun Aug 20 21:04:36 2023 -0400

        fix(HoverButtons): light/dark styling to match official site

    commit 61dcb4d307
    Author: Naosuke Yokoe <ankerasoy@gmail.com>
    Date:   Sat Aug 19 20:11:31 2023 +0900

        feat: Azure Cognitive Search Plugin (#815)

        * feat(AzureCognitiveSearchPlugin)

        * feat(tools/AzureCognitiveSearch.js): Add a new plugin (not structured
          version)
        * feat(tools/structured/AzureCognitiveSearch.js): Add a new plugin (structured version)
        * feat(tools/manifest.json, tools/index.js, tools/util/handleTools.js):
          Add configurations for the plugin
        * feat(api/package.json, package-lock.json): Installed a new package for the
          plugin (@azure/search-documents)
        * feat(.env.example): Add new environment variables for the plugin

        Here is the link to the corresponding discussion page:
        https://github.com/danny-avila/LibreChat/discussions/567

        * docs(AzureCognitiveSearchPlugin)

        * docs(features/plugins/azure_cognitive_search.md): Add a new document
          for the plugin

        * (fix:.env.example)

        * reverted extra whitespaces removed by the editor

        * docs(mkdocs.yml)

        * Add the Azure Cognitive Search Plugin's documentation item to
        mkdocs.yml.

    commit 3c7f67fa76
    Author: Danny Avila <messagedaniel@protonmail.com>
    Date:   Fri Aug 18 12:40:33 2023 -0400

        fix(abortMiddleware): handle early abort error where userMessage.conversationId is undefined. In this case, the userId will be used as the abortKey

    commit c74c68a135
    Author: Danny Avila <messagedaniel@protonmail.com>
    Date:   Fri Aug 18 12:10:30 2023 -0400

        refactor(MessageHandler -> useServerStream): convert all relating files to TS and correct typings based on this change: properly refactor MessageHandler to a custom hook, where it's passed a submission object to instantiate the stream. This is the bare minimum groundwork for potentially having multiple streams running, which would be a big project to modularize a lot of the global state into maps/multiple streams, particular useful for having multiple views in place

    commit 8b4d3c2c21
    Author: Danny Avila <messagedaniel@protonmail.com>
    Date:   Fri Aug 18 12:04:29 2023 -0400

        refactor(routes): convert to TS

    commit d612cfcb45
    Author: Danny Avila <messagedaniel@protonmail.com>
    Date:   Fri Aug 18 12:02:39 2023 -0400

        chore(Auth): reorder exports in Auth component
        fix(PluginAuthForm): handle case when pluginKey is null or undefined
        fix(PluginStoreDialog): handle case when getAvailablePluginFromKey is null or undefined
        fix(AuthContext): make authConfig optional in AuthContextProvider
        feat(hooks): add useServerStream hook
        fix(conversation): setSubmission to null instead of empty object
        fix(preset): specify type for presets atom
        fix(search): specify type for isSearchEnabled atom
        fix(submission): specify type for submission atom

    commit c40b95f424
    Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
    Date:   Fri Aug 18 16:11:00 2023 +0200

        feat: Disable Registration with social login (#813)

        * Google, Github and Discord

        * update .env.example with ALLOW_SOCIAL_REGISTRATION

        * fix some conflict

        * refactor strategy

        * Update user_auth_system.md

        * Update user_auth_system.md

    commit 46ed5aaccd
    Author: Patrick <psarnowski@gmail.com>
    Date:   Fri Aug 18 09:38:24 2023 -0400

        Show the response scores from Bing. (#814)

    commit 1dacfa49f0
    Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
    Date:   Thu Aug 17 20:32:31 2023 +0200

        update profile picture (#792)

    commit afd43afb60
    Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
    Date:   Thu Aug 17 12:50:05 2023 -0400

        feat(GPT/Anthropic): Continue Regenerating & Generation Buttons (#808)

        * feat(useMessageHandler.js/ts): Refactor and add features to handle user messages, support multiple endpoints/models, generate placeholder responses, regeneration, and stopGeneration function

        fix(conversation.ts, buildTree.ts): Import TMessage type, handle null parentMessageId

        feat(schemas.ts): Update and add schemas for various AI services, add default values, optional fields, and endpoint-to-schema mapping, create parseConvo function

        chore(useMessageHandler.js, schemas.ts): Remove unused imports, variables, and chatGPT enum

        * wip: add generation buttons

        * refactor(cleanupPreset.ts): simplify cleanupPreset function
        refactor(getDefaultConversation.js): remove unused code and simplify getDefaultConversation function

        feat(utils): add getDefaultConversation function

        This commit adds a new utility function called `getDefaultConversation` to the `client/src/utils/getDefaultConversation.ts` file. This function is responsible for generating a default conversation object based on the provided parameters.

        The `getDefaultConversation` function takes in an object with the following properties:
        - `conversation`: The conversation object to be used as a base.
        - `endpointsConfig`: The configuration object containing information about the available endpoints.
        - `preset`: An optional preset object that can be used to override the default behavior.

        The function first tries to determine the target endpoint based on the preset object. If a valid endpoint is found, it is used as the target endpoint. If not, the function tries to retrieve the last conversation setup from the local storage and uses its endpoint if it is valid. If neither the preset nor the local storage contains a valid endpoint, the function falls back to a default endpoint.

        Once the target endpoint is determined,

        * fix(utils): remove console.error statement in buildDefaultConversation function
        fix(schemas): add default values for catch blocks in openAISchema, googleSchema, bingAISchema, anthropicSchema, chatGPTBrowserSchema, and gptPluginsSchema

        * fix: endpoint not changing on change of preset from other endpoint, wip: refactor

        * refactor: preset items to TSX

        * refactor: convert resetConvo to TS

        * refactor(getDefaultConversation.ts): move defaultEndpoints array to the top of the file for better readability
        refactor(getDefaultConversation.ts): extract getDefaultEndpoint function for better code organization and reusability

        * feat(svg): add ContinueIcon component
        feat(svg): add RegenerateIcon component
        feat(svg): add ContinueIcon and RegenerateIcon components to index.ts

        * feat(Button.tsx): add onClick and className props to Button component
        feat(GenerationButtons.tsx): add logic to display Regenerate or StopGenerating button based on isSubmitting and messages
        feat(Regenerate.tsx): create Regenerate component with RegenerateIcon and handleRegenerate function
        feat(StopGenerating.tsx): create StopGenerating component with StopGeneratingIcon and handleStopGenerating function

        * fix(TextChat.jsx): reorder imports and variables for better readability
        fix(TextChat.jsx): fix typo in condition for isNotAppendable variable
        fix(TextChat.jsx): remove unused handleStopGenerating function
        fix(ContinueIcon.tsx): remove unnecessary closing tags for polygon elements
        fix(useMessageHandler.ts): add missing type annotations for handleStopGenerating and handleRegenerate functions
        fix(useMessageHandler.ts): remove unused variables in return statement

        * fix(getDefaultConversation.ts): refactor code to use getLocalStorageItems function
        feat(getLocalStorageItems.ts): add utility function to retrieve items from local storage

        * fix(OpenAIClient.js): add support for streaming result in sendCompletion method
        feat(OpenAIClient.js): add finish_reason metadata to opts in sendCompletion method
        feat(Message.js): add finish_reason field to Message model
        feat(messageSchema.js): add finish_reason field to messageSchema
        feat(openAI.js): parse chatGptLabel and promptPrefix from req.body and pass rest of the modelOptions to endpointOption
        feat(openAI.js): add addMetadata function to store metadata in ask function
        feat(openAI.js): add metadata to response if available
        feat(schemas.ts): add finish_reason field to tMessageSchema

        * feat(types.ts): add TOnClick and TGenButtonProps types for button components
        feat(Continue.tsx): create Continue component for generating button
        feat(GenerationButtons.tsx): update GenerationButtons component to use Continue component
        feat(Regenerate.tsx): create Regenerate component for regenerating button
        feat(Stop.tsx): create Stop component for stop generating button

        * feat(MessageHandler.jsx): add MessageHandler component to handle messages and conversations
        fix(Root.jsx): fix import paths for Nav and MessageHandler components

        * feat(useMessageHandler.ts): add support for generation parameter in ask function
        feat(useMessageHandler.ts): add support for isEdited parameter in ask function
        feat(useMessageHandler.ts): add support for continueGeneration function
        fix(createPayload.ts): replace endpoint URL when isEdited parameter is true

        * chore(client): set skipLibCheck to true in tsconfig.json

        * fix(useMessageHandler.ts): remove unused clientId variable
        fix(schemas.ts): make clientId field in tMessageSchema nullable and optional

        * wip: edit route for continue generation

        * refactor(api): move handlers to root of routes dir

        * fix(useMessageHandler.ts): initialize currentMessages to an empty array if messages is null
        fix(useMessageHandler.ts): update initialResponse text to use responseText variable
        fix(useMessageHandler.ts): update setMessages logic for isRegenerate case
        fix(MessageHandler.jsx): update setMessages logic for cancelHandler, createdHandler, and finalHandler

        * fix(schemas.ts): make createdAt and updatedAt fields optional and set default values using new Date().toISOString()
        fix(schemas.ts): change type annotation of TMessage from infer to input

        * refactor(useMessageHandler.ts): rename AskProps type to TAskProps
        refactor(useMessageHandler.ts): remove generation property from ask function arguments
        refactor(useMessageHandler.ts): use nullish coalescing operator (??) instead of logical OR (||)
        refactor(useMessageHandler.ts): pass the responseMessageId to message prop of submission

        * fix(BaseClient.js): use nullish coalescing operator (??) instead of logical OR (||) for default values

        * fix(BaseClient.js): fix responseMessageId assignment in handleStartMethods method
        feat(BaseClient.js): add support for isEdited flag in sendMessage method
        feat(BaseClient.js): add generation to responseMessage text in sendMessage method

        * fix(openAI.js): remove unused imports and commented out code
        feat(openAI.js): add support for generation parameter in request body
        fix(openAI.js): remove console.log statement
        fix(openAI.js): remove unused variables and parameters
        fix(openAI.js): update response text in case of error
        fix(openAI.js): handle error and abort message in case of error
        fix(handlers.js): add generation parameter to createOnProgress function
        fix(useMessageHandler.ts): update responseText variable to use generation parameter

        * refactor(api/middleware): move inside server dir

        * refactor: add endpoint specific, modular functions to build options and initialize clients, create server/utils, move middleware, separate utils into api general utils and server specific utils

        * fix(abortMiddleware.js): import getConvo and getConvoTitle functions from models
        feat(abortMiddleware.js): add abortAsk function to abortController to handle aborting of requests
        fix(openAI.js): import buildOptions and initializeClient functions from endpoints/openAI
        refactor(openAI.js): use getAbortData function to get data for abortAsk function

        * refactor: move endpoint specific logic to an endpoints dir

        * refactor(PluginService.js): fix import path for encrypt and decrypt functions in PluginService.js

        * feat(openAI): add new endpoint for adding a title to a conversation

        - Added a new file `addTitle.js` in the `api/server/routes/endpoints/openAI` directory.
        - The `addTitle.js` file exports a function `addTitle` that takes in request parameters and performs the following actions:
          - If the `parentMessageId` is `'00000000-0000-0000-0000-000000000000'` and `newConvo` is true, it proceeds with the following steps:
            - Calls the `titleConvo` function from the `titleConvo` module, passing in the necessary parameters.
            - Calls the `saveConvo` function from the `saveConvo` module, passing in the user ID and conversation details.
        - Updated the `index.js` file in the `api/server/routes/endpoints/openAI` directory to export the `addTitle` function.
        - This change adds

        * fix(abortMiddleware.js): remove console.log statement
        refactor(gptPlugins.js): update imports and function parameters
        feat(gptPlugins.js): add support for abortController and getAbortData
        refactor(openAI.js): update imports and function parameters
        feat(openAI.js): add support for abortController and getAbortData

        fix(openAI.js): refactor code to use modularized functions and middleware
        fix(buildOptions.js): refactor code to use destructuring and update variable names

        * refactor(askChatGPTBrowser.js, bingAI.js, google.js): remove duplicate code for setting response headers
        feat(askChatGPTBrowser.js, bingAI.js, google.js): add setHeaders middleware to set response headers

        * feat(middleware): validateEndpoint, refactor buildOption to only be concerned of endpointOption

        * fix(abortMiddleware.js): add 'finish_reason' property with value 'incomplete' to responseMessage object
        fix(abortMessage.js): remove console.log statement for aborted message
        fix(handlers.js): modify tokens assignment to handle empty generation string and trailing space

        * fix(BaseClient.js): import addSpaceIfNeeded function from server/utils
        fix(BaseClient.js): add space before generation in text property
        fix(index.js): remove getCitations and citeText exports
        feat(buildEndpointOption.js): add buildEndpointOption middleware
        fix(index.js): import buildEndpointOption middleware
        fix(anthropic.js): remove buildOptions function and use endpointOption from req.body
        fix(gptPlugins.js): remove buildOptions function and use endpointOption from req.body
        fix(openAI.js): remove buildOptions function and use endpointOption from req.body

        feat(utils): add citations.js and handleText.js modules
        fix(utils): fix import statements in index.js module

        * refactor(gptPlugins.js): use getResponseSender function from librechat-data-provider

        * feat(gptPlugins): complete 'continue generating'

        * wip: anthropic continue regen

        * feat(middleware): add validateRegistration middleware

        A new middleware function called `validateRegistration` has been added to the list of exported middleware functions in `index.js`. This middleware is responsible for validating registration data before allowing the registration process to proceed.

        * feat(Anthropic): complete continue regen

        * chore: add librechat-data-provider to api/package.json

        * fix(ci): backend-review will mock meilisearch, also installs data-provider as now needed

        * chore(ci): remove unneeded SEARCH env var

        * style(GenerationButtons): make text shorter for sake of space economy, even though this diverges from chat.openai.com

        * style(GenerationButtons/ScrollToBottom): adjust visibility/position based on screen size

        * chore(client): 'Editting' typo

        * feat(GenerationButtons.tsx): add support for endpoint prop in GenerationButtons component
        feat(OptionsBar.tsx): pass endpoint prop to GenerationButtons component
        feat(useGenerations.ts): create useGenerations hook to handle generation logic
        fix(schemas.ts): add searchResult field to tMessageSchema

        * refactor(HoverButtons): convert to TSX and utilize new useGenerations hook

        * fix(abortMiddleware): handle error with res headers set, or abortController not found, to ensure proper API error is sent to the client, chore(BaseClient): remove console log for onStart message meant for debugging

        * refactor(api): remove librechat-data-provider dep for now as it complicates deployed docker build stage, re-use code in CJS, located in server/endpoints/schemas

        * chore: remove console.logs from test files

        * ci: add backend tests for AnthropicClient, focusing on new buildMessages logic

        * refactor(FakeClient): use actual BaseClient sendMessage method for testing

        * test(BaseClient.test.js): add test for loading chat history
        test(BaseClient.test.js): add test for sendMessage logic with isEdited flag

        * fix(buildEndpointOption.js): add support for azureOpenAI in buildFunction object
        wip(endpoints.js): fetch Azure models from Azure OpenAI API if opts.azure is true

        * fix(Button.tsx): add data-testid attribute to button component
        fix(SelectDropDown.tsx): add data-testid attribute to Listbox.Button component
        fix(messages.spec.ts): add waitForServerStream function to consolidate logic for awaiting the server response
        feat(messages.spec.ts): add test for stopping and continuing message and improve browser/page context order and closing

        * refactor(onProgress): speed up time to save initial message for editable routes

        * chore: disable AI message editing (for now), was accidentally allowed

        * refactor: ensure continue is only supported for latest message style: improve styling in dark mode and across all hover buttons/icons, including making edit icon for AI invisible (for now)

        * fix: add test id to generation buttons so they never resolve to 2+ items

        * chore(package.json): add 'packages/' to the list of ignored directories
        chore(data-provider/package.json): bump version to 0.1.5

    commit ae5b7d3d53
    Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
    Date:   Tue Aug 15 18:27:54 2023 -0400

        fix(PluginsClient.js): fix ChatOpenAI Azure Config Issue (#812)

        * fix(PluginsClient.js): fix issue with creating LLM when using Azure

        * chore(PluginsClient.js): omit azure logging

        * refactor(PluginsClient.js): simplify assignment of azure variable

        The code was simplified by directly assigning the value of `this.azure` to the `azure` variable using object destructuring. This makes the code cleaner and more concise.

    commit b85f3bf91e
    Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
    Date:   Tue Aug 15 18:42:24 2023 +0200

        update from lang to localize (#810)

    commit 80aab73bf6
    Author: Danny Avila <messagedaniel@protonmail.com>
    Date:   Mon Aug 14 19:19:04 2023 -0400

        chore: rebuilt package-lock file

    commit bbe4931a97
    Author: Danny Avila <messagedaniel@protonmail.com>
    Date:   Mon Aug 14 19:13:24 2023 -0400

        refactor(ScreenshotContext): use html-to-image for lighter bundle, faster processing

    commit 74802dd720
    Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
    Date:   Mon Aug 14 17:51:03 2023 +0200

        chore: Translation Fixes, Lint Error Corrections, and Additional Translations (#788)

        * fix translation and small lint error

        * changed from localize to useLocalize hook

        * changed to useLocalize

    commit b64cc71d88
    Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
    Date:   Mon Aug 14 10:23:00 2023 -0400

        chore(docker-compose.yml): comment out meilisearch ports in docker-compose.yml (#807)

    commit 89f260bc78
    Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
    Date:   Mon Aug 14 10:12:00 2023 -0400

        fix(CodeBlock.tsx): fix copy-to-clipboard functionality. The code has been updated to use the copy function from the copy-to-clipboard library instead of the (#806)

        avigator.clipboard.writeText method. This should fix the issue with browser incompatibility with navigator SDK and allow users to copy code from the CodeBlock component successfully.

    commit d00c7354cd
    Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
    Date:   Mon Aug 14 09:45:44 2023 -0400

        fix: Corrected Registration Validation, Case-Insensitive Variable Handling, Playwright workflow (#805)

        * feat(auth.js): add validation for registration endpoint using validateRegistration middleware
        feat(validateRegistration.js): add middleware to validate registration based on ALLOW_REGISTRATION environment variable

        * fix(config.js): fix registrationEnabled and socialLoginEnabled variables to handle case-insensitive environment variable values

        * refactor(validateRegistration.js): remove console.log statement

        * chore(playwright.yml): skip browser download during yarn install
        chore(playwright.yml): place Playwright binaries to node_modules/@playwright/test
        chore(playwright.yml): install Playwright dependencies using npx playwright install-deps
        chore(playwright.yml): install Playwright chromium browser using npx playwright install chromium
        chore(playwright.yml): install @playwright/test@latest using npm install -D @playwright/test@latest
        chore(playwright.yml): run Playwright tests using npm run e2e:ci

        * chore(playwright.yml): change npm install order and update comment

        The order of the npm install commands in the "Install Playwright Browsers" step has been changed to first install @playwright/test@latest and then install chromium. Additionally, the comment explaining the PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD variable has been updated to mention npm install instead of yarn install.

        * chore(playwright.yml): remove commented out code for caching and add separate steps for installing Playwright dependencies and browsers

    commit 1aa4b34dc6
    Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
    Date:   Fri Aug 11 19:02:52 2023 +0200

        added the dot (.) username rules (#787)

commit 28230d9305
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Sun Sep 3 02:44:26 2023 +0200

    feat: delete button confirm (#875)

    * base for confirm delete

    * more like OpenAI

commit 2b54e3f9fe
Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
Date:   Fri Sep 1 14:20:51 2023 -0400

    update: install script (#858)

commit 1cd0fd9d5a
Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
Date:   Fri Sep 1 08:12:35 2023 -0400

    doc: Hugging Face Deployment (#867)

    * docs: update ToC

    * docs: update ToC

    * update huggingface.md

    * update render.md

    * update huggingface.md

    * update mongodb.md

    * update huggingface.md

    * update README.md

commit aeeb3d3050
Author: Mu Yuan <yuanmu.email@gmail.com>
Date:   Thu Aug 31 07:21:27 2023 +0800

    Update Zh.tsx (#862)

    * Update Zh.tsx

    Changed the translation of several words to make it more relevant to Chinese usage habits.

    * Update Zh.tsx

    Changed the translation of several words to make it more relevant to Chinese usage habits

commit 80e2e2675b
Author: Raí <140329135+itzraiss@users.noreply.github.com>
Date:   Mon Aug 28 18:05:46 2023 -0300

    Translation of 'com_ui_pay_per_call:' to Spanish and Portuguese that were missing. (#857)

    * Update Br.tsx

    * Update Es.tsx

    * Update Br.tsx

    * Update Es.tsx

commit 3574d0b823
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Mon Aug 28 14:49:26 2023 -0400

    docs: make_your_own.md formatting fix for mkdocs (#855)

commit d672ac690d
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Mon Aug 28 14:24:10 2023 -0400

    Release v0.5.8 (#854)

    * chore: add 'api' image to tag release workflow

    * docs: update DO deployment docs to include instruction about latest stable release, as well as security best practices

    * Release v0.5.8

    * docs: Update digitalocean.md with firewall section images

    * docs: make_your_own.md formatting fix for mkdocs

commit d3e7627046
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Mon Aug 28 12:03:08 2023 -0400

    refactor(plugins): Improve OpenAPI handling, Show Multiple Plugins, & Other Improvements (#845)

    * feat(PluginsClient.js): add conversationId to options object in the constructor
    feat(PluginsClient.js): add support for Code Interpreter plugin
    feat(PluginsClient.js): add support for Code Interpreter plugin in the availableTools manifest
    feat(CodeInterpreter.js): add CodeInterpreterTools module
    feat(CodeInterpreter.js): add RunCommand class
    feat(CodeInterpreter.js): add ReadFile class
    feat(CodeInterpreter.js): add WriteFile class
    feat(handleTools.js): add support for loading Code Interpreter plugin

    * chore(api): update langchain dependency to version 0.0.123

    * fix(CodeInterpreter.js): add support for extracting environment from code
    fix(WriteFile.js): add support for extracting environment from data
    fix(extractionChain.js): add utility functions for creating extraction chain from Zod schema
    fix(handleTools.js): refactor getOpenAIKey function to handle user-provided API key
    fix(handleTools.js): pass model and openAIApiKey to CodeInterpreter constructor

    * fix(tools): rename CodeInterpreterTools to E2BTools
    fix(tools): rename code_interpreter pluginKey to e2b_code_interpreter

    * chore(PluginsClient.js): comment out unused import and function findMessageContent
    feat(PluginsClient.js): add support for CodeSherpa plugin
    feat(PluginsClient.js): add CodeSherpaTools to available tools
    feat(PluginsClient.js): update manifest.json to include CodeSherpa plugin
    feat(CodeSherpaTools.js): create RunCode and RunCommand classes for CodeSherpa plugin

    feat(E2BTools.js): Add E2BTools module for extracting environment from code and running commands, reading and writing files
    fix(codesherpa.js): Remove codesherpa module as it is no longer needed

    feat(handleTools.js): add support for CodeSherpaTools in loadTools function
    feat(loadToolSuite.js): create loadToolSuite utility function to load a suite of tools

    * feat(PluginsClient.js): add support for CodeSherpa v2 plugin
    feat(PluginsClient.js): add CodeSherpa v1 plugin to available tools
    feat(PluginsClient.js): add CodeSherpa v2 plugin to available tools
    feat(PluginsClient.js): update manifest.json for CodeSherpa v1 plugin
    feat(PluginsClient.js): update manifest.json for CodeSherpa v2 plugin
    feat(CodeSherpa.js): implement CodeSherpa plugin for interactive code and shell command execution
    feat(CodeSherpaTools.js): implement RunCode and RunCommand plugins for CodeSherpa v1
    feat(CodeSherpaTools.js): update RunCode and RunCommand plugins for CodeSherpa v2

    fix(handleTools.js): add CodeSherpa import statement
    fix(handleTools.js): change pluginKey from 'codesherpa' to 'codesherpa_tools'
    fix(handleTools.js): remove model and openAIApiKey from options object in e2b_code_interpreter tool
    fix(handleTools.js): remove openAIApiKey from options object in codesherpa_tools tool
    fix(loadToolSuite.js): remove model and openAIApiKey parameters from loadToolSuite function

    * feat(initializeFunctionsAgent.js): add prefix to agentArgs in initializeFunctionsAgent function

    The prefix is added to the agentArgs in the initializeFunctionsAgent function. This prefix is used to provide instructions to the agent when it receives any instructions from a webpage, plugin, or other tool. The agent will notify the user immediately and ask them if they wish to carry out or ignore the instructions.

    * feat(PluginsClient.js): add ChatTool to the list of tools if it meets the conditions
    feat(tools/index.js): import and export ChatTool
    feat(ChatTool.js): create ChatTool class with necessary properties and methods

    * fix(initializeFunctionsAgent.js): update PREFIX message to include sharing all output from the tool
    fix(E2BTools.js): update descriptions for RunCommand, ReadFile, and WriteFile plugins to provide more clarity and context

    * chore: rebuild package-lock after rebase

    * chore: remove deleted file from rebase

    * wip: refactor plugin message handling to mirror chat.openai.com, handle incoming stream for plugin use

    * wip: new plugin handling

    * wip: show multiple plugins handling

    * feat(plugins): save new plugins array

    * chore: bump langchain

    * feat(experimental): support streaming in between plugins

    * refactor(PluginsClient): factor out helper methods to avoid bloating the class, refactor(gptPlugins): use agent action for mapping the name of action

    * fix(handleTools): fix tests by adding condition to return original toolFunctions map

    * refactor(MessageContent): Allow the last index to be last in case it has text (may change with streaming)

    * feat(Plugins): add handleParsingErrors, useful when LLM does not invoke function params

    * chore: edit out experimental codesherpa integration

    * refactor(OpenAPIPlugin): rework tool to be 'function-first', as the spec functions are explicitly passed to agent model

    * refactor(initializeFunctionsAgent): improve error handling and system message

    * refactor(CodeSherpa, Wolfram): optimize token usage by delegating bulk of instructions to system message

    * style(Plugins): match official style with input/outputs

    * chore: remove unnecessary console logs used for testing

    * fix(abortMiddleware): render markdown when message is aborted

    * feat(plugins): add BrowserOp

    * refactor(OpenAPIPlugin): improve prompt handling

    * fix(useGenerations): hide edit button when message is submitting/streaming

    * refactor(loadSpecs): optimize OpenAPI spec loading by only loading requested specs instead of all of them

    * fix(loadSpecs): will retain original behavior when no tools are passed to the function

    * fix(MessageContent): ensure cursor only shows up for last message and last display index
    fix(Message): show legacy plugin and pass isLast to Content

    * chore: remove console.logs

    * docs: update docs based on breaking changes and new features
    refactor(structured/SD): use description_for_model for detailed prompting

    * docs(azure): make plugins section more clear

    * refactor(structured/SD): change default payload to SD-WebUI to prefer realism and config for SDXL

    * refactor(structured/SD): further improve system message prompt

    * docs: update breaking changes after rebase

    * refactor(MessageContent): factor out EditMessage, types, Container to separate files, rename Content -> Markdown

    * fix(CodeInterpreter): linting errors

    * chore: reduce browser console logs from message streams

    * chore: re-enable debug logs for plugins/langchain to help with user troubleshooting

    * chore(manifest.json): add [Experimental] tag to CodeInterpreter plugins, which are not intended as the end-all be-all implementation of this feature for Librechat

commit 66b8580487
Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
Date:   Mon Aug 28 09:18:25 2023 -0400

    docs: third-party tools (#848)

    * docs: third-party tools

    * docs: third-party tools

    * Update third-party.md

    * Update third-party.md

    ---------

    Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>

commit 9791a78161
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Mon Aug 28 15:14:05 2023 +0200

    adjust the animation (#843)

commit 3797ec6082
Author: Ronith <87087292+ronith256@users.noreply.github.com>
Date:   Mon Aug 28 18:43:50 2023 +0530

    feat: Add Code Interpreter Plugin (#837)

    * feat: Add Code Interpreter Plugin

    Adds a Simple Code Interpreter Plugin.
    ## Features:
    - Runs code using local Python Environment

    ## Issues
    - Code execution is not sandboxed.

    * Add Docker Sandbox for Python Server

commit e2397076a2
Author: Alex Zhang <ztc2011@gmail.com>
Date:   Mon Aug 28 00:55:34 2023 +0800

    🌐: Chinese Translation (#846)

commit 50c15c704f
Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
Date:   Sat Aug 26 19:36:59 2023 -0400

    Language translation: Polish (#840)

    * Language translation: Polish

    * Language translation: Polish

    * Revert changes in language-contributions.md

commit 29d3640546
Author: Fuegovic <32828263+fuegovic@users.noreply.github.com>
Date:   Sat Aug 26 19:36:25 2023 -0400

    docs: updates (#841)

commit 39c626aa8e
Author: Danny Avila <messagedaniel@protonmail.com>
Date:   Fri Aug 25 09:29:19 2023 -0400

    fix: isEdited edge case where latest Message is not saved due to aborting too quickly

commit ae5c06f381
Author: Danny Avila <messagedaniel@protonmail.com>
Date:   Fri Aug 25 09:13:50 2023 -0400

    fix(chatGPTBrowser): render markdown formatting by setting isCreatedByUser, fix(useMessageHandler): avoid double appearance of cursor by setting latest message at initial response creation time

commit 9ef1686e18
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Thu Aug 24 20:24:47 2023 -0400

    Update mkdocs.yml

commit 5bbe411569
Author: Flynn <dev@flynnbuckingham.com>
Date:   Thu Aug 24 20:20:37 2023 -0400

    Add podman installation instructions. Update dockerfile to stub env (#819)

    * Added podman container installation docs. Updated dockerfile to stub env file if not present in source

    * Fix typos

commit 887fec99ca
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Fri Aug 25 02:11:27 2023 +0200

    🌐: Russian Translation (#830)

commit 007d51ede1
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Fri Aug 25 02:10:48 2023 +0200

    feat: facebook login (#820)

    * Facebook strategy

    * Update user_auth_system.md

    * Update user_auth_system.md

commit a569020312
Author: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Date:   Thu Aug 24 21:59:11 2023 +0200

    Fix Meilisearch error and refactor of the server index.js (#832)

    * fix meilisearch error at startup

    * limit the nesting

    * disable useless console log

    * fix(indexSync.js): removed redundant searchEnabled

    * refactor(index.js): moved configureSocialLogins to a new file

    * refactor(socialLogins.js): removed unnecessary conditional

commit 37347d4683
Author: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Date:   Wed Aug 23 16:14:17 2023 -0400

    fix(registration): Make Username optional (#831)

    * fix(User.js): update validation schema for username field, allow empty string as a valid value
    fix(validators.js): update validation schema for username field, allow empty string as a valid value
    fix(Registration.tsx, validators.js): update validation rules for name and username fields, change minimum length to 2 and maximum length to 80, assure they match and allow empty string as a valid value
    fix(Eng.tsx): update localization string for com_auth_username, indicate that it is optional

    * fix(User.js): update regex …

* Update package-lock.json

* Update SubmitButton.tsx

* Update SpeechRecognition.tsx

* fix: typescript error

* style: moved to new UI

* fix:(SpeechRecognition) lint error

* moved everything to hooks

* feat: support stt external

* fix(useExternalSpeechRecognition): recording the audio

* feat: whisper api support

* refactor(SpeechReecognition); fix(HoverButtons): set isSpeakling correctly

* fix: spelling errors

* fix: renamed files

* BIG FIX

* feat: whisper support

* fixed some ChatForm bugs and added the tts route

* handling more errors

* Fix audio stream initialization and cleanup in useSpeechToTextExternal

* feat: Elevenlabs TTS

* fixed some req issues

* fix: stt not activating on Mac

* fix: send audio blob to frontend

* fix(ChatForm): startupConfig var

* Update text-to-speech and speech-to-text services

* handle more errors correctly

* Remove console.log statements

* feat: added manual trigger with button

* fix: SpeechToText and SpeechToTextExernal + AudioRecorder

* refactor: TTS component

* chore: removed unused variable

* feat: azure stt

* feat: dedicated speech panel

* feat: STT button switch: fix: TextArea pr value adapted

* refactor: textToSpeech function and useTextToSpeechMutation

* fix: typo data-service

* fix: blob backend to frontend

* feat: TTS button for external

* feat: librechat.yaml

* style: spinner when loading TTS

* feat: hold click to download file

* style: disabled when apiKey not provided

* fix: typo startupConfig?.speechToTextExternal

* style: update icons

* fix(useTextToSpeech): set isSpeaking when audio finish

* fix: small issues with local TTS

* style: update settings dark theme

* docs: STT & TTS

* WIP: chat audio automatic; docs(custom_config): update to new .yaml version; chore: updated librechat.yaml version

* fix: send button disabled

* fix: interval update

* localization

* removed unused test code

* revert interval update to 100

* feat: auto-send message

* fix: chat audio automatic, default false

* refactor: moved all logic to hooks

* chore: renamed ChatAudio to conversationMode

* refactor: organized Speech panel

* feat: autoSendText switch

* feat: moved chataudio to conversationMode and improved error handling; docs: update localai model

* refactor: Auto transcribe audio

* test: AutoSendTextSwitch, AutoTranscribeAudioSwitch and ConversationModeSwitch.spec: refactor: removed hark

* fix: various speechTab fixes

* refactor(useSpeechToTextBrowser):: handle more errors

* feat: engine select

* feat: advanced mode

* chore: converted hooks to TS

* feat: cache TTS

* feat: delete cache; fix: cache issues

* refactor(useTextToSpeechExternal): removed unused import

* feat: cache switch; refactor: moved to dir STT/TTS

* tests: CacheTTS, TextToSpeech, SpeechToText

* feat: custom elevenlabs compatibility

* fix(useTextToSpeechExternal): cache switch not working

* feat: animation for STT

* fix: settings var not working

* chore: remove unused var

* feat: voice dropdown; refactor: yaml changes

* fix(textToSpeech): remove undefined properties

* refactor: Remove console logs and unused variable

* fix: TTS; feat: support coqui and piper

* fix: some STT issues

* fix: stt test

* fix: STT backend sending wrong data

* BREAKING: switch to react-speech-recognition, add regenerator-runtime/runtime in main.jsx

* feat: websocket backend

* foundations for websocket

* first pass elevenlabs streaming

* streaming audio

* stream changes

* input streaming implementation

* fix: client build errors

* WIP: streaming rewrite

* audio stream working but not the loop

* WIP: looping audio stream working

* WIP tts routes rewrite

* feat: track SSE runs by runId, which enables us to better track audio streams per message request

* chore: set activeRunId on data.created

* rate limit tts and only allow once

* WIP: streaming audio

* refactor(useSSE): simplify messageId/parentMessageId assignment in message stream

* delete unused component

* streaming working

* first pass but need to investigate forever pending bug

* optimize audio stream handling client and initial request

* fix(StreamAudio): null exception

* refactor(tts): add limiters for db polling and timeout promise by intervals and not elapsed time

* refactor(textToSpeech): reduce polling delay

* feat(StreamAudio): add caching

* refactor: rename global variable, add setIsPlaying, remove mediasource ref

* feat: use custom hook for audioRef to help determine audio end state

* fix: voices mutation -> query

* fix: voices mutation -> query 2/2

* feat: successful TTS for manual playback

* fix: tts voice init

* feat: playback rate

* feat: global audio toggles

* chore: Add renderIcon function for chat message hover buttons, update schemas with notes

* chore: add debug logging instead of console.logs

* fix: edge case undefined user id

* feat: Automatic Playback switch

* feat: add caching bump data-provider

* chore: tts add auth

* use global state for audio run

* feat: assistants support for TTS read aloud

* ci: uncomment tests for now until they are refactored

* stream audio tests are WIP

* refactor: make automatic playback false as default

---------

Co-authored-by: bsu3338 <bsu3338@users.noreply.github.com>
Co-authored-by: bsu3338 <bsu3338@yahoo.com>
Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
Co-authored-by: Berry-13 <marco13beretta@gmail.com>
Co-authored-by: Super User <root@Berry>
Co-authored-by: Marco Beretta <marco13beretta@proton.me>
2024-05-22 17:19:55 -04:00
Denis Palnitsky
ab6fbe48f1
📥 feat: Import Conversations from LibreChat, ChatGPT, Chatbot UI (#2355)
* Basic implementation of ChatGPT conversation import

* remove debug code

* Handle citations

* Fix updatedAt in import

* update default model

* Use job scheduler to handle import requests

* import job status endpoint

* Add wrapper around Agenda

* Rate limits for import endpoint

* rename import api path

* Batch save import to mongo

* Improve naming

* Add documenting comments

* Test for importers

* Change button for importing conversations

* Frontend changes

* Import job status endpoint

* Import endpoint response

* Add translations to new phrases

* Fix conversations refreshing

* cleanup unused functions

* set timeout for import job status polling

* Add documentation

* get extra spaces back

* Improve error message

* Fix translation files after merge

* fix translation files 2

* Add zh translation for import functionality

* Sync mailisearch index after import

* chore: add dummy uri for jest tests, as MONGO_URI should only be real for E2E tests

* docs: fix links

* docs: fix conversationsImport section

* fix: user role issue for librechat imports

* refactor: import conversations from json
- organize imports
- add additional jsdocs
- use multer with diskStorage to avoid loading file into memory outside of job
- use filepath instead of loading data string for imports
- replace console logs and some logger.info() with logger.debug
- only use multer for import route

* fix: undefined metadata edge case and replace ChatGtp -> ChatGpt

* Refactor importChatGptConvo function to handle undefined metadata edge case and replace ChatGtp with ChatGpt

* fix: chatgpt importer

* feat: maintain tree relationship for librechat messages

* chore: use enum

* refactor: saveMessage to use single object arg, replace console logs, add userId to log message

* chore: additional comment

* chore: multer edge case

* feat: first pass, maintain tree relationship

* chore: organize

* chore: remove log

* ci: add heirarchy test for chatgpt

* ci: test maintaining of heirarchy for librechat

* wip: allow non-text content type messages

* refactor: import content part object json string

* refactor: more content types to format

* chore: consolidate messageText formatting

* docs: update on changes, bump data-provider/config versions, update readme

* refactor(indexSync): singleton pattern for MeiliSearchClient

* refactor: debug log after batch is done

* chore: add back indexSync error handling

---------

Co-authored-by: jakubmieszczak <jakub.mieszczak@zendesk.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
2024-05-02 02:48:26 -04:00
Danny Avila
ecd63eb9f1
✨ feat: Assistants API, General File Support, Side Panel, File Explorer (#1696)
* feat: assistant name/icon in Landing & Header

* feat: assistname in textarea placeholder, and use `Assistant` as default name

* feat: display non-image files in user messages

* fix: only render files if files.length is > 0

* refactor(config -> file-config): move file related configuration values to separate module, add excel types

* chore: spreadsheet file rendering

* fix(Landing): dark mode style for Assistant Name

* refactor: move progress incrementing to own hook, start smaller, cap near limit \(1\)

* refactor(useContentHandler): add empty Text part if last part was completed tool or image

* chore: add accordion trigger border styling for dark mode

* feat: Assistant Builder model selection

* chore: use Spinner when Assistant is mutating

* fix(get/assistants): return correct response object `AssistantListResponse`

* refactor(Spinner): pass size as prop

* refactor: make assistant crud mutations optimistic, add types for options

* chore: remove assistants route and view

* chore: move assistant builder components to separate directory

* feat(ContextButton): delete Assistant via context button/dialog, add localization

* refactor: conditionally show use and context menu buttons, add localization for create assistant

* feat: save side panel states to localStorage

* style(SidePanel): improve avatar menu and assistant select styling for dark mode

* refactor: make NavToggle reusable for either side (left or right), add SidePanel Toggle with ability to close it completely

* fix: resize handle and navToggle behavior

* fix(/avatar/:assistant_id): await `deleteFile` and assign unique name to uploaded image

* WIP: file UI components from PR #576

* refactor(OpenAIMinimalIcon): pass className

* feat: formatDate helper fn

* feat: DataTableColumnHeader

* feat: add row selection, formatted row values, number of rows selected

* WIP: add files to Side panel temporarily

* feat: `LB_QueueAsyncCall`: Leaky Bucket queue for external APIs, use in `processDeleteRequest`

* fix(TFile): correct `source` type with `FileSources`

* fix(useFileHandling): use `continue` instead of return when iterating multiple files, add file type to extendedFile

* chore: add generic setter type

* refactor(processDeleteRequest): settle promises to prevent rejections from processing deletions, log errors

* feat: `useFileDeletion` to reuse file deletion logic

* refactor(useFileDeletion): make `setFiles` an optional param and use object as param

* feat: useDeleteFilesFromTable

* feat: use real `files` data and add deletion action to data table

* fix(Table): make headers sticky

* feat: add dynamic filtering for columns; only show to user Host or OpenAI storage type

* style(DropdownMenu): replace `slate` with `gray`

* style(DataTable): apply dark mode themes and other misc styling

* style(Columns): add color to OpenAI Storage option

* refactor(FileContainer): make file preview reusable

* refactor(Images): make image preview reusable

* refactor(FilePreview): make file prop optional for FileIcon and FilePreview, fix relative style

* feat(Columns): add file/image previews, set a minimum size to show for file size in bytes

* WIP: File Panel with real files and formatted

* feat: open files dialog from panel

* style: file data table mobile and general column styling fixes

* refactor(api/files): return files sorted by the most recently updated

* refactor: provide fileMap through context to prevent re-selecting files to map in different areas; remove unused imports commented out in PanelColumns

* refactor(ExtendFile): make File type optional, add `attached` to prevent attached files from being deleted on remove, make Message.files a partial TFile type

* feat: attach files through file panel

* refactor(useFileHandling): move files to the start of cache list when uploaded

* refactor(useDeleteFilesMutation): delete files from cache when successfully deleted from server

* fix(FileRow): handle possible edge case of duplication due to attaching recently uploaded file

* style(SidePanel): make resize grip border transparent, remove unnecessary styling on close sidepanel button

* feat: action utilities and tests

* refactor(actions): add `ValidationResult` type and change wording for no server URL found

* refactor(actions): check for empty server URL

* fix(data-provider): revert tsconfig to fix type issue resolution

* feat(client): first pass of actions input for assistants

* refactor(FunctionSignature): change method to output object instead of string

* refactor(models/Assistant): add actions field to schema, use searchParams object for methods, and add `getAssistant`

* feat: post actions input first pass
- create new Action document
- add actions to Assistant DB document
- create /action/:assistant_id POST route
- pass more props down from PanelSwitcher, derive assistant_id from switcher
- move privacy policy to ActionInput
- reset data on input change/validation
- add `useUpdateAction`
- conform FunctionSignature type to FunctionTool
- add action, assistant doc, update hook related types

* refactor: optimize assistant/actions relationship
- past domain in metadata as hostname and not a URL
- include domain in tool name
- add `getActions` for actions retrieval by user
- add `getAssistants` for assistant docs retrieval by user
- add `assistant_id` to Action schema
- move actions to own module as a subroute to `api/assistants`
- add `useGetActionsQuery` and `useGetAssistantDocsQuery` hooks
- fix Action type def

* feat: show assistant actions in assistant builder

* feat: switch to actions on action click, editing action styling

* fix: add Assistant state for builder panel to allow immediate selection of newly created assistants as well as retaining the current assistant when switching to a different panel within the builder

* refactor(SidePanel/NavToggle): offset less from right when SidePanel is completely collapsed

* chore: rename `processActions` -> `processRequiredActions`

* chore: rename Assistant API Action to RequiredAction

* refactor(actions): avoid nesting actual API params under generic `requestBody` to optimize LLM token usage

* fix(handleTools): avoid calling `validTool` if not defined, add optional param to skip the loading of specs, which throws an error in the context of assistants

* WIP: working first pass of toolCalls generated from openapi specs

* WIP: first pass ToolCall styling

* feat: programmatic iv encryption/decryption helpers

* fix: correct ActionAuth types/enums, and define type for AuthForm

* feat: encryption/decryption helpers for Action AuthMetadata

* refactor(getActions): remove sensitive fields from query response

* refactor(POST/actions): encrypt and remove sensitive fields from mutation response

* fix(ActionService): change ESM import to CJS

* feat: frontend auth handling for actions + optimistic update on action update/creation

* refactor(actions): use the correct variables and types for setAuth method

* refactor: POST /:assistant_id action can now handle updating an existing action, add `saved_auth_fields` to determine when user explicitly saves new auth creds. only send auth metadata if user explicitly saved fields

* refactor(createActionTool): catch errors and send back meaningful error message, add flag to `getActions` to determine whether to retrieve sensitive values or not

* refactor(ToolService): add `action` property to ToolCall PartMetadata to determine if the tool call was an action, fix parsing function name issue with actionDelimiter

* fix(ActionRequest): use URL class to correctly join endpoint parts for `execute` call

* feat: delete assistant actions

* refactor: conditionally show Available actions

* refactor: show `retrieval` and `code_interpreter` as Capabilities, swap `Switch` for `Checkbox`

* chore: remove shadow-stroke from messages

* WIP: first pass of Assistants Knowledge attachments

* refactor: remove AssistantsProvider in favor of FormProvider, fix selectedAssistant re-render bug, map Assistant file_ids to files via fileMap, initialize Knowledge component with mapped files if any exist

* fix: prevent deleting files on assistant file upload

* chore: remove console.log

* refactor(useUploadFileMutation): update files and assistants cache on upload

* chore: disable oauth option as not supported yet

* feat: cancel assistant runs

* refactor: initialize OpenAI client with helper function, resolve all related circular dependencies

* fix(DALL-E): initialization

* fix(process): openai client initialization

* fix: select an existing Assistant when the active one is deleted

* chore: allow attaching files for assistant endpoint, send back relevant OpenAI error message when uploading, deconstruct openAI initialization correctly, add `message_file` to formData when a file is attached to the message but not the assistant

* fix: add assistant_id on newConvo

* fix(initializeClient): import fix

* chore: swap setAssistant for setOption in useEffect

* fix(DALL-E): add processFileURL to loadTools call

* chore: add customConfig to debug logs

* feat: delete threads on convo delete

* chore: replace Assistants icon

* chore: remove console.dir() in `abortRun`

* feat(AssistantService): accumulate text values from run in openai.responseText

* feat: titling for assistants endpoint

* chore: move panel file components to appropriate directory, add file checks for attaching files, change icon for Attach Files

* refactor: add localizations to tools, plugins, add condition for adding/remove user plugins so tool selections don't affect this value

* chore: disable `import from url` action for now

* chore: remove textMimeTypes from default fileConfig for now

* fix: catch tool errors and send as outputs with error messages

* fix: React warning about button as descendant of button

* style: retrieval and cancelled icon

* WIP: pass isSubmitting to Parts, use InProgressCall to display cancelled tool calls correctly, show domain/function name

* fix(meilisearch): fix `postSaveHook` issue where indexing expects a mongo document, and join all text content parts for meili indexing

* ci: fix dall-e tests

* ci: fix client tests

* fix: button types in actions panel

* fix: plugin auth form persisting across tool selections

* fix(ci): update AppService spec with `loadAndFormatTools`

* fix(clearConvos): add id check earlier on

* refactor(AssistantAvatar): set previewURL dynamically when emtadata.avatar changes

* feat(assistants): addTitle cache setting

* fix(useSSE): resolve rebase conflicts

* fix: delete mutation

* style(SidePanel): make grip visible on active and hover, invisible otherwise

* ci: add data-provider tests to workflow, also update eslint/tsconfig to recognize specs, and add `text/csv` to fileConfig

* fix: handle edge case where auth object is undefined, and log errors

* refactor(actions): resolve  schemas, add tests for resolving refs, import specs from separate file for tests

* chore: remove comment

* fix(ActionsInput): re-render bug when initializing states with action fields

* fix(patch/assistant): filter undefined tools

* chore: add logging for errors in assistants routes

* fix(updateAssistant): map actions to functions to avoid overwriting

* fix(actions): properly handle GET paths

* fix(convos): unhandled delete thread exception

* refactor(AssistantService): pass both thread_id and conversationId when sending intermediate assistant messages, remove `mapMessagesToSteps` from AssistantService

* refactor(useSSE): replace all messages with runMessages and pass latestMessageId to abortRun; fix(checkMessageGaps): include tool calls when  syncing messages

* refactor(assistants/chat): invoke `createOnTextProgress` after thread creation

* chore: add typing

* style: sidepanel styling

* style: action tool call domain styling

* feat(assistants): default models, limit retrieval to certain models, add env variables to to env.example

* feat: assistants api key in EndpointService

* refactor: set assistant model to conversation on assistant switch

* refactor: set assistant model to conversation on assistant select from panel

* fix(retrieveAndProcessFile): catch attempt to download file with `assistant` purpose which is not allowed; add logging

* feat: retrieval styling, handling, and logging

* chore: rename ASSISTANTS_REVERSE_PROXY to ASSISTANTS_BASE_URL

* feat: FileContext for file metadata

* feat: context file mgmt and filtering

* style(Select): hover/rounded changes

* refactor: explicit conversation switch, endpoint dependent, through `useSelectAssistant`, which does not create new chat if current endpoint is assistant endpoint

* fix(AssistantAvatar): make empty previewURL if no avatar present

* refactor: side panel mobile styling

* style: merge tool and action section, optimize mobile styling for action/tool buttons

* fix: localStorage issues

* fix(useSelectAssistant): invoke react query hook directly in select hook as Map was not being updated in time

* style: light mode fixes

* fix: prevent sidepanel nav styling from shifting layout up

* refactor: change default layout (collapsed by default)

* style: mobile optimization of DataTable

* style: datatable

* feat: client-side hide right-side panel

* chore(useNewConvo): add partial typing for preset

* fix(useSelectAssistant): pass correct model name by using template as preset

* WIP: assistant presets

* refactor(ToolService): add native solution for `TavilySearchResults` and log tool output errors

* refactor: organize imports and use native TavilySearchResults

* fix(TavilySearchResults): stringify result

* fix(ToolCall): show tool call outputs when not an action

* chore: rename Prompt Prefix to custom instructions (in user facing text only)

* refactor(EditPresetDialog): Optimize setting title by debouncing, reset preset on dialog close to avoid state mixture

* feat: add `presetOverride` to overwrite active conversation settings when saving a Preset (relevant for client side updates only)

* feat: Assistant preset settings (client-side)

* fix(Switcher): only set assistant_id and model if current endpoint is Assistants

* feat: use `useDebouncedInput` for updating conversation settings, starting with EditPresetDialog title setting and Assistant instructions setting

* feat(Assistants): add instructions field to settings

* feat(chat/assistants): pass conversation settings to run body

* wip: begin localization and only allow actions if the assistant is created

* refactor(AssistantsPanel): knowledge localization, allow tools on creation

* feat: experimental: allow 'priming' values before assistant is created, that would normally require an assistant_id to be defined

* chore: trim console logs and make more meaningful

* chore: toast messages

* fix(ci): date test

* feat: create file when uploading Assistant Avatar

* feat: file upload rate limiting from custom config with dynamic file route initialization

* refactor: use file upload limiters on post routes only

* refactor(fileConfig): add endpoints field for endpoint specific fileconfigs, add mergeConfig function, add tests

* refactor: fileConfig route, dynamic multer instances used on all '/' and '/images' POST routes, data service and query hook

* feat: supportedMimeTypesSchema, test for array of regex

* feat: configurable file config limits

* chore: clarify assistants file knowledge prereq.

* chore(useTextarea): default to localized 'Assistant' if assistant name is empty

* feat: configurable file limits and toggle file upload per endpoint

* fix(useUploadFileMutation): prevent updating assistant.files cache if file upload is a message_file attachment

* fix(AssistantSelect): set last selected assistant only when timeout successfully runs

* refactor(queries): disable assistant queries if assistants endpoint is not enabled

* chore(Switcher): add localization

* chore: pluralize `assistant` for `EModelEndpoint key and value

* feat: show/hide assistant UI components based on endpoint availability; librechat.yaml config for disabling builder section and setting polling/timeout intervals

* fix(compactEndpointSchemas): use EModelEndpoint for schema access

* feat(runAssistant): use configured values from `librechat.yaml` for `pollIntervalMs` and `timeout`

* fix: naming issue

* wip: revert landing

* 🎉 happy birthday LibreChat (#1768)

* happy birthday LibreChat

* Refactor endpoint condition in Landing component

* Update birthday message in Eng.tsx

* fix(/config): avoid nesting ternaries

* refactor(/config): check birthday

---------

Co-authored-by: Danny Avila <messagedaniel@protonmail.com>

* fix: landing

* fix: landing

* fix(useMessageHelpers): hardcoded check to use EModelEndpoint instead

* fix(ci): convo test revert to main

* fix(assistants/chat): fix issue where assistant_id was being saved as model for convo

* chore: added logging, promises racing to prevent longer timeouts, explicit setting of maxRetries and timeouts, robust catching of invalid abortRun params

* refactor: use recoil state for `showStopButton` and only show for assistants endpoint after syncing conversation data

* refactor: optimize abortRun strategy using localStorage, refactor `abortConversation` to use async/await and await the result, refactor how the abortKey cache is set for runs

* fix(checkMessageGaps): assign `assistant_id` to synced messages if defined; prevents UI from showing blank assistant for cancelled messages

* refactor: re-order sequence of chat route, only allow aborting messages after run is created, cancel abortRun if there was a cancelling error (likely due already cancelled in chat route), and add extra logging

* chore(typedefs): add httpAgent type to OpenAIClient

* refactor: use custom implementation of retrieving run with axios to allow for timing out run query

* fix(waitForRun): handle timed out run retrieval query

* refactor: update preset conditions:
- presets will retain settings when a different endpoint is selected; for existing convos, either when modular or is assistant switch
- no longer use `navigateToConvo` on preset select

* fix: temporary calculator hack as expects string input when invoked

* fix: cancel abortRun only when cancelling error is a result of the run already being cancelled

* chore: remove use of `fileMaxSizeMB` and total counterpart (redundant)

* docs: custom config documentation update

* docs: assistants api setup and dotenv, new custom config fields

* refactor(Switcher): make Assistant switcher sticky in SidePanel

* chore(useSSE): remove console log of data and message index

* refactor(AssistantPanel): button styling and add secondary select button to bottom of panel

* refactor(OpenAIClient): allow passing conversationId to RunManager through titleConvo and initializeLLM to properly record title context tokens used in cases where conversationId was not defined by the client

* feat(assistants): token tracking for assistant runs

* chore(spendTokens): improve logging

* feat: support/exclude specific assistant Ids

* chore: add update `librechat.example.yaml`, optimize `AppService` handling, new tests for `AppService`, optimize missing/outdate config logging

* chore: mount docker logs to root of project

* chore: condense axios errors

* chore: bump vite

* chore: vite hot reload fix using latest version

* chore(getOpenAIModels): sort instruct models to the end of models list

* fix(assistants): user provided key

* fix(assistants): user provided key, invalidate more queries on revoke

---------

Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
2024-02-13 20:42:27 -05:00
Danny Avila
d20970f5c5
🚀 Feat: Streamline File Strategies & GPT-4-Vision Settings (#1535)
* chore: fix `endpoint` typescript issues and typo in console info message

* feat(api): files GET endpoint and save only file_id references to messages

* refactor(client): `useGetFiles` query hook, update file types, optimistic update of filesQuery on file upload

* refactor(buildTree): update to use params object and accept fileMap

* feat: map files to messages; refactor(ChatView): messages only available after files are fetched

* fix: fetch files only when authenticated

* feat(api): AppService
- rename app.locals.configs to app.locals.paths
- load custom config use fileStrategy from yaml config in app.locals

* refactor: separate Firebase and Local strategies, call based on config

* refactor: modularize file strategies and employ with use of DALL-E

* refactor(librechat.yaml): add fileStrategy field

* feat: add source to MongoFile schema, as well as BatchFile, and ExtendedFile types

* feat: employ file strategies for upload/delete files

* refactor(deleteFirebaseFile): add user id validation for firebase file deletion

* chore(deleteFirebaseFile): update jsdocs

* feat: employ strategies for vision requests

* fix(client): handle messages with deleted files

* fix(client): ensure `filesToDelete` always saves/sends `file.source`

* feat(openAI): configurable `resendImages` and `imageDetail`

* refactor(getTokenCountForMessage): recursive process only when array of Objects and only their values (not keys) aside from `image_url` types

* feat(OpenAIClient): calculateImageTokenCost

* chore: remove comment

* refactor(uploadAvatar): employ fileStrategy for avatars, from social logins or user upload

* docs: update docs on how to configure fileStrategy

* fix(ci): mock winston and winston related modules, update DALLE3.spec.js with changes made

* refactor(redis): change terminal message to reflect current development state

* fix(DALL-E-2): pass fileStrategy to dall-e
2024-01-11 11:37:54 -05:00
Marco Beretta
f19f5dca8e
🔥🚀 feat: CDN (Firebase) & feat: account section (#1438)
* localization + api-endpoint

* docs: added firebase documentation

* chore: icons

* chore: SettingsTabs

* feat: account pannel; fix: gear icons

* docs: position update

* feat: firebase

* feat: plugin support

* route

* fixed bugs with firebase and moved a lot of files

* chore(DALLE3): using UUID v4

* feat: support for social strategies; moved '/images' path

* fix: data ignored

* gitignore update

* docs: update firebase guide

* refactor: Firebase
- use singleton pattern for firebase initialization, initially on server start
- reorganize imports, move firebase specific files to own service under Files
- rename modules to remove 'avatar' redundancy
- fix imports based on changes

* ci(DALLE/DALLE3): fix tests to use logger and new expected outputs, add firebase tests

* refactor(loadToolWithAuth): pass userId to tool as field

* feat(images/parse): feat: Add URL Image Basename Extraction

Implement a new module to extract the basename of an image from a given URL. This addition includes the  function, which parses the URL and retrieves the basename using the Node.js 'url' and 'path' modules. The function is documented with JSDoc comments for better maintainability and understanding. This feature enhances the application's ability to handle and process image URLs efficiently.

* refactor(addImages): function to use a more specific regular expression for observedImagePath based on the generated image markdown standard across the app

* refactor(DALLE/DALLE3): utilize `getImageBasename` and `this.userId`; fix: pass correct image path to firebase url helper

* fix(addImages): make more general to match any image markdown descriptor

* fix(parse/getImageBasename): test result of this function for an actual image basename

* ci(DALLE3): mock getImageBasename

* refactor(AuthContext): use Recoil atom state for user

* feat: useUploadAvatarMutation, react-query hook for avatar upload

* fix(Toast): stack z-order of Toast over all components (1000)

* refactor(showToast): add optional status field to avoid importing NotificationSeverity on each use of the function

* refactor(routes/avatar): remove unnecessary get route, get userId from req.user.id, require auth on POST request

* chore(uploadAvatar): TODO: remove direct use of Model, `User`

* fix(client): fix Spinner imports

* refactor(Avatar): use react-query hook, Toast, remove unnecessary states, add optimistic UI to upload

* fix(avatar/localStrategy): correctly save local profile picture and cache bust for immediate rendering; fix: firebase init info message (only show once)

* fix: use `includes` instead of `endsWith` for checking manual query of avatar image path in case more queries are appended (as is done in avatar/localStrategy)

---------

Co-authored-by: Danny Avila <messagedaniel@protonmail.com>
2023-12-29 21:42:19 -05:00
Danny Avila
317cdd3f77
feat: Vision Support + New UI (#1203)
* feat: add timer duration to showToast, show toast for preset selection

* refactor: replace old /chat/ route with /c/. e2e tests will fail here

* refactor: move typedefs to root of /api/ and add a few to assistant types in TS

* refactor: reorganize data-provider imports, fix dependency cycle, strategize new plan to separate react dependent packages

* feat: add dataService for uploading images

* feat(data-provider): add mutation keys

* feat: file resizing and upload

* WIP: initial API image handling

* fix: catch JSON.parse of localStorage tools

* chore: experimental: use module-alias for absolute imports

* refactor: change temp_file_id strategy

* fix: updating files state by using Map and defining react query callbacks in a way that keeps them during component unmount, initial delete handling

* feat: properly handle file deletion

* refactor: unexpose complete filepath and resize from server for higher fidelity

* fix: make sure resized height, width is saved, catch bad requests

* refactor: use absolute imports

* fix: prevent setOptions from being called more than once for OpenAIClient, made note to fix for PluginsClient

* refactor: import supportsFiles and models vars from schemas

* fix: correctly replace temp file id

* refactor(BaseClient): use absolute imports, pass message 'opts' to buildMessages method, count tokens for nested objects/arrays

* feat: add validateVisionModel to determine if model has vision capabilities

* chore(checkBalance): update jsdoc

* feat: formatVisionMessage: change message content format dependent on role and image_urls passed

* refactor: add usage to File schema, make create and updateFile, correctly set and remove TTL

* feat: working vision support
TODO: file size, type, amount validations, making sure they are styled right, and making sure you can add images from the clipboard/dragging

* feat: clipboard support for uploading images

* feat: handle files on drop to screen, refactor top level view code to Presentation component so the useDragHelpers hook  has ChatContext

* fix(Images): replace uploaded images in place

* feat: add filepath validation to protect sensitive files

* fix: ensure correct file_ids are push and not the Map key values

* fix(ToastContext): type issue

* feat: add basic file validation

* fix(useDragHelpers): correct context issue with `files` dependency

* refactor: consolidate setErrors logic to setError

* feat: add dialog Image overlay on image click

* fix: close endpoints menu on click

* chore: set detail to auto, make note for configuration

* fix: react warning (button desc. of button)

* refactor: optimize filepath handling, pass file_ids to images for easier re-use

* refactor: optimize image file handling, allow re-using files in regen, pass more file metadata in messages

* feat: lazy loading images including use of upload preview

* fix: SetKeyDialog closing, stopPropagation on Dialog content click

* style(EndpointMenuItem): tighten up the style, fix dark theme showing in lightmode, make menu more ux friendly

* style: change maxheight of all settings textareas to 138px from 300px

* style: better styling for textarea and enclosing buttons

* refactor(PresetItems): swap back edit and delete icons

* feat: make textarea placeholder dynamic to endpoint

* style: show user hover buttons only on hover when message is streaming

* fix: ordered list not going past 9, fix css

* feat: add User/AI labels; style: hide loading spinner

* feat: add back custom footer, change original footer text

* feat: dynamic landing icons based on endpoint

* chore: comment out assistants route

* fix: autoScroll to newest on /c/ view

* fix: Export Conversation on new UI

* style: match message style of official more closely

* ci: fix api jest unit tests, comment out e2e tests for now as they will fail until addressed

* feat: more file validation and use blob in preview field, not filepath, to fix temp deletion

* feat: filefilter for multer

* feat: better AI labels based on custom name, model, and endpoint instead of  `ChatGPT`
2023-11-21 20:12:48 -05:00