LibreChat/packages/data-provider/src
Danny Avila 1662adc581
📺 feat: Google URL Context Param with Native YouTube Video Understanding (#13924)
* ✨ feat: Add Google url_context Param with Native YouTube Video Understanding

Mirror the web_search grounding wiring for a new Google/Gemini `url_context`
model param (resolves to the native `urlContext` tool). When enabled, YouTube
URLs in the latest user message are injected as Gemini video parts (fileData),
since the URL Context tool does not support YouTube.

* 🎞️ fix: Provider-aware YouTube injection limits for url_context

Address Codex review on the YouTube video-understanding path:
- Cap injected YouTube parts per request by provider/model (Vertex: 1; Gemini
  Developer API: 10 on 2.5+, 1 on earlier models) so multi-link messages cannot
  exceed the provider limit and get rejected.
- Set a video/mp4 mimeType on Vertex YouTube fileData (matching Vertex samples);
  the Developer API still omits it.

* 🧩 fix: Round-trip url_context for Google-compatible custom endpoints

Add url_context to openAIBaseSchema so the per-chat value persists for custom
endpoints configured with customParams.defaultParamsEndpoint: 'google', matching
how web_search is already picked there.

* 🚦 fix: Gate url_context tool to Gemini 2.5+ models

Per Google's URL Context supported-models list (2.5+/3.x only), skip the native
urlContext tool on earlier models (debug-log + no-op) instead of sending it and
triggering a provider 400. This also gates the coupled YouTube video-understanding
injection to 2.5+, since it keys off the resolved urlContext tool.

* ✂️ fix: Strip YouTube URLs from urlContext text; keep url_context out of OpenAI schema

- Remove url_context from the shared openAIBaseSchema (revert): it is Google-only
  and would otherwise leak as an unsupported param to OpenAI/Azure/OpenRouter
  requests. On Google-compatible custom endpoints url_context is enabled via admin
  addParams/defaultParams, same as web_search.
- When injecting YouTube video parts, strip the matched YouTube URLs from the prompt
  text so the urlContext tool (which reads URLs from text and cannot fetch YouTube)
  does not consume its URL budget on them. Non-YouTube URLs are left intact.

* 🎯 fix: Refine url_context model gating and YouTube injection edges

Address Codex round 4:
- Exclude non-text modality variants (image/live/tts) from URL Context support,
  mirroring the Google tool-combination modality exclusion.
- Use the resolved run model (model_parameters.model) for YouTube injection limits
  instead of the saved base model.
- Strip only the YouTube links actually routed to video (id-aware); keep over-limit
  links in the text so the model can still reason about them.
- Keep timestamped YouTube links (?t=/&start=) in the text so the moment cue survives.
- Recognize youtube-nocookie.com/embed links.

* 🎚️ fix: Exclude audio Gemini variants + preserve pre-id YouTube timestamps

Address Codex round 5:
- Add `audio` to the url_context modality exclusion so audio-only Gemini variants
  (e.g. gemini-2.5-flash-preview-native-audio-dialog) skip the tool instead of 400ing.
- Detect YouTube timestamps anywhere in the matched URL (incl. before `v=`, e.g.
  watch?t=90&v=<id>), so timestamped links are kept in the prompt text as intended.
2026-06-23 22:42:06 -04:00
..
react-query 🔖 fix: Decrement Bookmark Counts When Deleting Conversations (#13830) 2026-06-18 08:37:08 -04:00
types 🪢 fix: Paginate MCP tools/list to Load All Tools (#13840) 2026-06-20 11:04:06 -04:00
accessPermissions.ts 🔗 feat: Add Granular Access Control to Shared Links via ACL System (#13051) 2026-06-03 14:17:17 -04:00
actions.ts 🛡️ fix: Implement TOCTOU-Safe SSRF Protection for Actions and MCP (#11722) 2026-02-11 22:09:58 -05:00
api-endpoints.ts 🔐 fix: Gate Shared Startup Config By Link Access (#13897) 2026-06-23 08:28:37 -04:00
artifacts.ts
azure.ts
balance.spec.ts ⏱️ fix: Align Auto-Refill Next Date (#12980) 2026-05-06 21:40:18 -04:00
balance.ts ⏱️ fix: Align Auto-Refill Next Date (#12980) 2026-05-06 21:40:18 -04:00
bedrock.ts 🕐 feat: Add promptCacheTtl model parameter for 1h/5m cache duration (#13835) 2026-06-18 16:36:43 -04:00
cloudfront-config.spec.ts 🌩️ feat: Strict CloudFront signed cookie enforcement via requireSignedAccess (#13078) 2026-05-11 23:30:01 -04:00
codeEnvRef.spec.ts 🧱 refactor: typed CodeEnvRef + kind discriminator + principal-aware sandbox cache (#12960) 2026-05-08 12:29:43 -04:00
codeEnvRef.ts 🔐 feat: Mint Code API Auth Tokens (#13028) 2026-05-09 16:09:10 -04:00
config.spec.ts 🪨 fix: Preserve Bedrock Guardrail Config (#13381) 2026-05-28 21:38:53 -07:00
config.ts 🧠 feat: Configurable Reasoning Replay for Custom Endpoints (#13921) 2026-06-23 21:08:47 -04:00
createPayload.ts 🕰️ feat: Resolve Agent Prompt Time Variables in User's Timezone (#13815) 2026-06-18 08:39:56 -04:00
data-service.ts 🔐 fix: Gate Shared Startup Config By Link Access (#13897) 2026-06-23 08:28:37 -04:00
feedback.ts
file-config.spec.ts 🗜️ fix: Support Windows ZIP MIME Uploads (#13794) 2026-06-16 11:19:06 -04:00
file-config.ts 🗜️ fix: Support Windows ZIP MIME Uploads (#13794) 2026-06-16 11:19:06 -04:00
generate.ts ♊ feat: Model-Aware Max Output Tokens for Google/Gemini (#13390) 2026-05-29 08:09:32 -07:00
headers-helpers.ts 🍪 refactor: Refresh CloudFront Media Cookies (#13091) 2026-05-12 13:26:05 -04:00
index.ts 🔗 feat: Snapshot Files for Shared-Link Attachments (#13740) 2026-06-20 23:05:13 -04:00
keys.ts 🔐 fix: Gate Shared Startup Config By Link Access (#13897) 2026-06-23 08:28:37 -04:00
limits.ts 🧩 feat: Enable Model Spec Subagents (#13598) 2026-06-08 11:43:08 -04:00
mcp.ts 🔐 fix: Resolve Env Variables in MCP OAuth URL Fields (#13573) 2026-06-07 21:39:44 -04:00
messages.ts
models.ts 💬 feat: Conversation Starters for Model Specs (#13710) 2026-06-13 11:38:49 -04:00
parameterSettings.spec.ts ♊ feat: Model-Aware Max Output Tokens for Google/Gemini (#13390) 2026-05-29 08:09:32 -07:00
parameterSettings.ts 📺 feat: Google URL Context Param with Native YouTube Video Understanding (#13924) 2026-06-23 22:42:06 -04:00
parsers.ts 🐛 fix: resolve dayjs plugin ESM imports in data-provider (#13851) 2026-06-19 11:11:08 -04:00
permissions.ts 🔗 feat: Add Granular Access Control to Shared Links via ACL System (#13051) 2026-06-03 14:17:17 -04:00
request.ts 🎯 fix: Narrow Public Share 401 Bypass to the Share Endpoint Only (#12905) 2026-06-20 13:57:21 -04:00
roles.spec.ts 🔗 feat: Add Granular Access Control to Shared Links via ACL System (#13051) 2026-06-03 14:17:17 -04:00
roles.ts 🔗 feat: Add Granular Access Control to Shared Links via ACL System (#13051) 2026-06-03 14:17:17 -04:00
schemas.spec.ts 🗂️ feat: Add Private Chat Projects (#13467) 2026-06-03 15:29:18 -04:00
schemas.ts 📺 feat: Google URL Context Param with Native YouTube Video Understanding (#13924) 2026-06-23 22:42:06 -04:00
types.ts 🧠 feat: Configurable Reasoning Replay for Custom Endpoints (#13921) 2026-06-23 21:08:47 -04:00
utils.ts 🧯 fix: Prevent Env-Variable Exfil. via Placeholder Injection (#12260) 2026-03-16 08:48:24 -04:00