LibreChat/api/server/controllers/agents/openai.js
Danny Avila dfc3dfa57f 📍 feat: always-apply frontmatter: auto-prime skills every turn (#12746)
* 🔁 refactor: Rebase always-apply work onto merged structured-frontmatter columns

Phase 6 (disable-model-invocation / user-invocable / allowed-tools)
landed first on feat/agent-skills. Reconcile this branch with the new
mainline:

- Thread alwaysApplySkillPrimes through unionPrimeAllowedTools alongside
  manualSkillPrimes, applying the combined MAX_PRIMED_SKILLS_PER_TURN
  ceiling before loading tools.
- Add `_id` to ResolvedAlwaysApplySkill to match Phase 6's
  ResolvedManualSkill shape (read_file name-collision protection).
- Register 'always-apply' in ALLOWED_FRONTMATTER_KEYS / FRONTMATTER_KIND
  so Phase 6's validator recognizes it.
- Drop frontmatter from the listSkillsByAccess projection; the backfill
  helper remains as defensive code but its read path is no longer
  exercised on summary rows (no legacy rows exist — the branch never
  shipped), saving ~200KB per page.
- Retire the corresponding "backfills legacy on summaries" test.
- Plumb listAlwaysApplySkills through the JS controllers + endpoint
  initializer so the always-apply resolver sees a real DB method.

* 🧹 fix: Dedupe manual/always-apply overlap, share YAML util, tidy comments

Addresses review findings:

- Cross-list dedup: when a user $-invokes a skill that is also marked
  always-apply, the always-apply copy is now dropped so the same
  SKILL.md body never primes twice in one turn. Manual wins (explicit
  intent, closer to the user message). Dedup runs in both
  initializeAgent (so persisted user-bubble pills stay in sync) and
  injectSkillPrimes (defense-in-depth at splice time). New test cases
  cover single-overlap, partial-overlap, and dedup-before-cap.
- DRY: extract stripYamlTrailingComment to
  packages/data-schemas/src/utils/yaml.ts; packages/api/src/skills/import.ts
  now imports the shared helper. Also drop the redundant inner
  stripYamlTrailingComment call inside parseBooleanScalar — the call
  site already strips.
- Mark injectManualSkillPrimes as @deprecated in favor of
  injectSkillPrimes (kept for external consumers of @librechat/api).
- Document SKILL_TRIGGER_MODEL as forward-looking plumbing for the
  model-invoked path rather than leaving it as a bare unused export.
- Replace the stale "frontmatter is included" comment on
  listSkillsByAccess with an accurate explanation of why it was
  intentionally excluded.

* 🔒 fix: Include always-apply primes in skillPrimedIdsByName + clear alwaysApply on body opt-out

Two bugs flagged by Codex review:

P1 (read_file): `manualSkillPrimedIdsByName` only carried manual-invocation
primes, so an always-apply skill with `disable-model-invocation: true`
was blocked from reading its own bundled files, and same-name collisions
could resolve to a different doc than the one whose body got primed.
- Rename `buildManualSkillPrimedIdsByName` → `buildSkillPrimedIdsByName`
  (accepts both manual + always-apply prime arrays).
- Rename the configurable field `manualSkillPrimedIdsByName` →
  `skillPrimedIdsByName` throughout the plumbing (skillConfigurable.ts,
  handlers.ts, CJS callers, tests).
- Overlap resolution: manual wins on the rare edge case where the same
  name appears in both arrays (upstream dedup should prevent this, but
  defensive merging treats manual as authoritative).
- New tests: (1) gate-relaxation fires for always-apply primes, (2) `_id`
  pinning works for always-apply same-name collisions.

P2 (updateSkill): when a body update had no `always-apply:` key,
`extractAlwaysApplyFromBody` returned `absent` and the column was left
untouched. A skill that was once `alwaysApply: true` would keep
auto-priming even after its SKILL.md no longer declared the flag.
- Treat `absent` as a positive "not always-apply" declaration when the
  body is explicitly submitted; flip the column to `false`.
- Explicit top-level `alwaysApply` still wins (three-source precedence
  unchanged).
- New tests: body removes key → false, body has no frontmatter at all →
  false, explicit + body-without-key → explicit wins.

* 🧵 refactor: Collapse duplicate prime types + tighten parse + test hygiene

Sanity-check review follow-ups:

- Collapse `ResolvedManualSkill` / `ResolvedAlwaysApplySkill` into a
  single `ResolvedSkillPrime` canonical interface with two backward-
  compatible type aliases. Both resolvers feed the same pipeline stages
  (injectSkillPrimes, unionPrimeAllowedTools, buildSkillPrimedIdsByName);
  the per-source distinction lives on `additional_kwargs.trigger`, not
  on the resolver output.
- Move the `always-apply` branch in `parseFrontmatter` to operate on the
  raw post-colon text. The outer `unquoteYaml` was fine today because
  it's idempotent on non-quoted strings, but running it twice (once per
  line, once after stripping the inline comment) would be fragile if the
  unquoter ever grows richer YAML-escape handling.
- Add the missing `alwaysApplyDedupedFromManual: 0` field to the
  `injectSkillPrimes` mocks in `openai.spec.js` and `responses.unit.spec.js`
  so they match the full `InjectSkillPrimesResult` contract.
- Insert the blank line between the `unionPrimeAllowedTools` and
  `resolveAlwaysApplySkills` describe blocks.

* 🔧 fix(tsc): Cast mock.calls via `unknown` for strict tuple destructure

`getSkillByName.mock.calls[0]` is typed as `[]` by jest's generic default;
a direct cast to `[string, ..., ...]` fails TS2352 under `--noEmit` even
though the runtime shape matches. Go through `as unknown as [...]` like
the earlier test in the same file so CI's type-check step stays green.

* 🪢 fix: Propagate skillPrimedIdsByName into handoff agent tool context

Handoff agents go through the same `initializeAgent` flow as the primary
(with `listAlwaysApplySkills` now plumbed), so they resolve their own
`manualSkillPrimes` and `alwaysApplySkillPrimes` — but the
`agentToolContexts.set(...)` for handoff agents didn't carry
`skillPrimedIdsByName` into the per-agent context.

That meant `handleReadFileCall` fell back to the full ACL set + a
`prefer*` flag for handoff agents: same-name collisions could resolve to
a different doc than the one whose body got primed, and a
`disable-model-invocation: true` skill primed via manual `$` or
always-apply inside the handoff flow would be blocked from reading its
own bundled files.

Build the map via `buildSkillPrimedIdsByName(config.manualSkillPrimes,
config.alwaysApplySkillPrimes)` for every handoff tool context so
`read_file` behaves identically across primary and handoff agents.
2026-04-25 04:02:00 -04:00

921 lines
28 KiB
JavaScript

const { nanoid } = require('nanoid');
const { logger } = require('@librechat/data-schemas');
const { Callback, ToolEndHandler, formatAgentMessages } = require('@librechat/agents');
const {
EModelEndpoint,
ResourceType,
PermissionBits,
hasPermissions,
AgentCapabilities,
} = require('librechat-data-provider');
const {
writeSSE,
createRun,
createChunk,
buildToolSet,
scopeSkillIds,
loadSkillStates,
sendFinalChunk,
createSafeUser,
validateRequest,
initializeAgent,
getBalanceConfig,
extractManualSkills,
createErrorResponse,
recordCollectedUsage,
getTransactionsConfig,
resolveRecursionLimit,
createToolExecuteHandler,
buildNonStreamingResponse,
createOpenAIStreamTracker,
createOpenAIContentAggregator,
injectSkillPrimes,
isChatCompletionValidationFailure,
discoverConnectedAgents,
getRemoteAgentPermissions,
} = require('@librechat/api');
const {
buildSummarizationHandlers,
markSummarizationUsage,
createToolEndCallback,
agentLogHandlerObj,
} = require('~/server/controllers/agents/callbacks');
const { loadAgentTools, loadToolsForExecution } = require('~/server/services/ToolService');
const {
findAccessibleResources,
getEffectivePermissions,
} = require('~/server/services/PermissionService');
const {
getSkillToolDeps,
enrichWithSkillConfigurable,
buildSkillPrimedIdsByName,
} = require('~/server/services/Endpoints/agents/skillDeps');
const { getModelsConfig } = require('~/server/controllers/ModelController');
const { logViolation } = require('~/cache');
const db = require('~/models');
/**
* Creates a tool loader function for the agent.
* @param {AbortSignal} signal - The abort signal
* @param {boolean} [definitionsOnly=true] - When true, returns only serializable
* tool definitions without creating full tool instances (for event-driven mode)
*/
function createToolLoader(signal, definitionsOnly = true) {
return async function loadTools({
req,
res,
tools,
model,
agentId,
provider,
tool_options,
tool_resources,
}) {
const agent = { id: agentId, tools, provider, model, tool_options };
try {
return await loadAgentTools({
req,
res,
agent,
signal,
tool_resources,
definitionsOnly,
streamId: null, // No resumable stream for OpenAI compat
});
} catch (error) {
logger.error('Error loading tools for agent ' + agentId, error);
}
};
}
/**
* Convert content part to internal format
* @param {Object} part - Content part
* @returns {Object} Converted part
*/
function convertContentPart(part) {
if (part.type === 'text') {
return { type: 'text', text: part.text };
}
if (part.type === 'image_url') {
return { type: 'image_url', image_url: part.image_url };
}
return part;
}
/**
* Convert OpenAI messages to internal format
* @param {Array} messages - OpenAI format messages
* @returns {Array} Internal format messages
*/
function convertMessages(messages) {
return messages.map((msg) => {
let content;
if (typeof msg.content === 'string') {
content = msg.content;
} else if (msg.content) {
content = msg.content.map(convertContentPart);
} else {
content = '';
}
return {
role: msg.role,
content,
...(msg.name && { name: msg.name }),
...(msg.tool_calls && { tool_calls: msg.tool_calls }),
...(msg.tool_call_id && { tool_call_id: msg.tool_call_id }),
};
});
}
/**
* Send an error response in OpenAI format
*/
function sendErrorResponse(res, statusCode, message, type = 'invalid_request_error', code = null) {
res.status(statusCode).json(createErrorResponse(message, type, code));
}
/**
* OpenAI-compatible chat completions controller for agents.
*
* POST /v1/chat/completions
*
* Request format:
* {
* "model": "agent_id_here",
* "messages": [{"role": "user", "content": "Hello!"}],
* "stream": true,
* "conversation_id": "optional",
* "parent_message_id": "optional"
* }
*/
const OpenAIChatCompletionController = async (req, res) => {
const appConfig = req.config;
const requestStartTime = Date.now();
const validation = validateRequest(req.body);
if (isChatCompletionValidationFailure(validation)) {
return sendErrorResponse(res, 400, validation.error);
}
const request = validation.request;
const agentId = request.model;
// Look up the agent
const agent = await db.getAgent({ id: agentId });
if (!agent) {
return sendErrorResponse(
res,
404,
`Agent not found: ${agentId}`,
'invalid_request_error',
'model_not_found',
);
}
const responseId = `chatcmpl-${nanoid()}`;
const created = Math.floor(Date.now() / 1000);
/** @type {import('@librechat/api').OpenAIResponseContext} — key must be `requestId` to match the type used by createChunk/buildNonStreamingResponse */
const context = {
created,
requestId: responseId,
model: agentId,
};
logger.debug(
`[OpenAI API] Response ${responseId} started for agent ${agentId}, stream: ${request.stream}`,
);
// Set up abort controller
const abortController = new AbortController();
// Handle client disconnect
req.on('close', () => {
if (!abortController.signal.aborted) {
abortController.abort();
logger.debug('[OpenAI API] Client disconnected, aborting');
}
});
try {
if (request.conversation_id != null) {
if (typeof request.conversation_id !== 'string') {
return sendErrorResponse(
res,
400,
'conversation_id must be a string',
'invalid_request_error',
);
}
if (!(await db.getConvo(req.user?.id, request.conversation_id))) {
return sendErrorResponse(res, 404, 'Conversation not found', 'invalid_request_error');
}
}
const conversationId = request.conversation_id ?? nanoid();
const parentMessageId = request.parent_message_id ?? null;
const agentsEConfig = appConfig?.endpoints?.[EModelEndpoint.agents];
const allowedProviders = new Set(agentsEConfig?.allowedProviders);
// Create tool loader
const loadTools = createToolLoader(abortController.signal);
// Initialize the agent first to check for disableStreaming
const endpointOption = {
endpoint: agent.provider,
model_parameters: agent.model_parameters ?? {},
};
// `filterFilesByAgentAccess` is intentionally omitted: it calls
// `checkPermission` with `resourceType: AGENT`, but this route
// authorizes callers through `REMOTE_AGENT` (via
// `getRemoteAgentPermissions`), so including it would silently drop
// owner-attached context files for any remote user who has
// `REMOTE_AGENT_VIEWER` but not direct `AGENT_VIEW`.
const dbMethods = {
getConvoFiles: db.getConvoFiles,
getFiles: db.getFiles,
getUserKey: db.getUserKey,
getMessages: db.getMessages,
updateFilesUsage: db.updateFilesUsage,
getUserKeyValues: db.getUserKeyValues,
getUserCodeFiles: db.getUserCodeFiles,
getToolFilesByIds: db.getToolFilesByIds,
getCodeGeneratedFiles: db.getCodeGeneratedFiles,
listSkillsByAccess: db.listSkillsByAccess,
listAlwaysApplySkills: db.listAlwaysApplySkills,
getSkillByName: db.getSkillByName,
};
const enabledCapabilities = new Set(agentsEConfig?.capabilities);
const skillsCapabilityEnabled = enabledCapabilities.has(AgentCapabilities.skills);
const ephemeralSkillsToggle = req.body?.ephemeralAgent?.skills === true;
const accessibleSkillIds = skillsCapabilityEnabled
? await findAccessibleResources({
userId: req.user.id,
role: req.user.role,
resourceType: ResourceType.SKILL,
requiredPermissions: PermissionBits.VIEW,
})
: [];
const { skillStates, defaultActiveOnShare } = await loadSkillStates({
userId: req.user.id,
appConfig,
getUserById: db.getUserById,
accessibleSkillIds,
});
const manualSkills = extractManualSkills(req.body);
const primaryConfig = await initializeAgent(
{
req,
res,
loadTools,
requestFiles: [],
conversationId,
parentMessageId,
agent,
endpointOption,
allowedProviders,
isInitialAgent: true,
accessibleSkillIds: scopeSkillIds(
accessibleSkillIds,
ephemeralSkillsToggle ? undefined : agent.skills,
),
codeEnvAvailable: enabledCapabilities.has(AgentCapabilities.execute_code),
skillStates,
defaultActiveOnShare,
manualSkills,
},
dbMethods,
);
/**
* Per-agent tool-execution context map, keyed by agentId.
* Needed so the ON_TOOL_EXECUTE callback routes each sub-agent's tool calls
* to the correct toolRegistry / userMCPAuthMap / tool_resources.
* @type {Map<string, {
* agent: object,
* toolRegistry?: import('@librechat/agents').LCToolRegistry,
* userMCPAuthMap?: Record<string, Record<string, string>>,
* tool_resources?: object,
* actionsEnabled?: boolean,
* }>}
*/
const agentToolContexts = new Map();
agentToolContexts.set(primaryConfig.id, {
agent,
toolRegistry: primaryConfig.toolRegistry,
userMCPAuthMap: primaryConfig.userMCPAuthMap,
tool_resources: primaryConfig.tool_resources,
actionsEnabled: primaryConfig.actionsEnabled,
});
// Only run BFS discovery (and pay `getModelsConfig` upfront) when the
// primary has edges to follow — the common API case is single-agent.
let handoffAgentConfigs = new Map();
let discoveredEdges = [];
let discoveredMCPAuthMap;
if (primaryConfig.edges?.length) {
const modelsConfig = await getModelsConfig(req);
({
agentConfigs: handoffAgentConfigs,
edges: discoveredEdges,
userMCPAuthMap: discoveredMCPAuthMap,
} = await discoverConnectedAgents(
{
req,
res,
primaryConfig,
endpointOption,
allowedProviders,
modelsConfig,
loadTools,
requestFiles: [],
conversationId,
parentMessageId,
// The route enforces REMOTE_AGENT on the primary; every discovered
// sub-agent must clear the same sharing boundary, not the looser
// in-app AGENT one.
resourceType: ResourceType.REMOTE_AGENT,
},
{
getAgent: db.getAgent,
// Use `getRemoteAgentPermissions` so sub-agent authorization
// matches what the route's `createCheckRemoteAgentAccess`
// middleware does for the primary: AGENT owners with the SHARE
// bit are treated as remotely authorized even without an
// explicit REMOTE_AGENT grant.
checkPermission: async ({ userId, role, resourceId, requiredPermission }) => {
const permissions = await getRemoteAgentPermissions(
{ getEffectivePermissions },
userId,
role,
resourceId,
);
return hasPermissions(permissions, requiredPermission);
},
logViolation,
db: dbMethods,
onAgentInitialized: (agentId, handoffAgent, config) => {
agentToolContexts.set(agentId, {
agent: handoffAgent,
toolRegistry: config.toolRegistry,
userMCPAuthMap: config.userMCPAuthMap,
tool_resources: config.tool_resources,
actionsEnabled: config.actionsEnabled,
});
},
initializeAgent,
},
));
}
primaryConfig.edges = discoveredEdges;
// Determine if streaming is enabled (check both request and agent config)
const streamingDisabled = !!primaryConfig.model_parameters?.disableStreaming;
const isStreaming = request.stream === true && !streamingDisabled;
// Create tracker for streaming or aggregator for non-streaming
const tracker = isStreaming ? createOpenAIStreamTracker() : null;
const aggregator = isStreaming ? null : createOpenAIContentAggregator();
// Set up response for streaming
if (isStreaming) {
res.setHeader('Content-Type', 'text/event-stream');
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Connection', 'keep-alive');
res.setHeader('X-Accel-Buffering', 'no');
res.flushHeaders();
// Send initial chunk with role
const initialChunk = createChunk(context, { role: 'assistant' });
writeSSE(res, initialChunk);
}
// Create handler config for OpenAI streaming (only used when streaming)
const handlerConfig = isStreaming
? {
res,
context,
tracker,
}
: null;
const collectedUsage = [];
/** @type {Promise<import('librechat-data-provider').TAttachment | null>[]} */
const artifactPromises = [];
const toolEndCallback = createToolEndCallback({ req, res, artifactPromises, streamId: null });
const toolExecuteOptions = {
loadTools: async (toolNames, agentId) => {
const ctx = agentToolContexts.get(agentId) ?? agentToolContexts.get(primaryConfig.id) ?? {};
const result = await loadToolsForExecution({
req,
res,
toolNames,
agent: ctx.agent ?? agent,
signal: abortController.signal,
toolRegistry: ctx.toolRegistry,
userMCPAuthMap: ctx.userMCPAuthMap,
tool_resources: ctx.tool_resources,
actionsEnabled: ctx.actionsEnabled,
});
return enrichWithSkillConfigurable(
result,
req,
primaryConfig.accessibleSkillIds,
undefined,
buildSkillPrimedIdsByName(
primaryConfig.manualSkillPrimes,
primaryConfig.alwaysApplySkillPrimes,
),
);
},
toolEndCallback,
...getSkillToolDeps(),
};
const summarizationConfig = appConfig?.summarization;
const openaiMessages = convertMessages(request.messages);
const toolSet = buildToolSet(primaryConfig);
const formatted = formatAgentMessages(openaiMessages, {}, toolSet);
const formattedMessages = formatted.messages;
const initialSummary = formatted.summary;
let indexTokenCountMap = formatted.indexTokenCountMap;
/**
* Inject manual + always-apply skill primes so the model sees SKILL.md
* bodies for this turn — parity with AgentClient's chat path. OpenAI-
* compatible streaming uses its own tracker/aggregator shape, so the
* LibreChat-style card SSE events don't apply here; only the
* message-context part carries over.
*/
const manualSkillPrimes = primaryConfig.manualSkillPrimes;
const alwaysApplySkillPrimes = primaryConfig.alwaysApplySkillPrimes;
if (
(manualSkillPrimes && manualSkillPrimes.length > 0) ||
(alwaysApplySkillPrimes && alwaysApplySkillPrimes.length > 0)
) {
const primeResult = injectSkillPrimes({
initialMessages: formattedMessages,
indexTokenCountMap,
manualSkillPrimes,
alwaysApplySkillPrimes,
});
indexTokenCountMap = primeResult.indexTokenCountMap;
}
/**
* Create a simple handler that processes data
*/
const createHandler = (processor) => ({
handle: (_event, data) => {
if (processor) {
processor(data);
}
},
});
/**
* Stream text content in OpenAI format
*/
const streamText = (text) => {
if (!text) {
return;
}
if (isStreaming) {
tracker.addText();
writeSSE(res, createChunk(context, { content: text }));
} else {
aggregator.addText(text);
}
};
/**
* Stream reasoning content in OpenAI format (OpenRouter convention)
*/
const streamReasoning = (text) => {
if (!text) {
return;
}
if (isStreaming) {
tracker.addReasoning();
writeSSE(res, createChunk(context, { reasoning: text }));
} else {
aggregator.addReasoning(text);
}
};
// Event handlers for OpenAI-compatible streaming
const handlers = {
// Text content streaming
on_message_delta: createHandler((data) => {
const content = data?.delta?.content;
if (Array.isArray(content)) {
for (const part of content) {
if (part.type === 'text' && part.text) {
streamText(part.text);
}
}
}
}),
// Reasoning/thinking content streaming
on_reasoning_delta: createHandler((data) => {
const content = data?.delta?.content;
if (Array.isArray(content)) {
for (const part of content) {
const text = part.think || part.text;
if (text) {
streamReasoning(text);
}
}
}
}),
// Tool call initiation - streams id and name (from on_run_step)
on_run_step: createHandler((data) => {
const stepDetails = data?.stepDetails;
if (stepDetails?.type === 'tool_calls' && stepDetails.tool_calls) {
for (const tc of stepDetails.tool_calls) {
const toolIndex = data.index ?? 0;
const toolId = tc.id ?? '';
const toolName = tc.name ?? '';
const toolCall = {
id: toolId,
type: 'function',
function: { name: toolName, arguments: '' },
};
// Track tool call in tracker or aggregator
if (isStreaming) {
if (!tracker.toolCalls.has(toolIndex)) {
tracker.toolCalls.set(toolIndex, toolCall);
}
// Stream initial tool call chunk (like OpenAI does)
writeSSE(
res,
createChunk(context, {
tool_calls: [{ index: toolIndex, ...toolCall }],
}),
);
} else {
if (!aggregator.toolCalls.has(toolIndex)) {
aggregator.toolCalls.set(toolIndex, toolCall);
}
}
}
}
}),
// Tool call argument streaming (from on_run_step_delta)
on_run_step_delta: createHandler((data) => {
const delta = data?.delta;
if (delta?.type === 'tool_calls' && delta.tool_calls) {
for (const tc of delta.tool_calls) {
const args = tc.args ?? '';
if (!args) {
continue;
}
const toolIndex = tc.index ?? 0;
// Update tool call arguments
const targetMap = isStreaming ? tracker.toolCalls : aggregator.toolCalls;
const tracked = targetMap.get(toolIndex);
if (tracked) {
tracked.function.arguments += args;
}
// Stream argument delta (only for streaming)
if (isStreaming) {
writeSSE(
res,
createChunk(context, {
tool_calls: [
{
index: toolIndex,
function: { arguments: args },
},
],
}),
);
}
}
}
}),
// Usage tracking
on_chat_model_end: {
handle: (_event, data, metadata) => {
const usage = data?.output?.usage_metadata;
if (usage) {
const taggedUsage = markSummarizationUsage(usage, metadata);
collectedUsage.push(taggedUsage);
const target = isStreaming ? tracker : aggregator;
target.usage.promptTokens += taggedUsage.input_tokens ?? 0;
target.usage.completionTokens += taggedUsage.output_tokens ?? 0;
}
},
},
on_run_step_completed: createHandler(),
// Use proper ToolEndHandler for processing artifacts (images, file citations, code output)
on_tool_end: new ToolEndHandler(toolEndCallback, logger),
on_chain_stream: createHandler(),
on_chain_end: createHandler(),
on_agent_update: createHandler(),
on_agent_log: agentLogHandlerObj,
on_custom_event: createHandler(),
on_tool_execute: createToolExecuteHandler(toolExecuteOptions),
...(summarizationConfig?.enabled !== false
? buildSummarizationHandlers({ isStreaming, res })
: {}),
};
// Create and run the agent
const userId = req.user?.id ?? 'api-user';
// Extract merged userMCPAuthMap (needed for MCP tool connections across
// the primary and any discovered handoff sub-agents)
const userMCPAuthMap = discoveredMCPAuthMap ?? primaryConfig.userMCPAuthMap;
const runAgents = [primaryConfig, ...handoffAgentConfigs.values()];
const run = await createRun({
agents: runAgents,
messages: formattedMessages,
indexTokenCountMap,
initialSummary,
runId: responseId,
summarizationConfig,
appConfig,
signal: abortController.signal,
customHandlers: handlers,
requestBody: {
messageId: responseId,
conversationId,
},
user: { id: userId },
});
if (!run) {
throw new Error('Failed to create agent run');
}
const config = {
runName: 'AgentRun',
configurable: {
thread_id: conversationId,
user_id: userId,
user: createSafeUser(req.user),
requestBody: {
messageId: responseId,
conversationId,
},
...(userMCPAuthMap != null && { userMCPAuthMap }),
},
recursionLimit: resolveRecursionLimit(agentsEConfig, agent),
signal: abortController.signal,
streamMode: 'values',
version: 'v2',
};
await run.processStream({ messages: formattedMessages }, config, {
callbacks: {
[Callback.TOOL_ERROR]: (graph, error, toolId) => {
logger.error(`[OpenAI API] Tool Error "${toolId}"`, error);
},
},
});
// Record token usage against balance
const balanceConfig = getBalanceConfig(appConfig);
const transactionsConfig = getTransactionsConfig(appConfig);
recordCollectedUsage(
{
spendTokens: db.spendTokens,
spendStructuredTokens: db.spendStructuredTokens,
pricing: { getMultiplier: db.getMultiplier, getCacheMultiplier: db.getCacheMultiplier },
bulkWriteOps: { insertMany: db.bulkInsertTransactions, updateBalance: db.updateBalance },
},
{
user: userId,
conversationId,
collectedUsage,
context: 'message',
messageId: responseId,
balance: balanceConfig,
transactions: transactionsConfig,
model: primaryConfig.model || agent.model_parameters?.model,
},
).catch((err) => {
logger.error('[OpenAI API] Error recording usage:', err);
});
// Finalize response
const duration = Date.now() - requestStartTime;
if (isStreaming) {
sendFinalChunk(handlerConfig);
res.end();
logger.debug(`[OpenAI API] Response ${responseId} completed in ${duration}ms (streaming)`);
// Wait for artifact processing after response ends (non-blocking)
if (artifactPromises.length > 0) {
Promise.all(artifactPromises).catch((artifactError) => {
logger.warn('[OpenAI API] Error processing artifacts:', artifactError);
});
}
} else {
// For non-streaming, wait for artifacts before sending response
if (artifactPromises.length > 0) {
try {
await Promise.all(artifactPromises);
} catch (artifactError) {
logger.warn('[OpenAI API] Error processing artifacts:', artifactError);
}
}
// Build usage from aggregated data
const usage = {
prompt_tokens: aggregator.usage.promptTokens,
completion_tokens: aggregator.usage.completionTokens,
total_tokens: aggregator.usage.promptTokens + aggregator.usage.completionTokens,
};
if (aggregator.usage.reasoningTokens > 0) {
usage.completion_tokens_details = {
reasoning_tokens: aggregator.usage.reasoningTokens,
};
}
const response = buildNonStreamingResponse(
context,
aggregator.getText(),
aggregator.getReasoning(),
aggregator.toolCalls,
usage,
);
res.json(response);
logger.debug(
`[OpenAI API] Response ${responseId} completed in ${duration}ms (non-streaming)`,
);
}
} catch (error) {
const errorMessage = error instanceof Error ? error.message : 'An error occurred';
logger.error('[OpenAI API] Error:', error);
// Check if we already started streaming (headers sent)
if (res.headersSent) {
// Headers already sent, send error in stream
const errorChunk = createChunk(context, { content: `\n\nError: ${errorMessage}` }, 'stop');
writeSSE(res, errorChunk);
writeSSE(res, '[DONE]');
res.end();
} else {
// Forward upstream provider status codes (e.g., Anthropic 400s) instead of masking as 500
const statusCode =
typeof error?.status === 'number' && error.status >= 400 && error.status < 600
? error.status
: 500;
const errorType =
statusCode >= 400 && statusCode < 500 ? 'invalid_request_error' : 'server_error';
sendErrorResponse(res, statusCode, errorMessage, errorType);
}
}
};
/**
* List available agents as models (filtered by remote access permissions)
*
* GET /v1/models
*/
const ListModelsController = async (req, res) => {
try {
const userId = req.user?.id;
const userRole = req.user?.role;
if (!userId) {
return sendErrorResponse(res, 401, 'Authentication required', 'auth_error');
}
// Find agents the user has remote access to (VIEW permission on REMOTE_AGENT)
const accessibleAgentIds = await findAccessibleResources({
userId,
role: userRole,
resourceType: ResourceType.REMOTE_AGENT,
requiredPermissions: PermissionBits.VIEW,
});
// Get the accessible agents
let agents = [];
if (accessibleAgentIds.length > 0) {
agents = await db.getAgents({ _id: { $in: accessibleAgentIds } });
}
const models = agents.map((agent) => ({
id: agent.id,
object: 'model',
created: Math.floor(new Date(agent.createdAt || Date.now()).getTime() / 1000),
owned_by: 'librechat',
permission: [],
root: agent.id,
parent: null,
// LibreChat extensions
name: agent.name,
description: agent.description,
provider: agent.provider,
}));
res.json({
object: 'list',
data: models,
});
} catch (error) {
const errorMessage = error instanceof Error ? error.message : 'Failed to list models';
logger.error('[OpenAI API] Error listing models:', error);
sendErrorResponse(res, 500, errorMessage, 'server_error');
}
};
/**
* Get a specific model/agent (with remote access permission check)
*
* GET /v1/models/:model
*/
const GetModelController = async (req, res) => {
try {
const { model } = req.params;
const userId = req.user?.id;
const userRole = req.user?.role;
if (!userId) {
return sendErrorResponse(res, 401, 'Authentication required', 'auth_error');
}
const agent = await db.getAgent({ id: model });
if (!agent) {
return sendErrorResponse(
res,
404,
`Model not found: ${model}`,
'invalid_request_error',
'model_not_found',
);
}
// Check if user has remote access to this agent
const accessibleAgentIds = await findAccessibleResources({
userId,
role: userRole,
resourceType: ResourceType.REMOTE_AGENT,
requiredPermissions: PermissionBits.VIEW,
});
const hasAccess = accessibleAgentIds.some((id) => id.toString() === agent._id.toString());
if (!hasAccess) {
return sendErrorResponse(
res,
403,
`No remote access to model: ${model}`,
'permission_error',
'access_denied',
);
}
res.json({
id: agent.id,
object: 'model',
created: Math.floor(new Date(agent.createdAt || Date.now()).getTime() / 1000),
owned_by: 'librechat',
permission: [],
root: agent.id,
parent: null,
// LibreChat extensions
name: agent.name,
description: agent.description,
provider: agent.provider,
});
} catch (error) {
const errorMessage = error instanceof Error ? error.message : 'Failed to get model';
logger.error('[OpenAI API] Error getting model:', error);
sendErrorResponse(res, 500, errorMessage, 'server_error');
}
};
module.exports = {
OpenAIChatCompletionController,
ListModelsController,
GetModelController,
};