mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-08-31 08:56:48 +00:00
* 🔁 refactor: Rebase always-apply work onto merged structured-frontmatter columns Phase 6 (disable-model-invocation / user-invocable / allowed-tools) landed first on feat/agent-skills. Reconcile this branch with the new mainline: - Thread alwaysApplySkillPrimes through unionPrimeAllowedTools alongside manualSkillPrimes, applying the combined MAX_PRIMED_SKILLS_PER_TURN ceiling before loading tools. - Add `_id` to ResolvedAlwaysApplySkill to match Phase 6's ResolvedManualSkill shape (read_file name-collision protection). - Register 'always-apply' in ALLOWED_FRONTMATTER_KEYS / FRONTMATTER_KIND so Phase 6's validator recognizes it. - Drop frontmatter from the listSkillsByAccess projection; the backfill helper remains as defensive code but its read path is no longer exercised on summary rows (no legacy rows exist — the branch never shipped), saving ~200KB per page. - Retire the corresponding "backfills legacy on summaries" test. - Plumb listAlwaysApplySkills through the JS controllers + endpoint initializer so the always-apply resolver sees a real DB method. * 🧹 fix: Dedupe manual/always-apply overlap, share YAML util, tidy comments Addresses review findings: - Cross-list dedup: when a user $-invokes a skill that is also marked always-apply, the always-apply copy is now dropped so the same SKILL.md body never primes twice in one turn. Manual wins (explicit intent, closer to the user message). Dedup runs in both initializeAgent (so persisted user-bubble pills stay in sync) and injectSkillPrimes (defense-in-depth at splice time). New test cases cover single-overlap, partial-overlap, and dedup-before-cap. - DRY: extract stripYamlTrailingComment to packages/data-schemas/src/utils/yaml.ts; packages/api/src/skills/import.ts now imports the shared helper. Also drop the redundant inner stripYamlTrailingComment call inside parseBooleanScalar — the call site already strips. - Mark injectManualSkillPrimes as @deprecated in favor of injectSkillPrimes (kept for external consumers of @librechat/api). - Document SKILL_TRIGGER_MODEL as forward-looking plumbing for the model-invoked path rather than leaving it as a bare unused export. - Replace the stale "frontmatter is included" comment on listSkillsByAccess with an accurate explanation of why it was intentionally excluded. * 🔒 fix: Include always-apply primes in skillPrimedIdsByName + clear alwaysApply on body opt-out Two bugs flagged by Codex review: P1 (read_file): `manualSkillPrimedIdsByName` only carried manual-invocation primes, so an always-apply skill with `disable-model-invocation: true` was blocked from reading its own bundled files, and same-name collisions could resolve to a different doc than the one whose body got primed. - Rename `buildManualSkillPrimedIdsByName` → `buildSkillPrimedIdsByName` (accepts both manual + always-apply prime arrays). - Rename the configurable field `manualSkillPrimedIdsByName` → `skillPrimedIdsByName` throughout the plumbing (skillConfigurable.ts, handlers.ts, CJS callers, tests). - Overlap resolution: manual wins on the rare edge case where the same name appears in both arrays (upstream dedup should prevent this, but defensive merging treats manual as authoritative). - New tests: (1) gate-relaxation fires for always-apply primes, (2) `_id` pinning works for always-apply same-name collisions. P2 (updateSkill): when a body update had no `always-apply:` key, `extractAlwaysApplyFromBody` returned `absent` and the column was left untouched. A skill that was once `alwaysApply: true` would keep auto-priming even after its SKILL.md no longer declared the flag. - Treat `absent` as a positive "not always-apply" declaration when the body is explicitly submitted; flip the column to `false`. - Explicit top-level `alwaysApply` still wins (three-source precedence unchanged). - New tests: body removes key → false, body has no frontmatter at all → false, explicit + body-without-key → explicit wins. * 🧵 refactor: Collapse duplicate prime types + tighten parse + test hygiene Sanity-check review follow-ups: - Collapse `ResolvedManualSkill` / `ResolvedAlwaysApplySkill` into a single `ResolvedSkillPrime` canonical interface with two backward- compatible type aliases. Both resolvers feed the same pipeline stages (injectSkillPrimes, unionPrimeAllowedTools, buildSkillPrimedIdsByName); the per-source distinction lives on `additional_kwargs.trigger`, not on the resolver output. - Move the `always-apply` branch in `parseFrontmatter` to operate on the raw post-colon text. The outer `unquoteYaml` was fine today because it's idempotent on non-quoted strings, but running it twice (once per line, once after stripping the inline comment) would be fragile if the unquoter ever grows richer YAML-escape handling. - Add the missing `alwaysApplyDedupedFromManual: 0` field to the `injectSkillPrimes` mocks in `openai.spec.js` and `responses.unit.spec.js` so they match the full `InjectSkillPrimesResult` contract. - Insert the blank line between the `unionPrimeAllowedTools` and `resolveAlwaysApplySkills` describe blocks. * 🔧 fix(tsc): Cast mock.calls via `unknown` for strict tuple destructure `getSkillByName.mock.calls[0]` is typed as `[]` by jest's generic default; a direct cast to `[string, ..., ...]` fails TS2352 under `--noEmit` even though the runtime shape matches. Go through `as unknown as [...]` like the earlier test in the same file so CI's type-check step stays green. * 🪢 fix: Propagate skillPrimedIdsByName into handoff agent tool context Handoff agents go through the same `initializeAgent` flow as the primary (with `listAlwaysApplySkills` now plumbed), so they resolve their own `manualSkillPrimes` and `alwaysApplySkillPrimes` — but the `agentToolContexts.set(...)` for handoff agents didn't carry `skillPrimedIdsByName` into the per-agent context. That meant `handleReadFileCall` fell back to the full ACL set + a `prefer*` flag for handoff agents: same-name collisions could resolve to a different doc than the one whose body got primed, and a `disable-model-invocation: true` skill primed via manual `$` or always-apply inside the handoff flow would be blocked from reading its own bundled files. Build the map via `buildSkillPrimedIdsByName(config.manualSkillPrimes, config.alwaysApplySkillPrimes)` for every handoff tool context so `read_file` behaves identically across primary and handoff agents.
921 lines
28 KiB
JavaScript
921 lines
28 KiB
JavaScript
const { nanoid } = require('nanoid');
|
|
const { logger } = require('@librechat/data-schemas');
|
|
const { Callback, ToolEndHandler, formatAgentMessages } = require('@librechat/agents');
|
|
const {
|
|
EModelEndpoint,
|
|
ResourceType,
|
|
PermissionBits,
|
|
hasPermissions,
|
|
AgentCapabilities,
|
|
} = require('librechat-data-provider');
|
|
const {
|
|
writeSSE,
|
|
createRun,
|
|
createChunk,
|
|
buildToolSet,
|
|
scopeSkillIds,
|
|
loadSkillStates,
|
|
sendFinalChunk,
|
|
createSafeUser,
|
|
validateRequest,
|
|
initializeAgent,
|
|
getBalanceConfig,
|
|
extractManualSkills,
|
|
createErrorResponse,
|
|
recordCollectedUsage,
|
|
getTransactionsConfig,
|
|
resolveRecursionLimit,
|
|
createToolExecuteHandler,
|
|
buildNonStreamingResponse,
|
|
createOpenAIStreamTracker,
|
|
createOpenAIContentAggregator,
|
|
injectSkillPrimes,
|
|
isChatCompletionValidationFailure,
|
|
discoverConnectedAgents,
|
|
getRemoteAgentPermissions,
|
|
} = require('@librechat/api');
|
|
const {
|
|
buildSummarizationHandlers,
|
|
markSummarizationUsage,
|
|
createToolEndCallback,
|
|
agentLogHandlerObj,
|
|
} = require('~/server/controllers/agents/callbacks');
|
|
const { loadAgentTools, loadToolsForExecution } = require('~/server/services/ToolService');
|
|
const {
|
|
findAccessibleResources,
|
|
getEffectivePermissions,
|
|
} = require('~/server/services/PermissionService');
|
|
const {
|
|
getSkillToolDeps,
|
|
enrichWithSkillConfigurable,
|
|
buildSkillPrimedIdsByName,
|
|
} = require('~/server/services/Endpoints/agents/skillDeps');
|
|
const { getModelsConfig } = require('~/server/controllers/ModelController');
|
|
const { logViolation } = require('~/cache');
|
|
const db = require('~/models');
|
|
|
|
/**
|
|
* Creates a tool loader function for the agent.
|
|
* @param {AbortSignal} signal - The abort signal
|
|
* @param {boolean} [definitionsOnly=true] - When true, returns only serializable
|
|
* tool definitions without creating full tool instances (for event-driven mode)
|
|
*/
|
|
function createToolLoader(signal, definitionsOnly = true) {
|
|
return async function loadTools({
|
|
req,
|
|
res,
|
|
tools,
|
|
model,
|
|
agentId,
|
|
provider,
|
|
tool_options,
|
|
tool_resources,
|
|
}) {
|
|
const agent = { id: agentId, tools, provider, model, tool_options };
|
|
try {
|
|
return await loadAgentTools({
|
|
req,
|
|
res,
|
|
agent,
|
|
signal,
|
|
tool_resources,
|
|
definitionsOnly,
|
|
streamId: null, // No resumable stream for OpenAI compat
|
|
});
|
|
} catch (error) {
|
|
logger.error('Error loading tools for agent ' + agentId, error);
|
|
}
|
|
};
|
|
}
|
|
|
|
/**
|
|
* Convert content part to internal format
|
|
* @param {Object} part - Content part
|
|
* @returns {Object} Converted part
|
|
*/
|
|
function convertContentPart(part) {
|
|
if (part.type === 'text') {
|
|
return { type: 'text', text: part.text };
|
|
}
|
|
if (part.type === 'image_url') {
|
|
return { type: 'image_url', image_url: part.image_url };
|
|
}
|
|
return part;
|
|
}
|
|
|
|
/**
|
|
* Convert OpenAI messages to internal format
|
|
* @param {Array} messages - OpenAI format messages
|
|
* @returns {Array} Internal format messages
|
|
*/
|
|
function convertMessages(messages) {
|
|
return messages.map((msg) => {
|
|
let content;
|
|
if (typeof msg.content === 'string') {
|
|
content = msg.content;
|
|
} else if (msg.content) {
|
|
content = msg.content.map(convertContentPart);
|
|
} else {
|
|
content = '';
|
|
}
|
|
|
|
return {
|
|
role: msg.role,
|
|
content,
|
|
...(msg.name && { name: msg.name }),
|
|
...(msg.tool_calls && { tool_calls: msg.tool_calls }),
|
|
...(msg.tool_call_id && { tool_call_id: msg.tool_call_id }),
|
|
};
|
|
});
|
|
}
|
|
|
|
/**
|
|
* Send an error response in OpenAI format
|
|
*/
|
|
function sendErrorResponse(res, statusCode, message, type = 'invalid_request_error', code = null) {
|
|
res.status(statusCode).json(createErrorResponse(message, type, code));
|
|
}
|
|
|
|
/**
|
|
* OpenAI-compatible chat completions controller for agents.
|
|
*
|
|
* POST /v1/chat/completions
|
|
*
|
|
* Request format:
|
|
* {
|
|
* "model": "agent_id_here",
|
|
* "messages": [{"role": "user", "content": "Hello!"}],
|
|
* "stream": true,
|
|
* "conversation_id": "optional",
|
|
* "parent_message_id": "optional"
|
|
* }
|
|
*/
|
|
const OpenAIChatCompletionController = async (req, res) => {
|
|
const appConfig = req.config;
|
|
const requestStartTime = Date.now();
|
|
|
|
const validation = validateRequest(req.body);
|
|
if (isChatCompletionValidationFailure(validation)) {
|
|
return sendErrorResponse(res, 400, validation.error);
|
|
}
|
|
|
|
const request = validation.request;
|
|
const agentId = request.model;
|
|
|
|
// Look up the agent
|
|
const agent = await db.getAgent({ id: agentId });
|
|
if (!agent) {
|
|
return sendErrorResponse(
|
|
res,
|
|
404,
|
|
`Agent not found: ${agentId}`,
|
|
'invalid_request_error',
|
|
'model_not_found',
|
|
);
|
|
}
|
|
|
|
const responseId = `chatcmpl-${nanoid()}`;
|
|
const created = Math.floor(Date.now() / 1000);
|
|
|
|
/** @type {import('@librechat/api').OpenAIResponseContext} — key must be `requestId` to match the type used by createChunk/buildNonStreamingResponse */
|
|
const context = {
|
|
created,
|
|
requestId: responseId,
|
|
model: agentId,
|
|
};
|
|
|
|
logger.debug(
|
|
`[OpenAI API] Response ${responseId} started for agent ${agentId}, stream: ${request.stream}`,
|
|
);
|
|
|
|
// Set up abort controller
|
|
const abortController = new AbortController();
|
|
|
|
// Handle client disconnect
|
|
req.on('close', () => {
|
|
if (!abortController.signal.aborted) {
|
|
abortController.abort();
|
|
logger.debug('[OpenAI API] Client disconnected, aborting');
|
|
}
|
|
});
|
|
|
|
try {
|
|
if (request.conversation_id != null) {
|
|
if (typeof request.conversation_id !== 'string') {
|
|
return sendErrorResponse(
|
|
res,
|
|
400,
|
|
'conversation_id must be a string',
|
|
'invalid_request_error',
|
|
);
|
|
}
|
|
if (!(await db.getConvo(req.user?.id, request.conversation_id))) {
|
|
return sendErrorResponse(res, 404, 'Conversation not found', 'invalid_request_error');
|
|
}
|
|
}
|
|
|
|
const conversationId = request.conversation_id ?? nanoid();
|
|
const parentMessageId = request.parent_message_id ?? null;
|
|
|
|
const agentsEConfig = appConfig?.endpoints?.[EModelEndpoint.agents];
|
|
const allowedProviders = new Set(agentsEConfig?.allowedProviders);
|
|
|
|
// Create tool loader
|
|
const loadTools = createToolLoader(abortController.signal);
|
|
|
|
// Initialize the agent first to check for disableStreaming
|
|
const endpointOption = {
|
|
endpoint: agent.provider,
|
|
model_parameters: agent.model_parameters ?? {},
|
|
};
|
|
|
|
// `filterFilesByAgentAccess` is intentionally omitted: it calls
|
|
// `checkPermission` with `resourceType: AGENT`, but this route
|
|
// authorizes callers through `REMOTE_AGENT` (via
|
|
// `getRemoteAgentPermissions`), so including it would silently drop
|
|
// owner-attached context files for any remote user who has
|
|
// `REMOTE_AGENT_VIEWER` but not direct `AGENT_VIEW`.
|
|
const dbMethods = {
|
|
getConvoFiles: db.getConvoFiles,
|
|
getFiles: db.getFiles,
|
|
getUserKey: db.getUserKey,
|
|
getMessages: db.getMessages,
|
|
updateFilesUsage: db.updateFilesUsage,
|
|
getUserKeyValues: db.getUserKeyValues,
|
|
getUserCodeFiles: db.getUserCodeFiles,
|
|
getToolFilesByIds: db.getToolFilesByIds,
|
|
getCodeGeneratedFiles: db.getCodeGeneratedFiles,
|
|
listSkillsByAccess: db.listSkillsByAccess,
|
|
listAlwaysApplySkills: db.listAlwaysApplySkills,
|
|
getSkillByName: db.getSkillByName,
|
|
};
|
|
|
|
const enabledCapabilities = new Set(agentsEConfig?.capabilities);
|
|
const skillsCapabilityEnabled = enabledCapabilities.has(AgentCapabilities.skills);
|
|
const ephemeralSkillsToggle = req.body?.ephemeralAgent?.skills === true;
|
|
const accessibleSkillIds = skillsCapabilityEnabled
|
|
? await findAccessibleResources({
|
|
userId: req.user.id,
|
|
role: req.user.role,
|
|
resourceType: ResourceType.SKILL,
|
|
requiredPermissions: PermissionBits.VIEW,
|
|
})
|
|
: [];
|
|
|
|
const { skillStates, defaultActiveOnShare } = await loadSkillStates({
|
|
userId: req.user.id,
|
|
appConfig,
|
|
getUserById: db.getUserById,
|
|
accessibleSkillIds,
|
|
});
|
|
|
|
const manualSkills = extractManualSkills(req.body);
|
|
|
|
const primaryConfig = await initializeAgent(
|
|
{
|
|
req,
|
|
res,
|
|
loadTools,
|
|
requestFiles: [],
|
|
conversationId,
|
|
parentMessageId,
|
|
agent,
|
|
endpointOption,
|
|
allowedProviders,
|
|
isInitialAgent: true,
|
|
accessibleSkillIds: scopeSkillIds(
|
|
accessibleSkillIds,
|
|
ephemeralSkillsToggle ? undefined : agent.skills,
|
|
),
|
|
codeEnvAvailable: enabledCapabilities.has(AgentCapabilities.execute_code),
|
|
skillStates,
|
|
defaultActiveOnShare,
|
|
manualSkills,
|
|
},
|
|
dbMethods,
|
|
);
|
|
|
|
/**
|
|
* Per-agent tool-execution context map, keyed by agentId.
|
|
* Needed so the ON_TOOL_EXECUTE callback routes each sub-agent's tool calls
|
|
* to the correct toolRegistry / userMCPAuthMap / tool_resources.
|
|
* @type {Map<string, {
|
|
* agent: object,
|
|
* toolRegistry?: import('@librechat/agents').LCToolRegistry,
|
|
* userMCPAuthMap?: Record<string, Record<string, string>>,
|
|
* tool_resources?: object,
|
|
* actionsEnabled?: boolean,
|
|
* }>}
|
|
*/
|
|
const agentToolContexts = new Map();
|
|
agentToolContexts.set(primaryConfig.id, {
|
|
agent,
|
|
toolRegistry: primaryConfig.toolRegistry,
|
|
userMCPAuthMap: primaryConfig.userMCPAuthMap,
|
|
tool_resources: primaryConfig.tool_resources,
|
|
actionsEnabled: primaryConfig.actionsEnabled,
|
|
});
|
|
|
|
// Only run BFS discovery (and pay `getModelsConfig` upfront) when the
|
|
// primary has edges to follow — the common API case is single-agent.
|
|
let handoffAgentConfigs = new Map();
|
|
let discoveredEdges = [];
|
|
let discoveredMCPAuthMap;
|
|
if (primaryConfig.edges?.length) {
|
|
const modelsConfig = await getModelsConfig(req);
|
|
({
|
|
agentConfigs: handoffAgentConfigs,
|
|
edges: discoveredEdges,
|
|
userMCPAuthMap: discoveredMCPAuthMap,
|
|
} = await discoverConnectedAgents(
|
|
{
|
|
req,
|
|
res,
|
|
primaryConfig,
|
|
endpointOption,
|
|
allowedProviders,
|
|
modelsConfig,
|
|
loadTools,
|
|
requestFiles: [],
|
|
conversationId,
|
|
parentMessageId,
|
|
// The route enforces REMOTE_AGENT on the primary; every discovered
|
|
// sub-agent must clear the same sharing boundary, not the looser
|
|
// in-app AGENT one.
|
|
resourceType: ResourceType.REMOTE_AGENT,
|
|
},
|
|
{
|
|
getAgent: db.getAgent,
|
|
// Use `getRemoteAgentPermissions` so sub-agent authorization
|
|
// matches what the route's `createCheckRemoteAgentAccess`
|
|
// middleware does for the primary: AGENT owners with the SHARE
|
|
// bit are treated as remotely authorized even without an
|
|
// explicit REMOTE_AGENT grant.
|
|
checkPermission: async ({ userId, role, resourceId, requiredPermission }) => {
|
|
const permissions = await getRemoteAgentPermissions(
|
|
{ getEffectivePermissions },
|
|
userId,
|
|
role,
|
|
resourceId,
|
|
);
|
|
return hasPermissions(permissions, requiredPermission);
|
|
},
|
|
logViolation,
|
|
db: dbMethods,
|
|
onAgentInitialized: (agentId, handoffAgent, config) => {
|
|
agentToolContexts.set(agentId, {
|
|
agent: handoffAgent,
|
|
toolRegistry: config.toolRegistry,
|
|
userMCPAuthMap: config.userMCPAuthMap,
|
|
tool_resources: config.tool_resources,
|
|
actionsEnabled: config.actionsEnabled,
|
|
});
|
|
},
|
|
initializeAgent,
|
|
},
|
|
));
|
|
}
|
|
|
|
primaryConfig.edges = discoveredEdges;
|
|
|
|
// Determine if streaming is enabled (check both request and agent config)
|
|
const streamingDisabled = !!primaryConfig.model_parameters?.disableStreaming;
|
|
const isStreaming = request.stream === true && !streamingDisabled;
|
|
|
|
// Create tracker for streaming or aggregator for non-streaming
|
|
const tracker = isStreaming ? createOpenAIStreamTracker() : null;
|
|
const aggregator = isStreaming ? null : createOpenAIContentAggregator();
|
|
|
|
// Set up response for streaming
|
|
if (isStreaming) {
|
|
res.setHeader('Content-Type', 'text/event-stream');
|
|
res.setHeader('Cache-Control', 'no-cache');
|
|
res.setHeader('Connection', 'keep-alive');
|
|
res.setHeader('X-Accel-Buffering', 'no');
|
|
res.flushHeaders();
|
|
|
|
// Send initial chunk with role
|
|
const initialChunk = createChunk(context, { role: 'assistant' });
|
|
writeSSE(res, initialChunk);
|
|
}
|
|
|
|
// Create handler config for OpenAI streaming (only used when streaming)
|
|
const handlerConfig = isStreaming
|
|
? {
|
|
res,
|
|
context,
|
|
tracker,
|
|
}
|
|
: null;
|
|
|
|
const collectedUsage = [];
|
|
/** @type {Promise<import('librechat-data-provider').TAttachment | null>[]} */
|
|
const artifactPromises = [];
|
|
|
|
const toolEndCallback = createToolEndCallback({ req, res, artifactPromises, streamId: null });
|
|
|
|
const toolExecuteOptions = {
|
|
loadTools: async (toolNames, agentId) => {
|
|
const ctx = agentToolContexts.get(agentId) ?? agentToolContexts.get(primaryConfig.id) ?? {};
|
|
const result = await loadToolsForExecution({
|
|
req,
|
|
res,
|
|
toolNames,
|
|
agent: ctx.agent ?? agent,
|
|
signal: abortController.signal,
|
|
toolRegistry: ctx.toolRegistry,
|
|
userMCPAuthMap: ctx.userMCPAuthMap,
|
|
tool_resources: ctx.tool_resources,
|
|
actionsEnabled: ctx.actionsEnabled,
|
|
});
|
|
return enrichWithSkillConfigurable(
|
|
result,
|
|
req,
|
|
primaryConfig.accessibleSkillIds,
|
|
undefined,
|
|
buildSkillPrimedIdsByName(
|
|
primaryConfig.manualSkillPrimes,
|
|
primaryConfig.alwaysApplySkillPrimes,
|
|
),
|
|
);
|
|
},
|
|
toolEndCallback,
|
|
...getSkillToolDeps(),
|
|
};
|
|
|
|
const summarizationConfig = appConfig?.summarization;
|
|
|
|
const openaiMessages = convertMessages(request.messages);
|
|
|
|
const toolSet = buildToolSet(primaryConfig);
|
|
const formatted = formatAgentMessages(openaiMessages, {}, toolSet);
|
|
const formattedMessages = formatted.messages;
|
|
const initialSummary = formatted.summary;
|
|
let indexTokenCountMap = formatted.indexTokenCountMap;
|
|
|
|
/**
|
|
* Inject manual + always-apply skill primes so the model sees SKILL.md
|
|
* bodies for this turn — parity with AgentClient's chat path. OpenAI-
|
|
* compatible streaming uses its own tracker/aggregator shape, so the
|
|
* LibreChat-style card SSE events don't apply here; only the
|
|
* message-context part carries over.
|
|
*/
|
|
const manualSkillPrimes = primaryConfig.manualSkillPrimes;
|
|
const alwaysApplySkillPrimes = primaryConfig.alwaysApplySkillPrimes;
|
|
if (
|
|
(manualSkillPrimes && manualSkillPrimes.length > 0) ||
|
|
(alwaysApplySkillPrimes && alwaysApplySkillPrimes.length > 0)
|
|
) {
|
|
const primeResult = injectSkillPrimes({
|
|
initialMessages: formattedMessages,
|
|
indexTokenCountMap,
|
|
manualSkillPrimes,
|
|
alwaysApplySkillPrimes,
|
|
});
|
|
indexTokenCountMap = primeResult.indexTokenCountMap;
|
|
}
|
|
|
|
/**
|
|
* Create a simple handler that processes data
|
|
*/
|
|
const createHandler = (processor) => ({
|
|
handle: (_event, data) => {
|
|
if (processor) {
|
|
processor(data);
|
|
}
|
|
},
|
|
});
|
|
|
|
/**
|
|
* Stream text content in OpenAI format
|
|
*/
|
|
const streamText = (text) => {
|
|
if (!text) {
|
|
return;
|
|
}
|
|
if (isStreaming) {
|
|
tracker.addText();
|
|
writeSSE(res, createChunk(context, { content: text }));
|
|
} else {
|
|
aggregator.addText(text);
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Stream reasoning content in OpenAI format (OpenRouter convention)
|
|
*/
|
|
const streamReasoning = (text) => {
|
|
if (!text) {
|
|
return;
|
|
}
|
|
if (isStreaming) {
|
|
tracker.addReasoning();
|
|
writeSSE(res, createChunk(context, { reasoning: text }));
|
|
} else {
|
|
aggregator.addReasoning(text);
|
|
}
|
|
};
|
|
|
|
// Event handlers for OpenAI-compatible streaming
|
|
const handlers = {
|
|
// Text content streaming
|
|
on_message_delta: createHandler((data) => {
|
|
const content = data?.delta?.content;
|
|
if (Array.isArray(content)) {
|
|
for (const part of content) {
|
|
if (part.type === 'text' && part.text) {
|
|
streamText(part.text);
|
|
}
|
|
}
|
|
}
|
|
}),
|
|
|
|
// Reasoning/thinking content streaming
|
|
on_reasoning_delta: createHandler((data) => {
|
|
const content = data?.delta?.content;
|
|
if (Array.isArray(content)) {
|
|
for (const part of content) {
|
|
const text = part.think || part.text;
|
|
if (text) {
|
|
streamReasoning(text);
|
|
}
|
|
}
|
|
}
|
|
}),
|
|
|
|
// Tool call initiation - streams id and name (from on_run_step)
|
|
on_run_step: createHandler((data) => {
|
|
const stepDetails = data?.stepDetails;
|
|
if (stepDetails?.type === 'tool_calls' && stepDetails.tool_calls) {
|
|
for (const tc of stepDetails.tool_calls) {
|
|
const toolIndex = data.index ?? 0;
|
|
const toolId = tc.id ?? '';
|
|
const toolName = tc.name ?? '';
|
|
const toolCall = {
|
|
id: toolId,
|
|
type: 'function',
|
|
function: { name: toolName, arguments: '' },
|
|
};
|
|
|
|
// Track tool call in tracker or aggregator
|
|
if (isStreaming) {
|
|
if (!tracker.toolCalls.has(toolIndex)) {
|
|
tracker.toolCalls.set(toolIndex, toolCall);
|
|
}
|
|
// Stream initial tool call chunk (like OpenAI does)
|
|
writeSSE(
|
|
res,
|
|
createChunk(context, {
|
|
tool_calls: [{ index: toolIndex, ...toolCall }],
|
|
}),
|
|
);
|
|
} else {
|
|
if (!aggregator.toolCalls.has(toolIndex)) {
|
|
aggregator.toolCalls.set(toolIndex, toolCall);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}),
|
|
|
|
// Tool call argument streaming (from on_run_step_delta)
|
|
on_run_step_delta: createHandler((data) => {
|
|
const delta = data?.delta;
|
|
if (delta?.type === 'tool_calls' && delta.tool_calls) {
|
|
for (const tc of delta.tool_calls) {
|
|
const args = tc.args ?? '';
|
|
if (!args) {
|
|
continue;
|
|
}
|
|
|
|
const toolIndex = tc.index ?? 0;
|
|
|
|
// Update tool call arguments
|
|
const targetMap = isStreaming ? tracker.toolCalls : aggregator.toolCalls;
|
|
const tracked = targetMap.get(toolIndex);
|
|
if (tracked) {
|
|
tracked.function.arguments += args;
|
|
}
|
|
|
|
// Stream argument delta (only for streaming)
|
|
if (isStreaming) {
|
|
writeSSE(
|
|
res,
|
|
createChunk(context, {
|
|
tool_calls: [
|
|
{
|
|
index: toolIndex,
|
|
function: { arguments: args },
|
|
},
|
|
],
|
|
}),
|
|
);
|
|
}
|
|
}
|
|
}
|
|
}),
|
|
|
|
// Usage tracking
|
|
on_chat_model_end: {
|
|
handle: (_event, data, metadata) => {
|
|
const usage = data?.output?.usage_metadata;
|
|
if (usage) {
|
|
const taggedUsage = markSummarizationUsage(usage, metadata);
|
|
collectedUsage.push(taggedUsage);
|
|
const target = isStreaming ? tracker : aggregator;
|
|
target.usage.promptTokens += taggedUsage.input_tokens ?? 0;
|
|
target.usage.completionTokens += taggedUsage.output_tokens ?? 0;
|
|
}
|
|
},
|
|
},
|
|
on_run_step_completed: createHandler(),
|
|
// Use proper ToolEndHandler for processing artifacts (images, file citations, code output)
|
|
on_tool_end: new ToolEndHandler(toolEndCallback, logger),
|
|
on_chain_stream: createHandler(),
|
|
on_chain_end: createHandler(),
|
|
on_agent_update: createHandler(),
|
|
on_agent_log: agentLogHandlerObj,
|
|
on_custom_event: createHandler(),
|
|
on_tool_execute: createToolExecuteHandler(toolExecuteOptions),
|
|
...(summarizationConfig?.enabled !== false
|
|
? buildSummarizationHandlers({ isStreaming, res })
|
|
: {}),
|
|
};
|
|
|
|
// Create and run the agent
|
|
const userId = req.user?.id ?? 'api-user';
|
|
|
|
// Extract merged userMCPAuthMap (needed for MCP tool connections across
|
|
// the primary and any discovered handoff sub-agents)
|
|
const userMCPAuthMap = discoveredMCPAuthMap ?? primaryConfig.userMCPAuthMap;
|
|
|
|
const runAgents = [primaryConfig, ...handoffAgentConfigs.values()];
|
|
|
|
const run = await createRun({
|
|
agents: runAgents,
|
|
messages: formattedMessages,
|
|
indexTokenCountMap,
|
|
initialSummary,
|
|
runId: responseId,
|
|
summarizationConfig,
|
|
appConfig,
|
|
signal: abortController.signal,
|
|
customHandlers: handlers,
|
|
requestBody: {
|
|
messageId: responseId,
|
|
conversationId,
|
|
},
|
|
user: { id: userId },
|
|
});
|
|
|
|
if (!run) {
|
|
throw new Error('Failed to create agent run');
|
|
}
|
|
|
|
const config = {
|
|
runName: 'AgentRun',
|
|
configurable: {
|
|
thread_id: conversationId,
|
|
user_id: userId,
|
|
user: createSafeUser(req.user),
|
|
requestBody: {
|
|
messageId: responseId,
|
|
conversationId,
|
|
},
|
|
...(userMCPAuthMap != null && { userMCPAuthMap }),
|
|
},
|
|
recursionLimit: resolveRecursionLimit(agentsEConfig, agent),
|
|
signal: abortController.signal,
|
|
streamMode: 'values',
|
|
version: 'v2',
|
|
};
|
|
|
|
await run.processStream({ messages: formattedMessages }, config, {
|
|
callbacks: {
|
|
[Callback.TOOL_ERROR]: (graph, error, toolId) => {
|
|
logger.error(`[OpenAI API] Tool Error "${toolId}"`, error);
|
|
},
|
|
},
|
|
});
|
|
|
|
// Record token usage against balance
|
|
const balanceConfig = getBalanceConfig(appConfig);
|
|
const transactionsConfig = getTransactionsConfig(appConfig);
|
|
recordCollectedUsage(
|
|
{
|
|
spendTokens: db.spendTokens,
|
|
spendStructuredTokens: db.spendStructuredTokens,
|
|
pricing: { getMultiplier: db.getMultiplier, getCacheMultiplier: db.getCacheMultiplier },
|
|
bulkWriteOps: { insertMany: db.bulkInsertTransactions, updateBalance: db.updateBalance },
|
|
},
|
|
{
|
|
user: userId,
|
|
conversationId,
|
|
collectedUsage,
|
|
context: 'message',
|
|
messageId: responseId,
|
|
balance: balanceConfig,
|
|
transactions: transactionsConfig,
|
|
model: primaryConfig.model || agent.model_parameters?.model,
|
|
},
|
|
).catch((err) => {
|
|
logger.error('[OpenAI API] Error recording usage:', err);
|
|
});
|
|
|
|
// Finalize response
|
|
const duration = Date.now() - requestStartTime;
|
|
if (isStreaming) {
|
|
sendFinalChunk(handlerConfig);
|
|
res.end();
|
|
logger.debug(`[OpenAI API] Response ${responseId} completed in ${duration}ms (streaming)`);
|
|
|
|
// Wait for artifact processing after response ends (non-blocking)
|
|
if (artifactPromises.length > 0) {
|
|
Promise.all(artifactPromises).catch((artifactError) => {
|
|
logger.warn('[OpenAI API] Error processing artifacts:', artifactError);
|
|
});
|
|
}
|
|
} else {
|
|
// For non-streaming, wait for artifacts before sending response
|
|
if (artifactPromises.length > 0) {
|
|
try {
|
|
await Promise.all(artifactPromises);
|
|
} catch (artifactError) {
|
|
logger.warn('[OpenAI API] Error processing artifacts:', artifactError);
|
|
}
|
|
}
|
|
|
|
// Build usage from aggregated data
|
|
const usage = {
|
|
prompt_tokens: aggregator.usage.promptTokens,
|
|
completion_tokens: aggregator.usage.completionTokens,
|
|
total_tokens: aggregator.usage.promptTokens + aggregator.usage.completionTokens,
|
|
};
|
|
|
|
if (aggregator.usage.reasoningTokens > 0) {
|
|
usage.completion_tokens_details = {
|
|
reasoning_tokens: aggregator.usage.reasoningTokens,
|
|
};
|
|
}
|
|
|
|
const response = buildNonStreamingResponse(
|
|
context,
|
|
aggregator.getText(),
|
|
aggregator.getReasoning(),
|
|
aggregator.toolCalls,
|
|
usage,
|
|
);
|
|
res.json(response);
|
|
logger.debug(
|
|
`[OpenAI API] Response ${responseId} completed in ${duration}ms (non-streaming)`,
|
|
);
|
|
}
|
|
} catch (error) {
|
|
const errorMessage = error instanceof Error ? error.message : 'An error occurred';
|
|
logger.error('[OpenAI API] Error:', error);
|
|
|
|
// Check if we already started streaming (headers sent)
|
|
if (res.headersSent) {
|
|
// Headers already sent, send error in stream
|
|
const errorChunk = createChunk(context, { content: `\n\nError: ${errorMessage}` }, 'stop');
|
|
writeSSE(res, errorChunk);
|
|
writeSSE(res, '[DONE]');
|
|
res.end();
|
|
} else {
|
|
// Forward upstream provider status codes (e.g., Anthropic 400s) instead of masking as 500
|
|
const statusCode =
|
|
typeof error?.status === 'number' && error.status >= 400 && error.status < 600
|
|
? error.status
|
|
: 500;
|
|
const errorType =
|
|
statusCode >= 400 && statusCode < 500 ? 'invalid_request_error' : 'server_error';
|
|
sendErrorResponse(res, statusCode, errorMessage, errorType);
|
|
}
|
|
}
|
|
};
|
|
|
|
/**
|
|
* List available agents as models (filtered by remote access permissions)
|
|
*
|
|
* GET /v1/models
|
|
*/
|
|
const ListModelsController = async (req, res) => {
|
|
try {
|
|
const userId = req.user?.id;
|
|
const userRole = req.user?.role;
|
|
|
|
if (!userId) {
|
|
return sendErrorResponse(res, 401, 'Authentication required', 'auth_error');
|
|
}
|
|
|
|
// Find agents the user has remote access to (VIEW permission on REMOTE_AGENT)
|
|
const accessibleAgentIds = await findAccessibleResources({
|
|
userId,
|
|
role: userRole,
|
|
resourceType: ResourceType.REMOTE_AGENT,
|
|
requiredPermissions: PermissionBits.VIEW,
|
|
});
|
|
|
|
// Get the accessible agents
|
|
let agents = [];
|
|
if (accessibleAgentIds.length > 0) {
|
|
agents = await db.getAgents({ _id: { $in: accessibleAgentIds } });
|
|
}
|
|
|
|
const models = agents.map((agent) => ({
|
|
id: agent.id,
|
|
object: 'model',
|
|
created: Math.floor(new Date(agent.createdAt || Date.now()).getTime() / 1000),
|
|
owned_by: 'librechat',
|
|
permission: [],
|
|
root: agent.id,
|
|
parent: null,
|
|
// LibreChat extensions
|
|
name: agent.name,
|
|
description: agent.description,
|
|
provider: agent.provider,
|
|
}));
|
|
|
|
res.json({
|
|
object: 'list',
|
|
data: models,
|
|
});
|
|
} catch (error) {
|
|
const errorMessage = error instanceof Error ? error.message : 'Failed to list models';
|
|
logger.error('[OpenAI API] Error listing models:', error);
|
|
sendErrorResponse(res, 500, errorMessage, 'server_error');
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Get a specific model/agent (with remote access permission check)
|
|
*
|
|
* GET /v1/models/:model
|
|
*/
|
|
const GetModelController = async (req, res) => {
|
|
try {
|
|
const { model } = req.params;
|
|
const userId = req.user?.id;
|
|
const userRole = req.user?.role;
|
|
|
|
if (!userId) {
|
|
return sendErrorResponse(res, 401, 'Authentication required', 'auth_error');
|
|
}
|
|
|
|
const agent = await db.getAgent({ id: model });
|
|
|
|
if (!agent) {
|
|
return sendErrorResponse(
|
|
res,
|
|
404,
|
|
`Model not found: ${model}`,
|
|
'invalid_request_error',
|
|
'model_not_found',
|
|
);
|
|
}
|
|
|
|
// Check if user has remote access to this agent
|
|
const accessibleAgentIds = await findAccessibleResources({
|
|
userId,
|
|
role: userRole,
|
|
resourceType: ResourceType.REMOTE_AGENT,
|
|
requiredPermissions: PermissionBits.VIEW,
|
|
});
|
|
|
|
const hasAccess = accessibleAgentIds.some((id) => id.toString() === agent._id.toString());
|
|
|
|
if (!hasAccess) {
|
|
return sendErrorResponse(
|
|
res,
|
|
403,
|
|
`No remote access to model: ${model}`,
|
|
'permission_error',
|
|
'access_denied',
|
|
);
|
|
}
|
|
|
|
res.json({
|
|
id: agent.id,
|
|
object: 'model',
|
|
created: Math.floor(new Date(agent.createdAt || Date.now()).getTime() / 1000),
|
|
owned_by: 'librechat',
|
|
permission: [],
|
|
root: agent.id,
|
|
parent: null,
|
|
// LibreChat extensions
|
|
name: agent.name,
|
|
description: agent.description,
|
|
provider: agent.provider,
|
|
});
|
|
} catch (error) {
|
|
const errorMessage = error instanceof Error ? error.message : 'Failed to get model';
|
|
logger.error('[OpenAI API] Error getting model:', error);
|
|
sendErrorResponse(res, 500, errorMessage, 'server_error');
|
|
}
|
|
};
|
|
|
|
module.exports = {
|
|
OpenAIChatCompletionController,
|
|
ListModelsController,
|
|
GetModelController,
|
|
};
|