mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-08-31 17:03:24 +00:00
* 🧹 fix: Prune Orphaned File References on File Deletion Deleting a file via the Manage Files tab left its file_id in every agent's tool_resources.*.file_ids. Stubs accumulate until the frontend dedupe keys them as duplicates and blocks all new uploads (issue #12776). - Add removeAgentResourceFilesFromAllAgents in packages/data-schemas: a single updateMany/$pullAll across every EToolResources category. - Invoke it from processDeleteRequest after db.deleteFiles so every referencing agent is cleaned up, not just the one passed in req.body. - Wrap the cleanup in try/catch so a stale agent update cannot mask a successful file deletion. * 🧼 fix: Prune Orphaned File References on Agent Update Already-affected agents would stay broken even after the delete-time fix: the stubs sit on the agent document until something strips them. Heal them on the next save (issue #12776). - Add collectToolResourceFileIds + stripFileIdsFromToolResources helpers in @librechat/api — centralizing the tool_resources traversal used by the controller and the follow-up migration script. - In updateAgentHandler, check the effective tool_resources against the files collection. When orphans are found, either strip them from the incoming tool_resources (if the update sets them) or run the bulk cleanup (if the update leaves tool_resources untouched). * 🧰 chore: Add Migration to Clean Up Orphaned Agent File References Complements the delete-time and save-time fixes by healing agents that already accumulated orphan stubs before the upgrade (issue #12776). The script is idempotent — re-running it on a clean database is a no-op. - Add config/migrate-orphaned-agent-files.js following the existing migrate-*.js convention: --dry-run by default omitted (writes by default) and --batch-size= tuning knob. Streams agents via cursor. - Register migrate:orphaned-agent-files and :dry-run npm scripts. - Reuse collectToolResourceFileIds from @librechat/api so migration and runtime share the same traversal logic. * 🩹 fix: Address Codex/Copilot Review on Orphaned Agent File Cleanup Refines the #12776 fix series based on automated review feedback. - Scope save-time pruning to the current agent only. When a PATCH carries tool_resources, strip orphans from the incoming payload and pay the DB round-trip only then. Removes the collection-wide updateMany previously triggered when tool_resources was absent (Codex P2 / Copilot). - Wrap the orphan check in try/catch so a transient db.getFiles failure can't turn a good save into a 500 (comprehensive review #1). - Replace Object.values(EToolResources) casts with an explicit list of agent-side categories in both orphans.ts and agent.ts. code_interpreter belongs to the Assistants API and isn't a key of AgentToolResources — including it was a type lie and generated dead MongoDB clauses (comprehensive review #3, #8). - Export TOOL_RESOURCE_KEYS from @librechat/api and consume it in the migration script, dropping one duplicated definition (#4). - Cap migration results.details at 50 sample entries so the memory footprint stays bounded on deployments with thousands of corrupted agents (Codex P3). - Add migrate:orphaned-agent-files:batch npm script to match the convention set by migrate-agent-permissions / migrate-prompt-permissions (#7). - Add controller-level tests covering the three orphan-pruning paths: strip from incoming tool_resources, leave alone when tool_resources is absent, swallow db.getFiles errors and still save (#6). - Back pre-existing "should validate tool_resources in updates" test's file_ids with real File docs — the new pruning would otherwise strip them, and that test is about OCR conversion / schema filtering, not file existence. Register the File model in beforeAll so the fixture works. * 🩹 fix: Tighten TOOL_RESOURCE_KEYS Type and Align Migration Sample Output Two follow-ups from the second review pass. - Type data-schemas' TOOL_RESOURCE_KEYS as ReadonlyArray<keyof AgentToolResources> instead of readonly string[]. Data-schemas depends on data-provider, so the import is clean. Catches typos and aligns with the matching export in @librechat/api — doesn't guarantee exhaustiveness, but that's a TypeScript limitation, not a workspace one. - Align the migration's console output with DETAIL_SAMPLE_LIMIT: print every collected detail (up to 50) and, when more agents were affected than the sample size allowed, show a truncation notice. The old hard cap of 25 meant affected agents in the 26-50 range were collected but never shown. * ✅ test: Add Integration Coverage for Orphan Cleanup Paths (#12776) Exercise the delete-time and migration paths end-to-end against a real in-memory Mongo. Catches integration bugs the isolated unit tests on each layer couldn't. - api/server/services/Files/process.integration.spec.js — the primary repro: seed an Agent + File, call processDeleteRequest, assert the file_id disappears from every referencing agent's tool_resources while unrelated agents stay untouched. Also covers the no-op case and confirms a failure in the new cleanup step cannot roll back the file deletion itself. - api/test/migrate-orphaned-agent-files.spec.js — drives the migration module: --dry-run reports without writing, apply mode prunes across every tool_resource category, re-running is idempotent, and DETAIL_SAMPLE_LIMIT caps the in-memory sample on wide corruption. Mocks only the connect helper (the spec owns the mongoose instance) so the real migration code path — cursor, $pullAll, reduce — runs. * 🔒 fix: Run Orphan Cleanup Migration in System Tenant Context Codex P2 catch: under TENANT_ISOLATION_STRICT=true, the migration throws on the very first Agent.countDocuments() because the tenant isolation plugin fail-closes on queries without tenant context — which makes migrate:orphaned-agent-files unusable on the exact deployments most likely to have accumulated corruption. - Wrap the scan/prune body in runAsSystem so queries bypass the tenant filter (SYSTEM_TENANT_ID sentinel). The migration legitimately needs cross-tenant visibility — this is the same pattern seedDatabase and the S3 refresh job already use. - Add a regression test that spies on Agent.countDocuments() and asserts the active tenantStorage context is SYSTEM_TENANT_ID during the call. Pins the wrap against future regressions without the brittleness of toggling the strict-mode env var (which caches on first read). Note: the delete-time and save-time paths already run inside an authenticated HTTP request where tenantStorage.run is set by auth middleware, so the cleanup naturally scopes to the current tenant — which is the correct behavior there since file ownership is tenant-scoped. * 🧹 chore: Drop Unused path Import From Process Integration Spec Leftover from an earlier iteration that resolved the migration path via path.resolve before I switched to a relative require. The import does nothing now — removing it.
1023 lines
32 KiB
JavaScript
1023 lines
32 KiB
JavaScript
const { z } = require('zod');
|
|
const fs = require('fs').promises;
|
|
const { nanoid } = require('nanoid');
|
|
const { logger } = require('@librechat/data-schemas');
|
|
const {
|
|
refreshS3Url,
|
|
agentCreateSchema,
|
|
agentUpdateSchema,
|
|
refreshListAvatars,
|
|
collectEdgeAgentIds,
|
|
mergeAgentOcrConversion,
|
|
MAX_AVATAR_REFRESH_AGENTS,
|
|
collectToolResourceFileIds,
|
|
convertOcrToContextInPlace,
|
|
stripFileIdsFromToolResources,
|
|
} = require('@librechat/api');
|
|
const {
|
|
Time,
|
|
Tools,
|
|
CacheKeys,
|
|
Constants,
|
|
FileSources,
|
|
ResourceType,
|
|
AccessRoleIds,
|
|
PrincipalType,
|
|
EToolResources,
|
|
PermissionBits,
|
|
actionDelimiter,
|
|
removeNullishValues,
|
|
} = require('librechat-data-provider');
|
|
const {
|
|
findPubliclyAccessibleResources,
|
|
getResourcePermissionsMap,
|
|
findAccessibleResources,
|
|
hasPublicPermission,
|
|
grantPermission,
|
|
} = require('~/server/services/PermissionService');
|
|
const { getStrategyFunctions } = require('~/server/services/Files/strategies');
|
|
const { resizeAvatar } = require('~/server/services/Files/images/avatar');
|
|
const { getFileStrategy } = require('~/server/utils/getFileStrategy');
|
|
const { filterFile } = require('~/server/services/Files/process');
|
|
const { getCachedTools } = require('~/server/services/Config');
|
|
const { resolveConfigServers } = require('~/server/services/MCP');
|
|
const { getMCPServersRegistry } = require('~/config');
|
|
const { getLogStores } = require('~/cache');
|
|
const db = require('~/models');
|
|
|
|
const systemTools = {
|
|
[Tools.execute_code]: true,
|
|
[Tools.file_search]: true,
|
|
[Tools.web_search]: true,
|
|
};
|
|
|
|
const MAX_SEARCH_LEN = 100;
|
|
const escapeRegex = (str = '') => str.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
|
|
/**
|
|
* Validates that the requesting user has VIEW access to every agent referenced in edges.
|
|
* Agents that do not exist in the database are skipped — at create time, the `from` field
|
|
* often references the agent being built, which has no DB record yet.
|
|
* @param {import('librechat-data-provider').GraphEdge[]} edges
|
|
* @param {string} userId
|
|
* @param {string} userRole - Used for group/role principal resolution
|
|
* @returns {Promise<string[]>} Agent IDs the user cannot VIEW (empty if all accessible)
|
|
*/
|
|
const validateEdgeAgentAccess = async (edges, userId, userRole) => {
|
|
const edgeAgentIds = collectEdgeAgentIds(edges);
|
|
if (edgeAgentIds.size === 0) {
|
|
return [];
|
|
}
|
|
|
|
const agents = await db.getAgents({ id: { $in: [...edgeAgentIds] } });
|
|
|
|
if (agents.length === 0) {
|
|
return [];
|
|
}
|
|
|
|
const permissionsMap = await getResourcePermissionsMap({
|
|
userId,
|
|
role: userRole,
|
|
resourceType: ResourceType.AGENT,
|
|
resourceIds: agents.map((a) => a._id),
|
|
});
|
|
|
|
return agents
|
|
.filter((a) => {
|
|
const bits = permissionsMap.get(a._id.toString()) ?? 0;
|
|
return (bits & PermissionBits.VIEW) === 0;
|
|
})
|
|
.map((a) => a.id);
|
|
};
|
|
|
|
/**
|
|
* Filters tools to only include those the user is authorized to use.
|
|
* MCP tools must match the exact format `{toolName}_mcp_{serverName}` (exactly 2 segments).
|
|
* Multi-delimiter keys are rejected to prevent authorization/execution mismatch.
|
|
* Non-MCP tools must appear in availableTools (global tool cache) or systemTools.
|
|
*
|
|
* When `existingTools` is provided and the MCP registry is unavailable (e.g. server restart),
|
|
* tools already present on the agent are preserved rather than stripped — they were validated
|
|
* when originally added, and we cannot re-verify them without the registry.
|
|
* @param {object} params
|
|
* @param {string[]} params.tools - Raw tool strings from the request
|
|
* @param {string} params.userId - Requesting user ID for MCP server access check
|
|
* @param {Record<string, unknown>} params.availableTools - Global non-MCP tool cache
|
|
* @param {string[]} [params.existingTools] - Tools already persisted on the agent document
|
|
* @param {Record<string, unknown>} [params.configServers] - Config-source MCP servers resolved from appConfig overrides
|
|
* @returns {Promise<string[]>} Only the authorized subset of tools
|
|
*/
|
|
const filterAuthorizedTools = async ({
|
|
tools,
|
|
userId,
|
|
availableTools,
|
|
existingTools,
|
|
configServers,
|
|
}) => {
|
|
const filteredTools = [];
|
|
let mcpServerConfigs;
|
|
let registryUnavailable = false;
|
|
const existingToolSet = existingTools?.length ? new Set(existingTools) : null;
|
|
|
|
for (const tool of tools) {
|
|
if (availableTools[tool] || systemTools[tool]) {
|
|
filteredTools.push(tool);
|
|
continue;
|
|
}
|
|
|
|
if (!tool?.includes(Constants.mcp_delimiter)) {
|
|
continue;
|
|
}
|
|
|
|
if (mcpServerConfigs === undefined) {
|
|
try {
|
|
mcpServerConfigs =
|
|
(await getMCPServersRegistry().getAllServerConfigs(userId, configServers)) ?? {};
|
|
} catch (e) {
|
|
logger.warn(
|
|
'[filterAuthorizedTools] MCP registry unavailable, filtering all MCP tools',
|
|
e.message,
|
|
);
|
|
mcpServerConfigs = {};
|
|
registryUnavailable = true;
|
|
}
|
|
}
|
|
|
|
const parts = tool.split(Constants.mcp_delimiter);
|
|
if (parts.length !== 2) {
|
|
logger.warn(
|
|
`[filterAuthorizedTools] Rejected malformed MCP tool key "${tool}" for user ${userId}`,
|
|
);
|
|
continue;
|
|
}
|
|
|
|
if (registryUnavailable && existingToolSet?.has(tool)) {
|
|
filteredTools.push(tool);
|
|
continue;
|
|
}
|
|
|
|
const [, serverName] = parts;
|
|
if (!serverName || !Object.hasOwn(mcpServerConfigs, serverName)) {
|
|
logger.warn(
|
|
`[filterAuthorizedTools] Rejected MCP tool "${tool}" — server "${serverName}" not accessible to user ${userId}`,
|
|
);
|
|
continue;
|
|
}
|
|
|
|
filteredTools.push(tool);
|
|
}
|
|
|
|
return filteredTools;
|
|
};
|
|
|
|
/**
|
|
* Creates an Agent.
|
|
* @route POST /Agents
|
|
* @param {ServerRequest} req - The request object.
|
|
* @param {AgentCreateParams} req.body - The request body.
|
|
* @param {ServerResponse} res - The response object.
|
|
* @returns {Promise<Agent>} 201 - success response - application/json
|
|
*/
|
|
const createAgentHandler = async (req, res) => {
|
|
try {
|
|
const validatedData = agentCreateSchema.parse(req.body);
|
|
const { tools = [], ...agentData } = removeNullishValues(validatedData);
|
|
|
|
if (agentData.model_parameters && typeof agentData.model_parameters === 'object') {
|
|
agentData.model_parameters = removeNullishValues(agentData.model_parameters, true);
|
|
}
|
|
|
|
const { id: userId, role: userRole } = req.user;
|
|
|
|
if (agentData.edges?.length) {
|
|
const unauthorized = await validateEdgeAgentAccess(agentData.edges, userId, userRole);
|
|
if (unauthorized.length > 0) {
|
|
return res.status(403).json({
|
|
error: 'You do not have access to one or more agents referenced in edges',
|
|
agent_ids: unauthorized,
|
|
});
|
|
}
|
|
}
|
|
|
|
agentData.id = `agent_${nanoid()}`;
|
|
agentData.author = userId;
|
|
agentData.tools = [];
|
|
|
|
const hasMCPTools = tools.some((t) => t?.includes(Constants.mcp_delimiter));
|
|
const [availableTools, configServers] = await Promise.all([
|
|
getCachedTools().then((t) => t ?? {}),
|
|
hasMCPTools ? resolveConfigServers(req) : Promise.resolve(undefined),
|
|
]);
|
|
agentData.tools = await filterAuthorizedTools({
|
|
tools,
|
|
userId,
|
|
availableTools,
|
|
configServers,
|
|
});
|
|
|
|
const agent = await db.createAgent(agentData);
|
|
|
|
try {
|
|
await Promise.all([
|
|
grantPermission({
|
|
principalType: PrincipalType.USER,
|
|
principalId: userId,
|
|
resourceType: ResourceType.AGENT,
|
|
resourceId: agent._id,
|
|
accessRoleId: AccessRoleIds.AGENT_OWNER,
|
|
grantedBy: userId,
|
|
}),
|
|
grantPermission({
|
|
principalType: PrincipalType.USER,
|
|
principalId: userId,
|
|
resourceType: ResourceType.REMOTE_AGENT,
|
|
resourceId: agent._id,
|
|
accessRoleId: AccessRoleIds.REMOTE_AGENT_OWNER,
|
|
grantedBy: userId,
|
|
}),
|
|
]);
|
|
logger.debug(
|
|
`[createAgent] Granted owner permissions to user ${userId} for agent ${agent.id}`,
|
|
);
|
|
} catch (permissionError) {
|
|
logger.error(
|
|
`[createAgent] Failed to grant owner permissions for agent ${agent.id}:`,
|
|
permissionError,
|
|
);
|
|
}
|
|
|
|
res.status(201).json(agent);
|
|
} catch (error) {
|
|
if (error instanceof z.ZodError) {
|
|
logger.error('[/Agents] Validation error', error.errors);
|
|
return res.status(400).json({ error: 'Invalid request data', details: error.errors });
|
|
}
|
|
logger.error('[/Agents] Error creating agent', error);
|
|
res.status(500).json({ error: error.message });
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Retrieves an Agent by ID.
|
|
* @route GET /Agents/:id
|
|
* @param {object} req - Express Request
|
|
* @param {object} req.params - Request params
|
|
* @param {string} req.params.id - Agent identifier.
|
|
* @param {object} req.user - Authenticated user information
|
|
* @param {string} req.user.id - User ID
|
|
* @returns {Promise<Agent>} 200 - success response - application/json
|
|
* @returns {Error} 404 - Agent not found
|
|
*/
|
|
const getAgentHandler = async (req, res, expandProperties = false) => {
|
|
try {
|
|
const id = req.params.id;
|
|
const author = req.user.id;
|
|
|
|
// Permissions are validated by middleware before calling this function
|
|
// Simply load the agent by ID
|
|
const agent = await db.getAgent({ id });
|
|
|
|
if (!agent) {
|
|
return res.status(404).json({ error: 'Agent not found' });
|
|
}
|
|
|
|
agent.version = agent.versions ? agent.versions.length : 0;
|
|
|
|
if (agent.avatar && agent.avatar?.source === FileSources.s3) {
|
|
try {
|
|
agent.avatar = {
|
|
...agent.avatar,
|
|
filepath: await refreshS3Url(agent.avatar),
|
|
};
|
|
} catch (e) {
|
|
logger.warn('[/Agents/:id] Failed to refresh S3 URL', e);
|
|
}
|
|
}
|
|
|
|
agent.author = agent.author.toString();
|
|
|
|
// Check if agent is public
|
|
const isPublic = await hasPublicPermission({
|
|
resourceType: ResourceType.AGENT,
|
|
resourceId: agent._id,
|
|
requiredPermissions: PermissionBits.VIEW,
|
|
});
|
|
agent.isPublic = isPublic;
|
|
|
|
if (agent.author !== author) {
|
|
delete agent.author;
|
|
}
|
|
|
|
if (!expandProperties) {
|
|
// VIEW permission: Basic agent info only
|
|
return res.status(200).json({
|
|
_id: agent._id,
|
|
id: agent.id,
|
|
name: agent.name,
|
|
description: agent.description,
|
|
avatar: agent.avatar,
|
|
author: agent.author,
|
|
provider: agent.provider,
|
|
model: agent.model,
|
|
isPublic: agent.isPublic,
|
|
version: agent.version,
|
|
// Safe metadata
|
|
createdAt: agent.createdAt,
|
|
updatedAt: agent.updatedAt,
|
|
});
|
|
}
|
|
|
|
// EDIT permission: Full agent details including sensitive configuration
|
|
return res.status(200).json(agent);
|
|
} catch (error) {
|
|
logger.error('[/Agents/:id] Error retrieving agent', error);
|
|
res.status(500).json({ error: error.message });
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Updates an Agent.
|
|
* @route PATCH /Agents/:id
|
|
* @param {object} req - Express Request
|
|
* @param {object} req.params - Request params
|
|
* @param {string} req.params.id - Agent identifier.
|
|
* @param {AgentUpdateParams} req.body - The Agent update parameters.
|
|
* @returns {Promise<Agent>} 200 - success response - application/json
|
|
*/
|
|
const updateAgentHandler = async (req, res) => {
|
|
try {
|
|
const id = req.params.id;
|
|
const validatedData = agentUpdateSchema.parse(req.body);
|
|
// Preserve explicit null for avatar to allow resetting the avatar
|
|
const { avatar: avatarField, _id, ...rest } = validatedData;
|
|
const updateData = removeNullishValues(rest);
|
|
|
|
if (updateData.model_parameters && typeof updateData.model_parameters === 'object') {
|
|
updateData.model_parameters = removeNullishValues(updateData.model_parameters, true);
|
|
}
|
|
|
|
if (avatarField === null) {
|
|
updateData.avatar = avatarField;
|
|
}
|
|
|
|
if (updateData.edges?.length) {
|
|
const { id: userId, role: userRole } = req.user;
|
|
const unauthorized = await validateEdgeAgentAccess(updateData.edges, userId, userRole);
|
|
if (unauthorized.length > 0) {
|
|
return res.status(403).json({
|
|
error: 'You do not have access to one or more agents referenced in edges',
|
|
agent_ids: unauthorized,
|
|
});
|
|
}
|
|
}
|
|
|
|
// Convert OCR to context in incoming updateData
|
|
convertOcrToContextInPlace(updateData);
|
|
|
|
const existingAgent = await db.getAgent({ id });
|
|
|
|
if (!existingAgent) {
|
|
return res.status(404).json({ error: 'Agent not found' });
|
|
}
|
|
|
|
// Convert legacy OCR tool resource to context format in existing agent
|
|
const ocrConversion = mergeAgentOcrConversion(existingAgent, updateData);
|
|
if (ocrConversion.tool_resources) {
|
|
updateData.tool_resources = ocrConversion.tool_resources;
|
|
}
|
|
if (ocrConversion.tools) {
|
|
updateData.tools = ocrConversion.tools;
|
|
}
|
|
|
|
/*
|
|
* Strip orphaned file_id stubs from the incoming payload (see issue #12776).
|
|
* Scoped to updates that actually touch tool_resources: if the save does not
|
|
* modify that field, the delete-time cleanup in processDeleteRequest and the
|
|
* one-off migration already cover pre-existing corruption, so there's no
|
|
* reason to pay an extra DB round-trip here. Wrapped in try/catch so a
|
|
* transient failure in this integrity check never turns a good save into 500.
|
|
*/
|
|
if (updateData.tool_resources) {
|
|
try {
|
|
const referencedFileIds = collectToolResourceFileIds(updateData.tool_resources);
|
|
if (referencedFileIds.length > 0) {
|
|
const existingFiles = await db.getFiles({ file_id: { $in: referencedFileIds } }, null, {
|
|
file_id: 1,
|
|
});
|
|
const existingIds = new Set((existingFiles ?? []).map((f) => f.file_id));
|
|
const orphans = referencedFileIds.filter((id) => !existingIds.has(id));
|
|
if (orphans.length > 0) {
|
|
logger.warn(
|
|
`[/Agents/:id] Pruning ${orphans.length} orphaned file reference(s) from agent ${id}`,
|
|
);
|
|
stripFileIdsFromToolResources(updateData.tool_resources, orphans);
|
|
}
|
|
}
|
|
} catch (orphanCheckError) {
|
|
logger.warn(
|
|
'[/Agents/:id] Orphan file check failed, skipping cleanup for this request',
|
|
orphanCheckError,
|
|
);
|
|
}
|
|
}
|
|
|
|
if (updateData.tools) {
|
|
const existingToolSet = new Set(existingAgent.tools ?? []);
|
|
const newMCPTools = updateData.tools.filter(
|
|
(t) => !existingToolSet.has(t) && t?.includes(Constants.mcp_delimiter),
|
|
);
|
|
|
|
if (newMCPTools.length > 0) {
|
|
const [availableTools, configServers] = await Promise.all([
|
|
getCachedTools().then((t) => t ?? {}),
|
|
resolveConfigServers(req),
|
|
]);
|
|
const approvedNew = await filterAuthorizedTools({
|
|
tools: newMCPTools,
|
|
userId: req.user.id,
|
|
availableTools,
|
|
configServers,
|
|
});
|
|
const rejectedSet = new Set(newMCPTools.filter((t) => !approvedNew.includes(t)));
|
|
if (rejectedSet.size > 0) {
|
|
updateData.tools = updateData.tools.filter((t) => !rejectedSet.has(t));
|
|
}
|
|
}
|
|
}
|
|
|
|
let updatedAgent =
|
|
Object.keys(updateData).length > 0
|
|
? await db.updateAgent({ id }, updateData, {
|
|
updatingUserId: req.user.id,
|
|
})
|
|
: existingAgent;
|
|
|
|
// Add version count to the response
|
|
updatedAgent.version = updatedAgent.versions ? updatedAgent.versions.length : 0;
|
|
|
|
if (updatedAgent.author) {
|
|
updatedAgent.author = updatedAgent.author.toString();
|
|
}
|
|
|
|
if (updatedAgent.author !== req.user.id) {
|
|
delete updatedAgent.author;
|
|
}
|
|
|
|
return res.json(updatedAgent);
|
|
} catch (error) {
|
|
if (error instanceof z.ZodError) {
|
|
logger.error('[/Agents/:id] Validation error', error.errors);
|
|
return res.status(400).json({ error: 'Invalid request data', details: error.errors });
|
|
}
|
|
|
|
logger.error('[/Agents/:id] Error updating Agent', error);
|
|
|
|
if (error.statusCode === 409) {
|
|
return res.status(409).json({
|
|
error: error.message,
|
|
details: error.details,
|
|
});
|
|
}
|
|
|
|
res.status(500).json({ error: error.message });
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Duplicates an Agent based on the provided ID.
|
|
* @route POST /Agents/:id/duplicate
|
|
* @param {object} req - Express Request
|
|
* @param {object} req.params - Request params
|
|
* @param {string} req.params.id - Agent identifier.
|
|
* @returns {Promise<Agent>} 201 - success response - application/json
|
|
*/
|
|
const duplicateAgentHandler = async (req, res) => {
|
|
const { id } = req.params;
|
|
const { id: userId } = req.user;
|
|
const sensitiveFields = ['api_key', 'oauth_client_id', 'oauth_client_secret'];
|
|
|
|
try {
|
|
const agent = await db.getAgent({ id });
|
|
if (!agent) {
|
|
return res.status(404).json({
|
|
error: 'Agent not found',
|
|
status: 'error',
|
|
});
|
|
}
|
|
|
|
const {
|
|
id: _id,
|
|
_id: __id,
|
|
author: _author,
|
|
createdAt: _createdAt,
|
|
updatedAt: _updatedAt,
|
|
tool_resources: _tool_resources = {},
|
|
versions: _versions,
|
|
__v: _v,
|
|
...cloneData
|
|
} = agent;
|
|
cloneData.name = `${agent.name} (${new Date().toLocaleString('en-US', {
|
|
dateStyle: 'short',
|
|
timeStyle: 'short',
|
|
hour12: false,
|
|
})})`;
|
|
|
|
if (_tool_resources?.[EToolResources.context]) {
|
|
cloneData.tool_resources = {
|
|
[EToolResources.context]: _tool_resources[EToolResources.context],
|
|
};
|
|
}
|
|
|
|
if (_tool_resources?.[EToolResources.ocr]) {
|
|
cloneData.tool_resources = {
|
|
/** Legacy conversion from `ocr` to `context` */
|
|
[EToolResources.context]: {
|
|
...(_tool_resources[EToolResources.context] ?? {}),
|
|
..._tool_resources[EToolResources.ocr],
|
|
},
|
|
};
|
|
}
|
|
|
|
const newAgentId = `agent_${nanoid()}`;
|
|
const newAgentData = Object.assign(cloneData, {
|
|
id: newAgentId,
|
|
author: userId,
|
|
});
|
|
|
|
const newActionsList = [];
|
|
const originalActions = (await db.getActions({ agent_id: id }, true)) ?? [];
|
|
const promises = [];
|
|
|
|
/**
|
|
* Duplicates an action and returns the new action ID.
|
|
* @param {Action} action
|
|
* @returns {Promise<string>}
|
|
*/
|
|
const duplicateAction = async (action) => {
|
|
const newActionId = nanoid();
|
|
const { domain } = action.metadata;
|
|
const fullActionId = `${domain}${actionDelimiter}${newActionId}`;
|
|
|
|
// Sanitize sensitive metadata before persisting
|
|
const filteredMetadata = { ...(action.metadata || {}) };
|
|
for (const field of sensitiveFields) {
|
|
delete filteredMetadata[field];
|
|
}
|
|
|
|
const newAction = await db.updateAction(
|
|
{ action_id: newActionId, agent_id: newAgentId },
|
|
{
|
|
metadata: filteredMetadata,
|
|
agent_id: newAgentId,
|
|
user: userId,
|
|
},
|
|
);
|
|
|
|
newActionsList.push(newAction);
|
|
return fullActionId;
|
|
};
|
|
|
|
for (const action of originalActions) {
|
|
promises.push(
|
|
duplicateAction(action).catch((error) => {
|
|
logger.error('[/agents/:id/duplicate] Error duplicating Action:', error);
|
|
}),
|
|
);
|
|
}
|
|
|
|
const agentActions = await Promise.all(promises);
|
|
newAgentData.actions = agentActions;
|
|
|
|
if (newAgentData.tools?.length) {
|
|
const [availableTools, configServers] = await Promise.all([
|
|
getCachedTools().then((t) => t ?? {}),
|
|
resolveConfigServers(req),
|
|
]);
|
|
newAgentData.tools = await filterAuthorizedTools({
|
|
tools: newAgentData.tools,
|
|
userId,
|
|
availableTools,
|
|
existingTools: newAgentData.tools,
|
|
configServers,
|
|
});
|
|
}
|
|
|
|
const newAgent = await db.createAgent(newAgentData);
|
|
|
|
try {
|
|
await Promise.all([
|
|
grantPermission({
|
|
principalType: PrincipalType.USER,
|
|
principalId: userId,
|
|
resourceType: ResourceType.AGENT,
|
|
resourceId: newAgent._id,
|
|
accessRoleId: AccessRoleIds.AGENT_OWNER,
|
|
grantedBy: userId,
|
|
}),
|
|
grantPermission({
|
|
principalType: PrincipalType.USER,
|
|
principalId: userId,
|
|
resourceType: ResourceType.REMOTE_AGENT,
|
|
resourceId: newAgent._id,
|
|
accessRoleId: AccessRoleIds.REMOTE_AGENT_OWNER,
|
|
grantedBy: userId,
|
|
}),
|
|
]);
|
|
logger.debug(
|
|
`[duplicateAgent] Granted owner permissions to user ${userId} for duplicated agent ${newAgent.id}`,
|
|
);
|
|
} catch (permissionError) {
|
|
logger.error(
|
|
`[duplicateAgent] Failed to grant owner permissions for duplicated agent ${newAgent.id}:`,
|
|
permissionError,
|
|
);
|
|
}
|
|
|
|
return res.status(201).json({
|
|
agent: newAgent,
|
|
actions: newActionsList,
|
|
});
|
|
} catch (error) {
|
|
logger.error('[/Agents/:id/duplicate] Error duplicating Agent:', error);
|
|
|
|
res.status(500).json({ error: error.message });
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Deletes an Agent based on the provided ID.
|
|
* @route DELETE /Agents/:id
|
|
* @param {object} req - Express Request
|
|
* @param {object} req.params - Request params
|
|
* @param {string} req.params.id - Agent identifier.
|
|
* @returns {Promise<Agent>} 200 - success response - application/json
|
|
*/
|
|
const deleteAgentHandler = async (req, res) => {
|
|
try {
|
|
const id = req.params.id;
|
|
const agent = await db.getAgent({ id });
|
|
if (!agent) {
|
|
return res.status(404).json({ error: 'Agent not found' });
|
|
}
|
|
await db.deleteAgent({ id });
|
|
return res.json({ message: 'Agent deleted' });
|
|
} catch (error) {
|
|
logger.error('[/Agents/:id] Error deleting Agent', error);
|
|
res.status(500).json({ error: error.message });
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Lists agents using ACL-aware permissions (ownership + explicit shares).
|
|
* @route GET /Agents
|
|
* @param {object} req - Express Request
|
|
* @param {object} req.query - Request query
|
|
* @param {string} [req.query.user] - The user ID of the agent's author.
|
|
* @returns {Promise<AgentListResponse>} 200 - success response - application/json
|
|
*/
|
|
const getListAgentsHandler = async (req, res) => {
|
|
try {
|
|
const userId = req.user.id;
|
|
const { category, search, limit, cursor, promoted } = req.query;
|
|
let requiredPermission = req.query.requiredPermission;
|
|
if (typeof requiredPermission === 'string') {
|
|
requiredPermission = parseInt(requiredPermission, 10);
|
|
if (isNaN(requiredPermission)) {
|
|
requiredPermission = PermissionBits.VIEW;
|
|
}
|
|
} else if (typeof requiredPermission !== 'number') {
|
|
requiredPermission = PermissionBits.VIEW;
|
|
}
|
|
// Base filter
|
|
const filter = {};
|
|
|
|
// Handle category filter - only apply if category is defined
|
|
if (category !== undefined && category.trim() !== '') {
|
|
filter.category = category;
|
|
}
|
|
|
|
// Handle promoted filter - only from query param
|
|
if (promoted === '1') {
|
|
filter.is_promoted = true;
|
|
} else if (promoted === '0') {
|
|
filter.is_promoted = { $ne: true };
|
|
}
|
|
|
|
// Handle search filter (escape regex and cap length)
|
|
if (search && search.trim() !== '') {
|
|
const safeSearch = escapeRegex(search.trim().slice(0, MAX_SEARCH_LEN));
|
|
const regex = new RegExp(safeSearch, 'i');
|
|
filter.$or = [{ name: regex }, { description: regex }];
|
|
}
|
|
|
|
// Get agent IDs the user has VIEW access to via ACL
|
|
const accessibleIds = await findAccessibleResources({
|
|
userId,
|
|
role: req.user.role,
|
|
resourceType: ResourceType.AGENT,
|
|
requiredPermissions: requiredPermission,
|
|
});
|
|
|
|
const publiclyAccessibleIds = await findPubliclyAccessibleResources({
|
|
resourceType: ResourceType.AGENT,
|
|
requiredPermissions: PermissionBits.VIEW,
|
|
});
|
|
|
|
/**
|
|
* Refresh all S3 avatars for this user's accessible agent set (not only the current page)
|
|
* This addresses page-size limits preventing refresh of agents beyond the first page
|
|
*/
|
|
const cache = getLogStores(CacheKeys.S3_EXPIRY_INTERVAL);
|
|
const refreshKey = `${userId}:agents_avatar_refresh`;
|
|
let cachedRefresh = await cache.get(refreshKey);
|
|
const isValidCachedRefresh =
|
|
cachedRefresh != null && typeof cachedRefresh === 'object' && cachedRefresh.urlCache != null;
|
|
if (!isValidCachedRefresh) {
|
|
try {
|
|
const fullList = await db.getListAgentsByAccess({
|
|
accessibleIds,
|
|
otherParams: {},
|
|
limit: MAX_AVATAR_REFRESH_AGENTS,
|
|
after: null,
|
|
});
|
|
const { urlCache } = await refreshListAvatars({
|
|
agents: fullList?.data ?? [],
|
|
userId,
|
|
refreshS3Url,
|
|
updateAgent: db.updateAgent,
|
|
});
|
|
cachedRefresh = { urlCache };
|
|
await cache.set(refreshKey, cachedRefresh, Time.THIRTY_MINUTES);
|
|
} catch (err) {
|
|
logger.error('[/Agents] Error refreshing avatars for full list: %o', err);
|
|
}
|
|
} else {
|
|
logger.debug('[/Agents] S3 avatar refresh already checked, skipping');
|
|
}
|
|
|
|
// Use the new ACL-aware function
|
|
const data = await db.getListAgentsByAccess({
|
|
accessibleIds,
|
|
otherParams: filter,
|
|
limit,
|
|
after: cursor,
|
|
});
|
|
|
|
const agents = data?.data ?? [];
|
|
if (!agents.length) {
|
|
return res.json(data);
|
|
}
|
|
|
|
const publicSet = new Set(publiclyAccessibleIds.map((oid) => oid.toString()));
|
|
|
|
const urlCache = cachedRefresh?.urlCache;
|
|
data.data = agents.map((agent) => {
|
|
try {
|
|
if (agent?._id && publicSet.has(agent._id.toString())) {
|
|
agent.isPublic = true;
|
|
}
|
|
if (
|
|
urlCache &&
|
|
agent?.id &&
|
|
agent?.avatar?.source === FileSources.s3 &&
|
|
urlCache[agent.id]
|
|
) {
|
|
agent.avatar = { ...agent.avatar, filepath: urlCache[agent.id] };
|
|
}
|
|
} catch (e) {
|
|
// Silently ignore mapping errors
|
|
void e;
|
|
}
|
|
return agent;
|
|
});
|
|
|
|
return res.json(data);
|
|
} catch (error) {
|
|
logger.error('[/Agents] Error listing Agents: %o', error);
|
|
res.status(500).json({ error: error.message });
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Uploads and updates an avatar for a specific agent.
|
|
* @route POST /:agent_id/avatar
|
|
* @param {object} req - Express Request
|
|
* @param {object} req.params - Request params
|
|
* @param {string} req.params.agent_id - The ID of the agent.
|
|
* @param {Express.Multer.File} req.file - The avatar image file.
|
|
* @param {object} req.body - Request body
|
|
* @param {string} [req.body.avatar] - Optional avatar for the agent's avatar.
|
|
* @returns {Promise<void>} 200 - success response - application/json
|
|
*/
|
|
const uploadAgentAvatarHandler = async (req, res) => {
|
|
try {
|
|
const appConfig = req.config;
|
|
if (!req.file) {
|
|
return res.status(400).json({ message: 'No file uploaded' });
|
|
}
|
|
filterFile({ req, file: req.file, image: true, isAvatar: true });
|
|
const { agent_id } = req.params;
|
|
if (!agent_id) {
|
|
return res.status(400).json({ message: 'Agent ID is required' });
|
|
}
|
|
|
|
const existingAgent = await db.getAgent({ id: agent_id });
|
|
|
|
if (!existingAgent) {
|
|
return res.status(404).json({ error: 'Agent not found' });
|
|
}
|
|
|
|
const buffer = await fs.readFile(req.file.path);
|
|
const fileStrategy = getFileStrategy(appConfig, { isAvatar: true });
|
|
const resizedBuffer = await resizeAvatar({
|
|
userId: req.user.id,
|
|
input: buffer,
|
|
});
|
|
|
|
const { processAvatar } = getStrategyFunctions(fileStrategy);
|
|
const avatarUrl = await processAvatar({
|
|
buffer: resizedBuffer,
|
|
userId: req.user.id,
|
|
manual: 'false',
|
|
agentId: agent_id,
|
|
});
|
|
|
|
const image = {
|
|
filepath: avatarUrl,
|
|
source: fileStrategy,
|
|
};
|
|
|
|
let _avatar = existingAgent.avatar;
|
|
|
|
if (_avatar && _avatar.source) {
|
|
const { deleteFile } = getStrategyFunctions(_avatar.source);
|
|
try {
|
|
await deleteFile(req, { filepath: _avatar.filepath });
|
|
await db.deleteFileByFilter({ user: req.user.id, filepath: _avatar.filepath });
|
|
} catch (error) {
|
|
logger.error('[/:agent_id/avatar] Error deleting old avatar', error);
|
|
}
|
|
}
|
|
|
|
const data = {
|
|
avatar: {
|
|
filepath: image.filepath,
|
|
source: image.source,
|
|
},
|
|
};
|
|
|
|
const updatedAgent = await db.updateAgent({ id: agent_id }, data, {
|
|
updatingUserId: req.user.id,
|
|
});
|
|
|
|
try {
|
|
const avatarCache = getLogStores(CacheKeys.S3_EXPIRY_INTERVAL);
|
|
await avatarCache.delete(`${req.user.id}:agents_avatar_refresh`);
|
|
} catch (cacheErr) {
|
|
logger.error('[/:agent_id/avatar] Error invalidating avatar refresh cache', cacheErr);
|
|
}
|
|
|
|
res.status(201).json(updatedAgent);
|
|
} catch (error) {
|
|
const message = 'An error occurred while updating the Agent Avatar';
|
|
logger.error(
|
|
`[/:agent_id/avatar] ${message} (${req.params?.agent_id ?? 'unknown agent'})`,
|
|
error,
|
|
);
|
|
res.status(500).json({ message });
|
|
} finally {
|
|
try {
|
|
await fs.unlink(req.file.path);
|
|
logger.debug('[/:agent_id/avatar] Temp. image upload file deleted');
|
|
} catch {
|
|
logger.debug('[/:agent_id/avatar] Temp. image upload file already deleted');
|
|
}
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Reverts an agent to a previous version from its version history.
|
|
* @route PATCH /agents/:id/revert
|
|
* @param {object} req - Express Request object
|
|
* @param {object} req.params - Request parameters
|
|
* @param {string} req.params.id - The ID of the agent to revert
|
|
* @param {object} req.body - Request body
|
|
* @param {number} req.body.version_index - The index of the version to revert to
|
|
* @param {object} req.user - Authenticated user information
|
|
* @param {string} req.user.id - User ID
|
|
* @param {string} req.user.role - User role
|
|
* @param {ServerResponse} res - Express Response object
|
|
* @returns {Promise<Agent>} 200 - The updated agent after reverting to the specified version
|
|
* @throws {Error} 400 - If version_index is missing
|
|
* @throws {Error} 403 - If user doesn't have permission to modify the agent
|
|
* @throws {Error} 404 - If agent not found
|
|
* @throws {Error} 500 - If there's an internal server error during the reversion process
|
|
*/
|
|
const revertAgentVersionHandler = async (req, res) => {
|
|
try {
|
|
const { id } = req.params;
|
|
const { version_index } = req.body;
|
|
|
|
if (version_index === undefined) {
|
|
return res.status(400).json({ error: 'version_index is required' });
|
|
}
|
|
|
|
const existingAgent = await db.getAgent({ id });
|
|
|
|
if (!existingAgent) {
|
|
return res.status(404).json({ error: 'Agent not found' });
|
|
}
|
|
|
|
// Permissions are enforced via route middleware (ACL EDIT)
|
|
|
|
let updatedAgent = await db.revertAgentVersion({ id }, version_index);
|
|
|
|
if (updatedAgent.tools?.length) {
|
|
const [availableTools, configServers] = await Promise.all([
|
|
getCachedTools().then((t) => t ?? {}),
|
|
resolveConfigServers(req),
|
|
]);
|
|
const filteredTools = await filterAuthorizedTools({
|
|
tools: updatedAgent.tools,
|
|
userId: req.user.id,
|
|
availableTools,
|
|
existingTools: updatedAgent.tools,
|
|
configServers,
|
|
});
|
|
if (filteredTools.length !== updatedAgent.tools.length) {
|
|
updatedAgent = await db.updateAgent(
|
|
{ id },
|
|
{ tools: filteredTools },
|
|
{ updatingUserId: req.user.id },
|
|
);
|
|
}
|
|
}
|
|
|
|
if (updatedAgent.author) {
|
|
updatedAgent.author = updatedAgent.author.toString();
|
|
}
|
|
|
|
if (updatedAgent.author !== req.user.id) {
|
|
delete updatedAgent.author;
|
|
}
|
|
|
|
return res.json(updatedAgent);
|
|
} catch (error) {
|
|
logger.error('[/agents/:id/revert] Error reverting Agent version', error);
|
|
res.status(500).json({ error: error.message });
|
|
}
|
|
};
|
|
/**
|
|
* Get all agent categories with counts
|
|
*
|
|
* @param {Object} _req - Express request object (unused)
|
|
* @param {Object} res - Express response object
|
|
*/
|
|
const getAgentCategories = async (_req, res) => {
|
|
try {
|
|
const categories = await db.getCategoriesWithCounts();
|
|
const promotedCount = await db.countPromotedAgents();
|
|
const formattedCategories = categories.map((category) => ({
|
|
value: category.value,
|
|
label: category.label,
|
|
count: category.agentCount,
|
|
description: category.description,
|
|
}));
|
|
|
|
if (promotedCount > 0) {
|
|
formattedCategories.unshift({
|
|
value: 'promoted',
|
|
label: 'Promoted',
|
|
count: promotedCount,
|
|
description: 'Our recommended agents',
|
|
});
|
|
}
|
|
|
|
formattedCategories.push({
|
|
value: 'all',
|
|
label: 'All',
|
|
description: 'All available agents',
|
|
});
|
|
|
|
res.status(200).json(formattedCategories);
|
|
} catch (error) {
|
|
logger.error('[/Agents/Marketplace] Error fetching agent categories:', error);
|
|
res.status(500).json({
|
|
error: 'Failed to fetch agent categories',
|
|
userMessage: 'Unable to load categories. Please refresh the page.',
|
|
suggestion: 'Try refreshing the page or check your network connection',
|
|
});
|
|
}
|
|
};
|
|
module.exports = {
|
|
createAgent: createAgentHandler,
|
|
getAgent: getAgentHandler,
|
|
updateAgent: updateAgentHandler,
|
|
duplicateAgent: duplicateAgentHandler,
|
|
deleteAgent: deleteAgentHandler,
|
|
getListAgents: getListAgentsHandler,
|
|
uploadAgentAvatar: uploadAgentAvatarHandler,
|
|
revertAgentVersion: revertAgentVersionHandler,
|
|
getAgentCategories,
|
|
filterAuthorizedTools,
|
|
};
|