mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-08-28 12:44:28 +00:00
546f006e42
Three related changes that tighten the GitNexus CI/CD loop. Serialized deploys - Previous concurrency group was keyed by head ref with cancel-in-progress, which let deploys targeting different refs (e.g. main push + PR command) run in parallel. That's a data race: the prune-stale-indexes step computes active_names up front, so deploy A rsyncing /opt/gitnexus/indexes/LibreChat-pr-12580 can collide with deploy B pruning the same folder based on a pre-rsync view of the active set. - Collapse to a single global group gitnexus-deploy with cancel-in-progress: false. All deploys queue behind one another. A rsync/docker-compose restart is never killed mid-operation. The 20-minute job timeout bounds queue depth. PR completion feedback - Add a "index complete" comment step in gitnexus-index.yml that fires only when inputs.pr_number is set (i.e. the run came via the /gitnexus command). Posts success or failure with a link to the run and whether embeddings were generated. - Add a "deploy complete" comment step in gitnexus-deploy.yml that handles both trigger paths: workflow_run from a native PR auto-index (PR number recovered from the matrix entry whose runId matches the trigger run), and workflow_dispatch from the index workflow's bot- fallback path (PR number passed through as a new inputs.pr_number). - Plumb inputs.pr_number through the bot-fallback dispatch in gitnexus-index.yml so the deploy workflow knows where to comment for command-triggered runs. - Only comments on the PR that asked for the index, never broadcasts. Workflow rename - Drop the "DigitalOcean" suffix from the deploy workflow's display name and filename. The platform is still DO (.do/gitnexus/ still holds the compose + caddy config) but the workflow itself is platform-agnostic in form and the suffix was visual noise. - File renamed gitnexus-deploy-do.yml -> gitnexus-deploy.yml. - Concurrency group and all cross-references updated in lock-step. - permissions at deploy job level now includes pull-requests: write so the completion comment can post.
216 lines
9 KiB
YAML
216 lines
9 KiB
YAML
name: GitNexus Index
|
|
|
|
on:
|
|
push:
|
|
branches: [main, dev]
|
|
paths-ignore: ['**.md', 'docs/**', 'LICENSE', '.github/**']
|
|
pull_request:
|
|
branches: [main, dev]
|
|
paths-ignore: ['**.md', 'docs/**', 'LICENSE', '.github/**']
|
|
workflow_dispatch:
|
|
inputs:
|
|
embeddings:
|
|
description: 'Enable embedding generation (slow, increases index size)'
|
|
type: boolean
|
|
default: false
|
|
force:
|
|
description: 'Force full re-index'
|
|
type: boolean
|
|
default: false
|
|
# When invoked from the /gitnexus index PR command, the command
|
|
# workflow fills these so the index is built from the PR's head
|
|
# ref and uploaded under the PR-numbered artifact name.
|
|
pr_number:
|
|
description: 'PR number to index (set by /gitnexus command)'
|
|
type: string
|
|
default: ''
|
|
pr_ref:
|
|
description: 'PR head SHA or ref to check out (set by /gitnexus command)'
|
|
type: string
|
|
default: ''
|
|
|
|
permissions:
|
|
contents: read
|
|
actions: write # dispatch gitnexus-deploy.yml on bot-triggered runs
|
|
pull-requests: write # post completion comments for /gitnexus command runs
|
|
|
|
concurrency:
|
|
# When triggered by the /gitnexus command, group by PR number so rapid
|
|
# re-runs coalesce. Otherwise group by git ref as before.
|
|
group: gitnexus-${{ inputs.pr_number != '' && format('pr-{0}', inputs.pr_number) || github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
env:
|
|
GITNEXUS_VERSION: '1.5.3'
|
|
|
|
jobs:
|
|
index:
|
|
# Allow push + dispatch unconditionally; filter native pull_request
|
|
# events to contributors only. The /gitnexus command workflow does
|
|
# its own contributor-commenter check before it dispatches this
|
|
# workflow, so workflow_dispatch is always trusted here — including
|
|
# the case where the commenter wants to index a non-contributor or
|
|
# fork PR (the command uses refs/pull/<N>/head so checkout resolves).
|
|
if: |
|
|
github.event_name != 'pull_request' ||
|
|
github.event.pull_request.author_association == 'OWNER' ||
|
|
github.event.pull_request.author_association == 'MEMBER' ||
|
|
github.event.pull_request.author_association == 'COLLABORATOR'
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 25
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v4
|
|
with:
|
|
# When the /gitnexus command dispatches us with a pr_ref, it's
|
|
# a refs/pull/<N>/head ref that GitHub mirrors into the base
|
|
# repo for every PR, so checkout works for fork PRs too. When
|
|
# pr_ref is empty (native push/pull_request), fall back to the
|
|
# default ref actions/checkout would use.
|
|
ref: ${{ inputs.pr_ref || '' }}
|
|
fetch-depth: 1
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v4
|
|
with:
|
|
node-version: 24
|
|
|
|
- name: Cache npm store
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: ~/.npm
|
|
key: gitnexus-npm-${{ runner.os }}-${{ env.GITNEXUS_VERSION }}
|
|
restore-keys: gitnexus-npm-${{ runner.os }}-
|
|
|
|
- name: Run GitNexus Analyze
|
|
env:
|
|
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
run: |
|
|
FLAGS="--skip-agents-md --verbose"
|
|
|
|
# Decide whether to generate embeddings. Rules:
|
|
# push (main/dev) -> always embed
|
|
# pull_request -> embed ONLY when the PR changes files
|
|
# under paths that also trigger backend
|
|
# or frontend unit tests (api/, client/,
|
|
# packages/). Docs/config-only PRs skip
|
|
# embeddings to save ~3-5 min of CI.
|
|
# workflow_dispatch -> respect the explicit `embeddings` input
|
|
# (default false). This also covers the
|
|
# /gitnexus index [embeddings] command.
|
|
ENABLE_EMBEDDINGS=false
|
|
case "${{ github.event_name }}" in
|
|
workflow_dispatch)
|
|
[ "${{ inputs.embeddings }}" = "true" ] && ENABLE_EMBEDDINGS=true
|
|
;;
|
|
push)
|
|
ENABLE_EMBEDDINGS=true
|
|
;;
|
|
pull_request)
|
|
PR_NUM="${{ github.event.pull_request.number }}"
|
|
CHANGED=$(gh api "repos/${{ github.repository }}/pulls/$PR_NUM/files" \
|
|
--paginate --jq '.[].filename' 2>/dev/null || echo "")
|
|
if printf '%s\n' "$CHANGED" | grep -qE '^(api/|client/|packages/)'; then
|
|
echo "PR #$PR_NUM touches unit-test paths (api|client|packages) — enabling embeddings"
|
|
ENABLE_EMBEDDINGS=true
|
|
else
|
|
echo "PR #$PR_NUM does not touch unit-test paths — graph-only index"
|
|
fi
|
|
;;
|
|
esac
|
|
|
|
if [ "$ENABLE_EMBEDDINGS" = "true" ]; then
|
|
FLAGS="$FLAGS --embeddings"
|
|
fi
|
|
if [ "${{ inputs.force }}" = "true" ]; then
|
|
FLAGS="$FLAGS --force"
|
|
fi
|
|
npx --yes gitnexus@${{ env.GITNEXUS_VERSION }} analyze . $FLAGS
|
|
|
|
- name: Verify index
|
|
run: |
|
|
if [ ! -d ".gitnexus" ] || [ ! -f ".gitnexus/meta.json" ]; then
|
|
echo "::error::GitNexus index was not created"
|
|
exit 1
|
|
fi
|
|
echo "::group::Index metadata"
|
|
cat .gitnexus/meta.json
|
|
echo ""
|
|
echo "::endgroup::"
|
|
|
|
- name: Upload GitNexus index
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
# Artifact naming order of precedence:
|
|
# 1. /gitnexus command dispatch: inputs.pr_number -> pr-<N>
|
|
# 2. Native pull_request event: github.event.pull_request.number
|
|
# 3. Push or manual dispatch without pr_number: github.ref_name
|
|
name: >-
|
|
gitnexus-index-${{
|
|
inputs.pr_number != ''
|
|
&& format('pr-{0}', inputs.pr_number)
|
|
|| (github.event_name == 'pull_request'
|
|
&& format('pr-{0}', github.event.pull_request.number)
|
|
|| github.ref_name)
|
|
}}
|
|
path: .gitnexus/
|
|
include-hidden-files: true
|
|
retention-days: 30
|
|
|
|
# GitHub suppresses workflow_run events for workflow runs whose
|
|
# triggering actor is GITHUB_TOKEN (to prevent recursive chaining).
|
|
# That means when this workflow is dispatched by gitnexus-pr-command
|
|
# via `gh api workflow_dispatch`, the deploy workflow's workflow_run
|
|
# trigger never fires. Manually dispatch the deploy here in that
|
|
# specific case — user-triggered runs continue to rely on the
|
|
# existing workflow_run trigger, so we don't double-deploy.
|
|
- name: Trigger deploy workflow for bot-triggered runs
|
|
if: github.triggering_actor == 'github-actions[bot]'
|
|
uses: actions/github-script@v7
|
|
with:
|
|
script: |
|
|
core.info('Triggering actor is github-actions[bot]; workflow_run would not fire. Dispatching gitnexus-deploy.yml manually.');
|
|
// Pass pr_number through so the deploy workflow knows which
|
|
// PR to post its completion comment on (for /gitnexus
|
|
// command runs this will be set; for other bot dispatches
|
|
// it's empty and the deploy step falls back to matrix match).
|
|
await github.rest.actions.createWorkflowDispatch({
|
|
owner: context.repo.owner,
|
|
repo: context.repo.repo,
|
|
workflow_id: 'gitnexus-deploy.yml',
|
|
ref: 'main',
|
|
inputs: {
|
|
pr_number: '${{ inputs.pr_number }}',
|
|
},
|
|
});
|
|
|
|
# Reply on the PR when the /gitnexus command path runs so the
|
|
# requester knows the index step finished. This only fires when
|
|
# inputs.pr_number is set (command-triggered) AND the rest of the
|
|
# job succeeded. A separate comment posts from the deploy workflow
|
|
# when the live server has the fresh index.
|
|
- name: Comment on PR — index complete
|
|
if: always() && inputs.pr_number != ''
|
|
uses: actions/github-script@v7
|
|
with:
|
|
script: |
|
|
const outcome = '${{ job.status }}' === 'success' ? '✅ indexed' : '❌ index failed';
|
|
const prNum = parseInt('${{ inputs.pr_number }}', 10);
|
|
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
|
|
const embeddingsFlag = '${{ inputs.embeddings }}' === 'true' ? 'with embeddings' : 'graph-only';
|
|
const body = [
|
|
`### GitNexus: ${outcome}`,
|
|
``,
|
|
`PR #${prNum} was indexed ${embeddingsFlag}.`,
|
|
`[Index run](${runUrl})`,
|
|
'',
|
|
'${{ job.status }}' === 'success'
|
|
? '⏳ Waiting for deploy to serve the fresh index…'
|
|
: '_Index run failed — the previous index (if any) continues to be served._',
|
|
].join('\n');
|
|
await github.rest.issues.createComment({
|
|
owner: context.repo.owner,
|
|
repo: context.repo.repo,
|
|
issue_number: prNum,
|
|
body,
|
|
});
|