The executor aborted the whole plan on any node failure, which rules
out modeling optional semantics — a dependency declared with
required: false must be reported as skipped, not fail the operation,
and its dependents must still run. This is a prerequisite for planning
the start phase (dependency-condition waits) inside the reconciler.
Operation gains Optional: at the walker level, a failing Optional node
emits a Skipped event plus a warning and completes successfully, so
dependent nodes proceed. It is documented against the pre-existing
BestEffort flag, which is narrower (one specific expected error,
tolerated inside the operation itself) — the two docs now
cross-reference each other so they cannot be confused.
Covered by an executor unit test asserting a failing Optional node
does not fail the plan and its dependent still executes.
Groundwork for the start-in-plan convergence.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
waitDependencies returned nil when its context expired: every polling
goroutine hit the ctx.Done branch and reported success, so the
'timeout waiting for dependencies' translation after eg.Wait() was
unreachable in practice, and up --wait / start --wait could succeed
silently while nothing was healthy. The error path was only ever taken
by a race, when a health probe happened to be in flight at expiry.
The ctx.Done branch now distinguishes the two reasons the context can
end: a deadline is exactly the failure this function is asked to
detect and is surfaced (callers translate it into their own messages:
'timeout waiting for dependencies', 'application not healthy after X');
a plain cancellation (user interruption) keeps returning nil so
Ctrl-C does not masquerade as a dependency failure — preserving the
behavior of the restart/run/up callers that pass no timeout.
Both paths are now pinned by unit tests; neither message was covered
before.
Part of #14074 (C) and groundwork for the start-in-plan convergence.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
docker compose events <name> was passing service arguments straight to
the event filter without checking they exist in the project, so a typo
would produce an event stream matching nothing and block forever with
no output and no error.
Switch to projectOrName (which validates args via ToProject) before
calling backend.Events, consistent with logs, kill, and restart.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
When a locally-installed binary with the same name (e.g.
example-provider) sits in an earlier PATH directory than
bin/build/, exec.LookPath resolves the wrong binary. The
old tests appended the build directory to the end of PATH,
so any pre-existing system binary shadowed the test one.
Prepending instead ensures the freshly built binary always
wins, regardless of what is installed on the developer's
machine. CI is unaffected since no such system binary
exists there.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
doBuildBake mixed five concerns in a 300-line body (cognitive
complexity 88): progress display setup, translation of the project
into a bake file definition, temp metadata file allocation, bake
command construction, stderr rawjson streaming, and result
collection. Each now lives in its own function; the driver reads as
the sequence of those stages (cognitive complexity 17), and the
FIXME suppression is gone.
The bake variable was named to avoid shadowing the docker/cli
'build' package. No behavior change: prepareBakeBuild emits the same
config, and the stderr loop keeps the decoder-per-line semantics.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Cognitive complexity (gocognit, threshold 30 — the linter default) fits
this codebase better than raw cyclomatic complexity: it barely charges
guard clauses and early returns, and penalizes nesting instead. As a
result 8 of the 21 //nolint:gocyclo suppressions become unnecessary,
while deeply nested functions gocyclo never flagged are now caught.
The 19 functions still above the threshold keep a suppression, each
marked FIXME to complete the migration by restructuring them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Show an ENGINE column in `compose ps` default table output when the
com.docker.compose.engine label is present. Expose {{.Engine}} for custom
--format templates.
Rename api.EngineLabel to api.ContainerEngineLabel to avoid the
name collision with desktop.EngineLabel (com.docker.desktop.address).
Signed-off-by: Nick Sieger <nick@nicksieger.com>
Inline small functions that were extracted from their single call site
without a real boundary to justify it — no reuse, no dedicated test, no
responsibility of their own — so each caller now tells its whole story
top-down:
- getExecTarget, attachContainer, logContainer: one-line trampolines to
getSpecifiedContainer / doAttachContainer / doLogContainer, which
other call sites already use directly
- removeImage: single-statement wrapper, unlike its removeVolume
sibling which has actual logic
- checkSelectedServices: named like a validation, actually a filter;
the subtle rule (an unknown service is only an error with an explicit
compose file) now reads where options.Services is rewritten
- prepareLabels: mutated the map it received while looking pure at the
call site; the label writes are now visible in getCreateConfigs
- setDefaultTarget: mutation-by-pointer of the loop copy, now visible
in the loop of injectFileReferences
- buildVolumeOptions/buildTmpfsOptions/buildImageOptions: nil-guard +
field copies; the buildMountOptions switch now shows side by side
what each mount type propagates (buildBindOption keeps real logic and
keeps buildMountOptions under the gocyclo limit)
- displayDryRunBuildEvent: was longer than its only caller
- hasMore: read like a predicate, was a one-line spinner restart
- escapeDollarSign: wrapped a single bytes.ReplaceAll
- extractEnvCLIDefined: replaced by the canonical compose-go helper
types.NewMappingWithEquals().ToMapping(), as run.go already does
- isPullPolicyValid: rebuilt the valid-values slice on every call; now
a package-level list checked at the call site
- viewFromStackList: projection now sits next to the render closure
that consumes exactly its three fields
No behavior change; single gocyclo threshold untouched.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A running container may reference an image record that no longer
exists: under the containerd image store, `up --build` with identical
content moves the tag to a new index digest (provenance attestation
churn) and the daemon drops the old index the container was created
from — without compose recreating the container, by design (#13636).
`docker rmi -f` of a running container's image produces the same state.
`compose images` used to fail the whole listing on the resulting
NotFound. Degrade to what the container itself knows (image ID, and
repository/tag when the reference is not a raw ID) instead of failing.
Fixes#14014
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The 'builder does not support multi-arch' subtest expects the docker
driver to reject multi-platform builds, and skipped itself by checking
that buildx lists both linux/amd64 and linux/arm64. That heuristic
misses containerd-store hosts without binfmt emulation (plain CI
runners): only native platforms are listed even though the driver
happily cross-builds, so the build succeeds and the test fails with
'ExitCode was 0 expected 1'. Skip on the containerd image store
directly, detected from the daemon's DriverStatus.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Regression test for #14005, pulled-image scenario: with the source image
of a type=image volume already present in the local store, compose used
to rewrite the mount source to a digest the daemon can't resolve as a
mount source under the containerd image store (No such image).
TestImageVolume only exercises this path when a previous test happens to
have left the image locally; pre-pulling makes it deterministic. Also
asserts a second unchanged up doesn't recreate the service.
The test fails on main under the containerd image store and passes with
the fix from #14011.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Regression test for #14007: with the containerd image store, a local
multi-platform image holding the requested non-native variant must be
used as-is by the default missing pull policy — compose used to inspect
the image without a platform, compare the host variant's platform fields
to the requested one, conclude the image is missing and try to pull the
(unpublished) tag.
The test fails on main and passes with #14011, whose platform check
resolves the requested platform against the locally available manifests.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Compose manipulates two digest kinds that must never be conflated:
the platform-specific content digest (localContentDigest) is ONLY an
identity to compare a running container with a fresh build/pull, while
pinning image references in a reproducible compose model (publish /
config --resolve-image-digests) must keep the registry descriptor
digest — the multi-platform index — through ImageDigestResolver, or
the published file would be bound to the platform of whoever resolved
it. Document the distinction on both producers and lock it with a test
that fails if the resolver ever goes through a local image inspect.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
- dry-run up with a missing image must not hit the real daemon
- create twice with a non-native DOCKER_DEFAULT_PLATFORM is idempotent
- two services sharing an image with mixed platform pinning get their
own platform's digest label, idempotently, whatever pull ordering
- explicit pull refreshes ahead of a daily refresh window
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
- pullRequiredImages resolves each distinct pulled image once, after
all pulls completed, for the host default platform — the exact way
getLocalImagesDigests resolves already-local images. Resolving from
each pull's goroutine, for the pulled platform, let the recorded
digest depend on pull completion order when several services pull
the same tag for different platforms (last-writer-wins), and made
the first up after a pull recreate containers. Platform-pinned
services keep getting their own platform's digest from
serviceImageDigest. In dry-run nothing was actually pulled, so the
local inspect is skipped — it reached the real daemon through a
DryRunClient dispatch that no longer matched and failed with 'No
such image'.
- explicit 'compose pull' treats daily/weekly/every_N windows as due:
it is the only way to force a refresh ahead of the window.
- 'compose pull' no longer silently skips services declaring both
provider: and image:.
- pre_start hook images go through the same mustPull interpreter on the
up path as on the pull path, so refresh windows apply consistently.
- matchLocalManifest falls back to the inspect's flat platform fields
when the lone available manifest carries no ImageData (locally built
images), instead of reporting the platform unsatisfied and pulling a
possibly local-only image.
- scale resolves DOCKER_DEFAULT_PLATFORM without validating
build.platforms: a conflict on a service that isn't being built must
not abort the command. run needs nothing: createOptions.Apply
resolves and validates platforms on its path already.
- resolveImageVolumes documents the accepted one-time-recreate and
retag-race tradeoffs.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
compose pull switched on the raw pull_policy string: daily/weekly/every_N
never matched a case and fell through to an unconditional re-pull, and
the hook-image loop was a second interpreter that ignored the refresh
window entirely. Delegate the decision to the exact interpreter the up
path uses (mustPull), with hook images routed through the same decision
(build mapped to missing — a hook image can't be built as a fallback).
Two deliberate differences with up are kept and documented in
shouldPullImage: a service without an explicit pull_policy is always
refreshed (skipping it would turn an explicit compose pull into a no-op
once images exist), and a present latest tag is still refreshed under
missing/if_not_present — the tag is expected to move, and triggering the
pull lets the daemon negotiate with the registry, a manifest check with
no download when the local image is already current.
User-visible change (changelog): compose pull now honors
daily/weekly/every_N refresh windows instead of always re-pulling.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Image identities recorded for staleness detection were produced by
several independent paths yielding different digest kinds for the same
image: the platform check compared flat inspect fields while the digest
picked a manifest with the host matcher (never the service's pinned
platform), a wrong-platform summary just discarded still leaked its
digest into the label, bake substituted digests host-side in batch, and
the classic builder recorded the raw build-stream ID as-is. Any of those
mismatches makes the next up see a phantom image change and recreate
containers.
Converge every producer on one selection (matchLocalManifest /
localContentDigest): the shared parallel inspect feeds both the digest
and the platform check, platform-pinned services resolve THEIR platform's
manifest in-process (no extra API call), and both builders route through
canonicalBuiltDigest. Registry-only builds (push-only, multi-platform
without load) keep the builder-reported digest — volatile but honest, an
actual rebuild is still detected, where a stable placeholder would hide
real image changes. ensureImagesExists' final loop becomes the label's
single writer so the pinned resolution can't be overwritten, superseded
only by pull/build results already platform-resolved by their producers —
and when a pull or build refreshed the shared entry mid-run (a digest
resolved for whichever service triggered it), a service pinned on another
platform re-resolves its own with one extra inspect, in that case only.
With every producer converged, TestUpIdempotentContainerdStore is
un-skipped here.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
pullServiceImage returned the pulled image's raw inspect ID, while
getImageSummaries resolves already-local images through contentDigest
(the platform image-manifest digest). Both values feed the
com.docker.compose.image label that mustRecreate compares to detect
image changes, so the two paths disagreeing made the first 'up' after
the pulling 'up' see a phantom image change and recreate every
container once, with no change anywhere. Under the containerd image
store a tag@digest reference triggers this: the raw inspect ID is the
index digest, while contentDigest picks the platform manifest digest.
Resolve the pulled image through the same manifests-aware inspect and
contentDigest call getImageSummaries uses, so both sides of the
staleness comparison speak the same scheme.
Verified against a fresh docker:dind (29.7.0, containerd store) with a
tag@digest service: unpatched v5.4.0 recreates the container on the
second 'up'; with this fix the container survives repeated 'up' runs.
Existing behavior is preserved for engines without manifest support
(contentDigest falls back to the plain ID).
(Squashed with the follow-up lint cleanup from the same PR.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Max Malm <benjick@dumfan.net>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The e2e suite only ran on graphdriver daemons, where the different kinds
of image digests coincide — the blind spot that let #13636, #13998 and
#14005 through. Add one matrix entry enabling the containerd image store,
plus TestUpIdempotentContainerdStore: two consecutive `up` runs with no
change must not recreate any container. The test is red on this
configuration (the com.docker.compose.image label is written from the
index digest on the pulling run, then compared against the per-platform
manifest digest on the next run) and skipped until the next commit
resolves the pull-path digest.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
With the containerd image store, ImageSummary.ID holds the digest of the
platform-specific manifest so ServiceHash stays stable across attested
rebuilds (see contentDigest). resolveImageVolumes reused that same value
as the `type: image` mount Source, but the daemon only resolves a mount
Source by name/tag or top-level image ID, not by manifest digest — so
`compose up` failed with "No such image" whenever the volume's source
image was already present locally (always for a built image; on a
second run for a pulled one).
Keep Source as the resolved image name, and track the digest separately
via a new com.docker.compose.image-volume-digest label so mustRecreate
can still detect a rebuilt/updated source image independently of Source.
Fixes#14005
Signed-off-by: Ricardo Branco <rbranco@suse.de>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
`up --abort-on-container-exit` tore down the project when a one-off
container created by `docker compose run` exited. The container list
built in monitor.Start already excludes one-off containers via
oneOffFilter(false), but the event subscription did not, and the only
guard on incoming events is the service label, which a one-off
container inherits from the service it was run from.
Regression since v2.39.0 (last unaffected release is v2.38.2), and a
recurrence of docker/compose-cli#1955, fixed there by
docker/compose-cli#1987.
Signed-off-by: Branislav Osif <brano@osif.digital>
CI runs the tests as root, where the unreadable-directory test is always
skipped. Extract the WalkDir callbacks and inject the watch registration
so a synthetic permission error can drive those branches instead.
Signed-off-by: Endika Iglesias <endika2@gmail.com>
Containers are created from a project whose service environment is
resolved (env_file merged into environment), but since v2.22.0 `config
--hash` skipped that resolution, so hashes diverged from the
com.docker.compose.config-hash label for services using env_file.
Resolve the environment of the hashed services only, so a broken
env_file or platforms on an unrelated service still doesn't prevent
hashing, and honor --no-env-resolution as an escape hatch.
Fixes#14001
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
pruneDanglingImagesOnRebuild probed the name-keyed built-images map with
an image ID, so the spare-check never matched and every dangling image
carrying the project label was removed on each rebuild with --prune —
not just the superseded ones. Match dangling IDs against the map's
values (the freshly built image IDs) instead.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The official nginx images set STOPSIGNAL to SIGQUIT which dumps core.
Set it to SIGTERM to avoid dumping core on e2e tests when containers
running "sleep infinity" are stopped.
Signed-off-by: Ricardo Branco <rbranco@suse.de>
Pulling an oci:// resource resolved each layer digest through the
registry manifests endpoint, which answers 500 when the digest points
to a non-manifest blob. containerd v2.3.0+ (pulled in by buildx v0.36
and buildkit v0.32) no longer falls back to the blobs endpoint unless
manifests returned 404, so publish/pull of compose artifacts broke.
Fetch layers directly with the descriptors already listed in the
manifest instead of resolving them again.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Follow-up to #13603: scale, watch and shell completion loaded the
project without any tolerance option, so a missing env_file on a
service not involved in the operation aborted the command, while
up/exec/ps already tolerate this since #13156 and #13603.
Mirror the WithServices pattern: load with WithoutEnvironmentResolution
and resolve the environment once the project has been reduced to the
selected services, so targeted services still get their env_file
validated. Completion only needs names and never resolves. This also
aligns the config hash of scale-created containers with up-created
ones.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The runServices, runVolumes, runNetworks, runModels and runHash paths
called ProjectOptions.ToProject directly, bypassing the configOptions
wrapper that applies --no-consistency, --no-interpolate, --no-normalize,
--no-path-resolution, --profile filtering and env_file discarding.
Restore the variadic wrapper (mirroring configOptions.ToModel) and route
the five call sites through it.
Regression introduced by b80bb0586 (LoadProject API migration).
Fixes#13974
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The caller in pkg/compose/cp.go unconditionally defers
res.Content.Close() once the call succeeds. The dry-run client
returned a zero-value result with a nil Content reader, so
`docker compose cp --dry-run <ctr>:<path> <dst>` panicked with a
nil pointer dereference. Return an empty NopCloser instead, matching
the pattern already used for the other stream results in this file.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
addPreStartHookPulls skipped any hook image already present locally,
regardless of pull policy. As a result a service with pull_policy: always
had its own image force-pulled on every up while its pre_start hook images
were left stale — diverging from both the service image and the `pull`
command path (which already re-pulls hooks under always).
Skip the "already present" shortcut when the parent service is
pull_policy: always, so hook images get the same force-pull treatment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Address review feedback on pre_start hook image resolution:
1. GetDependentImages now skips a hook image equal to the service image
(resolved via GetImageNameOrDefault), so `config --images` no longer
prints a duplicate line and pullRequiredImages no longer schedules a
redundant pull for it.
2. pullRequiredImages (up/create path) now dedups dependent images by
reference via a `scheduled` set, so several hooks/services sharing the
same missing image don't schedule concurrent redundant pulls. The hook
pass moved to a helper (addPreStartHookPulls) to keep complexity in check.
3. The `pull` command no longer skips hook images under `pull_policy: build`.
A hook image is a registry image that can't be built, so only `never`
justifies skipping it — making `pull` consistent with the `up` path.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
pre_start hooks run as ephemeral init containers with their own image
(ServiceHook.Image), but that image was ignored by image resolution:
`config --images` didn't list it, `pull` didn't fetch it, and `up` failed
at runtime with "No such image" when it wasn't already present locally.
Add a GetDependentImages helper that returns a service's pre_start hook
images, and use it wherever service images are collected/pulled:
getLocalImagesDigests, pullRequiredImages (up path), the pull command, and
`config --images`. Hook images inherit the parent service pull policy.
post_start/pre_stop hooks run via ExecCreate inside the service container
and never use hook.Image, so they are intentionally out of scope.
Digest resolution/locking (--resolve-image-digests / --lock-image-digests)
is not covered: compose-go's WithImagesResolved only resolves service.Image
(needs an upstream change), and the --lock-image-digests override merges
pre_start by concatenation, which would duplicate hooks.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
collectObservedState indexed networks/volumes by compose label into a
single-valued map, so two live resources sharing a label (e.g. a leftover
after a rename) collided and the "winner" depended on the daemon's list
order — a nondeterministic `up` (spurious create/recreate events, possible
churn) on subsequent runs.
Make collection lossless and move the selection into the reconciler:
- ObservedState.Networks/Volumes become map[string][]Observed*: collection
records every label-sharing resource and makes no premature choice.
- selectNetwork/selectVolume deterministically pick the resource matching
the desired name (else the lexicographically smallest), returning the
others as orphans.
- reconcile resolves the observed state once (resolveObserved) into
single-valued resolvedNetworks/resolvedVolumes used everywhere, and warns
about orphans instead of acting on them — they are left untouched because
removing them could drop data or break unrelated workloads.
Adds selection unit tests, a collector aggregation test and a reconcile
conflict test (deterministic no-op + orphan warning across list orders).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A network rename does not require removing the old network — the new one
has a different name and is created independently. Yet the old removal
could block the whole operation: NetworkRemove fails when non-Compose
containers are still attached, and CreateNetwork depended on it.
Split the rename path from the same-name divergence path:
- Rename: CreateNetwork no longer depends on RemoveNetwork; the container
migration proceeds regardless. RemoveNetwork is marked best-effort and,
if the network is still in use (reported as a conflict), is skipped with
a warning instead of failing. Any other error (transport, Moby API) is
still propagated.
- Same-name divergence keeps the mandatory remove-before-create ordering.
Adds an Operation.BestEffort flag, honored by execRemoveNetwork, plus
reconcile and executor tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Address review findings on the network reconcile migration:
- createNetwork now treats a NetworkCreate conflict as success. A
concurrent `docker compose up|run` can create the same network in the
TOCTOU window between the observed-state snapshot and the create call;
the previous ensureNetwork retried on conflict, the plain create must
not fail hard.
- discoverUnmanagedNetworks/Volumes preserve the config-hash when the live
resource is owned by this project (project label present, key label
absent — e.g. written by an older Compose) so genuine divergence is
still detected. For resources we don't own the hash stays empty and they
are reused untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Treat a network rename (observed.Name != desired.Name) as a recreation
rather than an additive create: the old network is removed, the new one
created, and attached containers are migrated onto it (reconnected), so
they no longer stay on the previous network until recreated for another
reason.
Networks carry no data, so removing the previous network — instead of
leaving it dangling alongside the new one under the same compose label —
is safe and keeps subsequent runs deterministic. This is a marginal
behavior change from previous Compose releases (which created the new
network and left the old attachments in place) in exchange for the more
logical outcome.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Mirror the volume reconciliation work (#13962) for networks: move network
divergence detection and recreation out of the imperative pre-reconcile
path (ensureNetwork/resolveOrCreateNetwork/removeDivergedNetwork) into the
reconciliation plan.
- reconcileNetworks now owns creation of missing networks and, for a
network whose config-hash diverged, an explicit recreation sequence
(no user confirmation: recreating a network is not destructive):
stop containers -> disconnect -> remove network -> create network ->
reconnect containers. Attached containers keep their identity (they are
reconnected, not recreated), matching the previous behavior. If a
container is independently recreated by reconcileContainers, its removal
is ordered after the reconnect so they don't race.
- Renaming a network creates the new one additively and leaves the old one
untouched.
- collectObservedState discovers legacy/unlabeled networks by name and
records them as unmanaged matches (empty config-hash) so the reconciler
reuses them untouched; ownership warnings move to warnUnmanagedNetworks.
checkExternalNetworks keeps external-network validation/resolution.
- execCreateNetwork now issues a plain createNetwork; the imperative
ensureNetwork/resolveOrCreateNetwork/removeDivergedNetwork and the
connect/disconnect helpers are removed.
Adds reconcile, observed-state and executor tests covering network
create/diverge/rename, the entangled diverge+recreate case, legacy
by-name discovery and the ownership warnings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Rely on compose-go WithImagesResolved, which now resolves dependent
images — `type: image` volume sources and pre_start hook images —
with its already-digested guard, per-call memoization and
sibling-service detection (compose-spec/compose-go#894, #899), rather
than duplicating resolution logic CLI-side. The interpolated path gets
this for free; --no-interpolate maps the raw model onto a
pseudo-project keyed by service names to reuse the same resolution,
and --lock-image-digests keeps type:image volumes in its output.
pre_start hooks can't be carried into the lock override (hook lists
are appended on merge), so generating a lock warns that hook images
stay unpinned there.
As a side effect, `compose publish` now fails fast on unresolvable
dependent images.
Fixes#13827
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The additive rename path created the new volume but kept the old name in
the observed state, so hasVolumeMismatch never fired: existing containers
stayed on the old volume while fresh replicas mounted the new one
(split-brain), and later runs picked a nondeterministic winner between the
two equally labelled volumes.
Rewrite the observed volume name to the desired one after planning the
"renamed" create, so reconcileContainers migrates the existing containers
onto the new volume in the same up — restoring parity with the old
ensureVolume path — while still leaving the old volume and its data intact.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Two edge regressions from the switch to a label-scoped observed state,
both reported against the old ensureVolume path:
- A same-named volume created manually or by another project (no compose
label) was invisible to the observed state, so a VolumeCreate was
planned on every up: a hard failure if the driver differed, spurious
Creating/Created events otherwise. collectObservedState now discovers
such volumes by name (pre-label Compose semantics) and records them as
unmanaged matches with an empty config-hash, so the reconciler reuses
them untouched. The ownership warnings move to warnUnmanagedVolumes,
driven off the observed state; checkVolumes shrinks to external-only
validation (checkExternalVolumes).
- Renaming a volume hit the diverged path and, with up -y, deleted the
old volume and its data (VolumeHash includes Name), where it previously
just created the new one. When observed.Name != desired.Name the volume
is now created additively, leaving the old one untouched, with no prompt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
servicesUsingVolume only matched services mounting the volume directly, so
a service reaching it through volumes_from was not stopped/removed before
RemoveVolume. Docker materializes the inherited mount on the consumer's
container, so its removal would fail with "volume in use". Compute the
transitive volumes_from closure so every container referencing the volume
is removed first. (network_mode/ipc/pid: service:x share namespaces, not
mounts, and are intentionally excluded.)
Also reassign the result of Labels.Add in createVolume: it mutates in
place only when the map is non-nil, so discarding the return would drop
the config-hash label for a volume with no CustomLabels.
Addresses review feedback: documents why observed.Containers is cleared
without touching the observedContainersByService hashing snapshot, and
strengthens the cascade tests to assert the full plan ordering.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Move volume divergence detection and recreation out of the imperative
pre-reconcile path (ensureVolume/removeDivergedVolume) and into the
reconciliation plan, activating the dormant planRecreateVolume seam.
A diverged volume now produces an explicit, forward-only sequence:
stop containers -> remove containers -> remove volume -> create volume
-> create containers. Container re-creation is delegated to
reconcileContainers (affected services are cleared from the observed
snapshot so they are scheduled fresh, gated on the CreateVolume node),
and the recreation cascades to namespace/volume-sharing dependents.
User confirmation (recreate, data will be lost) is consulted while
building the plan via reconciler.prompt; declining leaves the volume
untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
`docker compose -f oci://<insecure-registry>/... up` failed against a
plain-HTTP registry unless --yes was passed:
failed to pull OCI resource "localhost:5000/test:interpolated":
Head "https://localhost:5000/v2/test/manifests/interpolated":
http: server gave HTTP response to HTTPS client
`up` loads the project twice. The first load goes through ToProject,
which built its OCI options from --insecure-registry correctly. Without
--yes, checksForRemoteStack then calls promptForInterpolatedVariables,
which re-loads the project through ToModel to list the interpolation
variables. That second load builds its own resource loaders via
remoteLoaders, and those passed an empty api.OCIOptions{}, dropping the
flag. Since the OCI loader always performs a network resolve, the
re-load spoke HTTPS to a plain-HTTP registry and failed before the
prompt could be shown.
The two construction sites had drifted apart, so rather than patching
the second one, both now share ProjectOptions.ociOptions(). `config`
and `viz` use the same ToModel path and are fixed as well.
Covered by an e2e case in TestPublish, which already runs an insecure
registry and an oci:// round-trip: it publishes a fixture carrying an
interpolation variable so the prompt fires, then runs `up` without
--yes and declines, asserting the re-load does not fail with
"server gave HTTP response to HTTPS client".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Domantas Petrauskas <dom.petrauskas@gmail.com>
With the containerd image store and BuildKit provenance attestations
(the default), a built image is stored as an attested index whose
top-level digest also covers the attestation manifest. That digest
churns on every build even when the runnable content is unchanged,
so compose recreated containers on every `up --build`.
Compare the digest of the "image" kind manifest instead, selected for
the target platform and restricted to locally available manifests, so
it is deterministic and reflects only config + layers. Both the build
and up sides of the staleness check go through the same selection, and
registry-only images keep the Bake-reported digest.
Fixes#13636
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
`docker compose config --no-interpolate <service>` and
`docker compose config --variables <service>` load the raw model
without applying service filtering, so the full model is rendered
regardless of the services passed as arguments. Filtering will not
be supported on these paths, so emit a warning to make sure users
are no longer misled by silently ignored arguments.
Fixes#13614
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
