When a legacy GGUF which requires our llama.cpp patch set is loaded, we do a
lazy conversion on disk to make it llama.cpp compatible. To support
downgrades, we temporarily hold both the new and old GGUFs and create a shadow
v1 manifest tag to prevent the downgraded server from deleting the new blobs.
Showing these in the list output is confusing to users and API consumers. The
legacy copies will be removed in a future release to recover the disk space.
Store named manifests as regular files on Windows, where symlinks can
be created successfully but fail when opened.
Repair existing symlinks lazily on read using the verified blob and
existing copy path. Keep locking consistent across platforms and avoid
reacquiring the store lock during pruning.
Fixes#18847
The Clef head read its weights with std::ifstream::seekg. The Windows
llama-server ships with libc++ 18.1.8, where that seek truncates the
offset to 32 bits. clef-flash stores the clef.* tensors at about 9.5GB,
so the head loaded backbone bytes as its weights and every
/v1/systemone request failed with "Clef: non-finite logit". clef:27b
keeps the head and output.weight below 2GB and was unaffected.
Read with ggml_fopen and an explicit 64-bit seek instead, as read_at
in llama/compat already does.
Fixes#18769
Forward GET /api/usage and GET /api/balance through the local server
using the signed-in account.
Add API documentation and tests for proxying, signing, and
disabled cloud access.
Implements the EmbeddingGemma2Model architecture on the MLX runner: 24-layer
bidirectional text encoder with PLE, shared gemma4 vision/audio towers,
mean-pool + L2 output. /api/embed accepts per-item media via input dicts.
* Reduce model lookup and MLX decision request overhead
Avoid decoding unrelated manifests when resolving existing model names.
Reuse small Metal scratch buffers between decision requests, releasing
larger working sets and clearing scratch after scoring errors.
* address comments
The create tests upload blobs which can cause problems when run in parallel on multiple systems. This change splits them out with an explicit tag to make it easier to control when they are run.
* mlxrunner: restore System One scoring
Carry the MLX implementation and its tests from the original System One
commit. The API/compiler and llama-server implementation already exist on
the release branch, so retain those versions. Adapt the optional model test
to the renamed decision package and its explicit rendering/context setup.
(cherry picked from commit 26e546ddb3)
* Enable MLX System One and add regression coverage
* mlxrunner: add Laya support for System One
Import Laya checkpoints and implement bidirectional decision scoring.
* mlxrunner: release cached buffers after System One scoring
* decision: match Tev1 and Nimble publisher prompt formats
Select Tev1 prompts using renderer metadata and fix JSON formatting, with regression coverage for GGUF and MLX.
* mlx: support Clef and Clef Flash decision models
Add model import, quantization safeguards, and text/image scoring with the Clef joint decision head.
* support vision capabilities for decision models
* Make Clef scoring cancellable between prefill chunks
Preserve joint-head inputs and test cancellation, cleanup, and recovery.
* mlx: separate decision preparation and readout from forward
Let serial and batched scoring share model-owned preparation and readout
without duplicating architecture-specific logic in the runner.
* mlx: add Strands Decider support
* implement caching
---------
Co-authored-by: ParthSareen <parth.sareen@ollama.com>
The [img] placeholders mixed internal markers into tool text: the native
llama-server path sends them as literal text, so an image-only result
gained a text part, and tool output containing [img] or [img-N] moved or
dropped image tags. api.Message cannot carry image positions without
in-band markers, so stop merging tool results that include images.
Text-only tool results, the only array form the Chat Completions spec
allows for tool messages, are still merged into one message (now with a
strings.Builder). Results with images keep the per-part split in order,
which preserves image positions, with the tool call id and name set on
every part.
Merging a tool result's parts collected its images separately from the
text, so renderers placed them all before the text: "Before: <A> After:
<B>" rendered as "<A><B>Before: After:". Write an [img] placeholder at
each image's position so renderContentWithImageTags and chatPrompt put
each image back where it was.
Show and list now return just "decision" for models declaring it, so clients don't offer them for general chat, tools, or thinking. The show response also omits the thinking field for these models. Scheduling and serving still check the full capability set.
A tool message with "content": [] produced no message at all, so the
tool call it answers had no result in the rendered prompt. Emit an empty
tool message that keeps its ToolCallID and ToolName, and skip it only
when it has no parts, call id, or name.
FromChatRequest splits array content into one api.Message per part. That
is intended for user content, but a tool message is a single result: the
split also dropped its tool_call_id and tool name, since those were only
copied when the message had tool_calls. Renderers that resolve tool
results by ID then see extra, unnamed results; for example, Kimi K3 fails
with "tool message 2 has no resolvable tool name" on a tool result with
two text parts, which the OpenAI spec allows.
Merge the parts of a tool message into one message with the text joined,
the images kept, and ToolCallID and ToolName set, matching how
function_call_output items are converted for /v1/responses.
