7 April 2026
MLX LM v0.31.2 updates caching, serving, and model workflows
MLX LM v0.31.2 is a first-party maintenance and feature release centered on prompt caching and batch generation, with additional generation controls, server improvements, benchmark support, model-related work, and bug fixes. The supplied release record is incomplete, so it should not be read as a full changelog.
MLX LM v0.31.2, published on 7 April 2026, concentrates on prompt-cache handling, batch-generation work, serving improvements, benchmarking, model support, and targeted fixes. The available first-party release record is partial, so users should treat it as an overview rather than a full account of every change.
01
What we know now
- 01
[1] Official MLX LM GitHub release record for v0.31.2, published 7 April 2026: https://github.com/ml-explore/mlx-lm/releases/tag/v0.31.2.
- 02
The release record highlights non-trimmable-cache prompt/message caching and batch-generator refactoring.
- 03
The same record lists generation penalties, cache and dataset fixes, server changes, benchmark delay support, and model-related updates.
02
Why this matters for Armenia
Relevant to developers in Armenia who use MLX LM for local AI workflows, particularly where caching, serving, benchmarking, and model compatibility affect development work.
03
Caching and batching
System and user-message caching is highlighted; batch generation was refactored.Serving
Allowed-origin support and missing content-length handling are listed.Generation and models
Penalties and model-related changes are included.Benchmarking
Delay support is listed for the benchmark tool.This is a high-level map of changes explicitly identified in the supplied release record, not a complete changelog.
04
Caching and batch-generation changes
For users with repeated prompts or multi-turn interactions, the headline cache changes may be the most directly relevant part of v0.31.2. The release notes point to improved handling of system and user messages in non-trimmable caches, while also recording several cache-related corrections. The notes do not quantify any speed or memory effect, so those outcomes should be tested in each deployment.
- The release highlights caching of system prompts and user messages for non-trimmable caches.
- It also highlights a refactoring of the batch generator.
- The change list includes fixes associated with cache checkpointing and a missing cache advance in a Qwen 3.5 path.
05
Generation and training-path maintenance
The release adds presence and frequency penalties, giving applications additional generation controls. Elsewhere, the change list records a dataset-related crash fix and implementation maintenance around RoPE-related components. The first-party notes do not specify interfaces, defaults, or behavioral details for the new penalty settings.
- Presence and frequency penalties are listed as an addition.
- The release includes a fix for a CompletionsDataset mask-prompt crash.
- It also lists a change intended to avoid mutating input in SuScaledRoPE and YarnRoPE.
06
Server and platform-facing updates
For projects exposing MLX LM through a server, v0.31.2 includes changes to origin configuration and request handling. The missing content-length item may matter for clients that do not provide that header. The supplied notes name the features but do not document deployment configuration or compatibility requirements.
- Server support for allowed origins is listed.
- The server is updated to handle a missing content-length header.
- The release notes also mention a move to device information described as metal agnostic.
07
Model, benchmark, and reliability work
The available record points to broader maintenance beyond the headline caching work. It names Nemotron support, a Qwen3 Coder parser fallback, support for delay in the MLX LM benchmark, and several reliability-oriented fixes. Because the summary ends mid-list, users should consult the official release page before assuming that these are the only relevant changes.
- Nemotron support is listed among the model-related work.
- A malformed-JSON fallback is listed for the Qwen3 Coder tool parser.
- Benchmark delay support is included.
- The notes also list test fixes and trainer-cache memory clearing.
08
Practical next steps
Teams considering this release can focus validation on the areas changed in the first-party notes.
- 01
Review prompt-cache behavior if an application uses non-trimmable caches.
- 02
Test generation settings that rely on presence or frequency penalties.
- 03
Check server deployments that need allowed-origin configuration or must accept requests without a content-length header.
- 04
Re-run benchmark workflows if they use delayed execution.
- 05
Verify model-specific workflows, including any Nemotron or Qwen-related paths used by the project.
09
Limits of this edition
The supplied release summary is truncated and does not establish the complete changelog.
The release record identifies changes but does not provide performance measurements or migration guidance.
Compatibility implications for individual applications and models are not established by the supplied material.
SRC
Source desk
Direct links to the material behind this selection. Seeing the source matters as much as reading the synthesis.