Liquid’s local vision model gets a faster way to write its answers

Liquid AI has released an experimental companion model designed to make a local vision model write its answers faster. Announced September 24, LFM2.5-VL-DSpark drafts possible continuations for the company’s LFM2.5-VL-3B model to check. Liquid reports substantial gains on Apple computers and an Nvidia server GPU, with an important limit: the technique speeds answer generation, while the work of encoding an image and processing the prompt remains.[1]

On an M5 Max, Liquid reports a 2.69-fold decoding speedup on an image-question benchmark, against 1.56-fold for the complete request. Both compare the same target model with and without its drafter. These are company results, not an independent test.[2]

For someone asking questions about a diagram or a photograph, the complete request is the experience that matters. A faster stream of words is welcome, but it may follow a substantial wait before the answer begins.[1]

A second model proposes; the first checks

Ordinary autoregressive generation builds an answer in sequence: the model produces a token, then runs again to produce the next. Speculative decoding inserts a cheaper model into that process. It proposes several tokens, and the larger model verifies them together. Accepted guesses save expensive sequential steps; rejected guesses require correction. The method can preserve the target model’s output distribution when its verification and sampling rules are matched correctly.[3]

The small model therefore does not get to substitute an unchecked answer. Its useful skill is agreement with the model doing the verification. A drafter that frequently guesses wrong consumes time without avoiding enough of the larger model’s work. That is why the speed of a small model on its own is an incomplete guide to the speed of the pair.[3]

Liquid’s roughly 280-million-parameter drafter uses information from the target model’s internal state. The reported tests use 16-bit weights; the company leaves performance with quantized weights outside this release’s scope. Someone running a smaller, compressed version should not assume the published multipliers will transfer unchanged.[1]

The image still takes time

The broader research behind the benchmarks suggests that this problem deserves its own tests. MMSpec, a March research preprint, assembled 600 samples across six multimodal task categories and evaluated ten speculative-decoding methods. Its authors found that throughput alone could obscure the latency a user experiences, and that techniques developed for text did not automatically work as well on image tasks. That study provides context for evaluating this release; it is not an independent test of Liquid’s September checkpoint.[4]

Consider an illustrative request that spends two seconds preparing its input and two seconds generating its answer. Making generation three times faster reduces the total from four seconds to about 2.67 seconds: a 1.5-fold improvement overall. The arithmetic explains why an impressive decoding result can coexist with a more modest reduction in waiting. These are hypothetical timings, not measurements of a particular computer.[3]

Different uses will expose that balance differently. A short description of a photograph leaves fewer output tokens over which to recover the cost of processing its image. A longer answer offers more generation work to accelerate. MMSpec’s emphasis on diverse tasks is useful precisely because a single rate cannot describe every mixture of image processing and text generation.[4]

Support has to reach the application

The integration work is already visible in public software. MLX-VLM provides a local vision-model runtime built around Apple’s MLX framework. Its documentation describes using a matching draft model and checking the proposed tokens against the target; DSpark’s documented path requires greedy decoding, with temperature set to zero. That constraint matters when reproducing a result or comparing it with a more varied sampling setup.[5]

The model card specifies MLX-VLM 0.7.2 or later. Its release notes include changes to vision processing and speculative decoding: the model download requires a compatible runtime.[6][2]

A llama.cpp change merged September 23 fixes compatibility when converting LFM2-VL targets for DSpark, including a positional-encoding issue. Application support depends on which runtime version it ships.[7]

LM Studio’s own documentation makes the trade-off explicit for speculative decoding generally: a second model needs extra computation and resources, and an unsuitable pairing can slow inference down. Its support for the technique is not, by itself, confirmation that this Liquid checkpoint works in every LM Studio release. The useful test remains the paired models, runtime version and actual task together.[8]

Local is a property of the request

Where the request runs is a separate choice. LM Studio documents offline use of downloaded models, including local document conversations and a local server; discovering and downloading new models requires a connection.[9]

Ollama also offers remotely run cloud models through its application, command line and API. The selected model and backend, rather than the desktop interface, determine where an image and prompt are processed.[10]

For this release, a useful comparison would hold the target model, image, prompt and sampling settings constant, then measure total completion time with and without the drafter. Recording the runtime version and memory use would make the comparison reproducible. Liquid’s company benchmarks provide a starting point for that test.[5][8]