DeepSeek’s vision experiment: images in agent tasks

The official examples interleave two images with text. JSON content blocks and a TXT format containing image paths produce identical prompts and input token IDs. A vision encoder and aligner connect image processing to the language model. Comparing two device screenshots illustrates a possible input structure; it is not a BitShovel test.

The model card reports multimodal agent benchmarks and notes that V4-Flash-0731 ignores image inputs in two comparisons. Scores depend on input conditions and evaluation settings, so they do not predict gains on every image-based task.

The MIT-licensed repository includes minimal PyTorch inference. Its README explicitly distinguishes reference code from a production serving engine. Setup includes installing dependencies, converting weights and running equivalent TXT or JSON examples. The generation loop uses straightforward autoregressive sampling; including the DSpark forward path does not establish a fully optimized serving loop.

Separate vLLM and SGLang guides cover serving. The vLLM example uses one 4×GB300 node, not a minimum-hardware requirement. This experimental release is distinct from the later V4.1 Flash release and pricing. Assess output quality, latency and resource use on the intended task.