Pixtral
By Mistral AI
Pixtral is Mistral AI's family of multimodal vision-language models, beginning with Pixtral 12B in September 2024, capable of processing interleaved images and text within a single model built on Mistral's language model architecture.
Definition
Pixtral is Mistral AI's family of multimodal vision-language models, beginning with Pixtral 12B in September 2024, capable of processing interleaved images and text within a single model built on Mistral's language model architecture.
Overview
Pixtral 12B was Mistral AI's first multimodal release, combining a 12-billion-parameter multimodal decoder with a newly trained 400-million-parameter vision encoder built from scratch specifically for the model, rather than reusing an off-the-shelf vision backbone like CLIP. This custom vision encoder was designed to natively support variable image resolutions and aspect ratios without the aggressive resizing or tiling that some other vision-language architectures require, and the model supports interleaving an arbitrary number of images with text within its context window, enabling multi-image reasoning tasks like comparing charts or following a sequence of screenshots. Mistral released Pixtral 12B under the permissive Apache 2.0 license, positioning it as a fully open-weight alternative to closed multimodal models like GPT-4o and Gemini, and to other open vision-language models like LLaVA and Qwen-VL. The model retained a 128K token context window inherited from Mistral's Nemo language model lineage, and benchmarked competitively on multimodal reasoning and document/chart understanding tasks relative to its parameter count at release. Mistral followed with Pixtral Large in late 2024, a larger 124-billion-parameter multimodal model built on top of Mistral Large 2, aimed at closing the gap with frontier proprietary multimodal models on complex document understanding, chart reasoning, and visual question answering, while still being available for research and, with appropriate licensing, commercial use. The Pixtral family reflects Mistral's broader strategy of releasing capable open-weight models across modalities (also including Codestral for code and Voxtral for audio) as an open alternative to the closed multimodal offerings from OpenAI, Google, and Anthropic.
Key Concepts
- Custom vision encoder trained from scratch rather than reusing an existing backbone like CLIP
- Supports variable image resolutions and aspect ratios natively
- Interleaves an arbitrary number of images with text in a single context window
- 128K token context window inherited from the Mistral Nemo lineage
- Released under the permissive Apache 2.0 open-weight license (Pixtral 12B)
- Pixtral Large (124B) extends capability for complex document and chart understanding
- Competitive multimodal benchmark performance relative to open-source alternatives