InternVL
InternVL is a family of open-source vision-language foundation models developed by researchers at Shanghai AI Laboratory and collaborators, designed to scale up the vision encoder component to better match the capacity of large language…
Definition
InternVL is a family of open-source vision-language foundation models developed by researchers at Shanghai AI Laboratory and collaborators, designed to scale up the vision encoder component to better match the capacity of large language models.
Overview
InternVL was introduced with the goal of addressing a common imbalance in vision-language models, where a very large language model is often paired with a comparatively small, off-the-shelf vision encoder (frequently a variant of CLIP), limiting the visual representation quality the LLM has to reason over. InternVL scales up the vision encoder itself to billions of parameters (InternViT), trained on large-scale image-text data, before aligning it with a large language model backbone, aiming to give the combined system stronger, more scalable visual representations rather than treating vision as a lightweight add-on. The InternVL project has progressed through multiple generations, including InternVL 1.5 and InternVL 2.5, introducing improvements such as dynamic high-resolution image processing (splitting large images into tiles processed individually, similar in spirit to techniques used by other leading multimodal models), stronger multilingual capability, and support for multi-image and video inputs. InternVL models have consistently scored competitively against leading proprietary multimodal models like GPT-4V and Gemini on public multimodal benchmarks covering document understanding, mathematical reasoning over images, and general visual question answering. As an open-source project, InternVL releases model weights, training code, and often the InternViT vision encoder separately, making its components reusable in other multimodal research projects beyond the core InternVL model itself. It's commonly cited alongside Qwen-VL and LLaVA as one of the leading open vision-language model families, and it has been particularly influential in research demonstrating that scaling the vision encoder, not just the language model, meaningfully improves multimodal performance.
Key Concepts
- Large-scale vision encoder (InternViT) scaled to billions of parameters
- Dynamic high-resolution image tiling for detailed visual input
- Multiple generations (1.5, 2.5) with progressively improved capability
- Multi-image and video input support
- Competitive benchmark performance against GPT-4V and Gemini
- Open-source weights, training code, and standalone vision encoder release