Phi-3.5
8B-parameter mini text model, a mixture-of-experts variant, and a vision model, designed to deliver strong reasoning performance relative to their compact size.
Definition
Phi-3.5 is a family of small, efficient language and multimodal models from Microsoft, including a 3.8B-parameter mini text model, a mixture-of-experts variant, and a vision model, designed to deliver strong reasoning performance relative to their compact size.
Overview
Phi-3.5 continues Microsoft's 'Phi' line of small language models, whose central bet is that carefully curated, high-quality synthetic and filtered web training data ('textbook-quality data') can let small models punch far above their parameter count on reasoning, coding, and math benchmarks. The family ships in three variants: Phi-3.5-mini (3.8B parameters, a dense transformer with a 128K context window), Phi-3.5-MoE (a mixture-of-experts model with 16 experts totaling about 42B parameters but only ~6.6B active per token), and Phi-3.5-vision (a multimodal model built on Phi-3.5-mini for image and chart understanding, OCR, and multi-frame video reasoning). Because the mini variant is small enough to run on a single consumer GPU or even on-device, Phi-3.5 targets scenarios where latency, cost, or offline operation matter more than having the absolute highest capability ceiling — mobile apps, edge inference, and cost-sensitive production services. The MoE variant offers a middle ground: higher effective capacity than the dense mini model while keeping per-token compute costs comparable to a much smaller dense model, since only a subset of experts activates for each token. All Phi-3.5 models are released with open weights under the MIT license on Hugging Face and Azure AI Foundry, making them popular for fine-tuning into specialized assistants, for research on data-efficient training, and as a baseline for comparing small-model efficiency across the open-source ecosystem alongside Llama, Mistral, and Gemma small variants.
Key Features
- Mini variant: 3.8B dense parameters with a 128K token context window
- MoE variant: 16 experts, ~42B total / ~6.6B active parameters per token
- Vision variant supports image, chart, and multi-frame video understanding
- Trained heavily on curated, synthetic 'textbook-quality' data
- Open weights under the MIT license
- Runs on a single consumer GPU or on-device for the mini model
- Strong reasoning and coding benchmark scores relative to parameter count
- Available via Hugging Face, Azure AI Foundry, and Ollama