Pythia
Pythia is a suite of large language models developed by EleutherAI, ranging from 70 million to 12 billion parameters, all trained on identical, publicly available data (The Pile) in the same order, specifically designed to enable controlled research into how LLMs learn and scale.
4 resources across 1 library
Glossary Terms(4)
OLMo
OLMo (Open Language Model) is a fully open large language model project from the Allen Institute for AI (Ai2), notable for releasing not just model weights but…
Pythia (model)
Pythia is a suite of large language models developed by EleutherAI, ranging from 70 million to 12 billion parameters, all trained on identical, publicly availa…
Cerebras-GPT
Cerebras-GPT is a family of open-source GPT-style language models, ranging from 111 million to 13 billion parameters, trained by Cerebras Systems on its own wa…
RedPajama
RedPajama is an open effort, led by Together AI with academic and industry collaborators, to reproduce the training dataset and models described in Meta's LLaM…