Red Pajama
2-trillion-token dataset and a family of trained models (RedPajama-INCITE).
4 resources across 1 library
Glossary Terms(4)
OLMo
OLMo (Open Language Model) is a fully open large language model project from the Allen Institute for AI (Ai2), notable for releasing not just model weights but…
Pythia (model)
Pythia is a suite of large language models developed by EleutherAI, ranging from 70 million to 12 billion parameters, all trained on identical, publicly availa…
MPT (model)
MPT (MosaicML Pretrained Transformer) is a family of open-source, commercially licensed large language models released by MosaicML in 2023, notable for using A…
RedPajama
RedPajama is an open effort, led by Together AI with academic and industry collaborators, to reproduce the training dataset and models described in Meta's LLaM…