MPT
MPT (MosaicML Pretrained Transformer) is a family of open-source, commercially licensed large language models released by MosaicML in 2023, notable for using ALiBi positional encoding to support very long context windows and for being trained with a strong focus on training efficiency.
3 resources across 1 library
Glossary Terms(3)
Cerebras-GPT
Cerebras-GPT is a family of open-source GPT-style language models, ranging from 111 million to 13 billion parameters, trained by Cerebras Systems on its own wa…
MPT (model)
MPT (MosaicML Pretrained Transformer) is a family of open-source, commercially licensed large language models released by MosaicML in 2023, notable for using A…
RedPajama
RedPajama is an open effort, led by Together AI with academic and industry collaborators, to reproduce the training dataset and models described in Meta's LLaM…