ALBERT
ALBERT (A Lite BERT) is a transformer-based language model introduced by Google Research in 2019 that reduces BERT's parameter count through factorized embedding parameterization and cross-layer parameter sharing, while introducing a sentence-order prediction pretraining task.
4 resources across 1 library
Glossary Terms(4)
XLNet
XLNet is a pretraining method and transformer-based language model introduced by researchers at Google and Carnegie Mellon University in 2019, which combines a…
ELECTRA
ELECTRA is a transformer-based language model pretraining method introduced by Google Research and Stanford in 2020 that replaces masked language modeling with…
DeBERTa
DeBERTa (Decoding-enhanced BERT with disentangled attention) is a transformer-based language model developed by Microsoft that improves on BERT and RoBERTa usi…
ALBERT
ALBERT (A Lite BERT) is a transformer-based language model introduced by Google Research in 2019 that reduces BERT's parameter count through factorized embeddi…