BERT Busters: Outlier Dimensions that Disrupt Transformers
Models of BERT family are overall robust to pruning, but they have an Achilles heel: the outlier dimensions, without which the quality of the model drops sig...
Posts by Year
2021
2020
What Types of Generative Models Are There?
The world is filled with data. Can we learn from this data to generate something new?
Question Answering for Artificial Intelligence (QuAIL)
QuAIL is a new challenging NLP benchmark that combines reading comprehension and commonsense reasoning.
The Dark Secrets of BERT
BERT and its Transformer-based cousins are still ahead on all NLP leaderboards. But how much do they actually understand about language?