Date: July 27, 2018
TLDR: I briefly review classic experiments in estimating the entropy of the English language, and compare the performance of state of the art deep learning models and humans at predicting the same chunk of text.
Date: September 22, 2020
TLDR: Post co-written with Akhilesh Gotmare about work at Salesforce where we used smaller language models as generative classifiers to guide generation from larger language models. We show that this method can make generations friendlier, reduce bias and toxicity, and achieve zero-shot controllable generation of unseen topics.