I am a deep learning research scientist at Salesforce where I lead language modeling research across domains. I developed GeDi as a method to control generation from large language models that can, for instance, make text generations more friendly, and less toxic. Lately I’ve been working on protein language modeling, where I designed the generation pipeline in a first of its kind research project that used language models to successfully generate antibacterial artificial proteins.
In my previous work as a PhD student at the University of Edinburgh, I invented multiplicative LSTM (mLSTM), a recurrent neural network architecture that found widespread use, including in Open AI’s sentiment neuron language model (before GPT/GPT-2/GPT-3), and the UniRep protein language model. I also developed dynamic evaluation, an adaptation approach that resulted in large improvements to the state of the art across language modeling benchmarks.