Department of Linguistics

Closing the NLP Gap: Progress in Low-Resource NLP and Providing Access for Language Communities

In the past decade, advances in natural language processing (NLP) have led to the development of powerful language technologies. However, the algorithms involved require large quantities of data that most of the world’s languages (“low-resource” languages) do not have. Research in low-resource NLP aims to resolve this issue, but much work remains, and the communities that speak “low-resource” languages have been consistently unable to benefit from efforts that are nominally intended to aid them.

In this talk, I present work which advances the state of the art in low-resource NLP and also addresses the practical issues which have prevented language communities from benefiting from modern NLP. First, I present the MicroBERT model architecture, which in low-resource settings improves the quality and cost of the fundamental model for modern NLP, the Transformer language model. Second, I present work focusing on NLP models’ potential to aid language documentation and revitalization efforts, which proceeds from the observation that the single most important missing piece for exchange between the NLP community and language documentation efforts is software for language documentation which can easily interface with NLP models. Finally, I conclude with a reflection on how both lines of work bring us closer to the goal of having modern NLP easily available to language communities that desire them, and an outline of future directions.