Effects of Transformer Distillation on Question Answering

This repository contains scripts, notebooks, data, and a report from a project exploring the effects of transformer model distillation on the question answering task. In particular, we fine-tune BERT, RoBERTa, distilBERT, and distilRoBERTa on the Stanford Question Answering Dataset (SQuAD) and compare model performance, model training time, and model inference time.