How To Build a Question Answering Bot with BERT
The idea behind transfer learning (Like Bert) is to take a model that was trained on a very large dataset, then fine-tune that model using the SQuAD dataset.
We can use BERT to extract high-quality language features from the SQuAD text just by adding a single linear layer on top. The linear layer has two outputs, the first for predicting the probability that the current subtoken is the start of the answer and the second output for the end position of the answer.
Training the model is relatively straightforward. The bottom layers have already great English word representation, and we only really need to train the top layer, with a bit of tweaking going on in the lower levels to accommodate our task for question answering.